I hope this doesn't sound rude, because I'm not meaning to but have you met many Tollers?? I've been researching Tollers for atleast 10 years now and have spent a lot of time with them in person. They're truly the enly thing I care about besides my family and dogs. I'll be getting a purebred in 2 years and will never be without one after that. In my opinion, they're not comparable to the Golden at all except for the fact that both love to retrieve. They're extremely active, always on the go and wanting something to do dogs. They constantly want to play, you can expect a ball to be dropped in your lap and have a Toller stare at you waiting for the ball to be thrown atleast once every 15-20mins. They're also extremely intense, highly intelligent (stubborn) dogs. Yes, they're willing to please but there must be something it it for them or else they see absolutely no point in doing what is asked a lot of the time. Many Golden people have Tollers also and they find that the Golden is lower maintenance and a whole lot more calm than the Toller. Many Golden people with Tollers that I've met, say they will stick with their Goldens and won't have another Toller. They're very reserved with strangers and prefer to watch from a distance, where as a Golden usually loves most everyone. They've also got their Toller scream, whcih sounds similar to a 2 year old child in a tantrum. It's not pleasant, but, Toller people put up with it because we love everything else about the breed. Many Border Collie people also have Tollers...Tollers are similar to the Border Collie, except instead of the drive to herd, they retrieve. They're almost just as active as BCs (A little less than the BC, but more than the Aussie) and have TONS of drive and energy. Now, I'm just speaking from my experience and maybe yours is different. It could also depend on the breeders, but this is generally ow the Toller is.
However, all of that being said, I think a Toller could possibly be a good match for you. They range in size from 17-21" at the withers and are between 37-55lbs usually...which is a little smaller than you'd planned but not too far off. The males are bigger, obviously. I personally feel the Aussie would fit you better but if you have any specific questions abut the Toller at all please feel free to PM me or post in here