Well, if they got loose and bred, particularly in places that don't have say weasels or mink, they could be a problem. That's why they are all sold neutered. Which should eliminate the problem. A single ferret can't do much harm, and even if they could, most escaped pets would die .. . its the ones that make it and breed that should be the problem.
So: Environmental Hazard: solved, they are fixed
Rabies: fixed, they have shots for them now AND how does an indoor animal catch rabies anyway?
Wild Animal: Learn your history and don't give me that crap about not enough anatomical difference between a polecat and a ferret . . . how much difference is there between a domestic rat and a Norway rat?
Vicious: Can be, but have you ever had one?
Its fear plain and simple. People are scared of them, don't like them and think they smell. So they are banned. I will note they are not pets for everyone and they should NEVER be alone with small children and babies. EVER. Even more so than a dog. That baby who had its toes eatten off, and they blamed the pit bull, and it turned out to the be ferret? No surprise. But frankly, if you let carnivores play with your children unsupervised, that says more about your parenting than the animals.