Why do you think a ban would do anything at all? 'Coz it worked so well with drugs....
The EU recently attempted to sign some sort of ban on designing AI derived robot soldiers. Which means said technology will be developed at will in other nations....until they invade us. The same is true for sex robots - they are here now, the tech will become easier and cheaper because it's derived from worthy developments (like AI careers for less abled or elderly people). They are not going anywhere.
Returning to the OP from another direction; it's the same old tired debate about what causes unwanted behaviour. We're told TV and films cause violenet behaviour. We're told porn causes rape. We're told fashion causes anorexia. And all of these rely upon the thing (sex robots, porn, movies) as being the 'new thing' that twists behaviour. Thing is, none of this is new. Evidence? Read the bible. Duteronomy 22:28-29 - explains how a man can meet and rape a virgin, so long as he has 20 silver sheckles to pay her father. She is then married to him and may not divorce. There you go, divine permission to rape away. But you think some guy who added a voicebox and more realistic feel to his blow-up-Betty is the problem?
There has been rape porn for ages. There is a lot of rape-based fantasy literature for women. Did this lead to a massive rise in women offering themselves up for rape (if that's not too oxymoronic). A lot of porn is based upon multiple partners; are we all swinging away every chance we get. No. Because (despite the views of many on this thread) even men (yes, those brainless penis driven halfwits) can tell the difference between human being and thing.
And, OP, yes those men may be sad and freaks. Perhaps because they struggle to connect with other humans easily and have thus been relegated to lonesome lives - by those people calling them sad freaks.
BTW, I'm in no way defending the invention of these things. I am, however, saying that before you all leap on your high horses, you kinda need some evidence that harm is caused. There's a place for people who embark on campaigns of ban-this and hang-the-bastards based upon nothing but their hunch that it doesn't seem right to them: Daily Fail.