OP, can you actually afford to set up on YOUR OWN?
Would you leave your husband, if this man wasn't on the scene, as it seems very unlikely, bearing in mind how long you've been with him, and the things you've put up with?
Ask yourself how you feel about setting up a new home on your own, and how you will cope if your husband gets nasty.
I think in these circumstances, I would want the new man to leave his wife first, and set up his own place, so that he can prove his commitment to you, if you are convinced that he is totally genuine. If he does this, please don't make the mistake of moving in with him, unless you know for sure that you can afford to run a home alone, because as other posters have tried to warn you, things might seem perfect now, but with an upset and angry wife, perhaps begging him to come home, and an angry and aggressive husband, likely threatening all sorts of damage to you both, things might suddenly change. He may feel guilty and go back to her, or you might start arguing, because one or other of your ex's cause aggro, you may even find that he has disgusting habits that you find you simply can't live with, so you DO need to be prepared to live alone both financially, and mentally, before you take the plunge into this relationship long term.
However, having said all this, in your shoes I would DEFINITELY leave your husband, as he clearly doesn't treat you well, and you DESERVE BETTER!