He sounds very nice!
Ask your psychiatrist where you stand on the custody front. If SS are not involved now, then why would they be involved if you were to split? Perhaps he is just threatening you? Playing on your low confidence and lack of self worth.
Courts these days don't normally take children away from their mothers without very good reason.
He sounds like a bastard. Just for saying that to your friend alone!
As the law stands, you would NOT have to buy him a house, that is ridiculous! He would have to leave and pay his share of the mortgage still. You cannot make him leave however, but you can start proceedings against him, if he is guity of adultery then you can divorce him pretty quickly.
I'm not saying that you should do all of this, after all I don't know this man. But I think you have suspicions of his fidelity anyway, and if a woman is suspicious then she is usually right. Also he sounds like a right bastard!