So you say you couldn't leave your awful husband because of your 4-year old son but the moment you start an affair you wanted to leave your husband for the other man.
Yeah, right.
I think the issue here is that you OP sound like someone who is terrified of being alone. You can only leave one crap relationship if you think you have another better one to go to. This affair is and always has been a crock of shit but you won't see it. You are clinging on like it's a lifeboat full of unicorns and roses, captained by your knight in shining armour. 
My plan was to move away from my husband for a bit in a shared accommodation and then move in with the other man as he assured me he loved my son. Hindsight is a great thing! Thank you for your patience! No way i will ever let a man come into my life until my son is grown up. Xx
Right. So you've gone from wanting to move in with this man and you've already introduced him to your child by the sounds of things
to saying you will now never be with anyone until your son is grown up. 
For goodness sake, get a grip. Life doesn't have to be all or nothing. SLOW DOWN. Stop lurching from one extreme decision/emotion to the polar opposite.
Why did your family not accept your husband? What was it about him they didn't like?
You gave up everything to be with him and yet within a couple of years he's apparently so awful that you've embarked on an affair with someone who doesn't sound much better. 
It seems to me you throw caution to the wind, always following your daft heart and not your head. You are getting completely swept up in the drama of it all, romanticising these bad relationships. First of all an 'against all odds' marriage to someone who, as it turns out, is horrible to you, and then a fairly seedy and run of the mill affair with a man who casually uses picks you up and drops you when it suits him and only calls when he wants money or attention.
Let the scales fall from your eyes and see that neither of these men are any good for you. Find the strength to be alone for a while. It's very character building.