Do you have any children with him? I'm guessing not or you would have mentioned them. I wonder what your past history is with men, have you always allowed them to treat you this way?
Because this is your fault too.
You have allowed this man to treat you like a faithful old labrador. You've allowed him to walk all over you.
You've come running when he has snapped his fingers.
You've done everything for him.
What you need is to change yourself.
You've put this man above yourself all through your four years together. You've got up at 6am to iron his shirts, presumably because he's incapable. I'm guessing you do all the shopping, the housekeeping, everything whilst he sits there like royalty.
Now he's getting his fun on the side, safe in the knowledge that he'll still have his ironing done and his meals cooked when he comes home.
You've allowed this.
You didn't make him become a bastard, but you've certainly not helped.
Sorry if that all sounds harsh, but I think you need more that just sympathy here. If you want to get out of this dreadful situation, you need to start making hard decisions and stand up for yourself, and that will be very hard for you to do.
I suggest your first step is a visit to your doctor. You suffer from low self-esteem and are no doubt depressed, so you need boosting. Given that this low self-esteem has originated from somewhere, I suggest you ask to see a counsellor.
Now surely there must be someone you can go to? Do you have any savings that will allow you to have a few days in a Travelodge even? You need to get out of the house and be alone for a while whilst you think things through.
He's frightened of you telling her husband in case it changes his cosy arrangement. It's up to you if you want to force this change, but if he does leave you, you'll have to move out anyway so you might as well start making plans to leave.
Go to the CAB, they'll give you financial advice. If you are not working, you could rent a flat and you'll receive housing benefit to help with the rent, you won't have to pay council tax and you'll get benefits. If you do work, then you may still be entitled to help. There are options there for you, you certainly won't end up on the street. Once you know where you stand financially you may feel strong enough to make one of those decisions.
I wish you well.