Actually it doesn't have to be lonely. You make new friends and life is enriched somehow, once you end things with the cheating spouse, you feel a huge weight lifted off your shoulders. I've found that.
What you have right now is a farce really though; having to check up on him, not trusting him. His cheating. Even if it is so far just an EA, it will eventually develop into more. He has form for this.
Oh and I am dating a gawjuss pilot right now and have 2 young kids, so I really wouldn't listen to men in your family. Life is what you make it and is this what you want for your life? Actually are these men all cheaters too, is it your Dad you've spoken to?
As I said on a thread only the other day but it's worth repeating. I know you want a stable father figure for your children, but it is equally important that your children see their mother being treated with love and respect, both by a partner and by herself. At the very least, they will understand that their mother values herself too highly to be treated poorly, and this is a far greater example to set for them. You're not doing your kids any favours by allowing this to continue imo. Hard to swallow, but I cannot believe for a moment that the home environment right now is loving, calm and peaceful for the kids with the sort of mistrust and others issues swirling about 
I suspect it may be that when the dust has settled and you have some distance on it, that you will see the relationship differently. Everything is clearer in retrospect. For when you give up the fear of losing him, you can calmly evaluate - is this what I want? And I found that to be so true. In fact I'm glad I'm not with my ExH now but I wouldn't have believed you at the time. I was desperate to make it work.
Take it or leave it. As a single parent with a 3 and 5 year old now, who was in your shoes not so long ago. It's honest advice, but only based on my experience.