"I know him so well and know he would never cheat" NOBODY knows for certain that another person will NEVER cheat, a lot of people don't even know THEY will NEVER cheat, I've yet to meet anyone who can honestly say they have NEVER at the very least come close over the age of 35.
"We were so happy first 3 or 4 years though and we love each other. Maybe we just need to compromise better." That's you starting to fall for the sunk costs fallacy - also if you were REALLY happy then you wouldn't have split the first time.
He is investing emotional energy in another woman he is attracted to even IF he isn't actually doing anything physically, energy he SHOULD be investing in your relationship.
"When we were on off for a while, I was in touch with one of my old flames, too. And I knew I just needed a friend and yes, an ego boost, so I don't 100% see him talking to her as being disloyal." That just shows YOU aren't fully committed to him either to be perfectly honest.
"I am sure he would not cheat as we discussed cheating many times and he has strict beliefs about it."
My ex husband was exactly the same - guess why we're divorced? He was highly critical of others infidelity (which was all around him as he was in the army) he is introverted, even shy in a crowd... Didn't stop him knocking up a mutual friend and colleague. It was SO out of character his family were genuinely concerned he might be ill!
Frankly at your age such naïveté is embarrassing.
Children of cheaters are more likely to cheat themselves too - not just my opinion several pieces of research back it up.
"he only does it as we have issues..." Those issues alone are reason enough to end this
"Having fun together, we do try" at this point in a relationship, when you have few real stressors, are when it really should be easy
"Money is a problem, too" you resent his spending, he resents that he's given up a well paid job to suit you.
Top 3 causes of marital/ltr disputes?
1 money
2 chores
3 infidelity
You've already got 2 ticked!
"And yes I want kids and marriage eventually but I am not desperate for it." You're 35, realistically you REALLY don't actually have a lot of time to mess about on the baby side of things! Myself and others that say so are often flamed for saying it but the reality is fertility declines, risk of mc, other pregnancy complications and having a disabled child all increase as you age. If he strings you along until you're 39 (which is 4 short years) and then dumps you for her or someone else because HE realises this relationship isn't ideal you're pretty much screwed! Unless you opt to go it alone with donor sperm (which is by no means guaranteed to work either) you're looking at
another 6 months to a year dating trying to find someone half decent (which is bloody hard as the good ones tend to be taken and the shit ones are all that's left!)
At least 1-2 years building the new relationship, hoping it works out
Another year or more ttc...
By which time you're 43 and your chances of motherhood are pretty minimal (however much this site tends to skew in favour of older mothers the reality is it's much harder to achieve motherhood after 35 let alone 40!!)
You're flogging a horse that's not only dead it's supplying the glue factory!