The he making you think you are going mad is classic gaslighting. Which is very harmful mentally.
I did similar, stayed after first affair as I felt very strongly that why tf must my life now be destroyed because of his actions. I also love my home and I didn't want the DC affected. That lasted for 5 years. Without me realizing, I did the pick me dance and I was the one who kept things going for the next few years.
Fast forward to me finding out about multiple affairs, and I just couldn't do it anymore. I had reached the point. But the affairs were actually the least of the problems... The worst part was the way I had been treated, the gaslighting, the lies, the moods etc etc. It really did a number on me and that part has been the most damaging.
We are divorced now, and the DC and I still live in the same house. The DC didn't get off unscathed mentally though, what they seemed to learn is that is was OK to live walking on eggshells...
And then my dd went into a relationship which was both physically and mentally abusive, but she didn't see it as a problem. That was the slap in the face for me as a mother, as I really felt that I taught her that it was OK to be treated that way iyswim.
It's not straightforward and it's not easy OP, this I know. As others have said, you don't need to decide anything now. But get yourself into a better position for long term. You say you don't have time for xyz, you have to make the time and put it into yourself. We often tend to put others first and realize a bit late thst we have left ourselves vulnerable.
I also placed a high value on material things. But that has changed so much now. We have what we need but tbh, not too much more, but what I am able to do is to sleep well every single night without things playing over and over in my head and trying to work out, did I just forget that he told my xyz, because I really don't remember it. (he hadnt). Or where is he? At work like he says, or not really.... Is that perfume I smell?
You can play him at his own game OP. Stash away what you can, don't believe much he tells you... Elevate yourself but keep your reasons to yourself.
🌻