I would forgive my husband for infidelity of a sexual nature far more readily than I would forgive him trying to take financial advantage of me. I think the difference is that sexual infidelity is perhaps a sign of weakness or giving in to impulses but taking financial advantage is quite calculated. Somebody has thought, I know, I can take financial advantage of my partner and this is how I'm going to do it. I could not imagine finding out somebody setting out to do me financial harm and carrying on the relationships with him. The stuff about keeping your enemies closer can be taken too far.
I have been married for over 30 years to the same man who, to the best of my knowledge, has never strayed in either respect. We still have very separate finances which we agreed at the very start but we split household costs and the costs of our children down the middle during 30 of those years My husband retired slightly early because of health concerns and I support us both now but that is my choice after 30 years of marriage. In the latter stages of his career, after the earlier part that made my mother think I'd be a young widow, he was less pressured than me and he did a lot of stuff that enabled me to concentrate on my career - he looked after the children a lot as he worked from home before it was a thing. Now he attends to a lot of the household administration and cooking while I continue to work so I come home to a hot cooked meal.
We both scraped by in earlier years to get our mortgage paid off. I ended up working up to the week before my first child was born and up to the day before my second and I was back at work six weeks later after a C-section both times. We drove old cars and the children had second-hand clothes and toys. Most of our furniture was second-hand and so on. I worked all hours like a dog to progress in my career he picked up the slack As a result of this, we are very financially secure for retirement and have a lovely large detached house in a very nice neighbourhood. We put both our children through university - one of them did medicine as a postgraduate in another city and we financially supported him through that.
The idea that if anything happened to my husband I would willingly hand over a share of my children's inheritance earned by my sacrifices (and the sacrifices of my husband) to a grifter like your partner would be completely unthinkable to me.
Your partner seems to have had quite a pleasant life. Previously, he lived in expensive housing. You say that since living with you he has had lovely trips/experiences with close family members and his children. He has paid off his car loan and saved about $20k. And yet he still wasn't satisfied. He thinks he should live in your house with making no contribution whatsoever to the holding costs of your house and that he should get a share of your house or some other house he wants you to purchase together. Meanwhile, he won't have the legal liability of a mortgage but with you ready to pick up the slack if he decides at approaching 50 to set up this new business of his.
As for him saying you are thinking of you rather than us, I'd fall over laughing if it wasn't so sad. He is thinking of he and him and not any us. When my husband received a concerning diagnosis, I realised that I'd been wailing about how scared and upset I was and I felt ashamed when I realised that I hadn't been thinking about him. He told me that I was entirely justified in thinking about how it might affect me and there was nothing selfish about that.
I would have "your partner" out of your house so fast he'd be at risk of friction burns on his heels. My sons would probably be flanking me holding pitchforks.