I remember your other threads.
I think you're over thinking this.
The thing is, you're an incredibly busy lady, I get that. But this man should be adding to your life, making it happier, making you feel special. Everything should improve because he's in your life.
Instead it seems that he's added pressure, extra work, his kids descending at weekends, etc.
It's not working, so end it.
The balance is skewed, you're giving, he's taking.
Stop worrying what other people, family think, it doesn't matter.
Like my nursing daughter says, 'Sometimes the juice just ain't worth the grind!'