You are assuming that they will stay together if you divorce him. But lots of women like OW want a BF and not a live in partner. The same as lots of men like your husband want a wife at home to do the house / kids / emotional and domestic labour AND the adrenaline rush of an affair.
Thats why people say that when a man leave his wife for his mistress, he creates a vacancy. It’s not that your husband preferred her to you - he wants both and feels perfectly entitled to have what he wants , when he wants it.
I suspect he s a man who is used to getting his own way, at work and at home.
You are also assuming that your husband will see a lot of his kids if you split up and I think you need to prepare yourself that this may not be the case.
I know you say “ he loves his kids, hes not a monster” and I’m sure he does love his kids in his own way. But nothing you have posted shows a lot of ACTIONS about him loving his kids , just a lot of words.
eg he gets angry at not being able to do what he wants all the time, he won’t look after his own kids on holiday and doesn’t enjoy spending tine with them.
That doesn’t say “ dad who will want 50:50 with his kids “ to me. It screams
“ dad who will threaten 50:50 to punish you and when he finds out how much child support he will save. Then panics when he realises that doesn’t want them during the week because of the hassle of childcare and he doesn’t want them at weekends because that time is his own. Then is surprised that you won’t agree to drop them off at his mums for a few hours at short notice on the odd weekend that he has spare time, so his mum can look after them and he can pop round act like dad of the year “.
I also think he’s the kind of man who will tell everyone that he wanted the kids more but you won’t let him and he doesn’t want to fight you in court because he’s so reasonable and you are a bit unhinged.
He sounds like a man who tells everyone he is a devoted father and he’d do anything for his kids. Rather than a man who actually will.
Of course I might be totally wrong and if I’m misreading things, please feel free to ignore everything I’d said.