Then unfortunately, you're stuck. You can't make your exH have them.
So I guess the answer to your question "Does that mean I should live the rest of my life in misery, being shouted at?" is probably 'yes' as long as you keep arguing/asking your exH to have them. And if that's the case, you're going to have to find some way to make peace with yourself over it. Because in the end, you're only upsetting yourself, you aren't upsetting your exH one tiny bit. And if he's doing it to punish you, you're actually making him happy by reacting. Sucks, but there it is.
I don't know the actual 'dynamics' of your conversations but I assume it's you ask, he says no, and an argument ensues. So why do you continue to butt your head against that stone wall? I mean, he's not going to change because you demand he does nor even if you ask nicely, is he? What do you think would happen if you simply stopped asking and bided your time. Would he eventually step up out of love for his children?
But also consider whether or not was a caring and involved father to begin with. Did he do 50/50 with you on caring for all of the children before you separated? Did he take them all out or care for them all at home on his own, even if you were there? Did he happily send you off for an evening with friends without frequently calling/texting about some 'disaster' or other? Because if not, it's not likely that he'll ever step up, and it would be more to do with his inability to parent rather than you going out.
Sometimes we do what we think we should do (or want to do) and we don't get the result we think we should get. This may be one of those times.