Just a heads up - a court order probably won't help at all with an exDP like this. Mine is similar and he actually took me to court, got everything the way he wanted it (long story) and still breaks the agreement every week 
we have been separated for 4/5 years now and at the beginning he was awful for this and (in retrospect) I was terrible for replying when I didn't need to.
My DF was a gem - him and DM had had an awful divorce so he was more than familiar with the situation and when I was wound up and ranting about ex he would just say to me "is it about contact arrangements? Is it to do with DS safety/immediate health concern? No? Then it isn't your problem, don't engage with it" he also told me to never ever respond when I was mad.
It was excellent advice that I follow even now. When ex comments on my parenting I just go "Ok" and let it be water off a duck's back while smiling sweetly. DS is a happy, bright, polite child and everyone who's ever looked after him has said I'm clearly doing something right so, the way I see it, ex's opinion just doesn't matter. When he attempts to alter the agreement re contact every week, DF taught me that if it suits me to say ok, if it doesn't just to reply with something like "sorry I can't help with that this time, you'll have to make other arrangements" - you don't owe him any explanation as to why or what you'll be doing instead. Then just don't engage further.
As your DC get older, they will begin to realise that it is him that is the cause of them feeling upset not you - DS (sadly) is already starting to realise this and commenting on it aged 6 despite me never bad mouthing his dad while he's in earshot.
Ultimately it is them still trying to exert control over you, and, frankly, he has none so stuff him 
just be thankful he is the gf's problem now and not yours. Keep taking deep breaths and count down the days until DC are old enough to facilitate contact themselves.