LOL at the Celtic words in the tattoos. I speak Irish and if I had a farm I would bet it all on the 'Celtic words' being misspelled and nonsensical...
You are dealing with a narcissist.
Your only way forward is to completely ignore him.
He will go nuts from your indifference.
What he can't stand is for anyone to drop out of his orbit. All those texts, etc., are designed to provoke you, keep you dangling, and keep you as a satellite of him.
Ignore, ignore, ignore, ignore, ignore, and then ignore some more.
Proceed to court. You may even get costs awarded. He will not be allowed to thumb his nose at the judge.
He is acting right now as if he can make up the rules and you have to obey them. Going to court would be a fitting comeuppance. It is sometimes a real shock to a narcissist to find there is an authority higher than them.
If mediation is suggested, REFUSE. That would be the road to insanity.
Get him on an every other weekend schedule with the children.
Go through your calendar and for every single holiday, decide who will have the children those days -- Easter, Christmas, Christmas Eve to 6 pm, various Bank holidays, New Years Eve, New Years Day. You have them on Mother's Day always and he has them on Father's Day always.
What are you going to do on the DCs' birthdays (I recommend they have a celebration meal in a restaurant with only parents present, or have a birthday party with other children, at a venue, and whoever has them on that day pays for the event).
Events and activities the children do on weekends will both parents be equally obliged to facilitate the DCs' social life on weekends when they are at one or other home? For instance, DS gets invited to a party that will take place on a weekend when exH has the DCs you want DS to be able to go and you want exH to take him. You get the invitation at your home -- do you forward it to exH or rsvp yourself? I recommend forwarding it but I also recommend he being obliged in a parenting agreement to put the children's social lives and other weekend activities like ballet, football, etc., ahead of lazing in bed or anything else he may have planned even if that means he doesn't get to drool all over the GF all weekend every weekend.
What about school holidays? How will you divide up parenting time or childcare expenses for school holidays? What about a summer holiday time when each parent can take the children for a holiday and if you want this, how long? What about permission to take them abroad?
Grandparents are they ever to be in the sole charge of any grandparent? If yes, for how long? How about finding alternative caregivers when they are in your care or his do you want you or him to have 'first refusal' or could he, for instance, dump the DCs with the GF if and when her charms begin to fade and he starts another affair.... Think about this one (remember, sauce for the goose is sauce for the gander, sadly).
Medical care -- who can make a decision about emergency care that may be needed? What about routine care? Are the children to be packed off for a weekend or holiday if they are ill? What is the definition of 'ill'?
Are the children to have a set or two of clothes at the home of the non-res parent so that the other parent won't have to pack suitcases for 4 DCs every other weekend? The other parent should buy the clothing (underwear, change of clothes for Sat and Sun, pajamas, socks, etc) and be responsible for laundry and replacing when the clothes get too small.
Who will pay for items like dancewear, swim lesson suits, gymnastics wear, football kit, school uniform and school shoes? These can be expensive.
If he can't make it some weekends (you can expect this to happen as he will tire of the responsibility and the GF will also get fed up of childcare and having her weekends filled with cooking, etc.,and also because he will want to mess with your mind, see if he can get a rise out of you) -- are you supposed to rearrange and throw your plans out of whack? I recommend if he can't make a weekend or other arranged parenting time he forfeits. That way you avoid communication and the danger of being dragged into upsetting negotiations and tiresome back and forth with him.
Dispute resolution -- you need to name a mediator who will try to help resolve disputes that arise once the agreement is up and running. There needs to be a proper procedure, and if costs are involved, they need to be equally shared by both parties. This will discourage frivolous complaints. 'Family Mediation' can help. There is a UK association..
You need to also secure the right to move wherever you want with them within the UK.