He’s using access and the threat of court to cause drama.
What would happen if you just agreed? I don’t mean roll over for any ludicrous demands, but for a week, sure, why not?
Either your child will have a lovely week with their dad, or they will miss you and tell him they want to come home and he’ll probably cut it short rather than deal with an upset and whinging child!
I know it’s not easy, but in cases like this, where there has been emotional abuse of you, but he’s still clearly capable of looking after the child (and if not, then the kid shouldn’t be there for 4 nights anyway!), the best course of action is to take the heat out of it and try to take his power away.
He wants a week, sure. He wants to go 50/50 - sure. He wants to take the kid on holiday, why not? You’d want to be able to do the same. Christmas days, alternate. One parent doesn’t get every year, no matter how miserable a day it might be for you, the child gets to spend it with a parent who wants them. That’s priceless.
How this usually goes is that the feckless dad realises he’s not having any impact and will try something else - probably returning the child earlier than agreed. (You’ll be out!) or changing plans last minute and not turning up.
Essentially, if it’s a game to him, don’t play.
If he’s serious, then you can try out different arrangements until you find one that works.
I know it’s hard. I always felt very aggrieved that my XH had been such a shit dad when we were married and now he wanted them for two nights in a row or to take them on holiday with him etc. It soon dwindled as he realised it was hard work looking after 3 DCs on his own!!
Good luck with it 