I did this years ago, & it's mostly worked out for all concerned...but it's been a tricky journey, legally & in terms of subsequent relationships all round.
Mine have always spent court ordered holidays with their father. Don't underestimate how limiting that can be - I've so often put fun travel ideas on the back burner because the kids couldn't join me (although I've got to do some fabulous solo travelling...).
My ex was an absolutely copper bottomed abusive, rapey, LTB, Disney dad dickhead, whereas yours doesn't sound actively awful.
It's a legal nightmare (& expensive) & the mood has shifted since I did it - I'd say if he wants to go to court, your chances are below 50% now.
My youngest has also now decided she might fancy 6th form in the UK, which means living with her dad, & this is a worry because honestly - leaving aside the fact that he's an absolute prick - I wouldn't trust him to keep a Swiss cheese plant alive. But she has, of course, the right to make that choice. So you need to be aware of that possibility - if your dc decide once they're older to bail on you, they can.
Feel free to pm me, but I would say if he's not on board you're probably buggered.
Dc & I have zero regrets, though. It's been an absolute blast & their opportunities have been amazing.