so his threat is that he'll get a court order? And that worries you enough to let him continue seeing them, even though you think he'll leave the country with them?
And what exactly do you think will happen if there is a court order????
A court order only tells him when he can/cannot have contact. nothing more. it is hardly a threat, and at the moment the only person who tells him when he does/does not have contact is YOU.
If you have serious concerns that he will abduct your children, then you STOP contact. NOW.
Write to him and tell him this (obviously it is better if you have proof of him saying this, but it doesnt matter if you dont) and then refuse to hand over the children. He will then instigate court proceedings and you will explain why he shouldnt have contact. the court/cafcass will decide on the risks and advise/order accordingly.
Alternatively if you are in fear of him, you go and see a solicitor tomorrow and explain the position. they are likely to then go to court without him on an urgent basis and get you an order stopping him from hurting you, and an order to say the children should live with you and sould not be taken out of the country by him, this order will last for a few days until the time that you have to attend court with him for the court to take a decision on how to proceed.
You should only do the latter if you are in fear for your safety. That course of action speeds matters up within the court and gets your hearing date sooner.
if you are not in fear of him, or believe (as you should, because the courts hope you will take this action first) that the police will help you if he does become violent, then you should follow my original advice of stopping contact and telling him now. he will then seek legal advice, file an application for contact in court, and you can then go to court (with or without a solicitor) to put forward your concerns.
Hope this helps.