@BusyWren - it is so helpful to realise that there are others in the same boat, honestly, right now there is nothing to stop you from saying to XH "this isn't working for the kids, it's not in DSs best interests to be left home alone for hours whilst you go to the gym / pub / nightclub, and DD would like to see you when she's not out with friends / studying / watching paint dry, so this is what's going to happen from now on....."
If he doesn't like it, and he decides to take you to court, he will have to attempt Mediation first (and you should receive an invitation to mediate) which you could then try, to put across your children's wishes and feelings, but no doubt he won't listen and will obtain a MIAM certificate from the mediator, and pay the £267 court fee to raise a C100 to start child arrangements proceedings.
The courts are currently inundated, no joke, so by the time you actually get your First Hearing (FHDRA) having spoken with CAFCASS (children and families court advisory service - all of which are ex Social Workers), your kids are going to be at least 6 months older than they are now.
The older they are, the more their wishes and feelings bear weight in the family court.... there aren't many judges / mags out there who make orders for 13 going on 14 year olds, I can assure you.