The financial arrangements and the XW not working aren't relevant.
I can see why the kids don't want to come to your house EOW. Leaving aside their opinion of your kids (I'm sure they're lovely, but they are strangers to these kids) of course they don't want to be 60 miles from home, in a stranger's home. Do they have bedrooms at his house in their town? I expect so. Would they have the same at yours? I expect not.
You say that this is about his tiredness not about you, as the kids think. But you also say that he used to go to them every Friday unless working - but now he spends most Fridays with you. I'm not saying he is wrong to make that choice, that he isn't allowed to have time for himself or ever change contact. BUT - be fair to these kids, of course it looks like it's about you, because he now spends the Fridays he had with them, with you!!
He's always worked away and gone home at weekends. So why is he too tired now? Has work picked up? Has he just got older? Are you sure he's not just more pissed off with the travel because he'd rather be with you than the kids? (that's not judgmental - I am divorced and have a new partner. There are nights I get back from work travel when I could just collect her in time - but I choose to let her stay with her dad so I can have time with my boyfriend)
Is he actually more tired because he used to stay in Town B, with 2x200 miie drive to Town A, but now he's doing some 140 mile drives midweek from Town B to you in Town C?
Honestly, I think he should still be seeing his kids Fri-Mon. He doesn't see them in the week. If he's too tired then he needs to employ someone in his business and reduce his income, or work Tue-Fri every other week, or something. Downsize the house in Town A. Or have less 2 week summer holidays and French colleges.
Yes, it's tiring working full time and having the kids at the weekend. But it's not unusual. And as he's self employed he has more scope than many to adjust his hours, albeit with impact to income.