As a civil servant, I have very rarely been able to take Christmas Eve off. It is expected we go into the office in the morning, but they close up at lunchtime.
DPs on both sides live 2.5 hours away, and it was further when we were going out and newly married as the motorway network wasn't finished so we still had to go through a number of large towns which were always busy that afternoon, as were the roads generally. So it wasn't unusual to take 4 hours or more to get "home" on 24th.
And as Creche was already closed, dd had to come with us to work, then pack up and go.
It was a pain - and we did stay in our own house a good few years because travel was too complicated when added to family stresses. And travelled more often in the years we could do it over the weekend.
You plan ahead, make sure you have a car picnic for snacks en route and get a hot coffee as you set off (and preferably have a decent meal for lunch before leaving). Know where to stop for rest and convenience breaks. Fill the car with petrol/diesel in advance, so breaks are for convenience or rest mainly and fuel is a bonus. Make sure the car is checked for tyre pressures, oil, washer fluid and couple of days before.
Have everything packed the evening before, and if possible, that includes the car. AIM to leave straight from work, and not need to go home in between. Get a good sleep on 23rd.
Think of things to keep dc entertained (it might not be a normal thing, but maybe playing a dvd is a good thing on this day, for example). Make sure you have dc well fed but not too full (so they are comfortable but less prone to car sickness), warm enough to maybe have a nap, (maybe bring a blanket to throw over them), and a list (in your head or on paper) of car games to play or songs to sing etc to pass time when they are awake. I spy, how many yellow cars, who sees a bus first....that kind of thing.
It's all doable - but tell manager that you need to leave as early as possible because you are travelling (and tell them how long) but cannot book leave - they may be able to let you leave earlier than usual (even for 24th) or even allow you to take leave after all.