My main question is if you are eating at about 1, why are you stuck in the kitchen until 4? If there are other adults there and you've done most of the cooking, someone else should be washing up, getting coffees and snacks etc surely? We've never done xmas dinner in evening but I always imagine the whole day becomes build up to it then, whereas I like the chilling out bit after.
In our house DH is main Xmas chef. We do as much prep as we can before the day, eg I normally make a freeze a sausage meat stuffing in advance, DH does bread sauce day before and reheats in microwave. Some peeling and chopping night before. We only have a turkey crown as usually 5 or 6 (one set of grandparents each year) of us, so cooking times not as long as some of these posters. He always has a schedule written out of what needs to be done when, and we pick a quite time in that to all sit down a give presents in the morning with mince pies or biscuits. (DC will already have had stockings and main gift from Santa first thing).
I'm in charge of most other hosting - simple breakfast, coffees and snacks, washing up dinner which grandparents usually take charge of with me drying and putting away, snack tea things for later etc. So although DH is in the kitchen most if morning (and prefers to be left to it than helped too much as we just get in his way!) He certainly isn't there until 4, and we do make time in schedule where we are all together in morning too.
Afternoon and evening are about board games, films or Xmas TV specials, a little walk if dry after lunch etc
However, I agree it's not just one day. Build up always nice, and I always particularly enjoy the week after when I am usually off work and we often have some Xmas get together with our siblings or the parents we didn't see on Xmas day, with less pressure on specific expectations but still lots of nice food and presents!