You aren't being silly at all. When money is a problem, all events that involve spending money are an added stress, especially in advance. This happens to DH and me every summer when all the families we know jet off somewhere beautiful and we stay at home or have a modest long weekend in the UK (still lucky to have that, but it is hard when children notice the disparity).
It really helps to let go of all conventional expectations of how Christmas is supposed to be. Hard with all the adverts and telly programmes showcasing massive excess, but anything said or portrayed by someone trying to sell you something doesn't count.
I grew up in a very poor country where most children got nothing.We got about 4 (mostly homemade) presents each. Sounds corny, but Christmas was still great fun. It was filled with love and togetherness and nice food. And for my mother, some fairly dodgy child-made necklaces, one or two of which she still loyally wears 40 years later!
However, you sound sensible and organised, so do remind yourself that you CAN do this and it WILL be fun. Add an extra dose of silliness to it! That's what I did for the 1st Xmas after my father died, and it helped a lot.
Why not try really silly joke presents (whoopee cushions etc), daft music playlists and 'Dad Dancing' competitions, printing corny jokes off the internet for people to read out-a favourite in our house-, games night, or fancy dress themes? Film nights with popcorn go down well, we did that and showed The Princess Bride recently, which the children loved.
Last year I told the family the dress code was 'tropical'. Seeing my usually very staid mother in a huge pair of pink plastic star-shaped sunglasses is now one of my best ever Christmas memories.