I don't think you were unreasonable OP, but if you are worried your colleges think badly of you now (not that they should) you can still try to smooth things over with them.
Perhaps the next time someone mentions the party you can say "yes it sounded lovely. I was sorry to miss it, I had something else going on that I couldn't get out of" and hopefully that will be all they need.
Or, if you are feeling brave, start the conversation with them and say "Sounds like you all had a great time at the party. I was sorry to miss it but I' just wasn't able to make it this year" and then go on to quickly ask them "how was your meal" or "what was the venue like" or something so the conversation moves on from what you were doing.
Christmas at work gets a bit out of hand in some places, at least in some of the places I've worked.
It's not a problem for me this year, only one night out and that's quite a relaxed one, but in the past a place I worked had the Christmas party for the whole region, which we had to travel about seventy miles to get to, then our office went out for a meal locally, then we had a secret santa day and then a day when we all had to take in something for a buffet in the office, one colleague did a santa dash that we were asked to sponsor and another sold christmas cupcakes at work to raise money for something else.
Finally we were asked to do Christmas jumper day for the charity the company supported and then on our last day the office were expected to all go out for drinks. We were a small enough office that anyone not taking part stood out like a sore thumb and it was all just so time consuming and expensive.
And that was all in just three weeks before we closed for Christmas for just three days.
This year DH has done a Christmas meal with drinks after, has a secret santa day with an office buffet to come next week, Friday is Christmas jumper day, he has a fundraising event and then again they want to go out for drinks on the last day. He's in the forces, and works with some fairly high ranking people, so saying no tends not to be an option because organising and taking part in things like this are remembered come write-up time.
We're spending more time and money with and on colleagues than we are with friends and family. He's had more nights out with colleagues this year than he has had nights out with me.
And then we do have all the other things to fit in too, as do most other people. It's all too much.
If it were up to me and we had to have a works night out, I'd rather it was mid year before the school holidays start, when there's not quite so much pressure on time and money.
So no, you are really, really not being unreasonable to say no to colleagues, although it might have been a good icebreaker for you. Is your company small enough that you could put a plate of mince pies out one day and say since you had to miss out on the party you thought you'd celebrate with a few cakes in the office at lunchtime? That might help break the ice with your colleagues and show them you aren't stuck up or anything.