I think there are two sorts of people- very Christmassy people, and people like us, OP. I don't really want to do secret Santa either-I don't like shopping, thinking of a gift, creating more tat to go to landfill, anything that feels like admin etc-but I have decided from now on to just humour the festive folk and do it with a smile. You're not going to win this one and it does seem to genuinely make them happy, and I'm all for that even if I like it spread out across the year rather than squeezed into two weeks in December.
I now have a plan. In January, I normally buy something generic in the sales-fluffy socks, whatever looks like a good deal, Christmas decorations perhaps, and then I put it into my drawer of gifts. If I know what the person I get for secret Santa might like, I'll get them that, which isn't too much bother, but if I have to think about it, they get my January bargain.
It's secret. Nobody knows who bought it. I don't overthink it. I'm also not ruling out redonating what I got in last year's secret Santa as a gift or, actually, back into the same secret Santa pot next year. It's secret! If the person who bought it got it back, they probably liked it. Win, win! I smile, am grateful for my gift and realise that not everyone gets lots of presents at Christmas. I don't need a gift but there might be one person in your workplace who will be genuinely delighted with theirs(probably, in all honesty, not one selected by me) and it's worth doing it for that.