It's a date. Depressingly, I shoehorn DH into conversations fairly quickly for this exact reason. I work away a lot and it's a hazard of the job. Tell him.
I can kind of see from where OP's pov but I think her mistake was agreeing to a weekend drink.
I remember once when I was in a similar position, I had been talking about going to the pub on Saturday to meet non-friends. A guy at work was listening and sort of hinted that he might come which I just kind of ignored, but he turned up, he seemed to think we were on a date...
When I was politely puzzled and distant with him, he told me I had a "bad aura" and was grumpy with me until I left.
I'm single, but when I'm at work, there to work, and yes part of that is being friendly - but really, no other ulterior motives, at all, would never cross my mind.
So friendly chat is just that, it wouldn't occur to me that anyone would see it as anything other than that.
I also compartmentalise work and home, I don't want new work colleagues to know too much about my private life.
Depressingly, as a single woman, near most everytime I chat to a new male colleague they always shoehorn in a GF/DW reference.
I mean, as if?
I'm there to work, not find my OTP. I find it kind of insulting really.
Or maybe asking "do you want a cup of tea' at 10am is code for "It's you, you are the one, I will not stop until you're mine"
Anyway, I'm Audi 5000...