I can completely understand how he feels—if I were in his position, I’d probably feel the same. The way you’ve described things makes it easy for him to doubt your intentions. Saying you were drunk and don’t know how long the kiss went on raises suspicions—it can look like you were, at least in the moment, okay with it. That’s why it seems mutual to him, not just a one-sided mistake
The way the night played out also adds to the suspicion. From the outside, it almost looks like your friend deliberately lagged behind and left you and the married coworker alone, as if to give you two a private moment. That gives the impression of attraction or consent, like you were given space for a private moment.
That makes it harder for others to believe it was entirely unwanted. Then afterward, you all continued the evening with food and a taxi. Most people—including police, if it were ever reported—would see that as inconsistent with being blindsided.
If the allegations were true,
Your coworkers aren't responsible people.
Your coworkers also didn’t have your best interest at heart. If you were that drunk, they should have made sure you went home early. Instead, they let the night carry on in a way that left you vulnerable.
Meanwhile, the man you’re interested in has his own questionable behavior—admitting to “platonic sleepovers” with an ex. (You may already know that he's involved in a relationship and are okay with it) if the roles were reversed, you wouldn’t accept that, so it makes sense he can’t accept your version of events either.
For your own stability, I’d recommend leaving this job, cutting off all coworkers permanently, and never mixing work with friendships or romance again. It creates unnecessary drama, risks your reputation, and can jeopardize your livelihood.
The biggest issue is drinking. If you’re blacking out or losing track of events, that’s not just bad luck—it’s a problem that needs attention. Therapy could help you process both your grief and your choices, and cutting out alcohol could save you from repeating this. For now, focus on healing yourself first.