So he’s the one behaving badly, but you feel you have to be the one that leaves?
A long time ago, when I was young and pretty, I had a boss who took a shine to me. It got awkward and embarrassing for me. He tried to kiss me when he was giving me a lift, amongst other things. He was older and married, but I decided to leave the job. When I gave in my notice he actually asked “is it because of me and I lied and said no. But of course it was. I was 24. Ironically, he was the HR Director.
When I was even younger, I worked in a shop where the only other person was a single much older man (in his 60’s). He knew I had left home and wasn’t flush for cash. He bought me a clock radio. I thought that was very kind. But then he got in touch with my flat mate and they chose me a “music centre”. Then he asked her for a photo of me, then I found him searching through my handbag when he thought I’d gone to the loo, so I left that job too.
Now I’m older, I am so cross with them both. Yes, the times were different. Now we have sexual harassment policies.
Please don’t leave or feel guilty for your other workmates. It’s HIS behaviour, not yours. You should see if you have any HR policies in your company that can sort this. If you can’t do it for yourself, then do it for all the other girls and women being intimidated by fuckwit males in the workplace.