RUN!
Get a different job if you can. Your boss is completely inappropriate and making you feel uncomfortable. I would bet anything he’s hoping to get you drunk so you loosen up - it’s why men buy women drinks at bars. I was gleefully naive for most of my life and it sounds like you may see the best in people as well, but having him even ask repeatedly for you to hang out outside of work is harassment and a clear indicator he is entitled, an out of touch ass who may be trying it on with you.
Having a drink with a boss outside of work hours is completely inappropriate unless it is in some sort of company event.
AND, you should never have to explain to anyone why you are saying no. Because this person has power and influence over you, he is quite clearly exploiting you to get what he wants out of the situation.
I would document the incidents and report him to HR. If you work for a company where that will make your job unbearable, then look for a new job right away. I don’t know how long you’ve worked for this man but his behavior is disgusting and completely inappropriate and in a post Weinstein/Epstein world if men did not know how to behave before they certainly should now.
The worst of it is I have been confronted with these types of issues my entire life and when pointing them out to men who are not that way, I often get met with disbelief by them. So, the bad ones are still bad and the good ones don’t believe the bad ones are that bad and I fear not much will change.
Same goes with some women not believing sexual-harassment exists because they have not been sexually harassed or are weirdly jealous because the person being harassed is more attractive than they are.
Please don’t let anyone gaslight you and trust your gut instinct. Any time you are having a feeling that you do not want to do something it is better to trust yourself and play it safe.
I hope you find an appropriate person to work for or you do not have to work for anyone in the future. If anyone says you are over reacting please know their opinions do not matter. You felt what you felt, it’s real and I wish I would have learned to trust my instincts sooner.
You deserve to live and work in a professional and safe environment.