Then you should have told him you weren't fine and walked away. You said the music was loud and that no one else heard him being verbal with you, so what did you expect your boyfriend to do, when he didn't realise anything was wrong.
You've already said you could have handled it differently if you hadn't had a bit much to drink.
You could have said, I was okay until you came over and out your arm on me without my permission. Then just walked off, without giving him a chance to say anything. But remember there are lots of idiots in life and you just need to know how to deal with them.
If this guy is a friend if your BF, you'll need to learn how to deal with him, especially as your man doesn't like confrontation. Some BFs would still contact their friend afterwards and say my GF told me what you said to her and you better leave her alone from now on.
Some BFs will just say, ignore him. My DH would say the latter and that's why when a family member of his annoyed me, I dealt with it head on. I told my DH afterwards and he just said, take no notice of him, he's foolish.
I know many other men would have had a word with the family member about it, but my DH avoids confrontation just like my dad does.
Maybe this isn't the man you want to settle with.