Yes depends a bit, what kind of behaviour are we talking about and is it just to you or is s/he the same towards everyone? What is the impact of the behaviour, i.e. is it just rudeness to colleagues or is it actually to customers/clients as well with the result that business is lost? Does the person's line manager know about the behaviour and is any action being taken/likely to be taken?
I too am really conflict avoidant but I've successfully worked with some people who are known as 'difficult'. Mainly I pick my battles and overlook a lot of stuff which is just lack of manners/superficial rudeness. I meet most rudeness with a sort of blank politeness, 'I see...pause....what do you think we should do about that?' to a big complainy shouting rant. I think generally people back off a bit if they realise they aren't going to get much reaction from me. It's fine to not always challenge every little thing and people will generally fundamentally not change the way they are no matter what you do so its crucial to not tie yourself in knots trying to change their behaviour.
Do take protective measures like avoiding being alone with the person, confirming contents of conversations in emails so that you have a record, whatever works for this particular individual.
Do have a safe venting place/person, ideally someone not at work, who you can say 'you won't believe what X just said to me...' to...
If I do have to directly talk to someone about their behaviour I use tactics I've been taught in 'constructive conversations' courses. You could see if there's anything available at your work similar but some of the techniques are:
-Always address the specific behaviour/its impact, don't be general or personal. E.g. 'I noticed you were shouting in that last conversation with Judy. Shouting is aggressive behaviour and Judy looked upset' not 'You are rude and aggressive and you've upset Judy". The latter tends just to lead to direct denial or tit for tat accusations. Direct examples are harder to contradict.
-Have a conversation as soon as possible after the event, but not in the heat of the moment. Try and pull people aside into 'neutral ground' and have conversations in a private space not in the open.
-Try and set expectations/turn the conversation towards the behaviour you do want to see not what you don't e.g. 'In future please do not raise your voice' rather than 'stop shouting'.
Good luck!