How do I avoid DHs brother without some big showdown?

[FreddieStarrAteMyHamster]

DH can't do conflict, he will do anything to keep the peace. His family live a few hours away and we usually see them about 4-5 times a year. Last year his brother did something terrible to DH (the thing impacts us all) that I cannot forgive, and while DH was upset at the time he has coped in his usual way by not confronting and then putting an alternate reality on the situation.
I have said to DH that I don't want to see his brother from now on. He won't say anything to him so we have managed this by just not visiting anyone (they all live so close it's impossible to make a trip without everyone knowing and turning up). We'll need to do a trip soon but it's making me anxious, if I see him I won't be able to be pleasant and DH and I will argue. DH went on his own last time with DD (with the excuse I was working but obviously I cannot use this forever).
WWYD?

It depends what it was, you could go and ignore or just confront him and get it over and done with. Its hard to say without knowing what it was.

Unfortunately your DH's stance has not helped him for that matter. Presumably he is like this because he has also learnt such behaviour from his own childhood.

Do you actually need to make a trip at all, are DH's family really worth seeing?. What are his parents like, do they know what their other son did?.

Who did DH and DD see last time family wise?. You are all part of the same package; not going may actually be playing into their hands.

No, agree DHs avoidance of the matter hasn't helped him. He's in his mid 40s so I can't change him now. This behaviour is a pattern his whole life, you are right. if I confront I'll be the bad guy and I really don't want to fall into that trap, I feel it's DHs job to say something to his sibling, not mine. The rest of the family are blameless on this specific matter although their whole dynamic is odd and there is constant bitching behind backs between them all although little is ever said to anyone's face (blended family issues). I would like to see them occasionally as before but the BIL, never again.

I actually doubt if anything at all positive would result from such an encounter if it were to happen (which it will not). Such people like his brother never apologise nor actually take any responsibility for their actions. His brother has clearly not apologised for what he did.

Are DHs parents actually worth seeing at all given their family dynamic?. If it is dysfunctional or if you are only really meeting them out of societal convention, exposing yourselves to all that actually does your own family unit no favours at all.

Thanks Attila, yes the brother isn't the type to express any self doubt or remorse. Hence I've just been avoiding. Feel as though the avoidance would have sat better with me if DH had just had it out with him though. argument-disagreement-never speak again more agreeable to me than just avoidance and the speculation that involves. It will come out one day and I feel we would have had the upper hand if DH had just dealt with this issue when it arose. A you can tell I'm cross with DH for being such a bloody doormat. Better that than a ruthless bastard like his sibling I suppose.
Some of the family are ok in their own way and it would be a shame for DD if we cut ties with all, also difficult to explain to her when she's older. I suspect I will have to go with him and if we see the brother I'll be icily polite and no more.