Andy I just wanted to add my support.
You have absolutely done the right thing.
DH and I have had untold problems over the years because his family just cannot (will not) detach from his ex.
When his mother chose not to come to our wedding because it would be too awkward for his ex, but went to hers a month later, it was the straw that broke the camel's back for them, I think and we are all NC now. (Not because of that.)
Before they went NC there was an event that I was very specifically not invited to because it would upset ex. I have no idea why, they'd been split up nine years when I met him. MIL wouldn't put up any pictures that I was in because it would upset ex, whose picture she had framed in her living room.
It's all a bit weird.
Anyway, this event, I told dh he should go as they were his family and I would support his decision etc. etc.
He refused for the same reasons you did and I have to say it meant the world to me that he stood by me, more than I thought it would actually.
So you've absolutely done the right thing, IMO.
I'm sorry about your dad but this is your brother's doing, not yours.