I can see it from both sides:
You've done a lot to help him out, so in a way, perhaps feel like he should reciprocate this? You're hurt that he's said no because the scales don't feel balanced, and also, let's be honest, because it puts your night out in jeopardy.
You are maybe also slightly hurt/jealous because you feel like he's putting his GF above you.
You're brother is grateful for your help, but should he be beholden to you for this? No, not really.
He has to be up early on Sunday, he wants an early night, you probably won't rock up at his house early evening, but ever likely, early morning.
He has two women in his life, who mean a lot to him, but whose relationships are very different. Naturally, he's going to side with his GF. He's probably completely oblivious that he's hurt you, as it was in no way his intention.
You maybe also have a touch of FOMO (Fear of Missing Out).
I think you're going to have to suck it up, else you're going to create a situation, and it will be you who comes out in the worst light for it.