YANBU - because generally, grandparents being at birthday parties is for the GP's benefit, not the DCs, they are normally having more fun with their friends and playing games, GP's won't be missed from the 2 year old's party, but then they wouldn't be missed from your DD's either. (Although you haven't said how old your DD will be)
It's the fact they consider your DD's birthday an event they want to attend and will enjoy seeing her enjoy her birthday, but don't feel the same about your DS that's a bit
- particularly if they made an effort with your DD's at the similar age.
There does often tend to be a sense that while parents are belittled for being abit 'PFB' - it often is more the extended family that really make a fuss about the first born of the grandchildren generation, then the 'novelty' has worn off by later babies. From what I've seen, it's grandparents/Aunts/Uncles who treat later DCs differently from first borns, not the parents themselves, but when the parents of the DCs point out that the extended family aren't behaving in the same way to all the DGC's/DN's, they get accused of being precious because the DCs won't notice. If the extended family hadn't made a fuss the first time round, the expectation wouldn't be there with later children.
Oh, and while DS might not notice who is and isn't there, your DD might well be old enough to start to notice that Granny and Grandad always comes to her parties, but not to her little brother's...