My brother (38) lives with my parents since his relationship with his ex broke down several years ago. That is not an issue as the house is more than large enough for everyone to have their own space. My brother has a son who is 9 and a step daughter who is 13.
He travels to his ex several times a week to assist with school run etc and has his son two weekends a fortnight and his step daughter occasionally with his son during holidays ( not an issue - he has been in her life since she was a toddler and calls him dad etc).
I still have my room at my parents house. Although, it is decorated to accommodate the 'style' of my husband and my DD who is 3. We rarely stay over apart from Christmas or if we are going on holiday as my parents are closer to the airport. I do visit my parents every Sunday - I enjoy this, my daughter enjoys this and my husband dips in and out depending on his mood.
The issue is, whenever my nephew (and 'step' niece on occasion) is there the kids all play nicely in the living room, in my old (now shared) room and in the garden etc. Basically, they get the run of the whole house. They play with my daughters toys but never share their own.
They will randomly decide in the middle of play or family time to hang out in their bedroom ( my brothers kids have their own room which is always messy and my brother has his own room which is always messy). What bugs me is that my DD is not allowed to go to their room ever. My brother even tries to stop her from going upstairs (which has more than just his bedroom(s).
I know my DD does not have a free pass to their bedroom, but i find it so exclusionary when they stop playing abruptly and she goes to follow her cousins and gets told by her uncle that she is not invited. There is no effort to redirect his kids. For example, yesterday they wanted to watch Sing 2 in the bedroom - so off they went, and my DD was in the garden playing football by herself until i joined her.
His reasoning is that the room is messy - but it has apparently been messy for 3 years! he doesn't mind his kids occupying her space and using her things but the favour is not returned and I hate the fact that she does not get the same free reign of her grandparents home as his kids.
When my DD has play dates at my brothers ex's house the play dates seem so much more fairer - when their mum is in charge. I feel like I need to cut grandparent visits down to the weekends when my niece and nephew are not in attendance so that my DD does not feel left out. To add, it is my brother who stops DD from going upstairs not her cousins
SO
IABU - DD needs to suck it up. Your brother has every right to restrict access to the rooms.
YANBU - It is out of order and you are right to cut down on grandparent visits in order to protect DD's feelings.