YANBU.
SIL came to babysit LO a few months ago. She had the run of the house but I closed the door to our bedroom, which in my family is a sign that the room is out of bounds. She had seven other rooms at her disposal.
I left her everything she could possibly need for LO and she had three contact numbers for me and DH, plus numbers for my parents if she needed any immediate help (we were an hour away and I know I sound PFB).
So she needed nothing in there, she had help available and we texted her a few times and spoke to her twice, both times she assured us that LO was fine and in bed.
Now he's lovely when he goes to bed, he has settled himself to sleep without crying since he was six months old and he sleeps right through the night. He has his bedtime stories downstairs and is more than happy to be put in his cot to go to sleep. If he's very tired he asks to go to bed. He goes to bed at 8pm. We told her all of this, she said that was fine. We told her he takes about five or ten minutes to go to sleep and he usually chatters to himself first, but to leave him be unless he started to cry. Again, she said fine.
When we got home, at gone midnight, LO was still up. She admitted he had never been to bed. She said she had tried to get him to sleep, but when she put him into bed he didn't fall asleep right away, so she got him back out because she could hear him talking.
So she brought him downstairs and they played for a bit, then she tried to nurse him to sleep, then decided he would fall to sleep if they both got into our bed and she waited for him to drop off but he didn't, so they staying in our bed while she read to him for a bit and then came back downstairs.
LO was crying when we got home, and she said she had decided to keep him up because we said not to leave him in bed if he cried. She said he had been asking to go to bed but she didn't want to take him while he was crying.
DH drove her home. I pulled my cats bum face a lot and took LO straight upstairs and put him into bed, where he fell asleep almost as soon as his head hit the pillow.
So after I got over being pissed off that LO had been kept up for more than four hours past his bed time (at 2.2 years of age) even though he had cried and asked to go to bed, I got pissed off that she had lied to us about him being in bed and hadn't phoned us to tell us she was having 'difficulties' in getting him to bed. And after I got over that I got pissed off that she had been in our room and in our bed for no good reason.
Which is my long winded and thread hijacking way of saying YANBU, especially if you are not home but come back to find a family gathering taking place amongst your intimate items.