Last time we told both sets of parents and grandparents plus siblings as we knew we'd want the support should the worst happen.
My mum then told every single person she knew (friends, extended family, work colleagues, checkout staff at Tesco) "I've got a secret about my daughter but I can't tell anyone". When I told her off it was ok apparently cos she hadn't ACTUALLY told anyone so she hadn't done what I had asked her not to do 
I was furious at the time. I appreciate now she was excited, but I told her I'd never tell her again if there was a next time! It just added to the pressure I was feeling that so many people knew.
We had an NHS scan at 6 weeks and a private one at 8 weeks that we went to alone. Both my mum and dh came to my 12 week scan.
I had another scan at 17 weeks with a consultant to rule out a condition, and even my dad insisted on coming to that one 😂 so there was 4 of us there for that one. Seems a bit silly looking back but it was a scary time for everyone and IF the scan had shown something, my parents both wanted to discuss it with the consultant as they knew what they were talking about as they'd been through it all before when I was a baby - if you see what I mean? Luckily everything was fine, so my dad just had a chance to marvel at how far technology has come on. Lol.
I just think you need to take in who YOU feel comfortable with. I've never been close to my mum, but having her there really bonded us and it was great to talk about what we had seen. I also know if we had had bad news, me and dh would have been in shock and mum would have stepped in and asked practical questions, as she's a health care professional herself. So her coming to all my scans worked for us, but that's not to say it'll work for everyone!
If you don't want anyone else there just say it's limited to 1 extra person in the room. That way they can't try and persuade you to change your mind.
I don't think I put my foot down enough in the early days of pregnancy about what I wanted tbh, and I'm paying for it now in some ways as I still don't get listened to about things regarding dd and she's 11 months. So I would say now is the time to be firm about how you want things to be so that you don't get walked all over once the baby comes.