Its not you, its him as well as his family.
The problem here, quite apart from his parents who are another issue entirely, is your man himself.
This comment:-
"He is very much a mummies boy (I love that he loves family) but it's coming between our relationship, he keeps secret from me with his parents and his mother hasn't liked me from day one".
Unfortunately your above assumption (I love that he loves family) is wrong and I would also say naive; he is totally and utterly in their grip as an adult and is still seeking his mother's approval in particular even now.
Your man now being unable to actually assert himself as an individual in his own right (due to his mother's conditioning) has and is causing real problems in your relationship now. He will always put them first, not you or your child and that will simply hurt him as well as you and your child. This was always going to happen in any relationship he has because these people have done real emotional and long term harm to him.
You come from an emotionally healthy functioning family, he and his family here are not; that is the fundamental difference between his parents and yours. You are a reasonable person at heart so you have tried to get along with his parents; unfortunately you are dealing with his parents who are clearly unreasonable, they will never be reasonable.
If you do choose to stay with this man you need to realise that he may well never change his attitudes towards his parents and will always put their and his interests above yours. Your man's behaviour towards you is putting great strain on you.
You at the very least will need to read "Toxic Inlaws" written by Susan Forward and raise your all too low boundaries with regards to them to date (why did you at all invite them to something as personal as a gender scan?) and reassert your own rights as a person and a mother in your own right. I would also not let your child be at all influenced unduly by his parents; look at the emotional damage they have done to your man. They will do that or similar to your child given the opportunity.