You're not blowing anything out of proportion. You're in a an abusive relationship. He is verbally abusing you, emotionally abusing you (telling you you're the dumbest person etc) shouting at you for "not doing as you're told" .....It's never ok for a man to speak to you like this, like you're a dog. I was in a relationship like this when I had a young baby and felt I was maybe "blowing things out of proportion". He has gaslighted me and psychologically torn me down so much that I doubted myself and thought maybe I was just being too sensitive. But none of this behaviour from him is normal, healthy or loving. When you step back and get perspective you will see looking back how wrong it is how he's treating you like this but he's gotten you used to it.
Read above thread in Relationships about going to a refuge with your baby. Trust me it doesn't stop at verbal abuse. Eventually he will grab you, shake you, slam doors, punch stuff around you to frighten you/let out his rage and keep you from addressing his abusive behaviour, doing scary thing to keep you walking on eggshells and "doing what he says" because you don't want to cause a fuss (ie. you think inside you can control his outbursts by doing what he says and tiptoeing around... - this is never healthy to feel this in a relationship - you deserve a normal loving relationship). My ex started exactly as you describe and it always escalates into physical - shoving, grabbing, restraining, frightening and intimidating etc - this is not healthy for your baby to be around. He eventually started doing it to my daughter as well - he couldn't contain his rage and he would grab her off me, shut me in a room, it started when I was pregnant but little things and I kept making excuses and "not wanting to blow it out of proportion" like you say. Fast forward 4 years and he has taken me to court to have joint custody of our daughter which he was awarded because there was never any record of his abusive behaviour to me because I didn't want to "blow it out of proportion". I didn't want to seem dramatic by calling the police. But I WISH TO GOD I had. Because now my beautiful and sensitive 4 year old daughter is court ordered to live with this man half the time and she says all the time that he hurts her, frightens her and shouts at her, but because he acts like the nicest guy ever in front of all teachers and social workers who had advised, they don't believe me/her. They think he's great. All because he had conditioned me into basically protecting him and thinking maybe the way he treated me was MY fault so I didn't have the confidence to do the scary thing and just report what he'd done to me and DD to the police and get it all properly recorded.
I spoke to Women's Aid but that wasn't quite enough evidence.
Believe me I know what I'm saying will sounds dramatic to you now as what you're saying is exactly how I was - in denial, new baby brain fog and very vulnerable, and you just want to keep the peace and the status quo, but it's worse to stay and say nothing and hope it will stop/get better/he will see/realise/change. Men like this never change.
I wish I had taken Mumsnetters advice on my post 4 years ago telling me to get a Non-Molestation Order, report abuse to Healthy Visitor, make plans to leave then once I was safe, make Police Report.
Now my daughter suffers because of my inaction and fear of him and him telling me I was "making a big deal out of nothing". He has eroded away your sense of what's right and wrong and he has a lot of power over you, I can tell this by what you've said. But you can and should get out of this situation immediately. Every day you stay you are in danger or his rage turning physical and you are putting your baby in an abusive home which then the baby will soak up all your stress too. Trust me. Throwing an object at someone is domestic abuse /assault too by the way, it doesn't matter that it didn't hit you. Verbal abuse is everything you described. The verbal, emotional and psychological abuse (sulking when he doesn't get his way, shouting, threatening abandonment, swearing at you, putting you down, calling you dumb, trying to GASLIGHT you - look up this phrase - to make you doubt your reality and make you think YOU'RE wrong in the head - it's all incredibly damaging long term and it will wear down your self esteem and confidence and it will limit your ability to be a calm, happy and present mother so it is hurting your baby as well.
Please please call Women's Aid phoneline 0808 2000 247 freephone 24 hour domestic violence helpline - they will not tell anyone, they will not do anything but listen and give you advice. They will be understanding and will help you see and get perspective that you are in an abusive relationship and how is acting is not normal or ok.
Then please make plans to leave for you and your baby. Please report everything you've said in this thread about the controlling (telling you to follow his instructions otherwise he shouts and swears at you) & abuse and how it is affecting you emotionally (tired, emotional etc ).
How do you think he will react if you say you'd like to separate and leave ? Likely he will get even more volatile so I would make silent plans to leave and just do it - a refuge will come pick you and the baby up whilst he is at work maybe or away this weekend. Then you can get your own life set up with the baby. And make sure you record with police his abusive behaviour so he cannot try and use the baby against you to control you anymore or try and get custody of baby to get revenge on your for leaving which is what my ex did as he couldn't handle the fact I got the courage to leave and he was terrified that I would tell people what he was really like. Take away all his power over you but informing professional people in authority who can help you and support you. If you tell ANYONE what you have said here, they will agree it is controlling and abusive. This is always a dangerous situation to be in - for your physical safety but mostly for your mental health too. Your priority needs to be you and the baby not tiptoeing around his rage.
best of luck and please please take action now xxxx