IMHO, the explanation you have given about his reaction after he was angry and you pointed out how it impacted you is Covert Narcissism. I recognise it because I'm living it. Unless he accepts responsibility for being angry and doesnt' try to make it about you "being afraid" or being "uspet" because you think he could hurt you. From every aspect you DO NOT accept him at any point creating a feeling in you that makes your own internal state change, such as fear, withdrawing from him, or defending yourself, or even worse trying to calm him.
It's his responsibility as a man to control his own mood/outbursts even in relation to something you are discussing in a slightly heated way.
A non controlling/narcissitic, emotionally intelligent man will know exactly when to step back and will pick a calm moment to discuss anything openly with you. He will also understand how his angry approach could affect you and 100% not blame you or make you feel guilty for raising it.
A controlling man will act on his impulse and respond with shouting, rage, or even sulking about his pride being hurt because you thought he could hurt you. There is no weakness in a any person standing back and saying "lets talk about this when we are calmer" Don't get into a cycle where you are treading on eggshells, and trying to change or keep the peace because you don't trust his reaction. This is endorsing his apalling behaviour and must not be allowed to happen. If it does - walk away, it WILL get worse and it WILL become bigger and take over your character.
Be careful, this man is not emotionally mature and this WILL 100% escalate. He is trying to make your response your problem which means... something wrong with your reaction. Your reaction in instinctive and is 100% valid. Men are intimidating when they shout or have sudden outbursts - end of.
Any men reading, learn to recognise your own triggers, learn not to blame women because they don't accept what is clearly bad behaviour and find a way to navigate communication without any form of aggressive or controlling response. Then we will believe you when you say that you would "never hurt us", you are hurting us emotionally and its even worse than physically.
Physically a woman will be more likely to walk away, but a manipulative man who controls responses ties you to him for a much longer more dangerous period until you forget who you were before him!