What a horrible situation, you must be so exhausted and anxious.
When a baby cries a lot, and can't be comforted, it can be extremely stressful. You can feel like an utter failure, especially if you are exhausted and have a list of things to do as long as your arm.
However your DH is an adult, and if he can't get a grip, he needs to get help or leave. He is not the primary carer of the baby so he it is a shame he has succumbed to the stress so significantly and it is not fair on you at all. My second son had reflux, and it was a very difficult time because he would vomit so often, and as you say we were best off keeping him upright, which was hard at night time especially. My son went dairy free and grew out of the reflux problems within a year but the first six months were hellish.
Can I ask, have you talked to DH about this when he is relaxed? Is he ever relaxed? Maybe not, with a 2 y.o. and a baby in the house. If you can catch him in a quiet moment, explain very calmly that you are very concerned about him, that you've noticed when he is with the baby he gets easily overwhelmed and that the behaviour is making you frightened. Remind him of the wall-punching incident. Ask him how he can be sure his behaviour won't escalate. If he engages with you, then ask him what he wants to do about it,making it clear that doing nothing isnt an option.
Then talk to him about your own situation, explain that you are obviously exhausted too, and struggling.
If it is an option, I would strongly recommend you ask his help to pack up and take you and both kids to either your parents, or his parents, to escape for a while. Or ask a parent or close relative to come and stay, for as long as possible really - a week or two would be great.
If you don't have any support at all, then ask your DH to leave for a bit, he could go and stay with a friend. And then get some help for yourself, if you can possibly afford it, even if it means you go without something else (cancel Netflix and go vegetarian to save money). Help might mean, a cleaner or sending your 2 year old to nursery or a childminder, or a night nurse to help a bit with the baby.
I don't think your DH is necessarily going to become violent toward you, but I think a LOT of people suffer from rage dealing with toddlers and babies, far more people than admit to it, people who are normally gentle and loving. Mums as well as dads. It can pass, it can be addressed, I'm not saying put up with it but I'm saying force your DH's hand if he won't do anything about it for himself.