Op this behaviour will escalate. My ex didn't punch walls, but he put his foot through a TV's, elbow through a window etc. He also ended pushing me, then choking me. He once kicked me in the face (he was a few steps above me).
If he kicks off, phone the police ASAP.
Can you ring social services, explain that his behaviour has escalated, you're terrified, and need help?
You're worth more than this. You don't deserve any of it, and the only cause, is your ex's own issues. It's not his job, to "put you in your place".
"I know in my head I would tell anyone in this situation to leave so why can't I?"
Because leaving is easier said, than done.
Leaving is the most dangerous time, so please be careful. Sometimes it's safer to wait and get your ducks in a row, (have alternate accommodation sorted, a place in a shelter or friends house etc), before you tell him it's over. Do you have anyone that could be present for that conversation? Or could you have it in a public place with lots of people about?
Or, you could always do a moonlit flit, let him come home to an empty house one day.