I agree with you, op, his reaction is the issue not the money. I couldn't comment on the financial side because I haven't seen your spreadsheets, so I don't intend to try.
He also said that if he paid me £100 a week and we had an argument, that he wouldn’t be entitled to anything back and would have essentially “lost” that money and be “throwing it away”.
This is so perverted, but in the context of what you have shared throughout this thread it's not that out of place.
He's behaving like somebody who thinks he's purchased you. Which is not normal.
Him not letting you pay for socialising, meals, etc is one of those things that at a superficial level sounds lovely and caring, but when you examine it a bit more closely it looks controlling instead.
Paying for meals and socialising - treats basically - means he gets to play the big shot, gets the ego boost of being able to play the generous and devoted boyfriend, whilst living in your home and watching your savings disappear.
It's all about the public face, which means people tend to respond with (as you've seen here) "oh but he's so generous, he can't be so bad really" when you try to seek their advice or support with his disrespectful behaviour. That's not an accident on his part.
The story about his ex is very convenient, and I agree with the pp who observed it sounds like the usual controlling man's line that gets trotted out to make you more compliant.
Likewise, making it so unpleasant for you and causing all this stress seems intended to make you less likely to push the issue or ever challenge him again if he's being unfair.
I can only tell you my observations as an outsider with more perspective than you can have as the person in the middle of this.
I think you would find the information here really useful in deciding whether his reaction is something that can be worked on, or a more serious indication of someone who is financially controlling: www.freedomprogramme.co.uk
I'm not recommending this because I think he's some evil monster or definitely being controlling, however there are significant warning signs here. The section on financial control covers coercive control (abuse) and contrasts that with how a respectful but imperfect partner would treat you and handle these issues. So you can take the information and weight it up for yourself.
What really concerns me about this op, is the attitude he has revealed towards you, the degree of manipulation that seems to be being exposed, and that by refusing to contribute and insisting on playing the big shot in public on things that don't help he is pushing you further into a financially precarious position (e.g. Running down your savings) that makes you more vulnerable and more dependent on him...
Ultimately, that's the aim of any form of controlling behaviour - to have power over you, however it can be obtained. It's not about monsters, or 24/7 blatant cruelty.