Any time we go away I book and pay for it. I go and stay at his at weekends and buy all the food and cook and clean up for him. I buy him stuff for his house that needs replaced. I give him money to help pay his bills.
For the love of Dog - STOP IT!
I also feel like I'm to blame for how his ex is treating him.
Don't you just ... 
Do you think the shiftless, selfish fucker wasn't a shiftless, selfish fucker to his ex as well?
Has it not entered your head that he has deliberately sold you his My Crazy Ex story, & has neatly triangulated you to do the Pick-Me Dance for him?
The more he sells you his tale of woe, the harder you dance.
This weekend I have booked and paid for a fancy b&b for us to stay in for Valentine's Day. I have to go through with it as it's non refundable.
Don't be daft. That's like saying you need to board a sinking ship, because you've paid for your ticket.
If you can't bear the thought of losing the money, go - just don't take him. Go solo, or with a friend.
Should I give him a final ultimatum then or never believe that he will change and finish with him after the trip?
Why do you think dancing even harder than you already have for 2 years is going to result in the miraculous personality transplant you yearn for?
Your man is a shiftless, selfish fucker, & no amount of fancy dinners are going to change that. You can spend as much money as you like on roses & champagne & soft lighting ... & guess what? - He'll still be a shiftless, selfish fucker.
Get you head straight on this one.
An income of £30k, paying CMS & sensible repayments on the £13k loan, would NOT make him this skint. You are his cash cow, & throwing more cash at him is not going to make him love you.
You need to forget about his ex, who is a total red herring.
He'll have been as honest about her as he was honest about taking you out to his social club, about taking you out at all when there is rugby to watch, about 'getting engaged', or about cohabiting.
He has spun you a load of old cobblers, & it's time for you to stop throwing good money after bad, good time after wasted time, & your own life on a liar who is never going to deliver on any promise, & who is using his ex as a shield.
Sorry this is worded so brutally OP. But your mum is right, & it's high time you realised it because this man is already making you miserable & the more you invest, the more miserable he will make you.
You'd do well to sack him off right now, & take your mum to the posh B&B this weekend instead. No wonder she can no longer discuss this with you, she must be pulling her hair out in frustration & sorrow, seeing you throwing yourself at this loser.