I find it a baffling that you are so resolute not to disclose the mystery spend. I presume it's because it somehow will impact the advice given.
I'm married and we have 3 accounts, mine, his, ours, and both pay an amount each pay day into the shared acc. But equally if the car packs up, or DIY house works, an unexpected cheeky holiday etc, whoever has it pays, there really is no mine or his.
Fwiw, I think it's absolutely fair enough that somebody wants to go on holiday with their partner before 2025. If you want to continue with current financial split arrangement then no matter what he agreed to it was unreasonable of you to suggest waiting until 2025 for holiday, 2 people working etc, there can't be any reason for it, except if maybe neither of you care about holiday much, or don't care much on spending time together.
Neither of you sound mature enough for a discussion either. All these lines in the sand, I told him I was paying my holiday fund on mystery spend, he agreed, he has money saved but hasn't offered to spend it, his family are rich, mine aren't (I don't see the relevance tbh), I won't go into credit card debt (absolutely, I agree), but you won't go on a cheaper holiday where there are children either.
Where's the compromise here?