Oof.
You've mentioned that he's asked permission which puts the pressure on you to be the bad guy if you say no. You are not his keeper and he should have the maturity and common sense to work out the answer all by himself.
My DH went away for a fortnight with a couple of friends without me and the kids BUT it was to visit a friend in another country who was going through a very low period and he wasn't the only one who had children and our children were much older at the time. People were still shocked that I "allowed" him to go. I didn't allow him, he's an adult human who made the choice himself.
What I'd ask you is: when is your 10 day Caribbean holiday with friends? When is your family holiday of the same price and distance?
I've never had to ask for permission for my own trips with friends but I've only ever been away for long weekends or a Mon-Friday. Also - those longer trips definitely didn't happen when there was a baby at home.
What's your support system like while he's away?
Is it a big financial issue? It would be for us.
When my Dad's been away on long trips with friends it's been for sports things like a world cup or Lion's tour but for each of these he's done he's taken my mum away on a really special holiday. She's away now with her sisters without him it's not one sided.
For you, the age of your children is a huge deal. That he booked this trip when your baby was newborn or about to be born (I forget) is BATSHIT CRAAAAAZY. How was his instinct not to cocoon his little family, to nurture his wife who's been through this huge physical and emotional rollercoaster that is pregnancy and birth and baby rearing, breastfeeding and broken sleep and all that? It's a major ick to skip away to the other side of the world (I assume you're in the UK usually?!) for no particular reason without you for a jolly.
It was hard enough physically (breastfeeding) and logistically to go on weekend hen dos away from my babies when they were 6 months old and it's really pissing me off that this guy hasn't even thought twice about it. And his friends not even thinking of you and the children. Fuck those guys. Jeez. They're so ignorant.
Look, you can have a lovely 10 day stretch at home without him. When my DH's away we don't eat half an much meat, I use more of the foods he's not a fan of, I get to watch what I want on TV, don't get woken up by his much earlier alarm. It's not all bad. Less of the compromising that's needed in a partnership I guess. But that's not with a 3 month old in the mix and that's where your guy's been a complete dicksplash.
Why couldn't it just be a weekend away?! Mental.