I have been the lower earning person in a similar couple when DH and i first started living together, we didn't completely merge finances until we were married. When we first moved in together he was earning vastly more than me, and was invited on a trip skiing in the US with a group of shared friends (although more his friends), we'd already commited to going on an overseas holiday later in the year for another friend's wedding, I couldn't afford to do both.
He actually did offer to pay for me, but I refused, I didn't feel ready to be 'kept' - once we'd got engaged and then married, then had DCs, it was different, we were a unit, but when you are just living together with no DCs, you aren't really a unit.
He went without me and I was rather jealous at the time, but i couldn't afford it and didn't make him feel bad about it. It was my choice, he had offered to pay.
The issue will be, will your DP make you feel bad about going without him? If he can't get the time off, then the difference in your wages doesn't really matter (other than that he can't really afford to go on his own later on if he needs you to offer to pay) - if you reposted saying you can afford to go away but your DP can't get the time off to join you, AIBU to go without him? most people would say YANBU.
If he's taking the lifestyle you offer but still begrudging you having things he can't have (be it because you can afford more than him even with you not expecting him to pay half the bills, or because he can't take the time off to do stuff even if he could afford it due to being SE), then he's not a keeper. People who only want you to have fun if it's fun with them aren't worth the effort.