I can see both sides of this. I think, on balance, you might be being unreasonable but I can absolutely understand why.
I probably wouldn’t want to take any children, let alone a surly 15 year old, on my honeymoon. I understand that you feel the 3 year old is too young to leave, though I really think if you wanted to, you probably could. I left my DD with my parents for 2 weeks for a work related course that was more or less mandatory. She had a whale of a time. I suppose it depends on the child and some will have not spent much time away from parents due to the pandemic.
However, I also think that if you marry after having children, the honeymoon is almost certainly going to be a bit different to the traditional honeymoon as you have other considerations and responsibilities. And if you are taking one child (even with childcare options- you might find your DC does not settle well, for instance), then it is even less like a traditional honeymoon. More like a family celebration.
I know your step-daughter is not behaving well. But, as you say you believe this to be coming from a place of anxiety and unhappiness, there appear to be reasons- though not an excuse- for this. So, what you do depends on whether you really believe she will be hurt. Do you, in your heart of hearts, believe that not inviting her (with strict rules- she cannot back out last minute, for instance) is going to cause her to feel hurt, left out/rejected etc? Or do you think it won’t cause her any upset at all?
If you do believe that it may cause longer term hurt to her and therefore there is a risk of harm to her relationship with her father, with you and with her sibling, then I think you need to take cognisance of that. Personally, then I think it would be extremely unwise to ignore that belief because your short-term gain (a DSD-free, and therefore more enjoyable, holiday) may come at the cost of the longer-term ability to improve relations with your DSD and between the half-siblings. If it does cause sufficient hurt, it may set you all up for years, or even a lifetime, of difficult relations. It may damage your STBDH’s relationship with his daughter. Only your and your fiancé can decide if the nice honeymoon outweighs these longer term considerations. For me, they would. I think I would also struggle a little to enjoy myself if I really thought I was causing hurt to my partners child and storing up trouble later down the line.
However, if you honestly and truly believe (not just hope/wish or would like to think- but absolutely are sure) that your DSD would not be put up nor down by not being invited, then that is a totally different situation and I would happily go without her.
I personally would be concerned that my partner, if he really believes his daughter will be hurt by being left out, is happy to do so to appease you. How would you feel if he did something similar to your DD in the future (obviously, I hope that would never be a possibility- but you see where I am coming from). It might only be over this wedding/honeymoon- but what message does that send to his daughter? And how will she know it is only about this specific situation (if it is)? Do you think it is likely to improve or worsen her feelings of insecurity about her place in her father’s priorities?
I genuinely think your DH (with your support) should have been trying to address the issues within your blended family- if he hasn’t already- so that your DSD does not feel so insecure that she can barely bring herself to speak to her younger sibling. I mean, unless she is exceptionally horrible person (and I’d be reluctant to ascribe that to a 15 year old- they can all be horrors, frankly), then that is almost certainly coming from a place of deep unhappiness/anxiety/insecurity. That’s not to say she cannot be brought to task for her behaviour, but she probably also needs some help too.
In short, you need to think about the longer-term impact of your choices.