You've said two things that I think warrant thinking about...
I guess it just feels really annoying that I’ve only just found the confidence to be who I really am sexually, and it’s come at a time when I’m happily married and therefore won’t get to explore it
That's not unusual, in my experience. We're the same age, and I have a few friends who feel the same. It's much easier to fantasize about what you could have been when you're safe and secure and happy. I've also found that babies tend to bring out that side of my friends, too - maybe because of how maternal it is, or maybe because it's quite hard-going? But the fantasy of another woman and how exciting and different that would be seems to be very tempting around that point...
There's a decent chance that it wouldn't be how you were expecting, and the reality would never live up to the fantasy.
But if it was just meaningless sex he was after then no I suppose I wouldn’t like it. But that’s because I’m insecure and would worry about him liking someone else more than me!
But also this. Again, in my experience, most of my female friends would consider themselves bisexual. Very few of my male friends would. Unless you know otherwise, it's far more likely that your husband would be having sex with other women than other men if you opened your marriage... And that is very likely to trigger the insecurity.
It's always rationalised that if your bisexual and experimenting with the same sex, it's better because you're after something that your husband can't provide - being female, a female body. And you probably believe that you wouldn't leave your husband for a woman, so that adds a level of "protection" from your side, so to speak, it lowers the risk.
That wouldn't be the case if your husband was sleeping with other women. It'd only be fair for him to have the opportunity if you do - one-sided open marriages don't tend to work - but that could then impact on your relationship quite strongly. You'd also have to work around any other insecurities that you might have - I have a friend who found it unexpectedly very painful when her husband's chosen partners had the body that she wanted, or when they were quite a bit younger and childfree... Those things might not bother you, but it's worth considering if anything would.
All that considered, there's a chance that 40 year old you won't actually fancy opening your marriage anyway, or that it won't feel as exciting when the baby days are over...
But you are definitely not alone in feeling like this. In my circle, at least, it's really common. I only know of that one relationship that tried opening things up - and it hasn't gone that well for them - but there will be others who do/have and it has worked for both parties. You just need to be really, really sure that it's worth the risk first, because you can rarely paper over that type of crack.