Yep, biphobia is unfortunately very common.
I'm bisexual and married to a woman. That's how my friends and family know me now, but if I say x male celeb is hot, I'll get a confused "but you're married to a woman" or "oh dear...you're not starting to miss a man are you" 
It really shouldn't be that confusing for people. Bisexual people are drawn to both sexes, but we are often labelled as greedy and or confused. It's incredibly annoying.
Anyway OP, it sounds as though you're probably also bi, but given you're married, there's not really much you can do to explore it, unless you're in an open relationship I guess.
Please don't jump for joy though, if he gives you the green light to experiment with a woman, because the only reason a lot of men do this is because they don't feel threatened by a woman, because there is no penis, which actually is massively insulting and often very dangerous, because they should feel threatened....because women are awesome 