FWIW, though I didn't have children, what you're describing sounds very similar to me.
I just sort of knew deep down, while knowing that it wouldn't be acceptable to some of my family. And I would keep second-guessing myself, thinking, oh, but if I were really gay, I'd be absolutely certain, or I'd feel powerfully attracted to random women, or whatever. Or the things @Josette77 describes. And, on the whole, I didn't feel those things.
It sounds to me as if what you are most sure about is not the presence of attraction to women, but the absence of attraction to men? If so, TBH I don't think the answer is that you're bisexual. It could be, as your DP says, that you're depressed and miserable and the absence of attraction/sex drive is a result of that. But it sounds as if it's not that, if you've always been semi-aware of it.
I don't really see how you can know for sure. All I know is (in my situation, which was much less high stakes as I had a briefer marriage and no children), I did leave, and met my (female) DP, and I am much happier about it all now. For me, looking back, yes it was entirely obvious I was gay, but I can only see it with hindsight.
I'm absolutely not saying that because that's my experience, therefore it must be yours. But if you're looking for someone to say that you could plausibly have the thoughts you're having and be gay, then yes, I know you can.