I hardly ever post on Mumsnet (long-time lurker!) but when I read your post I couldn't not post - I feel so much compassion for you. The way you describe your relationship sounds so much like the relationship I had with my ex-partner.
My ex-partner was also a bit older/more experienced than me (8 years older), was prone to depression (although his first/most serious episode was before we met, I think he had some mild symptoms at some points during our relationship), he was not at all physically close or affectionate with me, and had pretty much no libido and never initiated sex, and rejected me when I initiated sex.
Things also did not go well when I plucked up the courage to raise the sex issue - I felt that no matter how sensitively or tactfully I raised this, I was shut down by him - I felt like because he didn't think it was a big deal and was just the way he is, he was (inadvertently?) minimising my own feelings and the impact it had on me.
I also felt very ashamed about our lack of sex life - despite having wonderful close friends around me, I didn't even talk to my friends about it because I felt it was so stigmatising. But I also just felt really sad about it - it felt so sad to spend much of my twenties in an almost sexless relationship.
In the end, I ended the relationship. There were one or two other issues (nothing huge - but we were just not well matched). I'm fairly sure he would have been fine to carry on, but for me I just realised that after almost 7 years, my needs were not and were never going to me met in this relationship. For me, the optimistic choice was to end the relationship. I didn't want to close myself off to the prospect of being in a happier relationship where both myself and my partner were better suited to each other. It took me a long time to get to that point - maybe for the last 2-3 years I was seriously questioning the relationship - but I haven't looked back.
I am now in a relationship with a lovely, kind man, and I'm just so happy to now be in a relationship where I'm completely happy with the sex and affection side of things. After all those years it just feels wonderful - in contrast to how I felt this time last year.
Anyway, I'm sorry to ramble on about my own situation, but I thought it might help to realise that you're not alone. Of course, I completely understand our situations are not the same - I don't have children, and even without that, no two relationships are the same.
But anyhow, I think it's really good that you're reflecting on this, and I just hope you take the time to think about your own needs and wants. This could just be your posts but I wonder if you've spent a lot of time thinking about his feelings, his behaviours, his wellbeing, what you can do to accommodate him etc - rather than looking out for yourself and your own needs.
Wishing you all the best :)