I really think you must speak to him about it soon, at least to figure out if he's OK with no sex for 2.5 years... plus however long it could be put off in the future. You may find that he's actually desperate and lonely and depressed and sad, but he's stopped trying it on with you long ago because the rejection is far too painful to face again. That can look a lot like not wanting or caring, when it's quite the opposite.
I had a similar problem in my marriage (granted, it was not baby related so not exactly the same), and it nearly destroyed me as a human being, as well as nearly destroyed my otherwise extremely happy marriage. The guaranteed rejection was more than I could handle, so even though my body and soul were desperately crying out for him all the time, I simply could not bring myself to beg for it again. He never initiated any sort of affection or sex, so it died. My spirit was utterly crushed, and I was genuinely suicidal; I actually had a tidy and foolproof plan for killing myself. I nearly left home dozens of times, because it would have been a thousand times easier to lose everything we'd worked for and be on my own in some dreary bedsit than to be near the man who should have loved me but, in my mind, did not. My self esteem was non-existentent, I was utterly hopeless and completely alone in the world, a pathetic and flimsy shell of my usual self, and it was honestly one of the worst things I've ever been through.
Sex is absolutely not merely about the physical release -- not even for men. In an established, long-term partnership between mature adults, it's so much more than simply getting off. It's about the emotional and intellectual need for intimacy and feeling loved and wanted and needed, feeling worthy of love, feeling able to make your partner happy. It's about allowing yourself to be vulnerable with one another and being able to trust each other with that vulnerability. Not having sex in a marriage tends to (of course not always, but often) breed distance, even apathy, for both parties. One or both partners may feel unloved, undesireable, ugly, useless, and all sorts of other hideous feelings that can seep in and then eventually take over every aspect of their self image.
And emotional love usually easily overcomes any physical "imperfections" (for lack of a better word atm) you may percieve in yourself, and luckily that is often particularly true of men. I drank and ate entirely too much to dull with the pain of my sexless marriage and got shockingly fat, yet my husband apparently still always thought I was beautiful and loved me very much. I haven't lost all of the weight yet and I definitely still personally feel very ugly and unsexy and unattractive and ashamed of my body, but that all goes away in the moment, and neither of us notices how awful I look. He doesn't care how I look; he loves me as I am. My self esteem is magically so much higher now that there is love and trust and intimacy in this house again, and a feeling of belonging again.
I highly recommend you work this out - not just for him and/or for your marriage, but for yourself as well. Intimacy breeds intimacy, and I'd wager you yourself would find that you've been missing that intimacy of the non-sexual sort, that feeling of being in love, of being in a solid partnership.
If you speak to him and find that he does indeed want and miss sex, I think you should look into "sexless marriage" or "dead bedrooms" on the internet. I particularly recommend trying to understand how the person who does want and miss sex feels, how the cessation of marital intimacy affects them, and why they need - not want, NEED - sex and its closeness.
Of course, all of that rambling is irrelevant if your husband can honestly say that he doesn't mind not having sex. But I think you must speak to him about it - and get a truthful answer to the question - before that elephant in the room crushes you both.
I didn't even realise how bad it was, nor why, for some time. It took a lot of learning, introspection, and personal exploration for me to understand and therefore explain to my husband that I really wasn't just some relentless pervert who only thinks about one thing, but that what I needed was the intimacy and the love and the bond that comes along with sex. He simply had no idea - it had never crossed his mind - how much I believed we had lost and how badly I was despairing. It had never occurred to him that it was even a problem, much less one that was literally killing me.