I'm struggling with the same issue for different reasons-in fact both of us would probably like another child, but I find the practical diffculties overwhelming and worry about our two kids. Because I'm a total prat this manifests itself by my brooding over pregancy and babies in front of my partner, then going all 'yeah but' when she responds with enthusiasm for being pregnant. You should think yourself lucky you haven't got a stupid DH like me, but an open and sensible sort of man.
I really think if one partner says no, even if it's just because they want the spare room for their collection of Star Wars dolls, then the answer for you two has to be 'no', and the only question is whether you want another baby more than the relationship. One member of my family sabotaged her partner's contraception to try and force his hand, and the result is both revealed themsleves as total shits-she's now a single mum and nauseatingly self-pityingly aggrieved, her second child is blanked by his 'Dad'.
The earlier termination horrifies me, though. I would assume that me and my partner fucked in a way where I knew she might get pregnant, the choice became hers (because otherwise I'm saying things about her choice over her body which make the sex itself disgusting and exploitative). YANBU to think he owes you big time. But that won't solve the future problems of your relationship with each other and an unwanted child.