'so you only have one then?' or 'you just have one then?'
I have never been able to find a good answer to this that doesn't make me sound rude or bitter but that also puts people in their place with their implication that I am somehow 'lacking'!
I chose to have one child and love it so I'm not sad at having one. Some witty answers please!