"Why not?" in answer to "No, I don't have children". No-one ever questions a parent's decision to have kids, do they? Or suggest they'll change their mind? Or that they might regret it when they're older?
And yet I've had all of this and so. much. more. My particular favourite was "You don't know what love is until you've had a child". My response that they must have lived a pretty loveless, awful existence prior to children was, apparently, unfair 
I'll tell you the most hurtful comment though:
Oh, we just met up with the kids.
This from a group of old friends, who I've known for absolutely donkey's years and who have all, one by one, had children. Great, I'm genuinely pleased for them, especially the ones who've struggled with infertility or other problems. But they've gradually formed a new group which is made up of exactly the same people...except me. I've seen photographs of them on FB doing the same things as we used to do but with the children - obviously - in tow. When I've mentioned that I didn't realise they were going to the pub garden for the afternoon, or meeting up for a coffee with one of the group who was back in the area for the afternoon, I get "Oh, we just met up with the kids". No you didn't, you met up with each other and your kids just happened to be there, unless you all actually sat separately and ignored each other while the children (several of which are still only babies) interacted together. You just didn't invite me because you only want to socialise with other parents now.
I can totally understand not inviting a non-parent to, say, soft play but to be excluded from just sitting in a pub garden because I don't have kids and they do? It's incredibly hurtful.