Until fairly recenbtly women continued having babies until they hit menopause, which could be in their fifties. I think the only real issue is the risk of defects increasing over the age of 35. The rest is just pressure from society.
When people make odd or negative comments about pregnancy, in my experience its often due to their own personal issues, more often than not envy. Any negative comments I have had were from women who had never had a happy stable relationship and so not had the choice to have a baby or when to have a baby, or from people who felt the oppurtunity for a family had passed them by