It could be, too, Anchovy, that in the past, when older people had babies, it was NOT usually their first or second ones.
Most of the time, when a woman in her 40s had a baby, she'd had quite a few already, like my grandmother.
Also, there usually wasn't a period of heavy partying, recreational drug use/alcohol consumption or recreational smoking that many women nowadays had in their 20s or 30s, all of which can affect egg quality.
Or years and years of pill/contraception use or multiple casual partners, all of which have now risen the number of cases of STDs in men and women in their mid-30s and up and some of which can compromise fertility.
Yes, older people have always had babies, but usually it was because it couldn't be avoided - lack of birth control, marital rape legal, etc.