I think there are multiple reasons.
For one thing, in prior generations, people simply didn't have the options to sample the 'banquet of life' the way we do now, even those of us without vast incomes/wealth. A few generations ago, most people had limited education, limited career mobility, limited opportunities to travel, not much else to do. So the natural progression was to marry, settle down, pop out offspring at intervals because reliable contraception wasn't available. Gender roles were more distinct and men didn't experience the drudgery of childrearing as women did.
Also sexual mores are different. My elderly neighbour, born in 1920 and now deceased, was a bit unique in that she was a musician from about the age of 10, and in her late teens to late 20s, toured with an all-woman band. They had lots of bookings and were in demand but it was still a sort of vaudevillian life staying in cheap lodgings, budgeting strictly for food, etc. I loved to hear her tales and once remarked that men must've been plentiful (it was wartime) and she said yes but that she and her mates had firm policies about not drinking and that "we all were virgins when we eventually married." (I believe her, not so sure I believe it about all 8 or 10 of them!) She said "Women get married because they want kids and men get married because they want regular sex." Old-fashioned perhaps but not totally off the mark.
Nowadays, men have myriad lifestyle options, can partake in sports/hobbies/clubbing or just enjoying life. Air travel is cheap and affordable if they desire to see the world. Social "norms" have changed and no one looks askance if someone isn't "settled" at any particular age. There's no external pressure. And they can get all the sex they want without being married.
Also in many cases, women are happy to set up housekeeping, foot half or more of the bills, even bear offspring, without a commitment. And no matter how oafish, slothful, lazy, abusive, boozy or dim the man, as we see here on Mumsnet it seems that somewhere there is always a woman who will put up with anything to be paired with them. So they have little incentive to strive or shape up into decent hardworking providers. And very little incentive to desire domesticity that requires any effort or pledge on their part.
I know it's considerd a "dated" notion but if women would raise their standards, perhaps men would raise their behaviour. Why continue to mate with such losers? Better to be solo and childfree than to settle for some begruding, resentul or disinterested partner.