The real question is why there are more young people in this situation than their used to be. Of course there have always been people unable to work.
I have some doubts that it is mainly about young people being more ill with diseases etc. There could be some small element of that, but it's not, IMO, significant. And that will include autism. I don't think autism cases have actually gone up that much.
I'd look at three main things:
Changes in types of employment available meaning that people who used to find work in certain types of jobs no longer can find anything they can do. (Related to this, potentially, changes in education?)
The mental health crises in young people.
A change in attitude to work among young people.
I think these are probably all factors, but I think the latter two are connected to some extent. There is a definite feeling among a good number of young people that if they are feeling shitty, have any pain, or are facing some kind of adversity, they cannot work. I suspect it's a true belief for many of them and it is actually really bad for their mental health.
I have a lot of serious mental health issues in my family and extended family, including older generations, so I'm not unaware they can be very real and sometimes debilitating - I've had family members who have died from mental health problems. I'm not trying to imply they can't be serious.
But the current explosion among young people is not an organic phenomena, and it's not been brought on by "hard times" - those have existed in the past as well, we aren't at some uniquely bad place in history, even if we just look at the last 150 years. It's to do with something we are doing.