The Warnock Report was earlier than the 1970s and suggested 20% IIRC, so I do think that there was a 20% that there was an awareness of even in the 1970s. Some of them BTW were given names like "Educationally Sub-Normal" which is not especially helpful and no longer used today. We didn't have an understanding of the more specific diagnoses like Dyslexia until later on.
The thing that probably makes this make more sense is that the majority of that 20% would not have been in mainstream schools. For example OP said:
Other medical needs can be things like cerebral palsy where the child needs support from an adult to do regular physio.
I don't think children with needs like regular physio would have been in mainstream schools until fairly recently. In some ways the inclusion of children with needs like this in mainstream makes absolute sense - there's no reason for them to be in a specialist school if this is their only need and they can access standard education otherwise. However, it does make it difficult for the school because presumably this physio can't be done in the classroom so it takes an adult away from the class for regular periods and this probably isn't properly funded. Also there might be more children reaching school age with these kinds of disabilities. In the 1950s, my grandparents had a child with CP and he died before he was 1 year old, which was presumably more common in the past.
I understand that there has been a measurable increase in the number of children with significant medical needs and disabilities, due in part to things like better survival rates of premature birth, and these children are now taking up the majority of spaces in SEN schools (which makes sense, as they obviously need that support) but this means that children who would have previously been sent to special schools are now kept in mainstream because there is no space for them elsewhere, and the accommodations they need may not be practical or realistic for a mainstream school to implement. I would guess this is also related to the mass closure of special schools/inclusion policy in the late 90s? As in the schools were closed because it was thought it would be better to meet the needs of "less complex" pupils in MS schools - arguably likely to be true if the schools had ever been given enough resources, funding, staff and training in the first place. But the combination of fewer places in special schools and higher numbers of children with the most complex needs has meant that not only are children with low support needs in MS schools but also children with more moderate needs, who were never expected to be there in the first place.
I wonder also if access to early intervention has reduced because the poster saying they work in a nursery and years ago children wouldn't come to nursery without being potty trained etc. But there must have BEEN children with SEN in the past too, even if they existed in smaller number. So why weren't they ending up in nursery? I feel like in the past, health visitors had more time and smaller caseloads and did more developmental checks, more children seemed to get speech therapy early on, which might have then sent them on a split path - with some improving and their other communication/behaviour/development improving as a result and they went on to nursery and school without much incident, whereas some would have more complex needs identified and perhaps never would have gone to a mainstream setting.
Plus when DS1 (now 17) was entering nursery, I was told that he MUST be potty trained and was not allowed to attend in nappies. I only managed to get him dry about a month before he started, so I remember being worried about it. I wonder if perhaps I hadn't managed, what I would have done - maybe I would have kept him at home until he was school age? Or if he had had some SEN that made it difficult, I probably would have taken him to the doctor who would have been able to refer him. Friends who are still in the UK (I am not) have had a child referred and they were literally on the waiting list for years while their undiagnosed autistic child attended mainstream nursery without being able to speak, interact normally with other children or use the toilet. It seems to me like if they had had an earlier assessment of needs, they might have been offered a place in an SEN nursery. Or not. I don't actually know, but that's what my assumption would be.
I've seen anecdotal stories as well from MNers on long threads like these, where their child was identified with some difficulty at a younger age and told they wouldn't manage in MS school so offered a place at a specialist school. But these days parents seem to have to fight for this, nobody is being offered it, and in the meantime their child is on roll at a MS school and either attends with huge problems or is at home half the time with EBSA, all of which doesn't seem to work for either the child, the teacher, the parent(s) or the rest of the class.
Although I'm abroad so can't speak to the UK system, I feel this is exactly what has happened with my own DS2. He was struggling with behaviour throughout the local version of nursery and although he did have some speech therapy and I did take him for some developmental testing and they identified some problems including ADHD, it seems like ADHD is so common there is no blanket assumption that they can't manage in a mainstream school - which in many ways, fair enough. But then he did start school and it was just immediately a disaster. Again it is nonfunctional for him, for his teacher, for me and for the rest of his class, and there seems to be no automatic plan or solution and this has completely floored me. I was obviously massively naive about this, but I think I assumed that either his needs are pretty standard and so the school would be able to handle them, or his needs are extreme and should have been identified by nursery or the testing he had before he started school, and we should have been advised about alternative options, or the school would immediately identify that they can't meet his needs and put some kind of emergency plan in place, e.g. moving him to the remedial class, (which hasn't happened) or some kind of temporary arrangement while more assessments are done, which seems to be what we are doing - I went back to the doctor, they brought someone in from their side - but this all took months and in the meantime he's just been left in class in the scenario which isn't working. I have a meeting next week to find out what's happening next.