Yes your post screams autism at me too, especially when you say he has got worse since Y3.
Infant education is very visual and hands on, but when they go into KS2, everything becomes much more complex and abstract and expectations of children are much higher.
Our son was really struggling by Year 3/4. Lessons were much more about talking and listening, and because his processing skills were so much slower than the average child, the normal delivery of a lesson was too fast for him to cope with, and it stressed him out. He did have some support at school, so was able to hold together until he got home and then all hell broke loose.
I cannot believe your son was discharged by SLT at 4 as he clearly has receptive language issues (understanding and processing information) which have not been addressed, as he is still struggling to understand everything which will be contributing to the anxiety and subsequent melt downs at school.
As PP above said sensory issues are a common trait in those with autism and with the genetic link present, I would certainly ask for an ASD assessment.
I would also ask for the school to bring in an Educational Psychologist to observe and assess him and ask for Speech and Language Therapy and Occupational Therapy assessments. As NHS waiting lists are so long, it may be quicker getting private assessments done if you are in a financial position to do so.
As your son is 10 and will be transferring to secondary school within the next year or so, it may be worth considering an EHC Needs assessment. You could ask the school to start the process or apply yourself.
Information here:
www.ipsea.org.uk/ehc-needs-assessments
Bear in mind though that some local authorities refuse the first request as a matter of course, and then concede when parents appeal. 