I really feel for you. You must have a specific learning difficulty of some kind regarding processing, short term memory etc. Most likely dyscalculia and dyspraxia, I would think. Your written English is very good. I think that, with coaching, you could pass English Language GCSE because not that much of it requires memory skills, it's more about what you can write in the moment. But Maths is a different case and, without it, not many employers will take you.
I do agree with you that it's unfair for some jobs. Farm labouring, fruit picking, security, shop floor work, care work, majority of catering/hospitality jobs and some other manual jobs have different essential skills/personality traits which can't be assessed by Maths and English exams. I wonder if it's the memory difficulties which are your real barrier.
I'm trying to think of something that might work for you. Maybe waitressing as you can write the orders down? Care work if you set alarms that tell you what you need to do when?
I'm really sorry, it must be so frustrating for you because you're clearly an intelligent person who wants and deserves valuable work to do. I feel like there should be learning aids out there to help you access the world better but I confess that I don't know specifically what they are.