I know it sounds implausible, Caster8. Why would be be teaching children for so many hours in school if they could just learn things themselves? But it's true.
To take your maths example:
Insofar as maths actually is an essential subject, it surrounds us in our daily lives. In that case, it would be difficult to reach adulthood without having been exposed to maths constantly and getting a decent grasp on it. Few of us set out to teach our babies to talk through a formal programme. We know that they will hear language from the day they are born, and will learn to talk spontaneously. The same is true of maths.
But what about calculus? It isn't likely a child would acquire a mastery of that subjects through daily life, so surely we have to teach it to them? The answer to that is, if calculus isn't a feature of the child's life then he doesn't need it yet, so there's no need to force it on him. If he finds he needs it later (for example because he's decided to be an engineer), he will learn it then. He'll learn it willingly, because he sees the point. He'll learn it efficiently, because he can use it immediately in the context which interests him.
My husband, a roofing carpenter, decided in his 30s that trigonometry would help him in his job, and learned all he needed in a few weeks. He's quite competent at it as he uses it almost daily. My class at school spent an entire year on trigonometry; I doubt that many of my classmates now remember anything of it or that they have ever used it since. What use was that year to us? It only taught some of us to fear and dislike maths. For the few who were interested, it was very slow going because we were learning alongside people who didn't share our enthusiasm.
Here's a psychologist who thinks he's found evidence that introducing formal maths too early is a waste of time and even impedes children's mathematical understanding: www.psychologytoday.com/blog/freedom-learn/201003/when-less-is-more-the-case-teaching-less-math-in-school
This echoes my experiences at school and at home. You ask why schoolchildren don't learn huge amounts from their out-of-school experiences. They do, but sometimes less than HE children because compulsion at school actually makes many of them switch off. In "How Children Fail", the schoolteacher John Holt described the primal fear reaction which he observed in many children in his class. Their objective was not to do the maths correctly, it was to escape from the situation they were in with the least pain possible.
I remember helping some of my classmates who struggled with maths. There were questions they could not answer such as "4 x 1/2", and I remember wondering how it was possible for anyone not to understand such a thing. Anyone with four halves of chocolate bars in front of her would know how many she had altogether, so why could my schoolmates not answer the question? It was because they were in such a state of fear that they were unable to think, and because they had never learned to connect their daily experience with this abstraction on the paper... and then they had been presented with more and more paper-based exercises which confirmed them in their belief that maths was impossible.
My younger child cannot answer "6 + 1", but she understands that her six year old friend will be seven on her next birthday. She does not need to memorise "6 + 1 = 7". In time, when she is ready for abstraction, she will learn what "6 + 1" represents, and then it will be trivial for her as she will already have years of practical experience to fall back on. I am not going to risk switching her off to maths or confusing her by demanding that she master number bonds or use a number line now. Her teenaged sister can do all the calculations essential to her daily life because she has learned them through daily life. It really is that easy.