Working in a school within a subject which is often dismissed as "not needed" by those with low aspirations, I have picked up on a number of reasons why poverty and low aspirations often went hand in hand.
Some simply couldn't afford not to drop school after the age of 16. Benefits for children stop or are vastly reduced by that point, so they are needed to earn a wage immediately in order to keep going.
Some have seen many of their older relatives have a good life starting off in jobs that didn't use to require qualifications (childcare being a prime example) and just don't know how much rules and requirements have changed. Those are the ones that are often astonished when they first look at post-16 courses, by which time it is often too late.
Some are told their religious duties come before everything else (yes, this was very much a thing in one school I worked at) and they heavily rely on being able to work with relatives or family friends when they leave school.
Some have very high aspirations from the media they consume (pro footballer or singer come to mind) but don't understand that those require money to be able to take to pro levels. A former neighbour child was a karate progidy and his parents ended up travelling across the country almost every weekend for competitions and expert training. Many of our children don't understand the need for tenacity and money when looking at popular pro careers because so often a rise from poverty to fame is being portrayed as achievable in the media when the reality is very different.
Some just don't have a concept of money and what life costs because they never had pocket money, never saw their parents deal with money in person (card payments look and feel very different to cash transactions) or don't relise that their lifestyle incurs debt. I have had many conversations with students who did not have a clue that they needed to pay for e.g. council tax, water or even cleaning products.
Others had horrible parents and peers who would tell them where their place was and what the consequence of aspirations would be. I was repeatedly told I'd never make it in my chosen career by my family and my peers would bully me for trying to do well at school. It took an awful lot of grit to keep going and work alongside school because only my teachers would help me keep an eye on my goals while most of my environment just sneered.
And yes, some are just plain lazy. That can be a personality thing; one of my children is lazy by nature and needs a kick up the arse most days to get on with stuff while the other is a ball of energy. It can also be down to never having to work for anything until they are adults - so many of my students don't know how to cook basics, do laundry or even just ensure they have all their equipment because their parents do everything for them.
So there are many reasons why there is a divide and not all fit the common stereotype.