Op, I'm sorry to hear of the experience you've had, and no YANBU
I think one of the reasons some people struggle to accept that its not clear cut is because it's assumed to be willpower. Its not.
The example I found most helpful for this was a training course I went on where they described childhood as the foundations and ground floor of building a house. An ideal childhood would have solid foundations and all the bricks in place, ready to start building upwards. For an abusive childhood, take away bits of the foundations, or make them shallow, or in unstable ground, or remove them completely. Same with the bricks - the more severe the abuse was, remove more bricks. You've got the start of the house but it's patchwork.
Now imagine adulthood - the next storey up - and it's obvious why adulthood would be more fragile for some. Some people might have resources - friends, education etc - that means they can add in a few bricks and make the walls a bit stronger. Some people might find they can't reach the gaps, but they find other ways to shore it up, there's still gaps of bricks but they'll add in a crossbeam or some props and the wall will be stronger.
For those people who have a strong house, they already have what they need to keep building, they've got a head start.
For those who have a house that needs work, if they can strengthen it, they can keep building. But it will probably take longer, because they've had to spend extra time and energy fixing things.
For those who've got no way of plugging the gaps or shoring it up, they might spend all their time trying to protect themselves against the elements and building stories 3/4/5 might seem like a pipe dream.
Sorry if that comes across as trite but I honestly found it a useful way of thinking about it. Our childhood builds our personalities, it affects our development - emotional and cognitive - it impacts our temper, patience, self esteem, self belief. Someone who has had a difficult childhood is not going to have the same resources to 'pull themselves up by the bootstraps' as someone else might have.