Lots of different issues here.
Household income often determines if any adult can work part-time, regardless of what the exact working arrangement is. Households with 2 working adults will usually have higher incomes with those with one and of course, more skilled and qualified people tend to be paid more. Given the middle class are often more skilled and qualified, their salaries tend to be higher and if there are 2 adults in a family, 1.5 salaries might well be enough to live on.
The people who probably struggle most are single parent families, especially if their skill set or qualifications or child caring responsibilities limit the types of jobs they can do and money they can earn. As others have said, many of those jobs are more rigid in terms of hours, being done under supervision and less open to flexibility, regardless of how many hours they might be.
I think it's a reality that some jobs have more scope for flexibility than others. Customer facing roles need a person present at set times and many of those tasks cannot be done from home or at other flexible times. These tend to be the lower paid jobs so more occupied by the working class or lower paid workers. There doesn't seem to be a lot that can be done about the nature of those jobs not being flexible.
I'm sure there are some lower grade jobs that could have more flexibility in offices. Home working and flexible starts etc are perhaps slower to be given to lower pay grades - and this could be addressed more. Employers do need to know workers will do the work and not take advantage - which is the case with all flexible working arrangements - but do people feel that lower grade workers are less trusted with regards to these things? Are lower grade workers less able to self-start and deal with difficulties as they crop up, than higher grade workers? There probably is a perception of this and it is possible that attitudes to work might differ amongst different grades if worker leading to employer reluctance. WHat do you think?
Regarding holidays etc, employers do need to look more at the bigger picture and allocate fairly. People without children or without flexible working shouldn't always be at the back of the queue - it is up to the management to handle this and not the workers who have flexible contracts themselves. Management need to take responsibility and ensure flexible working doesn't disadvantage others who don't do it.
At the end of the day though, in a system where people earn differing amounts and do different kind of jobs, those with higher skills will often earn more than those who lack them. Those people will have more choices than those who don't about types of work and indeed working patterns. Families with 2 high earners will have more options than those with just one. It is capitalism - having resources and income gives you choices which those without don't have.