Again this depends. for one you say "working" well you just mean employment. I do work at home which is cleaning, cooking, and childcare including education, so all this is work that others would be paid for, and it's still work just like their work is, only not paid via PAYE, instead I'm paid in resources; home, clothes, food, security.
If you mean employment I have always found that very difficult for a number of reasons which are personal to me but certainly not exclusive to me. I would also find it hard as in difficult to use childcare instead of be there for my child myself in instances that are not school, and I did feel this way about school which is why I home educated for the first two years of school.
Going to work, traveling, being on someone else's schedule, spending all day out of the house unable to relax or have breaks, get a coffee, chat to friends when I wanted I would find difficult.
My husband enjoys work and the social aspect and he also seems to get a real kick out of being a provider (does that make him a regressive sexist person? Yes. Is that an issue for us? No)
But in the past employment was necessary to my mental health because without it I lacked structure and my mental health has had issues over the years, like many people.
So it all comes down to how we want to spend our time. I absolutely love spending time at home, on my own schedule, doing work I thoroughly enjoy and would literally do for nothing, with a day spanned out ahead of me knowing that at 3pm I get to see my child and hear all about her day in school. Knowing that if she were to not be able to deal with school I would be able to instantly change that for her and do home ed again.
That freedom and flexibility is what I value - and I'm pretty sure this is what most people value and if work gives that to others that's fantastic, it did to me! Now, it doesn't, it would hinder both those things which I hold to be most valuable to my personal enjoyment of life.