It is bloody difficult!
I have worked since DD was 6 months old - a year FT+ and the rest 3 days a week. Have just ditched my job and about to be self employed consultant.
I think you have to accept that if both parents work FT, then some elements of home life suffer - whether that be cleaning, sick kids having to go to grand parents (which not everybody has on the doorstep), no time for hobbies or friends, quality time with your OH, etc.
On the other hand, if one of you chooses to be SAH, then you have to accept the disadvantages of that, i.e. financial vulnerability, isolation, sometimes boredom, no money for hobbies (like my expensive ones anyway!), etc.
I do think it's a matter of looking at your circumstances and agreeing with your OH what works for all of you. In your case, if you were to work, then your DH would probably have to do more at home. Does he want to do that? Do you want him to do that?
In prep for DD going to school next year, I am about to go self employed. My job has crap holidays and although my employers are very good at giving me time off for sick kiddo, in service days etc, atm I can't go to all the school events (once a week atm, which is quite enough, never mind twice), and the job is just not well suited to being done PT. It's not going to change anytime soon, and going s/e suits me well for reasons other than childcare - although it will be a massive help in that regard.
Do you fancy being s/e? Is there anything you could do in that line which could bring in some dosh, give you some independence etc, but still allow you to be there for school holidays? I know lots of folk on MN say just use holiday clubs etc, but certainly where I am geographically, there are VERY few child minders, and ever fewer after school clubs, holiday clubs etc.
Tbh, I think that with 4 DC being a SAHP is still FT or thereabouts - even if they are all at school. If you have 1, and they go to school, I understand that maybe you might legitimately have some spare time you could use to work - if you could get past the holiday 'problem'. I can only imagine the volumes of shopping, cooking, cleaning, laundry etc.
If you don't have to financially, and you don't particularly want to, then there is no pressing need to get a job.
Obvs, if you have to for financial or other reasons, then it's a different matter and yes, you could totally work it out. But yes you would be permanently exhausted. I also feel like I am letting EVERYBODY down. My DD at home, and my clients and colleagues at work.
Human beings are not robots.