I have done it. Senior management in the NGO sector, travel abroad and within home country. Next step would have been CEO. My priority was job satisfaction rather than status or money, but they came with the package and were very pleasant 
Then I adopted DD, and my priorities changed. Although the adoption was the effect of changing priorities rather than cause.
Having said that, looking after DD includes being able to provide for her financially as well as emotionally, plus having the emotional stability to meet her needs - both of which in my case do involve working outside the home. Financial needs include plans for an income once I retire, not just immediate needs. My emotional well being requires a life in the wider world, the structure of going to work, and the interaction with other adults. Having said that, I could get that by working on a voluntary basis if I didn't need the money.
I have been lucky in finding PT work - while it has been far more junior than before, in each job I have learnt something new that adds to my CV. I have thought about offering pro bono consultancy to NGOs to keep my links and expertise up to date, but for various reasons have decided not to. I don't think it would have been difficult to find organisations who would have liked my input though, it's just not the right time for personal reasons. If I ever want to go back to that career I can probably use that route in.
Luckily the NGO sector is a bit more used to the idea of career breaks and flexible working, plus there are not the same demands in terms of continuing professional development than if I had been e.g. a doctor.
But as others have mentioned, it's not either/or. Perhaps being stricter with boundaries would do it - do you enjoy your job apart from the way it affects your family life? (Part of the reason I found it easy to change was that I was burnt out with constant fundraising and juggling financially because the need is always greater than the resources).
Would the same job in a different company allow you more flexibility? Could you freelance or do consultancy work, while planning not to make much money (so you can turn down work if you want to).
From my experience and reading this thread, those who have been happy down shifting have done it with a clear idea of what they want, rather than what they don't want, if that makes sense?