The thrill of getting a paper published, getting a large grant funded or the euphoria after a large conference lecture has been particularly successful is great. However, I found that as my research group increased in size, I was doing less and less hands-on research and more and more admin (project management, financial management, personnel management, grant writing, paper writing, sitting on different committees for endless hours, coursework marking, exam setting and marking, reports on student performance in exams, statistical analysis of student performance in exams, redesign and re-validation of the undergraduate degree, responsibilities to journals - peer-review, editorial decisions etc, just to name a very small part of it), that my job was no longer about doing active research, but about managing and facilitating others to do the research I loved. This is a necessary progression in academia. If you want to move up the career ladder, you find yourself taking on more admin and chasing more money (despite the misconception of some PhD students, the university does not give you any money for research, so everything done must be funded by money you find and running a science lab is expensive business) whilst the active research is being done by your group.
In addition, you are only as good as your last paper/conference lecture and extensive travel is an essential component of the job. If you are not visible, you cannot network and get your work (and yourself) noticed. Whilst I was on maternity leave with ds2 I attended 4 international conferences. There is no possibility to take a year out completely if you have a research group to manage and if you want to remain visible in your field. After having ds1 I attended my first conference after having him when he was 4.5 months old. The conference was in the States and meant a full week away.
Also bare in mind that academics do not get the same holidays as undergraduates. There are no holidays every 6 weeks and no long break over the summer. This means your children will need to be in holiday club during school holidays.
If you are only looking at academia to do a PhD, it is a completely different story to being a full-time, permanent academic. Doing a PhD is a time where you get a focus almost exclusively on your research (though this is all consuming, it is a luxury that academics don't get later in their careers). However, don't think that a PhD will immediately open doors to an academic position. In my field, most people in permanent positions will have done at least 5-6 years on temporary postdocs first (hence the reason why most University Lecturers are in their 30's before getting a permanent job).