BackforGood Unless you are clergy, or the child of clergy, you have absolutely no idea of what it is to grow up as that. And I say that with kindness, and respect, and Christian love. You may be in a church and "know" clergy families, but you have no idea of what it is to live it. And of course there are going to be different experiences, I know loads of different types of clergy families spanning over 30 years.
But I'd hazard that most if not all would say that their personal cost of the vocation (because it is a vocation, it's not a job) is to their whole family. Because when you accept ministry as your calling, you're accepting it for your family too. And it is a 24 hour job. 365 days a year. Because there's always somebody who wants or needs something.
Like greybow said, it is harrowing as a child to have phone calls come through about death, and funerals, often left as answerphone messages. Or to have people turning up and knocking on your door, wanting and demanding, whether it be time, energy, money, shelter, because any person who wants something can find your address or phone number in the parish magazine. The being held to a higher standard purely because you are the child of a vicar. The being visible, again, because your parent is a "pillar of the community" which dictates your behaviour and experience as a child.
You never know if your parent is going to be available to you, everything, and I do mean everything if subject to change because someone else might need them. The dead, dying, family of the bereaved, the needy, those in mental health crisis, those who are struggling, children are struggling, parents are struggling, those with no family, the sick, the family of the sick etc, your parent is constantly available to all these people, all of the time. And that's without the normal services, the sermon writing, children's church, going into school to do assemblies, the Diocesan meetings, parish Council, funerals, funeral prep, aftercare of families, weddings, baptisms etc, morning prayer, bible study groups, evensong, sunday service, notice sheet, food bank, clothing bank. And those are just off the top of my head 
I do agree that its largely about boundaries, emotional and physical, but as a child, you can't impose those for other people on behalf of your family. Especially when being an emotional support for those who require it is part of the job. And they come straight to the source, because "the office" is the home.
I hope I've articulated this well, it's late, it's been a difficult, emotional week for me and i am exhausted with grief at the moment. So if I have come across as rude then I apologise in advance as that was not my intention.