I think it depends a lot on your particular industry.
I have just had 3 years out and am trying to get back into work now.
Slightly different in my case because I went back after 6 months mat leave, but was made redundant a year later. The industry I work in was in a downturn, and because I didn't want a job which involved travelling away from home for weeks at a time my options were limited. So I wound up drifting into a career break.
Finding it really difficult to get back into work now, I'm afraid to say...
But it might not be too bad for you if your industry is doing well when you return, and if their is a decent demand for workers with your skills.
I wonder if the kind of industry where part time work is common is also more likely to be the kind of industry that is accepting of those returning from career breaks? There may be some correlation there, maybe?
What will help is doing something (beyond childcare) during your career break. eg, strategic volunteering, helping to run a toddler group, online courses etc.
In your position, I would work part time rather than not at all if I could. You probably wont see your career progress much during this time, but you will keep your foot in the door. Plus a couple of days away from your baby is not nearly as hard as putting them in nursery 5 days a week, especially if they are with grandparents (who they will be able to develop a close bond with, and who will be in their lives long term).
Make sure that the grandparent care is definitely there though; you would not be the first mother who makes her return to work dependant on the promises of grandparents only to find that they have changed their minds/are no longer able to provide regular childcare.