I don't know the city well enough to be able to answer that to be honest - I would only be doing what you would and Googling it! But I prefer being in a town to being in the city - I have a sense that with a small child I would always rather be somewhere that is (or seems) more friendly.
With that in mind, you could also look at Plymstock - I have friends there who love it and have young families.
But Ivybridge feels less 'part of the city' to me than Plymstock does. Obviously I can only recommend what I know personally, so you will get a totally different opinion from others I'm sure! Also, you and I could be very different in terms of what is important to us. I have a two year old and these are the things I like about being here:
Easy driving distance to the beach and walking distance to the moor.
Easy to get into the city if I need to.
Enough shops in the town that I don't need to go into the city much at all (but bear in mind I am not a big window shopper/clothes shopper)
Great children's centre
Cinema
Library services are good
Big (i.e. huge) Tesco very nearby (five mins by car) if I need it, but also three smaller supermarkets in walking distance
Garden centre and farm shop in walking distance
Several parks and well-kept play areas
A couple of nice cafes where I sometimes meet with friends
There's an OFSTED outstanding rated nursery very near my house and two primary schools that I have heard good things about. There are other primary schools in the town but I don't know anything about them.
Although we live on an estate, it doesn't feel cramped - space around the houses and wide road
House prices are lower here than a lot of places in Devon - we moved here from a town about 15 minutes' drive further north because we could get so much more for our money
Lots of community stuff going on if you choose to get involved, and about three free magazines that pop through your door each month telling you about them
Interesting things for children to visit like the fishing lake and the donkey sanctuary
Feels close to the countryside because you can see the hills rising up around the area, even when in the town.
You could also look at Lee Mill as a possibility, but you would be more reliant on your car/buses if you were based there as it is a sort of extension of Ivybridge.
The best thing you could do is come down for a few days and have a look around the general area to get a feel for it. Is that a possibility?