As a tourist, assuming you stick to tourist type activities and dont' venture into the less savoury areas, you should be fine. And it's a wonderful place to visit and brilliant with children (except lots of restaurants don't have high chairs so take a portable one).
Stay in guest houses or hotels in mainstream areas, rent a car or take Uber's and you'll have a wonderful time. The only real safety thing is don't walk around at night. South African cities are not walking=friendly overall, and especially at night (you can wander around the waterfront sure, but I wouldn't walk int he city etc. For context, I lived a 15 minute walk from my office and I still drove, not least because I didn't want to be walking in the evenings with a lap top).
For the kids, there are loads of things to do. The Atlantic side (camps bay etc) is more flashy and smart, but the water is colder and there will be lots of posher, child unfriendly restaurants. However, family friendly restaurants are everywhere and you'll be amazed as your child is taken off to the kitchen to make pizza/ encouraged to hit soft play or invited to face painting while you're eating!
Toddlers LOVEc the scratch patch - basically a massive ground filled with polished semi precious stones in weird shapes. They buy a bag and go "hunting for treasure". The waterfront is generally fun with shopping, play areas, restaurants etc.
Visit the Penguins at boulders beach for another fun, kid friendly activity.
If you want to visit a wine farm and stay, many are kid friendly and very relaxing. We stayed at Spier and enjoyed it immensely but there are loads.