Oh for the love of heaven. I hope you do come back to the thread OP, and I'm sorry that I didn't see it before. Jeez, this ridiculous idea that you can't or mustn't expose little children to anything other than the things they already know well! Sounds like a smashing way to produce a(nother) generation of parochial, narrow-minded fearful xenophobes.
We've been to Vietnam, specifically to Hanoi and Halong Bay, with a 3 and 8 year old - this February in fact. (My 3 year old told everyone we were going to 'visit Nam' - like she was some grizzled vet). It was lovely. We took them to the temples, we walked a lot, even went to HCM's tomb, which was a fascinating experience for them. They particularly loved Halong Bay, which is an astonishing place.
The Vietnamese love children, it seems - especially European looking children. One of mine has blonde curly hair and she was constantly cooed over by giggly teenagers. We had a lovely time.
Yes, it's a dirty place (it has a smell that is occasionally revolting, truth be told). Yes, it's busy and haphazard and you have to be aware of your surroundings, but as a tourist it's fine. I wouldn't necessarily suggest a fully independent backpacking tour of the hinterland, unless that's your thing, but as a touristy tourist, it's a great place. The food is awesome, everything is dirt cheap and it's really interesting. Hanoi is full of Western chain restaurants, if you're looking for some familiarity occasionally too. I assume HCMC is similar.
My big recommendations are - just walk around, it's amazing. The traffic is horrendous (and so noisy!) but you'll soon work out how to handle it). More than that, though - do the boat trip at Halong, at least one night and two if you can. It's otherworldly.
Go - experience, it'll be great.