This is more of a question than a suggestion though I have a few favourite places I take dd.
Greenwich Maritime Museum is v. nice with wave machines and lots of stomping areas for mobile babies and there's a lovely play park in the beautiful park outside when the weather's finer.
Tate Modern can be good but can get v. busy.
The Imperial War Museum is good, especially for boys as it's full of tanks and boats and some reconstructions of things like bunkers which are quite exciting.
But as you'll see these are all museums and I'm a bit stuck to find other purely baby friendly places.