You'll see from my user name that we're not local to London. But I lived there for a while, know the city well, and have been a few times with DW and DD (neither of whom know the place at all).
When DD was about three DW had another meeting in London so I took the littl'un to London Bridge. We left the station, wandered down through Borough Market (picked up a few free samples, bought a couple of things) and headed down to the river. She'd seen a book about London, so was excited to recognise Tower Bridge.
Then we walked along the South Bank towards Waterloo. Saw the Golden Hind, views of St. Paul's, went into Tate Modern (which she enjoyed more than expected), watched some street performers and finished up at the South Bank Centre where we met up with my wife. Cost almost nothing, plenty to look at and chat about. She was able to walk some of it, and we had a buggy for when she got tired. Was a successful day out. Repeated it in Feb (DD now aged 7) and again had a good time but with much more chat and questions. It's worth doing a bit of prep work so you can, for example, use St. Paul's as a way of talking about the Great Fire, or use the Globe to introduce Shakespeare. Second time around, she loved the idea of the South Bank as some kind of semi-lawless Elizabethan badlands where cool but slightly dodgy stuff happened, compared with very sedate Westminster with its royal palaces and houses of parliament (this was largely inspired by Blackadder and the Barking Mad Bishop of Bath & Wells; you don't need lists of dates or famous people, but a plausible story works well).
Another day out that worked: train into Waterloo. Over Hungerford Footbridge to look at Cleopatra's Needle (DD interested in ancient Egypt). Also pointed out Big Ben, and the views down the Thames towards St. Paul's (that 'I saw that in a book' stuff again). Up Villiers Street into Trafalgar Square - lions, fountains, Nelson. Then along the Mall towards Buckingham Palace, with a detour into the park to see the pelicans. I think she was about 6 at the time. Again, minimal cost. We got lucky with a load of coaches coming and going from Buck House, and DD convinced herself she'd seen a real-life princess going to visit (might have been Sophie Wessex, probably wasn't).
The big things were knowing where I was going and what routes would work on foot (central London is smaller than you think, so unless you like sweating in a hole you can avoid a fair bit of the tube), and having a clear idea of what we might see and why it would be interesting. DD is very interested in art, so we always go to one of the galleries. I'll look up a few paintings before we go, so we can pick out some highlights, look at them, talk about them and not get overloaded with a million different pictures. I skimped on that prep work before the Natural History Museum and suffered for it: these places are vast, full of kids and if you don't have a plan you're going to struggle to make them work. If you're a tube ride away (rather than 3 hours and £130+ on LNER) you can plan 'little and often' trips, especially if you're going to target free attractions.