The Thames Clipper boats are fun. They're TFL-run so cost a lot less than the tourist boats but it's the same journey just minus the commentary. Also if you.end up going East e.g. Greenwich, the Docklands light railway trains are driverless (kids love to sit at the front and "drive" and have some good views. All TFL transport is best paid by tapping your debit card, and cost caps will be applied automatically. Just make sure you use the same card each time. No change for 6 year old, they just walk through the wide gate with you. You should get the hang of the tube map quickly enough, everywhere is well signposted and there are always helpful staff around.
Big Ben is right by Westmimster pier or station. From there you can walk through St James Park (lots of wildlife including the famous pelicans, and a good playground) to Buckingham Palace.
If you time it right, the Changing of the Guards is a spectacle worth seeing. Dates and times here. https://changing-guard.com/dates-buckingham-palace.html. Then from the palace it's a short walk to Trafalgar Square.
All of the above is manageable in a morning (I used to do it alone with 2 young children). You're not far then from Leicester Square, Covent Garden, Soho etc (bit of a walk, or a short hop on a bus or tube if little legs are tired). I'd personally highly recommend Covent Garden for fun shops, decent cafes.and lots of street performers. The transport museum there is great too but a little pricey.
Then for the other day if you want to take it fairly easy perhaps stay around the South Bank area, always plenty going on. Borough Market is an excellent food market close to your accommodation or go a bit further West for the London Eye, Sea Life, plenty of food and drink places and street performers.
Other good options would be Kensigton (Science Museum - free but well worth paying extra for Wonderlab and Natural History Museum); or Greenwich (lovely park, Maritime museum - free, Royal Observatory- free but you pay a few £ for the planetarium shows, Cutty Sark - costs - and the Greenwich foot tunnel & Meridian line). I'd allow a full day each for either area.
Have fun!