I think it’s an excellent age to visit London with your 11 year old, will be harder work with your one year old of course but everywhere is at that age 😬.
Definitely yes to HP studios and the Tower. Paddington would be a great theatre option for your 11 year old - it’s genuinely uplifting, but maybe don’t include the baby on that one.
Getting the Uber boat along the Thames is a fun thing to do, you can see all the key sights easily. A good route is from Embankment to Battersea Power Station. Your 11 year old is the right age to get a lot from the one hour audio tour of the Houses of Parliament, you’d need to book. Walking through St James’s Park and along the Mall to see Buckingham Palace is an interesting route.
Museums - proceed with caution in August, they’re really busy. My top tactic is to limit a visit to an hour or so and be really targeted about what you’ll see. So decide to go and see the Egyptian section at the British Museum for example and get in and out. The V&A might work well for you, there’s a paddling area and garden in the centre which is a bit of an oasis. The Tate Modern is pretty special, again I’d plan what you’ll see in advance and be targeted.
Re where to stay, you’d don’t want to add a long tube journey to your day so I wouldn’t stay too far out.
The south of England does not have many clean rivers/lakes suitable for children sadly so I’d aim for at least some hotels with pools.
London does however have great water play areas for children, especially Granary Square fountains in King’s Cross and the Diana fountain in Hyde Park.
Along the Thames to the west of London is lovely, water generally clean and some paddling spots. Walking on the Thames Path around Cookham/Marlow for example is a good day out.
Re beaches, I love Devon but it’s a long drive. Beaches are often pebbly near London but you could consider West Wittering or Dorset beaches if you want sand. Kent has good sandy options, for example Broadstairs or Margate.