Yes, very important not to turn up in London and ask where the Harry Potter studio is. It's not there!
When we've visited I've stayed at the Holiday Inn Hemel Hempstead which is pleasant and has a pool. However, it is a motorway junction hotel so may not be great for you if you don't have a car. (I'm sure you could easily taxi it from there to Harry Potter, though).
Try to get tickets for the first time slot of the day at Harry Potter. You first see a short movie and are taken through the Great Hall, and then they open the doors on the first sound stage, basically a massive warehouse-sized room full of wonderful stuff. The second time we went, ds and I bombed straight to the end of this first room, to where the green screen area is where you can be videoed on a broomstick and fun things like that, because the first time we went the queue was an hour and a half long. We had massively overreacted (in fact the people manning the green screen bit were somewhat shocked to see anyone there!) - even when we finished no-one else had even arrived However, it's still worth doing it that way round, to beat the queues - the sound stage is huge and you can visit stuff in any order, the only limitation is that once you exit the sound stage you can't go back in. So don't feel the need to follow the herd.
HP is going to take up a big chunk of one day, so whether you go into London at all is up to you. If you do, there are the obvious very touristy things like the London Eye (plus next door the Aquarium, the London Dungeon and Shrek's Adventure). However, keeping up the Harry Potter theme you could also visit the Millennium footbridge which features at the start of Half Blood Prince. This is between St Paul's cathedral and Shakespeare's Globe and near HMS Belfast and Tate Modern. Alternatively I have seen actual 'Harry Potter tours' advertised, which take you to the locations used in the various films, I have no idea how good these are.
It will be a lot cheaper to base yourselves outside London and head in by train. Once you're there, under 11s are free on London transport, and you can use a contactless card to pay (NB I don't know if it has to be a UK-registered contactless card, worth checking).