If you go to Hyde Park you stand a chance of seeing the Household Cavalry practicing on horseback. There is the Serpentine lake with boats. Over the road is Kensington Palace Gardens which has a beautiful Peter Pan inspired playground with pirate ship, tee pees etc and there is a nice statue of Peter Pan also. There is of course the Palace (though that is not terribly interesting for children) and another pond and a bandstand. You might also see some wild parakeets which have bred from escaped pets.