Do not feed bread to the ducks or take nuts for the squirrels Bread is bad for ducks, and the squirrels are viscous beasts.
Diana Memorial Playground is great, but can get very, very busy so it's best to go early. The Holland Park Adventure Playground is less busy and also great - it may be getting refurbished at the moment though.
For fountains - you can either go to the paddling pool and fountains in the centre of the V&A museum (South Kensington), or you can also try Granary Square - this is a new area but very chic with lots of cafes and a great way to kill some time with the kids playing in the fountains.
For markets, Borough Market is much nicer than Portobello Market, although it's in Borough and not Kensington. It's mainly food though rather than tat. Maltby Street Market is also nice, but much, much smaller.
If you do go to Kensington Palace, you can visit the Orangery for a nice cup of coffee. Pedalos on the Serpentine are also possible, but that's the City side of Hyde Park.