Went to the Natural History Museum in Portsmouth today. It was a great small museum. Especially good for a nearly 3 year old who likes dinosaurs as the museum has a large model dinosaur and some smaller dinosaur related exhibits.
There is a butterfly house section (May-Sept) where children if standing still enough, can have a butterfly land on them.
The museum is spread over several floors, though is fairly small so only takes half an hour say to visit. Perfect for young children who get bored quickly.
The venue is not suitable for buggies. I would also say try to avoid carrying anything as some of the walkways are quite narrow.
NHM Portsmouth was a nice place to visit whilst also doing other things in the area.