There's loads of outdoors stuff you can do. There are some lovely streams for the kids to paddle in - lots of people go to 'Brockenhurst beach' near the Balmer Lawn hotel, but that's next to the main road so it's much nicer to go out into the forest on the other side of Brockenhurst at Puttles Bridge as there is a lovely shallow stream there in the forest near the car park. You sometimes have to share it with the ponies though!
You can hire bikes at various places - there are loads of easy, flat tracks on the forest. There are also riding stables at Brockenhurst and elsewhere if the older one fancies that. They have Shetlands for the little ones. They take them out for a hack on the forest and the rest of you can walk alongside.
Beaulieu is a really pretty village and they might enjoy Beaulieu house and the motor museum - they've got a new kids play area there and there's a monorail. Lots of donkeys, ponies and cows wandering around the village. Exbury Gardens near Beaulieu are also pretty and they have a steam train that the kids love. You could also try Bucklers Hard, where you can go on a boat trip on the Beaulieu river.
Depending on where you are staying in the forest, the beaches are in easy reach. The sandy beach at Mudeford spit is beautiful (get the ferry across from Mudeford Quay - you have to get there early for parking as it gets really busy) or there are shingle beaches at Milford/Barton and Lepe.
If you want shops/cafes/pubs etc, Lymington is a lovely town. Market day is Saturday so you have to get there early for parking that day.