I am on the Isle of Wight (Cowes) and the first thing to say is that you will be able to buy practically everything you need once you get here! We do have several of the main supermarkets here on the island - Tesco, Sainsburys, Lidl and Co-op. So I would recommend travelling light - or as light as possible when you have two small children! Ours are 3.7 years and 16 months old.
I am not sure where Haven has sites on the island but really, nowhere is especially far from anywhere assuming you have a car? (If not, it is much MUCH cheaper to buy a rover ticket on the bus rather than paying for individual journeys which cost a fortune!)
We have been having some lovely weather - we have been at the beach for the past few weekends! Children friendly include Ryde, Sandown, Ventnor and Shanklin for sand. Colwell Bay and Compton are also nice and less 'touristy'.
If you are looking for play parks, some of our favourites include: Gurnard seafront, Bridge Road in Cowes, Sandown by the Heights leisure centre (also probably the best public pool as the shallow end is the whole length of the 'baby' pool) and Yaverland in Sandwon - the far end by the zoo.
Animal places include the zoo in Sandown, Amazon World (our 3yo loves the sloths and the armadillos), Coleman's Farm and Fakenham Farm.
If you need an indoor soft play then you could try Space Island in Lake and JR's place on the Industrial Estate by Newport but I recommend The School House in Cowes - it is a proper cafe/restaurant with a free indoor play centre which is great for little ones but not for over 5s or so which makes it even better for the small ones! We were there for over 90 minutes last weekend and practically had the place to ourselves for half an hour over lunch! Then we went downstairs to the restaurant and had proper food for lunch at proper grown up tables! And yes, it is free for the kids to play!
Almost everywhere is child friendly - we always take our girls out to eat and have not had any problems as far as I can recall.
Also, regarding staying at Haven - are you in a caravan? We have done Eurocamp in Holland and France so figure same set up. Is the baby still in a moses basket or are you planning on a travel cot? These tend not to fit in the main bedroom in a caravan - space is usually limited. Also check for bedside rail for older child.
(Not wanting to be pushy but, if travel cot or bedside rail are needed, we do have these items in our loft that we could easily lend you if helpful!)
Overall, just try to avoid comparing child friendly holidays with the resort or city breaks you used to enjoy. I am sure you will have a lovely time here - and the children will have fabulous time!