The Garlic Farm. Absolutely lovely, and close to Arreton which is an incredibly pretty rural area, close to just about everywhere as the island is diddy and it's pretty well in the middle. They have lots of options for places to stay and a very nice restaurant on site, too - great coffee, fabulous breakfasts. Get up early enough to get a table by the terrace, and you can watch red squirrels feeding.
Lots of the places by the sea are very old style touristy, which can be fun for a day trip but less so to stay in, IMO. Bembridge is lovely though, and if you want somewhere quiet but not rural, by the sea, it's a good bet. But as it's small and charming it's really expensive, from memory. Godshill is inland, and like a village from an Enid Blyton or Miss Marple tv series - super pretty, but also a total tourist trap.
Personally, I'd stay in Ventnor if I didn't want rural. Very foody, and clustered against a steep hill down to the sea so exceptionally pretty, but doesn't feel like a toy town in the same way as some of the other places can. Feels like people still live there - same with Cowes and Ryde but they're also just normal towns to some extent. Bembridge and Ventnor feel distinct as a village and small town in that specific part of the world, and so does Arreton.
If you message the wonderful people at the ice cream place, Crave, via their Facebook page they will advise on the best places to stay and eat around Ventnor. They've ranked best ice cream place in the country on Tripadvisor and their lovely kind service is a large part of why - they know everyone and everywhere. We always ask them where is currently best to eat and they've never steered us awry! Sure they can do the same about places to stay, too. Incidentally, if you ask them to make any flavour at all, ready for your arrival, then they can and will. My kids arrive on the island and we go straight to Crave, where two of their slots have tubs of their requests ready to scoop. And that's the norm - they do it for anyone.
Love the Isle of Wight. It's really special.