As someone who was born on the IOW, lived there for 26 years (and still have the majority of my family there), I have to say YABU - sorry😁
As you’ll have seen, the island is a beautiful place in the summer and it’s wonderful to visit for a staycation. Living on the island is a very different reality. For one, getting on and off the island is not only expensive, but the boats are extremely unreliable. It depends on your circumstances, but if anyone in your family would be intending to commute off of the island to work even occasionally (or you regularly go away for family holidays which require airport access), I would strongly advise against moving to the IW. I spent a good few years commuting to work from the island and the stress was insane. You’re frequently faced with delays and cancellations either due to the weather, technical issues with the boat etc. Also, boat timings don’t line up with any of the main trains e.g. to London, which is also annoying.
Trying to get deliveries to the island is also tricky - Amazon won’t even deliver liquids there for some bizarre reason! Also, any heavy items like sofas and bits of furniture can either be very expensive to deliver or you’ll find that a lot of well known brands will not deliver there at all. Popular services like Uber etc have not reached the island. Other than that, you may have noticed that there are barely any decent high street shops on the island and more seem to be shutting down each time I visit!
In the winter it can be quite a depressing place. There’s not an awful lot to do, particularly as quite a few touristy related places close over the winter period. Many children find it incredibly boring and whilst the island is generally safe, there’s quite a lot of petty crime largely from bored youths with nothing better to do. Most islanders will tell you that crime rates are on the up (have a look at the Island Echo website), though I guess that’s the same in most areas sadly.
Theres very much an aging population and most of the people who move to the island are either retired or nearing retirement and looking to slow down. For this purpose it’s perfect, but I wouldn’t encourage a younger family to move to the island, personally. Career prospects there are pretty dire! Also, if you value your privacy you may also struggle living on the IW - it’s very much a case of everyone knowing everyone!
Sorry for the long post but hope this helps!