Hang on, this is YOUR inheritance from YOUR parents, right? Your DH sounds quite controlling trying to exert his wishes onto how it will be spent, especially talking about all the advantages for his parents.
I was in the same position as you and dithered about buying a holiday home. DH just said he'd go along with whatever I wanted to do with it, it was my decision.
I decided against a holiday home because:
It's extremely tying in so many ways. You're stuck feeling you have to "get your money's worth" frrom it and spend ALL your holidays there, and I like to go to lots of different places. Some people like the idea of spending all their holidays in the same place each time but that's not for us, we like to do lots of different types of holiday.
The seaside type places that I would have wanted are too far away to go for the weekend and as I work in a school I'm tied to school holidays, which are the busiest times in those sort of areas.
We can hardly get tradesmen who want to work on our own house never mind from 200 miles away. The responsibility of maintenance was too much.
Don't always know very far in advance when we'd all be available to use it, and didn't want the potential aggro of family having a sulk if we said they'd have to wait to "book" a week because we needed to wait to hear when DH could have his annual leave, or when the kids' prom or other commitment was or something.
Too much extra expense generally. Extra bills (which you can bet no family freeloaders would offer to pay towards), insurance hassles, council tax etc. The only way to cover the extra cost would be to rent it out as a holiday rental and then that defeats the object as the busiest times for those would be the only times we were able to be there.
As the kids got older they grew out of English bucket and spade type of holiday and were more interested in travelling abroad so in the school holidays they wanted to hang out with their mates, they wouldn't have wanted to spend weeks at a time in a seaside holiday town with less to do for teenagers than there is here in our city, and crapper public transport. Plus things like driving lessons kicked in. Hard to to that when you're living in 2 different places.
Now they are at uni, they are off getting cheap flights to Europe with their mates for a few days, or doing internships all summer. One has a foreign girlfriend so has travelled to her home country. It would be a mistake to assume that as uni students they could spend their summers staying there and working, even if that's what they say they'd want to do now. Things happen, circumstances and feelings change.
If you both work full time, when are you going to find the time to sort your OWN house, do your OWN garden, see your own friends and family, run errands locally, if you're away at the other place so much?
And of course, the most important till last, the ethics of it all. It would just stick in my throat to be the cause of local people being priced out of their own area. It's noticeable setimes when you go to a popular tourist spot in this country, and the service in pubs or restaurants is bad because they can't find any staff (because they've flogged the cheap staff accommodation, and the locals have had to buy somewhere much further away). Also, a holiday town dies when half the homes are empty some of the year, and proper high street shops struggle. Have you ever been to a seaside high street? They're full of tourist tat shops and Fat Face.
If I were you, I would teach myself about investing, that's what I've done. You can do it very simply and make better returns than with ordinary savings accounts. Some of my money I keep in higher interest bonds and then use the interest that I get each year to put in a holiday fund. We have had some lovely holidays that the then teenagers will remember forever. 2 California road trips, weekends away in nice hotels, with no washing up, no maintenance or gardening to do like you would have to do in a holiday home.
Just having the choice and freedom is way more advantageous to me than the bind of a holiday home.