One thing we need to think about is the enormously increased cost of providing services in rural areas.
Even things like bin collection cost far more in the countryside. Things like adult social care, hospital services and public transport for those who age out of driving, become horribly, horribly expensive to provide in the countryside - bankruptcy-inducing for local governments, actually. Rural buses are massively loss-making and cost a fortune. Care workers have to spend long periods of time dragging around to this place and that, meaning that you require huge workforces to deal with a relatively small population (whereas when elderly people live in, say, retirement flats in time, you can use warden control system to keep people independent for longer, and a careworker can nip round and help a whole bunch of people in the same flat very quickly).
Holiday home owners don't make much call on these services, for obvious reasons. Long-term "generationally rural" communities do make some call on these services of course, but it really helps that younger members of communities/extended families can help the elders out; for example, it matters less if the bus service isn't great if the younger members of the community can give lifts to the elders and pick up their shopping for them etc.
The real danger point comes when communities become hollowed out by retirees who have moved in from outside,or when nearly all the homes get bought up by second-home owners, leaving a rump of "local" elderly people who are now alone with no younger "locals" to support them. These people will make substantial demands on local services (especially social care and needing things like lots of buses) which will be all the greater if they do not have local extended family or strong local networks to help them. This is going to become a serious issue going forward if we are not careful.
I can already hear the whinging of elderly voters who have retired to the countryside for lifestyle reasons: "I can't drive any more! The council needs to run more buses! Why aren't they providing more buses?" Because there is no fucking money, Susan. Did you not think about this before retiring to a pretty village?
This kind of thing is why we should be cautious about laissez-faire attitudes towards relocation and demographics - "Everyone has the right to live where they want to! Nobody owns a village in the Lake District!" Yes, but, do we want people living full-time in the Lake District villages at all? If so, it's in everyone's interest to ensure that there are communities with people of many different ages (including the young) and strong social/family networks, to ensure that people have the community spirit and physical ability to help each other out and manage the challenges of rural life together. Otherwise, we are going to paying eye-popping costs for this in the near future.