I watched a Ben Fogle episode where a couple moved to Wales and did this. They paid 35k for the land and moved several horse transportation vehicles onto the land and turned them into homes.
One of the conditions of them staying there is that they have to be 65% self sustaining within a five year period, otherwise they have to take all accommodation down and are no longer allowed to live there.
https://www.channel5.com/show/ben-fogle-new-lives-in-the-wild/season-13/wales
I just googled them and it turns out their neighbours, who they never got along with, have bought the land with the lane that allows them to access their property from and are now accusing them of trespassing.
https://www.mirror.co.uk/news/uk-news/family-fear-being-blocked-home-29050206
Obviously, this is Wales and not England, but the principle remains: you buy a plot of land and you always have to deal with other landholders and local councils constantly changing goalposts.
After living in the countryside for several years, I have to say, they're a 'special' kind of folk. In that, I mean, they want the land for what they want and anyone coming in from outside who has different ideas, plans or ways of living is seen as some sort of alien entity. It's a sort of 'it's always the way things have been done and of course things shouldn't change' mentality.
Apologies for any countryside dwelling folk who are of a different mindset! Perhaps it's just my particular part of the country (deep farming country).
Saving up for a pre-existing smallholding feels like a safer plan, imo. And of course you have to live there if you want to keep animals.