There could be all sorts of reasons people haven't planted the gardens.
They may not know anything about gardening - not everyone is brought up being taught what to do in the garden, what plants are suitable for specific conditions, how to look after them.
They may not have the money to spend - seeds don't cost much, but for most you can't just scatter them on the earth and hope for the best. It takes some preparation, and looking after - preparation requires tools, which cost money.
People may be using the space as a children's play area and either not have space for plants too, or think that plants would get damaged.
People may not have time - they may have other commitments - work, family to look after, caring for an ill relative etc.
People may not be physically able to do the work - you can't tell by looking at someone what conditions they have.
I enjoy gardening and have some level of knowledge (no way an expert). I have previously taken a lot more care of the garden then I do now. Although I may look fine, and have spent time doing some jobs in the garden , over the last few years, circumstances have stopped me doing what i would like. I have been unwell myself, so whilst i may want to do the garden finding a time when I have the energy, it is not too wet, and I am not working/elsewhere has meant it hasn't always happened.
Even when I have spent time in the garden, there has been a limit to what I can do - either due to time, energy or physical ability due to an injury. Add in helping to care for other family members, means it is even less likely to happen. I do what I can, but need to be realistic in what i can achieve. Have tried getting someone to do some bigger jobs in the garden for me, and am looking to have some of it changed to make it easier to maintain, but finding someone that will even come and quote to do the job isn't as easy as it sounds.
Whilst I agree to a point with what you have said, you may not be taking the reality that some take into account. Also you only said you have tried potato plants, but not what else you have planted, so it is unclear what effort you have put in.