Diet - a couple of thoughts.
First of all, I wouldn't worry about the work. You can get black weedsheeting on ebay for cheap and cover the entire site, weighing it down with bricks. You can then deal with small sections as and when you feel ready without feeling like the rest of it is becoming a jungle that is getting more and more difficult to deal with. This is what I did. I also think that some fighting with weeds is par for the course when you start. You CAN do this and what is more I think you will enjoy it! Why not give it a go? (Also - I think I spy some raspberries in the corner there?)
However, I would worry about the isolation. I think this is a matter of how it 'feels' - all of those instinctive things about a place that make you feel at home or in danger. Your site sounds a bit different. And having someone pull a knife like that is threatening, and I think sometimes the fact that it's a child makes it even more worrying, because they don't have the same understanding or empathy as adults do (honestly, his Mum wants to give him a good talking to and ground him for about 6 months). So I'm not surprised you are bearing this in mind.
I would be fascinated to hear how other women feel about this. I'm very interested in how we, as women, experience public space differently from men, partly as a result of how space is configured, and partly because of the way we learn about 'threat'. My allotment is fairly isolated. It is up a country road, next to a very large golf course on one side and the grounds of a stately home (since converted into flats) on the other. It's a long way from houses, and it's on a dogwalking trail, which means that the gender balance in the surrounding few hundred metres is about 80% male, 20% female. I am sometimes there on my own. If you'd asked me before this moment of reflection, I would have said that I definitely felt safe. However, when I think about it, I do have an eye to who is around me much more when I am there alone, and I keep my phone handy too. I rarely even take it when DH is there with me.
To come back to Diet - one solution to both issues might be to get a friend to help you with the allotment. Several of the plots at my site are run by two couples, not one and the women seem to come together as a kind of social thing. That would give you companionship, a bit of space to yourself and a bit of security.