First off - many thanks for the responses, varied and helped open my eyes! Some to what I am or maybe doing wrong, some to making me realise I need to explain bits further to get an informed response.
Let me start with the existing garden - the side area I am mentioning - I am not going to be the one to "sort it" nor is DW - this area needs machines, bricklayers etc just to get to a state of - now what. This is beyond my capabilities or more probably willingness, I know that it will be a huge effort that even trying my best is likely to fall short of even minimum standards. It involves wall building, drainage, hardcoring - I know nothing in these areas. However I have weedkillered the area, that worked then they grew back. I think some comments are right - we should both knuckle down and sort the weeds.
HorryIsUpduffed - you have a point - I used to have a rep of starting things and not seeing them through. This was very common when I was growing up. This is now very uncommon. Beyond the yearly "I ought to exercise more, I ought to get fitter, I ought to diet" I now rarely give up on ideals (in fact now supping on a nice pint of home brew that I was previously told I would give up the idea as a fad). So it is a fair point, I am fighting against a years old me compared to a recent history me there!
I have now viewed the plot itself. I am not an expert. But it does seem like as good as it gets. If I were to decide to do nothing but maintain, I need to cut the hedges (mostly hawthorn) down to 5ft (from about 8 - 10ft). That is the only law laid down. The plot has a shed that includes shelving, planting area, and further storage, a separate greenhouse, water, 5 water butts, hose pipe laid throughout, an eating apple tree, a plum tree, a pear tree, more blackberry bushes than 10 people could shake a stick at, nothing other than long grass and a bit of said blackberry bush incursion that needs addressing, is 420 sq yards - it is a dream. It is also blockaded in all sides (so my DS - 2.5 YO) can't escape - and he loved it on the 30 minute viewing.
Now - I am pretty certain I want to go for this, It is close to home, big (but the smallest plot on this allotment), has 3 established fruit trees, I believe our son will love the area.
However - I agree, I think I need to come to some compromises/ground rules with DW.
So I appreciate others may not have had the whole family thing work out. I don't think mine is actually going to be anything like my dream! However, I think I can take my DS there some of the time - probably looking at doing lighter work and involving him, in exchange for DW accepting that I will also need time there either without him or with her in attendance to then get on with the more involved work.
Further thoughts appreciated.