I would start with something the size of a half plot allotment - about 120 square metres. This will give you a good amount of work to begin with, should produce a decent crop in the first year, but should be really manageable since you have a lot on your plate with 3 little ones already!
Work out your structure to start with. Are you doing raised beds or more old-fashioned rows? Where will your paths go? Bear in mind that you need to get between plants to weed and crop them, so huge, deep beds with no space to move can quickly become a PITA if you are doing this by hand.
What do you want to grow? Bear in mind that rotating your crops really helps the soil, which also helps the yield - so a mix of heavy feeders, legumes, roots and brassicas is a good idea.
Preparing the ground: since you are up against it in terms of available energy and time, I would do a combination of dig and no-dig. Cover half the ground you're not going to use immediately with cardboard and dump loads of horse manure on top. This will kill the existing weeds and suppress new ones and will rot down over the next few months while you work on the other half. A lot depends on the kind of soil you have here. If it's heavy clay, I would turn it to the depth of a spade, dump compost and manure on top and let the weather help you break it down. You can then fork it over/use a cultivator after a few months before planting things out in March-May. By that time, hopefully the weeds will be dead under the other half, and you can just till the soil and plant into that as well!
Most veg like sun and lots of it, but you can use a slightly more shaded, part-sunny part of your site to plant fruit. The advantage with getting this in early is that many fruit trees and bushes take a couple of years to get going.