Lots of good ideas here. I think in general my main takeaway from being an inexperienced numpty dealing with youngsters is that no-one is perfect and there will always, always be hiccups, but try as far as you can to set them up to succeed, so things like plan ahead and don't make life unnecessarily stressful for yourself (the last thing in from the field thing has been extensively covered), make use of help from more experienced horses and people where ever you can (so things like following a quieter/steadier horse in from the field or out on the roads) and don't be afraid to use aids like a monty roberts collar that will give you that bit of extra emergency control when you need it.
Groundwork is absolutely crucial especially for a child's pony, personally I'm not a huge fan of lunging as the main form of groundwork, it's useful in some circs. e.g. as a quick leg stretch if you don't have time to ride, to get the hoolies out of a very fresh one or to build muscle and topline when done correctly, but personally I wouldn't use this as a foundation for correct training with a youngster esp a small pony, unless you are very skillful which I certainly am not personally, by doing loads of lunging all you really do is teach them to spin round and round you in tight circles in an enclosed area which is bad for young joints, gets them much fitter than you need and doesn't do much to prepare them mentally or physically for their eventual job. Do you know how to long line OP? If not I'd get someone to teach you and then I'd get him out and about on long lines in straight lines, just in walk is fine, you can start in the arena but then ideally go round and round the yard, up and down local lanes and bridleways and yes, back and forth to the field loads and loads of times until it's boring to him. You can have someone at his head to start with if you have a helper but eventually he should learn to take his cues and confidence from behind, unless he's to be a lead rein pony eventually of course. Practice lots of transitions back and forth from halt, turns, straight lines and curved lines with him, over ground poles and raised poles/cavaletti, despooking, obstacles etc wherever you can as well, always thinking forward and off the aids, keep him busy and always paying attention to you rather than daydreaming and getting into mischief and it will really stand him well. But like I say, set him up for success, don't expect it will all be perfect overnight and if you sometimes need to go back a step (like leading everywhere in a bridle not a headcollar until his manners are drilled into him) to eventually go forward again, your aim is the long term here so providing the overall trajectory is positive don't doubt yourself.
Experienced help is the other crucial thing, with my very first youngster I felt so silly paying someone to come and show me how to successfully get her to stand tied up on the yard or walk past the spooky bush without turning herself in knots, I felt I really should be able to manage such basic things myself but my instructor was so skillful and sorted so many things/taught me techniques/ways to think about training I'd never had to think about with my previous, older horses it ended up being the most worthwhile thing I've ever paid for...
Good luck!