It's hard going isn't it. We have good and bad days and i always hit the parenting books. Here are some of the things that have helped me.
I do two mins on naughty step and he rarely he gets off these days - if he does i just quietly/calmly put him in a boring room, strapped in buggy or in car seat for 3 mins. if he is still kicking up a stink i leave him longer till the drama is over. won't let them out till peace reins even if it means we end up being late. we have been late many many times in the past!!
Dr christoper green of the fab book' toddler taming' says that kids often kick off as a last min attempt to over throw any new found parental authority. Quietly stand firm and leave him in the buggy till he has a better attitude. Afterwards breifly explain why he was on the step and move on/change subject. Both my two year olds have understood the naughty step and responded well to it. I know 2 seems young but they do understand more than we give them credit for.
i think proffessionals say to ignore the bad - withdraw attention and then reward the good. Can you think of small rewards en route to nursery. Half a choc button or a fruit flake at every lampost to those that sit in buggy nicely/walk well?
Negative attention (shouting) is better than no attention from a childs point of view. Keep calm and try to catch/praise him being good. Talk to him how you would like to be spoken to and give good quality attention. Try to hold the kids in good regard.
Distraction and sillyness also work at times. Can you turn walking into a game? Pretend sons are nice pet dogs and you are thier mummy dog. Throw in the odd bowl of invisible dog food or chocolate dressed as a doggie treat enroute.
Get a micro scooter for the three year olds. Costs a bit but it's fab and unique in design. Please read reviews. I got one and my 2 year old whizzes everywhere now at top speed. I can just about keep up with him.
Also consider a bike maybe?
They do get easier honestly.