Sparkle I've got experience in this as a child {I was incontinent from urinary reflux} and my own daughter currently.
She is 3 and has been potty trained for a year and often regresses, at times constantly wetting even though she knows perfectly well, physically speaking, how to go to the loo.
I'd strongly recommend against punishing his behaviour. As you've noted it won't actually change his behaviour and instead the negative responses could easily turn the issue into an ongoing battle of wills between you and create stress for him that actually perpetuates, rather than the helps the problem.
I'm afraid you can't 'make' him not do it, it's a phase, hopefully a short lived one, so patience is the answer.
Since in his case he has just started, it seems most likely he is not ready. IE even though he can physically do it, he still wants to be cleaned up / babied by you. You can consider putting him in nappies again and trying again later.
What will help is positive reinforcement. Start the day with getting him excited about not wetting today. Every time he successfully goes to the loo, let him know what a great job he is doing. Make it so that the attention he is looking for he gets when he does keep himself dry, not when he wees elsewhere.
You can also try taking him every hour / whatever time period would suit him as far as not wetting himself in between, to help bladder train him, if it's a case of him not being fussed about going because he is busy with something else.
I know you've not had many responses here yet, but it IS really common, so don't be to upset with your son or yourself {altough I know it's incredibly frustrating!} What he is doing is quite 'normal' not 'naughty'. :)