He sounds super touch sensitive - and I think you have hit the nail on the head when you put it down to the very early weeks of his life and a poor breeder.
We have a Labrador who was exactly the same with regard to touch and mouthing and over excitement for exactly the same reasons.
Growling in a pup that young is nothing at all to worry about - it is a display of their general discomfort, so a warning that they don't know how to cope with what's going on rather than anything agressive. He's not going for you in the sense of an agressive dog - rather in the way of an over stimulated (by touch) and confused puppy.
Is your dog also very keen on food?
Confrontation in this situation isn't the way forward - jolly your dog out of it instead. Lots of silly, calm, singsong voice while you're drying him and avoid common touch sensitive areas for the time being (top of head and back). I also used a very small piece of towel so she didn't feel overwhelmed by that too. Little, matter of fact and often worked with toweling her down - and lots of a very cheerful, singsong, soothing 'who's a silly girl, biting and growling me? it's never going to work, biting and growling you have muddy legs and they need to be clean sort' of nonsense. The words are irrelevant it's all in the tone of voice. Try not to over compensate either - that will also freak him out!
It is very hard to work through issues caused by a poor start; I am a pretty experienced dog owner and have found our poor start girl a real challenge.
She needed to be taught in a thoroughly positive way what to do and how to behave. She doesn't have the equipment to work out for herself how to react to a new situation. Our girl also struggles with a very low thyroid function which can have a negative affect on dog behaviour.
We also found that rewarding behaviour with treats for her wasn't the way forward. Food was far too big a deal for that. Gentle strokes on the chest and calm praise get the most out of her. She so BADLY wants to be good!