DS1 will fortunately wear hats, gloves and coats these days as he absolutely does not wear trousers. There is one pair of leg warmers that he will wear in colder conditions when he's out for prolonged periods. I've been sworn at in the street because DS1 was happily wearing shorts at 1oC, the irony being that the person who verbally abused me had an exposed skin head so no wonder he was feeling the cold. 
DS2 happily makes mainstream clothing choices. I was a lot warmer in my youth. I still work up a good glow if I'm active, but rest cold.
Interstingly when DS1 has been on a thermal imaging camera, he glows white hot compared to DS2. He genuinely, rarely feels the cold. The danger is that if he does get cold, it can be quite sudden and intense so other than his knees, he has access to warm layers. When we camp, he has finally grasped the message that it's better to stay warm than to warm back up.
I'm not irresponsible. I battled through the toddler years miserably unaware that DS has ASD and sensory issues. I eased up when he was around 3-4 and better able to articulate himself and a couple of years of battling every single time we left the bloody house in inclement weather had achieved nothing but stress. These days, I see my maternal duty as providing access to what he needs, not enforcing.
I've known children dropped off for nights out in poor weather in literally their Cub/ Brownie uniform. There's a massive difference in whether that child feels the cold or not. It is very much case by case. There's also a massive difference in how long you're exposed to the elements. 5 minutes is very different to hours. Babies too young to shuck their layers do need particular attention, but too hot can be worse than too cold.