OP, I think that your neighbour is a busy body with too much time on their hands, I'd swiftly tell them to wind their neck in and leave.
As for the cold, if you were out walking with your baby fever an hour or more the baby would be in the same conditions as laying in a parked pram in your garden, no difference so as long as baby is wrapped up and you've said they are, I don't see a problem.
As for the fox lunatics worriers, I can't remember the last time I saw a fox in daylight, let alone with the balls to walk through streets and pounce on prams, I wouldn't worry about that.
Cats jumping on prams? Probably not unheard of but I've not heard of or witnesses anything like it so it wouldn't stop me.
Being distracted by other children is a possibility, but you could say the same for other situations in the home, OP used her judgement.
My biggest worry would be baby being snatched, but I think I am particularly neurotic about this.
I am not surprised a social worker has said that they'd investigate, in my experience they do more harm than good more than half of the time and are quite irrational, that's just my experience.
I love how scandinavian countries have different parenting ways to us and think that they have good ideas/ways of going about things.