I think it's the colour that is bringing any dressing gown vibes, so you need to contrast that. Black. Which is usually the answer in my book anyway, but black, no scarf, or anything else that will fight with the hood, the hood is excellent, lean into it, don't chop it off.
Other than that, wide leg trousers and trainers, heeled boots or shoes, joggers and a cashmere sweater and trainers and hair up, baggy jeans and a slim fitting polo neck, whatever, but you will need to anchor it with some black and something else to signal that you're not out because there was a fire alarm in the hotel, like a good solid pair of earrings.