I layer it up rather than getting one of those expensive running jackets. In the snow I was wearing running tights, a running t-shirt, a hi-vis long sleeved top and a buff (or a hat on the really cold days). All in wicking fabric which will help keep you warm.
Sports Direct is very cheap, I've got most of my kit from there.
I wear just below the knee running pants - almost all year. I do have a pair of winter running tights for when it really does feel too cold to expose flesh. I wear running tees (once you start entering races you get one of these in practically every race: handy) and when it is cold I wear a thin long sleeved running top under (don't know where I got mine but I think Primark do them cheaply enough). I have a running jacket but it's more wind proof than water proof. I think you have to accept a bit of wetness! Aldi and Lidl do great running stuff & I got DD luminous long running socks recently for winter time in Aldi (to highlight her and keep her warm). They also do luminous running gloves in Tesco for less than a £5. Once you're under way you won't notice the cold but I have a peaked cap for rain - once it's not blowing into your face you'll be fine. Anything so long as it's not cotton in the rain (Gets too heavy/stays too wet) Oh and I have a Buff - a sort of tubular scarf which you can pull up over your mouth & nose in icy conditions or turn into a hat or just as a scarf - really handy and versatile. AND Buff's can also be luminous!! (so I guess I'm pointing out the need to be visible in poor weather at all times of the day.