It does if you do enough of it!
I used to work as a riding instructor at a busy stables- so I was on the go at least 5 days a week from 8.30 til 5. I wasn't doing much cardio, but I was constantly standing/walking/lifting/riding- I lost about a stone in the first year, and about half a stone over the second summer, mostly through exercise as my diet was crap, although admittedly I wasn't eating huge amounts due to being so busy. I also had a great core that made me look naturally slimmer.
After I left that job, I did try to keep my fitness up, but when you are in a sedentary job for most of the day, it's just impossible to exercise to the same level. If I went for a run, that might burn 300-500 kcal depending on distance, speed and hills, but that would be all I'd do that day, and I'd often eat more during the day due to boredom/having time to notice I was hungry! I also had rest days from running, where I was often completely sedentary, which didn't help! I didn't gain much weight in this time (maybe 5lbs) so I think the running did help me keep the weight off.
The change in exercise did change my shape, and I had proper hamstring muscles for the first time in my life.
Now I'm doing a PGCE, which is pretty intense and I haven't found that much time for exercise. I eat a lot of convenience food, so my diet is poor too. If I started exercising regularly, it would make a positive difference to my body- but I think I'd have to do at least an hour a day to make any difference to my weight- and I just don't have time. It would be much more time effective to cut my calorie intake by say 500kcal a day!
However, if I went back to being physically active all the time, I definitely would lose a lot of weight again. But unfortunately, 30 minutes in the gym 3-4 times a week (which is what a lot of people talk about when they say exercise) won't make a hugely noticeable difference.
Also, replacing fat with muscle can mean you don't see results on the scales straight away, even though your body looks better and long term it will allow you to eat more calories without gaining.