OP, when we lived in Oakham, I used the trains every week in the first two three years, when we had only one car, though DH could walk to work initially. Buses in small towns stop running at 5
and the route which is 10 min by train takes forever by bus(50 min). Though there is only one train company running on this particular line, so being monopolists there are no deals, they can charge extortionate prices. Still after living there I wouldn't consider living anywhere with no train station.
Oakham is quiet-ish, but I know lots of people who chose it over Stamford, also compared to Deepings there is more stuff to do, and there is Rutland Water cycling distance away.
Also Uppingham in Rutland is charming, I actually liked it better than
Oakham, but it didn't have a train station, only one expensive supermarket(Co-op) and it IS tiny. Schools are really good though.
Visit both Stamford and Oakham and see for yourself, it's less than 15 min drive between them.
As for Stamford. Funnily enough, I had an acquaintance, only for a year, her DC was in the same reception class as my DC. We were invited to her DC's birthday party, they were nearly the only ones who organised the party and its theme themselves (parents and friends+family), all the other parties we went to had a brought in entertainer. Anyway, I thought it was really fun and original, and the kids of course joined in, but parents were all standing around a bit po-faced. I could tell straight away her and DH were not locals 
Turned out later they were renting in Stamford and her DH was commuting to Stevenage, and by the end of the academic year they moved away, to Hitchin! She was SOO excited to leave! She said Hitchin was more hip and them, and Stamford was nice, but a rich people's bubble. (which does sound a bit strange to me now that I've heard of Hitchin's property prices on this thread). I had never heard of it until she mentioned it.
We are certainly not rich, but also we are not born and bred locally.
Incidentally, myredcardigan, you must had lived here before we moved here, we did in 2010, and by then it seemed that every fourth person in Stamford was not Stamfordian/local originally! Maybe it used to be insulated, but it had such a powerful influx of newcomers(esp. people from London and Cambridge), and I've witnessed it a lot myself in the last five years, so maybe its feel has somewhat changed? I don't know, it's difficult for me to judge, I grew up in a very busy European capital(I'm not English, but DH is), and my only experience of living here was Bath(amazing, but couldn't afford to stay), Oakham and Stamford. I was so crushed by small town living in Oakham and had to change culturally and all my habits and what I enjoy etc, that even a move to Stamford felt like a massive upgrade - so busy! so cosmopolitan ;) (relatively! but Oakham was 99% white British) has an Art Centre! cinema! DIY stores
several supermarkets
There is even a direct train to London in the morning, could be there in under an hour. Etc.