It depends how remote village-y you want to go. Also, depends on how well equipped your village needs to be. There's some very pretty places to the east of Swindon, but within easy reach for things like supermarkets and rail links, such as Wanborough, Bishopstone, Liddington and so on. However, they don't have much in the way of amenities barring a couple of good pubs.
Alternatively, if you want a village type atmosphere with a bit more to it, it could be worth you looking at Wroughton, or even Wootton Bassett.
Shrivenham is really nice but it's pricy. South Marston is a bit cheaper but again, not a great deal there beyond nice houses and pubs.
Personally, I'm not mad keen on most of the villages to the north of Swindon, like Cricklade and Purton (just personal preference), but Blunsdon and Highworth are quite nice and you'd be in catchment for a good state secondary then.
Swindon gets a lot of really bad press, and yes it's got its grotty bits, like anywhere else. However, it's actually got quite a lot going for it once you settle in and know the secret password! There's a growing number of decent places to eat, we get some decent theatre and ballet on a reasonably regular basis, there's a thriving Literature festival and open air theatre in the summer, lovely parks... we moved here 15 years ago and we've never left!