Hi, we also have four kids, and live in village nr Totnes. I would say that for us (and everyone likes different things) it's a great place to bring up kids. I would definitely be really thorough in your research about location. Torbay is not at all similar to up this way, and then different villages have different vibes too.
If you like Totnes (we love it here), but need more for your budget, villages that have good feel and have families in contrast to retirement or holiday rentals include- Ashburton (really lovely now, lots of people live here 'instead' of Totnes, lots of little shops and nice cafes. lovely town houses with high ceilings and large gardens), South Brent (more Devon locals, as well as incomers, people rate the primary, lots going on in village, other side of A38, so more rain, but cheaper). Buckfastleigh is a bit marmite, but cheaper houses, and has really pulled itself up recently, so might get bargain. Used to not have great rep though.. might be the next Ashburton?
Rattery, Dartington (really sought after, great village, nr Dartington hall with festivals, cinema, arts, gardens etc)., Staverton, Harberton and Harbertonford all good bets... lots of families live here then socialise, shop, etc from Totnes.
As far as schools, private, private alternative, or local? All the village schools are CofE around here, with mixed reps. None bad, mainly quite small in villages, but a different feel to each. In central Totnes the Grove is the primary many feel is better academically, but Dartington is also popular (although in temp. accom. right now). Village schools too many to mention, ask if interested in any of them and I might know more!
The secondary system is Comps and Grammar schools around here. The Grammars being co-ed or also one each in Torbay for Boys or Girls. Mixed rep depending who you ask. KEVICCs which is in Totnes is a large comp, and gets a fairly good rep, especially for arts. In Ashburton there's South Dartmoor, which has a different feel, but not much between them really. Privately, ask if you want to know, and also a number of alternatives, i.e. Steiner just outside Totnes, or Park (primary) on the lovely Dartington estate, Sands (secondary) in Ashburton. The Dart free school possibly going to open next year in Newton Abbott? So lots of choice!
As far as where to live, we really stretched ourselves to get the location we wanted, and I'd say for us that was the right decision, Totnes has such a great feel, and so much going on, there is so much on offer for our kids too, I would rather the compromise on house than town....
Do let me know or Pm if you've got questions, or want to ask about a specific school or location and I'll do my best to help. (We rented for a year while we househunted, and although I hated it at the time, I think it did help us rule out some places and find out more about our options). We still feel moving here was the best decision we have ever made for our family...Good luck!