I also live in Caversham. We moved from South West London in July. We love it here, based on all the things you said you liked - nice parks, villagey feel, groups, etc. I also have family here, and can honestly say I am happier here than in London. Reading town centre is easy enough to walk to (and it's a nice walk, in the daylight at least, although as someone said, a bit dark at night, but plenty of people do it) and it has pretty much all the major shops you need. The best thing is that I don't go there that often as it's nice to stroll into Caversham and there's enough there to keep DS entertained - little toy shop, river walk, nice cafes, even a small soft play centre in the gym.
Prices have shot up recently, because at the moment - though this could change - there aren't enough sellers. Consequently getting a house is even more stressful than usual. £250k for a 3 bed could be difficult in central Caversham, although as somebody mentioned, the areas around King's Road and Queen's Road are possible (but will be snapped up quickly; plus the third bedroom is often very small in these terraced houses). I'm afraid I can't really advise about Caversham Park, as I don't really know the area. Emmer Green is nice, though. We are trying to buy here for the same budget, except we're also considering 2 bedroom houses, and we've been looking several months. Buy-to-let landlords are a particular pain as they have snapped up several properties from our hands at the asking price or more, or otherwise we've been outbid by other depserate first-time buyers who happen to have the extra £5k to cover the ridiculous rise in stamp duty (I was weeping into my tea about that last week, but feeling slightly more upbeat now )
The 'best' schools are currently deemed to be Caversham Primary and Emmer Green Primary - I think they're rated outstanding, and The Hill Primary is also meant to be good. The catchment area for Caversham Primary is notoriously difficult to get into, and most of the houses there are larger and a lot more expensive, unless you go for a flat. The catchment school for most houses in central Caversham is Thameside, which apparently isn't amazing (rated 'average' by Ofsted I think) BUT these things change over time, and it's always worth looking at the school with an open mind.
Good luck - let me know if you move here, as I also have a 3 year old :)