Spotty - ignore JoantheFennels remarks about the smell from the river. I am looking at it as I sit here typing. I have never, ever, heard anyone complain about it smelling. I hardly think people would stroll along the Embankment on pleasant evenings in such numbers, or sit outside the Embankment hotel or the rowing club balcony enjoying a quiet drink if there was an almighty pong in the air. I also have mild asthma and no way has it been worsened by any miasma from the river.
I prefer this end of town, i.e. the Castle Road area. The houses are mainly in single family occupancy whereas the Saints area tends to be a bit "flatty" - no offence to flat dwellers out there!
You don't mention your children's ages but the Castle Road area is served by Castle Lower, with an "Outstanding" from OFSTED. Not necessarily the be-all -and-end -all I know, but our grandchildren love it there and are doing very well. The three tier system is a bit of a pain, but one of the local middle schools, Goldingon, has also been given "outstanding", and the other one, Newnham, has a new head who is reputed to be very good.
Do you know about "The Paddock"? The houses at the east end of Waterloo Road, a chunk of the Embankment and the southerly parts of Rothsay and Bushmead enlose an area which is for their private use, invisible from the road. There used to be a tennis court there, maybe still is, it is a large pleasant area like a private park. Understandably houses with access to it tend to be a bit pricier.You can spot it on Google earth.
I do agree that the shopping is not brilliant, but at least from the Castle Road area it is a short and pleasant stroll into town; if you don't like Milton K you can go to Cambridge and Park-and-Ride or get a bus.
Our elder son is currently looking for a new house in SE london. Favourite is on sale for £600,000 with about £200,000 needing to be spent on it. We think he is insane since for that money, as you have seen, you can get a mansion in Bedford. Downside is a weekly season on the train = £100, nevertheless...
Might have to name change to FanofBedford