Just after people's thoughts as DH and I discuss this issue endlessly and go round in circles and we've no one else to chat to at the moment!
We currently live in a really nice house in a nice area (we appreciate we are very lucky!)
Good points are:
close to station for commute,
close to good school (like 5 minute walk)
really nice outlook over a park (other than in the summer when people like to hang out there at all hours and occasionally cause trouble)
About enough space for us at the moment
Features and decor all nice (high ceilings etc) although with a house of this age it feels like there's a never-ending to do list of maintenance and decorating.
Downsides are
Semi-detached and we would really love a detached house
Smallish garden. As it is a semi it has 'garden' on 3 sides but it's mostly quite awkward and has only a small flat bit of grass that we had to get a landscaper to put in a great cost! Difficult to see how we could we could improve it
Garage/parking isn't great as it was a 1970s add on. Could possibly be changed for lots of money.
Not very private - being semi detached means our garden is overlooked and the park at the front means there are members of the public at the front all day long.
House is over 3 floors - I've found it a nightmare with babies and toddlers but it is easing slightly as they are getting a bit older (now 2 and 4).
I think our ideal home would be detached, larger private gardens and driveway (isn't everyone's?). They are like hen's teeth in this area so it's likely we'd have to take a hit on the location and be further from station/school. We'd love a bit more space to have a playroom which could maybe become a music room or gym as the children get older. That sort of thing.
It would also probably cost us minimum 70k in stamp duty and fees as the budget for a house like that is going to be upwards of £750,000. Could afford up to about £1m but would rather not go to that. The last few houses that looked like the sort of thing we're after sold for £975k and £1.2m so it may be unrealistic anyway 
My question is therefore do we forget about moving and use the money for stamp duty on improving the house? A loft conversion is appealing as we have an amazing loft but we'd be over four floors. A garden room could make our bottom level a bit more useful and 'wow' but we'd be eating into the little garden space we have.
I also find it hard to shake off the idea that we are lucky to live where we do and should just thank our lucky stars (which we do!) that we have a large enough house with a garden...
I am just musing things over for chat really. Move or improve?!