A few things which stand out to me are:
The kitchen cupboards - the top cupboards don't look as if they match the bottom ones.
The living space is too small for the size of the house - the living room/dining area/kitchen looks cramped. Could you put the dining table in the garden room to free up some space in the living room?
We have a townhouse (and love it) but as a PP mentioned they are not suitable or practical for all families.
I also agree with another PP who commented on the unusual layout of your townhouse. Our home has the kitchen, dining room and utility room on the bottom floor; living room, master bedroom and bathroom on the middle floor and then 3 bedrooms and a family bathroom on the top floor. Our children are teenagers so they have the entire top floor to themselves (and I don't have to look at all their clutter on a regular basis!) but I would not have fancied the idea of this house if they were babies/toddlers - all that climbing up and down stairs in the middle of the night would have driven me to distraction.
Your home is decorated tastefully so the issue is not that people are worried about having to spend a fortune to correct bad decor. Generally, the problem is usually the price - a house (regardless of what an estate agent tells you) is only truly worth what someone else is willing to pay for it.