Okay, but it is really the comparison that is the issue.
House A - detached, on a hill, stunning views. Small, cottage style and is subject to very tight planning regs so no real scope to develop/extend.
House B - semi detached, down the hill from House A, same views, but not quite as stunning (due to fact down the hill a bit). Bigger house (room size), very usable space. Potential to extend (not hugely, but enough).
We have an (unaccepted) offer in on House A, but are in weird position that owners are no longer actively advertising it, but still have our offer on the table - they haven't actually withdrawn it from sale (as dp said if they had we probably would not be in this dilemma).
I am trying to think about how I would feel if House A didn't exist!