Should I buy a house that needs an extension? Panicking

[Polkadotdash]

Advice needed please.
We’ve had an offer accepted on a house that only works for us if we extend it with a two storey addition at the back of the house. It’s a terrace with a small garden, which will be made 3m smaller by the extension.
We went on a visit with an architect yesterday and they seemed unimpressed with what we were thinking and thought it was unlikely we’d get permission for a two storey extension, despite three other houses having done it in the street. This was because the neighbouring houses are so close to the one we are buying. We also will have no rear access to the garden so the building work will all need to be done through the front of the house. The architect’s eyebrows shot up when we said we were planning to live there whilst the work was done.
I’m now freaking out and have got serious cold feet about going ahead.
My husband won’t talk to me and says it’s up to me to call the purchase off.
We’re currently renting (at extortioniate cost) having sold last year. We’ve been looking at houses for two years. I’m fed up of looking. This house seems to be the best we can afford in a very expensive area.
Will it matter if the garden is tiny? Maybe 4m by 4m. It’s currently horrible, like a paved prison yard with mesh fencing at the top of the wood fencing.
What should I do?

A couple of thoughts OP - we have been through all this.

We pulled out of a house we were v v keen on after discovering we were unlikely to get planning permission for the extension we wanted to do. In that case I had made an appt with the council and sat opposite a planning officer. He said he couldn't say for sure because it would have to go through due process etc etc but in his view it was very unlikely. Could you seek an urgent appointment? I went along armed not with architect drawing but with print outs of what neighbours had done. And yes some had done two storey extensions - which was my ammunition - and all he said was that the council were reluctant to repeat mistakes of the past. We told the EA and they were not at all put off. They are only interested in a sale and knowing we were properly keen to buy a property simply switched to bombarding us with other stuff. Could you talk to the council?

The vendor's disappointment is nothing to do with you and should not cloud your judgment IMHO. Buyers remorse is corrosive and will hang over your lives potentially for decades.

Next thing - we bought a small house in a nicer area and the builders have been in for eight months. We moved out. There is rear access. And it's still been quite nightmarish. Big pressure, stressful decisions to make, unexpected costs because there always are, silly family disagreements about paint colours, do we need new windows etc etc. In the nicest way possible you so both need to be 100% sure you're doing the right thing at the outset or I worry that resentment would fester and the relationship will come under even more pressure during the months of chaos, cost and uncertainty.

Plus t took a long time actually to find builders so you need to work out whether you're happy to exist in the house for up to a year before a project could actually start. So many people are extending rather than moving that there is pressure everywhere - on planning decisions, quantity surveyors, builders etc. We are west London and really struggled to get good builders to tender. Many had two or three jobs already lined up and they can afford to pick and choose. Good trades from e Europe are thinking about going back there. Contractors need to keep them on side. They don't want a job where the owners are on site needing a working loo every day and micro-managing etc etc. They won't only jack the cost up - they won't bid for the work at all.

Upshot - there is always another house. See if the council will give you and DH some advice.