I would do it.
If you stay where you are you will still have a big mortgage (and work could dry up there....!). If you can radically cut your outgoings, you can live on much lower wages.
But, ME saying that is one thing. I know I could, because DH and I are both very capable and tough, and don't mind discomfort. Are you BOTH completely on the same page? Are you prepared to put the work in, and maybe spend years in an unfinished house - or at least one that's not finished exactly as you want?
With regards to the house, practicalities first;
Can you even get the mortgage?
What exactly does the house NEED to be safe and livable? We bought a house that was v.similar to your picture, and had it checked before purchase;
Structure & chimneys = fine
Roof = fine
Wiring = old but safe.
Plumbing = old, (awful bathroom) but fully functioning
Boiler = last legs, so we changed it
Radiators = Pipes on outside of walls, but working and OK.
This left us with a shabby old house (1923) that was a mess, but liveable, (if you have quite low standards!) We did the work gradually, using money to get in the trades that we just couldn't do. We learnt a lot from Youtube videos/mags/books, and bought very carefully. You could get a long way with £50K, obviously prioritising rewire/heating if needed etc.
We bought loads of stuff from Ebay. Here are some of our best bargains:
Carpet.
Excellent quality. Light grey. Enough to do the stairs, landing and all the bedrooms. Only been down for 2 weeks in a new build. £80
Large DG Window & skylight
Bought for extension. New. They were mis-measures, and being sold by builder. £50
Bi-fold Doors.
Another builders's mistake. New. £400
Engineered Oak Flooring
From another new build. Enough to do our large living room and hall. £70
Kitchen
Top quality Solid wood. 2 years old. Including sink and hob £400
We had to fetch most of this stuff ourselves, but it was so worth it. Check out used kitchens on Ebay now, and you will see what bargains there are. You can also find ex-demo bathrooms and kitchens online, and companies that specialise in selling on used, high-end kitchens.
It could be done, but it depends on you both getting stuck in, and being realistic about what you can do. Yes, you can spend loads, but you can save too. Some of the prices quoted on here are for SE builders, doing everything. We did a ground floor extension. It's one large room. It has a pitched roof with a skylight, bifold doors going out, a large window, and double doors leading back into the house. Oak flooring, and wire gallery style lights. It's very nice, even if I do say it myself! It cost around £18K. Our neighbour got builders in to do something similar. Theirs was £37K. Ours was cheap because we dug the footings (which is £££) ourselves, and a builder friend and his son did the rest. We had it screeded, wired and plastered, but fitted skirting, painted and put down the floor ourselves and bought all the windows/doors as above