As others have said, costs of repairing/restoring would be astronomical, for very little space. I can't imagine that it would meet building regs - what about fire escapes etc!!!
It's a listed buulding, so you'd be limited as to what you could do.
Also, no guarantee that planning permission would be given again.
It's been bought and sold and put on the market again recently.
@Piglet John - it was built for a rich nobleman as an observatory and as a "castellated romantic feature" - presumably that means a folly. It's on the buildings at risk register: www.buildingsatrisk.org.uk/details/898956
That report was written in 2016.