We had exactly the same dilemma.
We are listed and were told no to wind and solar. Our attic is part boarded so no assistance for insulation. No other government assistance because due to listing we can't address priority issues around heat loss (doors, windows etc).
As I remember ground source heat pump works for underfloor heating, which we can't have.
No mains gas.
Options were wood pellet, lpg, oil or electric. Am not convinced of the eco credentials of wood pellet (know 2 houses with it who love it). Plus have had solid fuel before and was a nightmare.
I don't think electric has improved sufficiently to justify the cost.
We opted for most efficient oil we can. Buy oil in a cooperative, so minimum order 500l (roughly 45p per litre at present). Insulation everywhere we can plus thick curtains. Plus 2 woodburners.
5 bed detached house and use approx 1800l of oil per year.
Old houses, especially listed, are not environmentally friendly. Worth a chat with your listed building officer in case there are other options.