The boring things make an enormous difference immediately - so up your loft insulation (check out how to do this correctly for an old house), identify and block drafts, put curtains over the doors, thick insulated curtains on the windows, secondary glazing at the windows, etc etc etc.
I'd agree with Frogetyfrog that ground source heat pump would be well worth looking at - but you need to have underfloor heating for it to work best, and in an old house, you may not be able to lift the floors to lay underfloor heating. If you've greened your house in advance, you'll get a grant for installing the heat pump, and will apparently get some money year on year in the next budget.
Our neighbours have a wood-pellet powered boiler, but I think you need a big store right next to your house (to store the wood pellets in) for this to work, which may not be appropriate for an old house. (We also looked at this, but don't have the room for the wood pellet store)
We have the original floors, so have oil heating and radiators. It doesn't cost a fortune to run, but we have a Honeywell heat management system that puts every room on an individual heat timer, so we only use the oil that we need to. This cost about £1000 upfront, but pays for itself quickly.