Yup, I think your summary of potential actions sounds about right. The bath/shower thing is tricky - although an immersion heater may be more efficient in heating the water, the bath will require more water than the shower, so it's a trade off.
I'm guessing you have no gas central heating? So while your 'leccy bill is high, it represents most of your utility bill? Problem is that 'leccy is a very expensive (and inefficient) way to heat things. If you did have gas I'd recommend waiting for a year or two then getting a combined heat and power boiler, which would probably be perfect for your needs.
A GSHP can deal with all your space heating needs and can probably be used to pre-heat water, so you'll require less energy to get all your hot water needs. But do remember you will still need electricity to run the pump. You can probably work on a heating co-efficient of at least 1:4, ie for every unit of electricity into the system, you get four units of heat out. So a key question when considering its effectiveness is whether the heating/hot water is such a large portion of your energy use that the pump's energy requirements will be less than the energy displaced by not having to heat things electrically, IYSWIM. While it's more reliable than solar thermal, the trade off is that it is much more expensive, because of the ground works needed.
I'm sure you can get ground mounted solar thermal panels - have a look here in fact you can probably rig up a system yourself with some black painted radiators, but it won't be as pretty or efficient!
Ranty rant...I do quite enjoy this stuff, though . I'm a bit of an energy nerd, basically - my company does research and development of new products for the cleantech market, so I'm constantly spending my time chatting to engineers about the numbers associated with energy conservation and generation.