Heat pumps basically create "free" energy (in fact they extract energy from the input air). The amount of free energy they create (called the COP or coefficient of performance) can be as high as about 4x the energy that is put in. But can be as low as 2x.
Unfortunately the amount of free energy is also dependent on the output and input temperature of the pump. The lower the input temperature compared with the output, the less free energy is created. What this means is that in winter the COP is much lower than it is in summer. This leads to a new measure of performance, SCOP (seasonal coefficient of performance) which takes into account the seasonal variation in temperature.
If you are competing with mains gas, the key is to get your SCOP better than the price difference between electricity and gas. This corresponds roughly to an SCOP of about 3 (electricity is about 3.5x the price of gas, but gas is only about 90% efficient in heating in a new gas boiler).
Getting the SCOP to this level is not impossible, but you probably won't get a huge amount more than this. A house in a relatively mild climate on the south coast might do pretty well but in the centre of Scotland much less so.
Heat pumps require bigger radiators (because they heat the water to lower temperatures so effectively you run the radiators at lower temperature for longer to put the same amount of energy into the house). This means retro fitting to an old house can be more difficult. Sometimes it is just not practical to get the amount of required radiator in a room and you need to move to wet underfloor heating instead.
They require better insulation because a heat pump generally will deliver much less power than a gas boiler, generally the energy input is made the same as a gas boiler by running them at lower power for longer than a gas boiler. Insulation is used to prevent energy leak, but also to ensure that the temperature of the house can be changed in a reasonable amount of time.
So my advantages/disadvantages against mains gas would be :
i) Probably similar running costs if best SCOP is achieved. At best running costs won't be a huge amount cheaper.
ii) More expensive to install - especially if radiator retrofit and insulation retrofit is required. Heat pump itself probably similar to boiler in terms of unit (with grant). Grants may neutralise some of the cost.
iii) Will become increasingly green as electricity supply decarbonises.
iv) Maintenance vs. gas boiler currently unknown. Seems more difficult to get service engineers currently.
v) Operating lifetime vs. gas boiler currently unknown.
vi) No flue emissions (gas boiler emits largely CO2 and water vapour).
vii) Generally noisier in terms of fans.
viii) Worst in terms of instantaneous heating (for example they do not do as well as gas if you switch the heating off for 3 days while you are away, then want to return and heat the house up quickly, thus costing you more money).
Taking into account all of the above, my feeling is if you are trying to retro fit a heat pump instead of mains gas to an existing property that has not been designed for heat pumps then it is almost certainly not worthwhile, or at least should be investigated very carefully evaluating the running costs and the installation costs vs. mains gas prices atm. Of course there is nothing to say that in the future the government may change the electricity:gas pricing ratio.
I think in all cases, if you are thinking of getting a heat pump it is worthwhile commissioning an independent consultant (make sure they are independent and not selling for one of the heat pump companies) to figure out whether you would be better spending your money on a pump or on something like solar instead.