Easy and cheap, with the best payback because it cuts heat loss so much, is mineral wool loft insulation. I will only use the soft variety treated with Ecose, which is brown and des not shed irritant dust and fibres. It is actually made by Knauf, but widely sold as an own-brand. It will be badged "Ecose" on the wrapper.
If there is old yellow fibreglass in the house, I would seriously put on goggles, gloves, a mask and overalls, and strip it out into bin bags which I would lower carefully onto an old blanket under the loft hatch. I would use a builders canister vac to clean out the dust in the loft, and I would take off my overalls and PPE standing on the blanket and bag them up for laundering before going into the bathroom for a thorough shower or bath, including hair wash. It is that bad.
Next, full length insulated curtains on every window, and on Portiere Rods over the doors (this is a cunning design that lifts them when you open the door so they don't drag on the floor).
Block all disused fireplaces, leaving a brick-sized ventilation hole to prevent damp from internal condensation ( yes, it is a thing)
You might not be able to afford good curtains yet, but if you have just moved in, fit good curtain rails throughout, screwed to the wall in a way that I can explain in DIY, before you decorate. Look up standard sizes for ready-made curtains to decide the height. Even a horrid cheap curtain, or sheet of decorators polythene dustsheeting, will cut heat loss.
Do not put radiators under windows. Heat loss due to thermal gradient is enormous with old windows, and worse still if the radiator is under them.
If you have to use electric heaters, fan heaters give a quick burst, but are expensive to run. I recommend oil-filled radiators, which give a slow, even heat and can be low power, or turned down, to keep a bedroom tolerable and dry even on a cold night. Better still, they get no hotter than a teapot, and will not start a fire or scorch a jumper or newspaper you accidentally drop on them. Fan heaters, and especially exposed element electric fires, are very much more dangerous on this regard.
Do not buy miniature heaters that fit on a socket and are advertised on the 'net. They are not safety approved and are very dangerous.
Finally...
All electric heaters are equally efficient. They all turn 1kWh of electrical energy into 1kWh of heat energy. Do not be misled by adverts that try to make you think a magical expensive heater will give you more heat for less money. It is a physical impossibility, like trying to pour two pints of milk out of a one-pint jug.