Ah.
That's the top of an electric immersion heater, it is used when the boiler is out of order. It does not affect the boiler. Sadly it will not affect gas usage. The immersion heater has its own thermostat, which shoiuld usually be set at about 55C. Do not insulate over the cap.
It sounds like you have an older heating system, probably with a cast-iron boiler, often an old Potterton, and gravity circulation to the cylinder. The circulation between boiler and cylinder will not be pumped, and will not have an electric valve to stop it when the cylinder comes up to temperature. You might find that the hot water from the taps gets uncomfortably hot during winter when the boiler is on for long periods to heat the radiators.
If you look at the big pipes that go into the or front side of the cylinder, one near the bottom, and one above it, about half-way up, there is a faint chance that you will see a big brass thermostatic valve with a plastic cap, often blue, on it where the lower one joins the cylinder. However these valves are often jammed, and cost over £100 to buy, so not much hope here either.
You will save a bit of energy by turning down the boiler thermostat so that it does not overheat the cylinder, this will also make the rads less hot so you may need to turn it up again in cold weather. Otherwise, insulate the cylinder with as many jackets as you can get round it, and the hot pipes, especially the big ones between the cylinder and the boiler.