There's a huge amount of misinformation about IHT here that the OP would do well to ignore.
From the information given, it's not possible to ascertain whether the gift is liable to IHT. You'd need to know exactly what other gifts the grandfather made, if any, and when these were made; the full value of the estate, whether he was widowed and has any transferable nil rate band available from his late wife etc etc etc.
It might be that he hadn't made any previous gifts and could use 2 x £3,000 annual gift exemptions (this can be carried forward for one year if unused) so £6,000 of the gift would have been tax free and the remaining £4,000 would be added back into the estate. If his residual estate was valued at less than £325,000 then there's no IHT to pay.
But like I said, it's impossible to work it out based on the information given.
Assuming the gift was a gift and not a loan, you're not obliged to pay it back and even if for some reason you wanted to, it couldn't just be distributed by the executors as part of the estate because it no longer belonged to your grandfather anyway.
This
Not enough info to say if IHT is due from yourselves or just from the estate, very much depends on other gifts, when they were made, available NRB and so on. If no other gifts were made in the past 7 years and the gift to you was below the GF's available NRB then you and your husband will have no personal liability to tax. If this hasn't been clarified yet by the executors then it needs doing. As for the rest of it, the will should be followed as per the GF's wishes and not those of the beneficiaries.