Things like the Koch-i-Noor diamond are tricky, because you can no longer identify who it originally belonged to, so there is no meaningful way it can be “given back” to anyone.
It was allegedly originally mined somewhere south-eastern India, but that could just be a legend. Equally, legend says it changed hands several times due to local conquests and wars. Things get a bit clearer when sources suggest it was seized by the first Mughal Emperor, who invaded what is now India from modern day Uzbekistan, and colonised the country.
An Iranian ruler started raiding the Mughal territories in the early 1700s and took the diamond by conquest then. The ruler’s son then had to give the diamond to an Afghanistan warlord in return for his military support. It stayed in Afghanistan for a bit, but was then again taken by force in the 1800s in return for assistance by the Sikh rulers (in modern day Pakistan) when the Afghan leaders became involved in wars with the East India Company. It stayed with the Sikhs for some time until they too got into a war with the East India Company, and at the conclusion of that war it was “given” to Queen Victoria as part of the peace treaty.
So, it’s been in the UK for the last 180 years or so, as they were merely the last in a long series of regimes to claim it as a result of aggression. But India, Iran, Pakistan, and Afghanistan now all claim it. Giving it back to Pakistan as the last documented owner before the UK wouldn’t work, as that territory also took it by force - as did the one before that, and, and, and…Good luck untangling that…