But, the government is determined to hold the games, so my opinion doesn't matter
It’s not the government, it’s the IOC.
If Japan cancels, the financial penalties to the IOC are crippling. Huge. They cannot cancel. Even in normal times it’s usual for a host city not to break even.
Only the IOC can cancel or postpone. What happens in Japan locally is of no consequence to them, neither is what happens to athletes. They are the ones who benefit financially, so it is not in their interest to cancel.
There was talk of the IOC getting athletes to sign a waiver that there was no comeback if they became severely ill, disabled or died from covid while at the olympics. Don’t know if that happened.