I understand the theory is that Starmer would not be upset if there were no debates, because he has little to gain from them given his current poll lead - a strong performance is unlikely to increase his lead.
However he understands they are likely to happen and is fine to do them.
It is not the case he is scared to have them.