"As you also probably know, she would have been the first person to insist that I still come to be with you all instead of going to her."
He is referring to the Queen.
Is he saying this is why he is not at Balmoral with the rest of the RF? Is this what he means by "going to her"?