I think he must take the lead here, he switched the switch & was cold for a week, he then dropped the bomb, then has had another week "to think" if there is anything he wants to retrieve.
You haven't pushed, but you haven't got anywhere else to go, so if he needs more time he must find his alternative space. (don't make it too easy)
He has avoided you, & seems not to consider your feelings at all. is he always so childish ? not really grown up behaviour.
Anyway, if he sits on the sofa, wordless & waits for you to talk it out of him, I think you need to say, "this is your game, play on".
remember that You are the one who has had the stress at fertility clinic, he said he wanted kids, he won't ever know what it was like for you & it is just a tad one sided to decide it's not what you wanted when in fact you said you did !
also only 6 days ago you said I questioned him about whether I did something, or if he is upset about something but he denied anything was wrong. I asked if he loves me, he said he did.
So he now has the occasion to spit it out. tell you what he wants, or doesn't want, whether he has just written of all those years overnight, & if there is a OW, or potential OW, or not. because lying now, whether it is all over, or he want to work it out, is pointless.