This is how weak men behave to justify cheating and their appalling behaviour so they don't look like the guilty party in all this, when in reality he is just a low life cheat without an ounce of integrity. He is no longer the man you thought he was, he is not your friend and certainly not on your side.
He has gaslighted the hell out of you and no doubt when agreeing to take him back he expected to resume sexual relations with you. By denying your reality (his cake eating) he has put your mental and physical health at risk ( please get an STD check).....you have also been denied informed consent.
He is several steps ahead of you despite being in limerence. He has no intention of ending affair as suffered no consequences, you let him back.
I'm not sure of your work and living conditions but can only suggest you wise up, read all you can, give him something to think about by telling the tosser to take a hike.
You certainly need to do the 180°, grey rock him and start making plans. Please don't do the 'pick me dance' - he is not worth another minute of your heartbreak and it is so demoralising.
He will continue to behave like a shit until you seize some control back OP. Stay strong, you will get through this. Take care, eat sleep and depend on your family and friends.
I threw mine out after similar circumstances. Dumped his clothing at OW's and totally blocked him. I also informed his work he'd been using their office to meet illegally during lock down. He lasted a week with OW and despite begging to come back, tears and tantrums I was done.
You should be too. Nobody should ever be expected to put up and shut up. Seize back the power OP.