You need to treat this as a whole issue, not just him taking his rings off.
If you'd had a few weeks of discussions, is it fair to say that there is no saving this now? He seems fairly certain that he wants to move on.
His behaviour is consistent with that, and will only become more so. Taking his rings off for work is the first step, but he will eventually stop wearing them altogether, he might already be lining up a new relationship, and he'll start telling people. He'll stop wanting to share your bed, and your life. You need to be prepared for that.
Take your own rings off. Make arrangements for him to sleep elsewhere. Stop making the day-to-day the same, because it's not. You're just letting yourself get stuck behind, whilst he makes new plans. Visit a solicitor and find out where you stand with everything. Tell someone that you can trust, because I'd bet that there is someone at work that he will be sleeping with/having a relationship with shortly - he's probably showing them that he's actually ended it by going into work with his rings off.
Take steps forward. It wil hurt all the more if you stick your head in the sand - It might feel like a solution, but as you've seen, it's not.