Forgiveness, it sounds like an EA. He kept OW and their messages a secret, which was lying by omission. He admits that he would hate your doing the same. The secrecy indicates feelings on his part. Why else would he hide her? It sounds like he is minimizing.
Were they texting after hours and on weekends? While he was at home and out with you?
You might consider having him leave for a while to give you space to process this blow. Such a consequence would tell him that you mean business, and he would experience how losing you would feel.
It would be best if they did not continue working together, as his boundaries with her are still weak. Can he change jobs?
You do need to implement a structure for affair recovery. This would include full transparency with phone, emails, computer, phone bills, etc., as well as his answering questions and taking your hurt/anger whenever you need to vent. He should not attend any work nights out if she is present.
It is also recommended that the offending partner attend individual counseling to investigate the weak boundaries, poor coping skills, and sense of entitlement that led him to seek ego-boosts outside his relationship. Ideally, he would learn how to strengthen his boundaries to safeguard against repeating.
You would benefit greatly from individual counseling by having a safe space to express yourself in your journey of healing.
I am very sorry that you are going through this nightmare. Personally, I wouldn’t continue my marriage if my husband was having a secret relationship.