Faulk you have come a long way since your first post. You have been very brave in confronting this because it's clear that you are terrified of losing him.
However, the irony of it is that if you want to stay in a relationship with this man, you have to ask him to leave.
He has to understand that he may lose you. That he has broken all your trust and lost any respect you had for him. He needs to realise that you might not even want to be with him anymore.
Only then, when he truly accepts that, through his own choices, his own actions, he has risked everything, will he make a proper and genuine commitment.
If he doesn't commit to you fully, openly, honestly and take full 100% responsibility for what he has done, there is no way that he can start to rebuild a relationship with you.
All this has to be done by him. It has to be his idea and he has to genuinely want to do this. There is nothing you can do because it's not your problem to fix.
Sure, you can make him show you his phone, you can make him go to relate, you can make him give you all his passwords, etc. But you making him is not him choosing for himself is it?
What about the music lessons, have they stopped? How does he behave when you talk about it, is he contrite and supportive or does he get fed up, defensive or try to minimise or shift the blame?
Tell him to leave whilst you have some time to think about what you want. Honestly, it will scare the life out of him. And if he runs to her then you have your answer and can start to move on.
If he doesn't then and only then is the time for you to consider rebuilding this relationship together.