Don't sneak to look at his phone.
It's very possible he is having an online affair, so he wouldn't need to go and see anyone.
I'd stop asking him anything and start investing time in yourself. You've discussed the reduction in affection and change of habit and the secrecy with his phone. Just leave him to it.
When I had something I wasn't happy about with my DH..(concerning another female, but not an affair), I just told him I was simply going to withdraw and detach from him emotionally. ....in order to protect myself.... and the result would be in the end I'd just have nothing left for him.
He knew how serious I was and he stopped what he was doing.
I don't have access to his phone, nor he mine. I couldn't confirm details, but I laid my cards on the table.
You should:
• Socialise with friends
• Do things that make you feel good. Like spa days or a simple walk or hike to clear your head.
• Keep yourself busy. Perhaps get a hobby.
Don't let your life revolve around him and just observe his actions from a distance, while getting on with your life.
You'll eventually decideif you want to carry on this way, or if your detached enough to not even care.