It’s not his job to review yours or anyone else’s reasonable adjustments unilaterally.
Reasonable adjustments should be agreed between the workplace and yourself with independent backup from occupational therapists or other professionals as required. Both parties should regularly review the adjustments to ensure they are working effectively - eg if you have a back problem your standing desk hasn’t developed a mechanical problem, the interpreters for a deaf person are delivering well and keeping them up to date, are you using outdated screenreading software and need something new, and so on.
It’s not for him to review and decide that a person is or isn’t in the right job based on their disability and the adjustments needed. That’s not the purpose of reviewing adjustments and he doesn’t have sole power of yea or nay, the onus is corporate.
I get this. I am disabled. I am part of my work’s disability network. I know how hard it is to manage and upskill and advocate for yourself, both on my own account and hearing colleagues’ stories. I have a lifetime’s experience.
You sound quite intimidated and already afraid of rocking the boat even without the latest incidents. You absolutely should escalate to HR as he’s the problem, not you. I get this - I see it often enough among disabled colleagues who absolutely don’t want to rock the boat but then things escalate. Among other things, there’s the concern over getting another job - not so easy for us. So I understand. BUT -
Things won’t get better if you try to manage it yourself on the quiet without drawing attention. It needn’t be a grievance but you could document it in an informal but confidential conversation - but it sounds as if he’s already starting to manage you out by treating you unfairly. That does need escalating and you do need to advocate for yourself.
Or your boss will - and is - taking advantage of your disability work ethic to pile more and more on you until you collapse with the stress of managing your work and your disability. Then you’re managed out.
It’s an uncomfortable and confronting truth I’m afraid and I’m encouraging you with my lived experience as a disabled person and as an activist and advocate for others - please do escalate. His behaviour is unfair and breaks the law and you aren’t playing dirty or telling tales by raising a concern with HR - on a confidential basis if necessary.
Please do feel empowered to raise matters with ACAS and talk them over. That doesn’t mean a grievance is started if that’s your concern - you can just ask them for advice. It’s good, free and confidential advice which will leave you clearer about your legal rights. Ask them about use of personal phones, your legal right to time off, and your right to be treated fairly at work as a woman and a disabled employee. That might help you navigate going forward.