If I was going to give the benefit of the doubt, (and I’m not saying he deserves it, this may not apply in your situation, it’s just a different perspective to consider), it’s no excuse, but it can be really difficult to live with and care for/provide for someone with a disability. It’s a lot of responsibility, it can be really mentally draining, and it can really wear a person down.
My grandma is in a similar situation with my granda, she is the only one who works to keep a roof over their head and food on the table, he needs support day to day, and it does have an impact on you as a person to be a provider and a carer day in day out- not everybody is capable of it or cut out for it. She loves him, absolutely, but I could easily see where if she had spent the day at work and then came home and there were dirty dishes left on the side, he put something away as she was using it, and then turned a light off when she was in the room, where she would be very irritated by that. It’s one of those things that when you have a base level of calm, content, happy, it doesn’t really bother you. For example right now, if DH turned a light off when I was in the room I would just call out that I was in there and that would be the end of it. However, if I had been at work all day today while DH was at home, if I had some resentment about the fact he was not also working, then I got home to dirty dishes etc, I can’t say I wouldn’t be really irritated too.
It may just be that you are not compatible together unfortunately, as I say, being the provider and carer for your partner isn’t for everyone, and if it’s not for him then it will only build resentment.
On another note, it would be worth looking into to PIP/DLA/LCW again to see if you are eligible for anything. If you struggle enough that you are unable to work then you should be eligible for something, speak to Citizens Advice possibly for assistance and advice on applying. You don’t have to stay if you don’t want to, there is always a way x