You are right - it IS too much :( I lost my Dad to dementia, it is an awful thing.
The problem i found with SS is that if you do their jobs for them they are more than happy to stand back and let you do it, even at the expense of your own physical and mental health. I think you need to make a stand - what would happen if you simply were not around? They would have to deal with things - while i understand that you want the best for your brother, maybe it is time to take a step back. Is the fact that you are willing to help influencing their decisions, they need to be making them as if he needs to get 100% of his care from them. You can support him in other ways, as you are able - don't let them put it onto you.
This happened with my dad - my mum coped for as long as she possibly could, but it got to the point where she couldn't prevent him from wandering, he was getting violent and was still physically strong - it was awful. My poor mum, the social worker was as useful as a chocolate teapot - he turned up at my mothers house dressed like rupert fecking bear - i kid you not, it was like he was in fancy dress and asked "what do you want us to do?" and he was not being helpful iyswim. My poor mother had had enough and literally had switched off and wouldn't talk - so he wouldn't help 
All he could do was turn to me and say "well you are going to have to have him........." I was pregnant, i already had a teenage DD who was extremely distressed by my DF's condition, literally did not have room and (irrelevant really) kept snakes at the time and my dad was petrified of them. We had to fight to get him into a care home, had to take the first home that came along, our relief soon turned to despair as the home was just awful and in the end we had to find him somewhere else, so more stress - especially as we were forced into making a quick decision and had to find a home quite far away as the home he was in evicted him because we dared to complain. Some "care" homes are literally in it for the money, employ unsuitable people and don't give a shit about their patients - except they call them "service users"
Looking back, we should have done things differently - we should have insisted on respite care, we should have insisted on carers coming to the house to help, we should have made a plan earlier, but we listened to SS and the CPN tht the support would be there . The only bit of advice i can give you is while your mum is still capable of voicing what she wants to do then have "the conversation" Try and get those things in place as much as is practical and then set about looking at care homes for her now rather than waiting til a point that she will need them. If she lives alone that may come sooner than you think. They are all so starkly different. Again - do not let social services force you into doing their jobs for them, because they will - not because they are lazy, although my experience of them is that they are so incredibly well versed in buck passing, its like a script.
So i know it sounds like i am telling you to do MORE, but i think you need to take the steps now to ensure the appropriate care package is in place - making it clear that you cannot actually be the carer. It is not feasible or practical and you are not able to do it. You really should only have to be supporting your mum and your brother emotionally - it is down to the authorities to provide practical support - that'l be what you pay your taxes for and your parents will have paid their taxes and national insurance for over the years.