I think that all parties here are very guilty of not thinking this holiday arrangement through at all.
OP, when you are all at home it sounds as though you and DP are working as a team to care for MIL. That is great if it suits you that way.
DSS is around mainly in university holidays, otherwise usually not. Suddenly, the rest of you are going on holiday and leaving him in SOLE charge of what sounds like an increasingly difficult elderly relative (MIL). MIL is very likely frustrated and depressed at her declining faculties and increasing reliance on other people, in addition to having cantankerous tendencies and the possible onset of memory problems.
DSS, having recently returned from uni, will probably be seeing much more of a decline than you do, but for the fortnight you are away there will be NO teamwork for him. He is not exactly experienced in care work, yet is being thrown in at the deep end and will be on duty 24 hours a day for two weeks. It could well seem very daunting.
I don't think it is a fair ask. Get carers a couple of times a day and meals on wheels for lunchtime. DSS could go to his job, but would be home in the evening anyway, so company for her then and around if a problem occurred.
I think you have ploughed on ahead and possibly asked too much.
When I was in my late teens my grandma had to spend a few months living with us following major surgery for ovarian cancer. During that time we all cared for her and even as a team it was very hard. Whenever I had to spend just a few hours on my own with her I found it terrifying. I was so scared about how to cope. I knew then that I was not cut out to be a carer.
I think you have plumped blindly for the easiest and cheapest, but wrong, option to allow you to have this surely much needed holiday, and it is unfair on your DSS.