I understand exactly how you feel. I only have one ds but I have felt that I have lost all the time I used to take for granted with ds so I can give dsc quality time instead and indeed when my ex sees ds, I feel envious of the long car journey they get alone together! 
I am luckier in that we don't have dp's 4 children all the time although one only goes to mums one night a week and inevitably we seem to end up doing "adult" things then like going out to dinner with friends. Ds is supposed to go to his dads on those nights too but he has got very unreliable and I have to admit, it suits me as I can get a few hours with ds if we're not going out.
I think you have to lay it on the line with your dp that you need time with your kids alone and he needs to make that happen. My dp will take his kids out for a fry up or go shopping and although its only a hour or so, it does help as does picking up ds from after school clubs and going for a costa, get new clothes, cinema etc.
My dsc all get on well with ds but they do all cling into him for entertainment, so i have been in the situation many times where ds is not alone for one minute over a weekend because there is always one of dsc with him. However, there would also always be one dsc by my side too and I can't tell you how suffocated and trapped that has made me feel!
One of dsc doesn't come to ours anymore, the eldest has a new part time job and dsd2 has an occasional weekend job too so it's got easier to get sometime with ds.
I often crave a lazy morning with just ds and myself and making pancakes and chatting and slobbing around but I do accept that it's going to not really happen very often or for very long.
I do think you need to talk to your dp and make some changes.
I feel incredibly guilty if I go out with ds and leave dsd2 at home but I still do it. I might feel guilty but it helps me feel less resentful.