You do not deserve to be shouted at.
You are not a bad step-mum.
You are doing the right thing by distancing yourself.
Your DH needs to stop making excuses for her, parent her and actually get to the bottom of what the heck is going on with her. It may well be that she has some form of special needs it may be that she's just behaving the way she is because she can, and your DH is enabling her. Does she behave oddly anywhere else? Have any of your friends or family said anything quietly to you about her?
The reason I ask that is because I had issues for years with my eldest SD, let's call her Carol (still have issues come to that, but that's another story).
When we first got together and were in the meeting each other's families stage, my Dad remarked that she was an "odd little girl". She was always perfectly behaved when they were around and he couldn't put his finger on what it was, but just felt she was odd. When she played up at home DH just shrugged and said it was just "Carol being Carol" She was socially awkward and often inappropriate. She did stuff that was just plain weird. When she was 11 she came to live with us full time which meant switching schools, and without going into a long boring ramble, we went through 2 years of hell. After 2 years of enduring intense scruitiny by social services, numerous tests, dedicated help at school etc etc it was decided by the "experts" that she was very slightly on the autistic spectrum, and had learning difficulties but that most of her behaviour was learned from her Mother (again, we won't go there). Bottom line was she didn't have any mental health issues so we were on our own in trying to deal with it. Lovely.
During this period DH was forced to accept that he had unwittingly been enabling her behaviour and change how he approached her. He had to step up, and I had to step back and disengage to a large extent. Needless to say that did not go down well with Carol who was used to having her Dad wrapped round her little finger and me bending over backwards not to upset her.
We had a bumpy few years and then she seemed to come right - until she turned 18 and was once again forced to change by being willing to take her first steps into the adult world. She refused, DH didn't cave, she threw a tantrum and went to live with her Mother, where she still lives. I became the font of all evil and source of every problem known to man and a really bad person. She also punished her Dad by basically cutting him out of her life unless she wanted something, then she became "Daddy's little girl" again. I ceased to exist, which suited me fine. Interestingly in the last 6 months she has apparently been trying to change, and wants to visit this weekend (I'm soooo looking forward to that!) so we'll see how it goes.
Anyway. Bottom line is you don't have to like your SD, it's ok to not like people who treat you like the dirt beneath their feet. You are also not required to enable her behaviour. Disengagement is a sanity saviour, and it will help your relationship with your DH because you won't resent him for enabling her. It will also help his relationship with his daughter because it will force him to parent her, which even if she resents, she will respect. She will love him, but if he lets her walk all over him, she won't respect him, and neither will you. But that's his problem, not yours.
The only other thing I would say is don't let her behaviour ruin what you have with your DH. You entered into the relationship because you love him, not because you want to parent his children. Yes you want to help them, but not to the detriment of everything else.
ps: sorry for the really long post!