Yes, I am an adoptive mother and yes, I do know how challenging traumatised children can be (though I do object to the sweeping stereotype in your post: my adopted dd is charming and wonderful, thank you very much).
Your SIL may not have been the right person to adopt this girl: perhaps she doesn't have the considerable resources needed to parent a child with such extreme problems. Or she may have simply been worn down by years of trying to cope with inadequate support. You say she won't accept she is struggling - perhaps she knows you are hostile and simply won't acknowledge it to you?
Either way, you need to get your head round the fact that this child is as much part of your family as your own children. You cannot send an adopted child 'back' any more than you can send your birth child back.
You asked for advice and here it is: if your SIL is not getting help and would like to have some, she could post a thread here and talk to experienced adopters who have gone through similar. You, in the meantime, need to work out how to protect your dc, while also trying to be as supportive as you can to SIL and dn.
If you were hoping we could advise you on how to tear your niece away from her parents in order to send her to an institution, well, that's not going to happen is it?