I would ask her to write a letter to you stating everything she feels is wrong and making her unhappy - ask her - and mean it, to put everything the family could do to make it better. Tell her you if she is willing to do this, you will read her letter carefully, reflect on what it says and write back.
It seems the communication channels are closing between you, she is shutting you out. She may not be able to put into words what is wrong, but a letter will give her the chance to spill out her feelings without interruption.
Whatever she writes, even if it is the worst thing you can imagine and very hurtful. Put your own feelings to one side and see what and where you can help. If there is something more happening you will at least know about it. It might be angry (but anger is just sadness as we all know) it might shock, don't be upset. Just let her get it all out if she is willing. Send her a letter back telling how much she means to you, and that will never change.
She will remember your efforts to look after her, she will remember treating you badly and still you carried on loving her, she may be in a bad place with her mental health but can not ask for help. As long as you carry on being loving and kind, keeping the door wide open, that is all you can do, and that is more than enough. Most teens grow out of this, and she will be no exception. I have never forgotten my mother banging on the door of a horrible drug fuelled party and telling me she loved me, and was taking me home. I hated her at the time, but I still think of it now 30 years later. Don't give up on her.