Of course it is awful seeing your DD upset.
Have you started a proper adult conversation with the mother, without her mumbling, and finding out exactly what the issue is. Your DDs friend might not know the full reason ?
The thing is, and I mean this in the nicest way, if you DD struggles to follow instruction the other mum might not feel confident supervising her, but unless you have a discussion with the mum yourself, you won't know.
I can't look after my own godson due to his Autism, it's not that I don't want to, it's because he won't cope with it, so we always go to their house. He loves seeing our DS but I couldn't give him a positive experience in our home.
If it brings joy to your DD, unless it's an unhealthy friendship, just let it continue and stop the expectation that it needs to be returned, it's an experience you either choose your DD to have or not.
But I am assuming if you thought it unhealthy it would have stopped by now, you just sound hurt and angry that your DD is being excluded. I would talk to the mum for the benefit of my DD