Don't post often, but I have seen a few debates of this sort on MN. I think it is the context, in this case, that OP has given (many times) that makes me say YANBU.
"The only issue is when we are having a group discussion (usually about what we are planning to do) and they then carry on the conversation in a different language. So a group conversation about something that impacts us all is then carried on to the exclusion of some of those involved."
True, as Liara (and others) have pointed out, the mid-conversation discussions OP has (in her birth language) with her daughter, are likely completely innocuous and, while perhaps not exactly off-topic, may be an 'aside' by the friend to her daughter.
"(in English) Shall we meet up for a playdate on Thursday?"
"(in mt) Oh, but don't you have to work for that homework you have to hand in on Friday?"
"No, it's OK, I can get it done on Wednesday."
"(in english) Sure, let's meet on Thursday."
"Not sure how that is excluding your dd, as the bits in mt are completely irrelevant to her."
To that I ask, but who gets to decide what is relevant to which members of the group having the whole conversation?
In the example Liara gave, the OP's DD could have had the same homework, but unlike her friend hadn't made alternative plans to do it, so that reminder would have been very relevant to her, had she been able to understand it.
I am deaf and there are certain people who continually feel they have the right to decide that some parts of a conversation (that we are mutually involved in) are not relevant to me! They may or may not be correct about the relevance, but (as part of a conversation that we are mutually involved in) it is not up to them to decide the relevance to me of their own (or other people's) 'asides' when within that group conversation. IMO.