I think Op handled it well. She took action, but did it in a controlled, measured and not angry way and had a conversation with bother the children and the other mother to explain what had happened.
To those who think he shouldn't have been taken home, I wonder what a child would need to do for you to consider this an appropriate course of action, or if you think that because a child is 5, there cannot be anything at all that they could do which would warrant being sent home due to their age? Do you think they cannot take any responsibility for their actions and that the parent in charge needs to bear all responsibility for absolutely everything and anything they do?
I think a 5 year old is quite different to a toddler. I wouldn't expect a 5 year old to draw on the wall nor expect to have to tell them not to do it. I would expect a 5 year old to need a reasonable amount of supervision on a play date and to expect they might need directing to activities and have a fairly short attention span, but I would also expect to be able to leave a pair of 5 year olds in a living room or bedroom for a short while without them doing damage.
Accidents happen. We all understand that. The question comes down to whether something is an accident which happens due to a mistake and lack of judgement which comes from being 5 (annoying, but part of having 5 year olds round) or if it's an action which a child will have clearly known was wrong (not acceptable iof it causes damage and was wilful).
The handling of these events is important. OP didn't shout at the child or parent. She was measured and calm. She didn't punish the child but simply took them home. It all sounds exactly right. No doubt the other mother did feel a bit embarrassed by it, but that's not to say she thought it was wrong. I think that when we are parents and send our children into the bigger world of other people's homes or school, we have to accept that we will receive 'feedback' from the other adults who take responsibility for our child, which will be both positive and not always so pleasing, but also that we have to let those other adults make decisions about what is acceptable and how they will react to it. And we need to usually graciously receive that feedback.
Other parents might have chosen to ignore the writing on the wall and not tell the other parents or to brush it off as acceptable due to age. That would be their choice, as OP's actions as her choice. Personally, I'd think a hosting parent who turned a blind eye and didn't mention this to the other parent would be wimping out and not doing the child any favours - how is anyone ever going to learn if they aren't picked up on poor behaviour?