@Minimumstandard
Taking them elsewhere is like a military operation and really stressful unless there are more small children at whoever is hosting's house.
And it's still stressful unless the host's children are worse behaved than yours. DS is quite well-behaved for his age now, but when he was younger, we would only visit people with feral children. If their children were drawing on the walls or tipping cereal on the floor, I felt less bad when DS was crumbling cake into the sofa.
I love this, and couldn't agree more.
I vividly remember going to a young couple's flat. We were overseas working together, a group of 3 families with kids inlcuding mine, and this couple. They had set up some toys and colouring in their bedroom, so we could sit and chat a bit.
I was uneasy about unsupervised kids in their bedroom, but when I said maybe that wasn't a good idea, young couple got cross and said I was being controlling and it was their house. I immediately removed the felt pens, and said, sorry, ds (2.5) will draw on the wall, why don't we keep the pens and paper out here where the grown ups are. But the young wife insisted that the kids could play in her room.
Then one of the other children asked for the pens and the young wife took the pens back into the bedroom. I didn't see this at the time.
Sure enough, ds drew all over the walls. Wallpaper, rented flat.
They were very cross. I was furious. I was expected to pay (a lot, it needed repapering) to clear up a mess of their own creating. They said I should have more control over my toddler.
Yep, that 2 year old that I told you not to give pens to, and tried to tell you not to let play unsupervised in your room, but you insisted, and told me I was a control freak, that one that I was trying to control 
3 years later they had a toddler themselves and he wrote to me and apologised, he hadn't realised how hard being a parent was....