God, no way. Not on this earth. DS is just about able to manage my Mum in his own home. I'm due quite soon so a very good friend I trust implicitly, and DS pleads to visit after school (her son is in his class, they've been close friends since toddlers)and sobs when I collect him for home, had him for a sleepover on Friday. She called me at 10.30 because he wanted to come home, and when she dropped him off he clung to me for a good few minutes. DS visits MIL for a few days now, but with his dad. He still gets a bit tearful about being away from me and his home at night, though, apparently. He just doesn't know her well enough to get any security from her. And he's five!
Two? Two, and grownup wants are all anyone is bothered about?!
I refused to let him go on hols with them, aged 1.5 and the partner had a go at me! And your DH allowed this individual to treat you in this manner, and even to make demands that your pre-verbal toddler visit them alone when he barely knows them?
You should always, always follow your instincts, IMO. Nobody knows your child better than you. My son adores my mother, even when I find her a pain, and he's fine with her. He would not be with anyone else - and to be honest, I wish he would be. It would make life easier! But at two, no way would I send a little kids off like that to a strange house and people he barely knows. He'd feel completely abandoned, and he's a small person, not a toy to be shared.