When I first read this my thoughts were "yes, yabu".
But. I thought about my ex mil (ok, we weren't married but you know what I mean). She was an absolute cow to me, for no reason whatsoever until the day she found out I was pregnant.
Up until I had ds and for a few months afterwards. Then they offered to look after him while I worked and without going into it too much they made my life hell. They basically took over and slated me for everything they possibly could, lied to me and treated ds as their own. This included things such as taking him to the doctors all the time when there was nothing wrong with him, telling me the gp had said he had various allergies (I checked, no such thing was ever said). Changing him out of his own clothes as soon as he got there into second hand cast offs (nothing wrong with second hand btw, but he had his own lovely new pfb clothes), throwing away the fresh meals we had cooked for him and taking him to McDonalds.....That woman was the reason I had to start taking anti depressants and a big factor in the split between me and her darling son.
After about 3 years it ended the day ds's grandfather came running out to the car, yanked open the door and told me I was neglectful and making ds's life hell for trying to get him out of needing a dummy. I blew my stack, said a few things I shouldn't have and he never went there again.
I told them they could see him, but I would not be taking him to their house and they would not be setting foot in mine (was living apart from ds's dad by this time). If his dad wanted to take him to see them or they wanted to pick him up...yes.
Funnily enough they never bothered, can't say I missed them much .