DH just invited her just now so no such luck this time as he knows I have no plans
You need to sit DH down and tell him needs to start listening to you.
There will come a point where he has to choose - you or his mother.
I don't think you should have agreed to DH and DS going visiting together to hers. That rewards her bad behaviour with access to DS and you out of the way.
Her behaviour is incredibly rude - staying for 8 hours is a complete imposition, and being someone's mother and grandmother doesn't make it less so. This is not what normal people do to others when they want to establish friendly relationships with them. Tell your DH that.
Following you into the bedroom when you go there with the baby is bizarre and unpleasant. You need to shut the door firmly behind you when you go in with the baby in future. If she opens it and trails in after you, get a lock installed. She has no business doing that. There is no possible reason for it, and you shouldn't let her get away with it.
Your problem here is that when your DH talks about the two of you getting along, what he means is you acting like a doormat and letting his mother get far too involved in your relationship as spouses, in your relationship as parents, and in your baby's life.
An associated problem is that DH seems to think of DS as some sort of toy that everyone should get equal access to. The baby isn't a toy, and no, everybody doesn't get equal access. He's behaving like a child and allowing his mother to get away with behaving like a spoiled child.
............
As an aside, is there any particular reason your DH doesn't drive?