My mil was just like this when I had my first dc. It got to the point where I would hide upstairs and pretend to be out, when she called, just because I needed some space and couldn't bear another long afternoon of her monopolising my baby.
Luckily my dp was supportive and recognised that she was driving me crazy. We did ask her not to come around so often and to phone first. Tbh, I wasn't as tactful as I might have been, which I regret, because she really was motivated by love for the baby, but otoh, she wasn't very tactful towards me, in trying to take over my baby! In my mil's case, her own IL's had been very domineering and I think she was trying to get back some of what she'd missed out on with her own child.
You have my sympathy. Your dh is kind of right, in that if you don't clear the air, your future relationship may be strained. However, he has no right to insist that you speak to her, against your will - that could well make matters worse and result in an argument.
I think I would tell my dh that I will speak to her about it, and that she and he, might not like what gets said, but seeing as he's insisting, he'll just have to suck it up, however it goes. Tell him you expect to be backed up by him - you two are a unit and if he wants this conversation to go ahead, he has to support what you are trying to say.
Don't be rude or deliberately unkind, but point out that all your motherly instincts are screaming at you to hold your own baby and it makes you feel very uneasy if someone refuses to hand the baby back, when asked. Now would be a good time to gently lay out some ground rules, like phoning before visiting etc.
Things will settle down - your baby knows that you are his mum. This is a time of great adjustment for the whole family and eventually things calm down, you just have to explain what you feel comfortable with, as kindly as you can manage.
don't let them browbeat you or lay on the guilts - your mil has had a baby herself, she should know how new mothers feel and be sensitive to it.