What some people are forgetting, is that not all mothers/MIL are kind, caring people. You just have to read these forums to see there are plenty of toxic parents, who shouldn't be allowed any contact with their grandchildren, let alone "time alone" with them.
Fathers don't grow the baby in their bodies for 9 months, so don't have the same bond as mothers do. It's biology. Therefore, they should respect their partner's wishes.
This particular MIL, has shown she cannot be trusted with the baby, and because she's being emotionally manipulative, getting her son on side to badger OP, in order to get what she wants, regardless of the distress this is causing, it should be obvious that she is not a suitable person to leave the baby with. It's not rocket science.
OP, next time, if it helps, imagine you've got all of the pps here who have supported you, standing right behind you when you tell H/MIL NO!