If I was upstairs and baby started crying it would be full-blown screaming by the time I'd detached from pump and got downstairs (which would no doubt annoy neighbour more than pump). So it makes sense on many levels to pump in same room.
Yes, it makes sense to stay in the room if you've decided not to budge on the issue and can provide examples of how you've made accommodations when a complaint is filed. From there it is up to the LL to add better insulation if reasonably low noises are still being heard. There are inexpensive ways to do this and something like that should fall to him to fix, just like a leaky pipe.
But saying a baby will scream the house down if you're not there as a reason is fucking absurd. Why wouldn't your DH get up with baby? Ear plugs block out noise, but they won't block out a screaming baby right next to you. If he's sleeping through a baby crying right next to him, he's "sleeping". There's a bigger problem than "needing" to pump in the room.
I FF and DH was happy to take night shifts while I slept because he's a night owl anyway. I'd grab her in the mornings and he'd grab her during the midnight feeds.
But even when I tried BFing, he would get up to bring me DD so I could feed her and then rock her back to sleep when I was done. When I was struggling he ran out and got breast pads, cream, and I believe he made calls to the lactation consultant for me to ask questions. You should not be doing this all on your own. A baby is a team effort. You need the support of your partner and you need him to let you sleep. When I wanted to switch to formula he supported me and was thrilled to take over so I could finally get more than a 2 hour nap.
So, no. A woman does not typically take over all baby duties while she's on maternity leave. That time is for your body to heal physically and emotionally from having a baby. This is his child too, he needs to pitch in.