Fairy, I can totally understand what you're feeling. I was there too this time last year. I couldn't explain why, but even though my baby slept reasonably well at night, if he cried at all, even for 1 min it would send me into oblivion in my head. I couldn't stand it and would get so angry.
I used to wish someone would knock on the door and say they were taking him away. I thought a lot of other stuff that I would be too embarrassed to write here, but that was the PND talking in my head.
Personally for me, looking back, a lot of people said it was just because of sleep deprivation, and I waited too long to get help because I wanted to believe it was that, and it would pass soon, but it didn't. Getting out and about didn't help either. Yes, someone coming over to help for a couple of hours was great, but when they left, I felt like I was back at square 1.
Has the HV done the edinburgh score with you?
PND is more than baby blues or just feeling tired. A difficult baby can set it off, but it takes more than just a few hours time away from the baby to get better. I think it would be worthwhile speaking to the GP. But also, perhaps call someone at a PND support place like House of Light or PANDA's? The good news is that PND is something that gets better over time, but the quicker you talk to someone it can help.
I was lucky and found a PND support group that met face to face. Just spending 2 hours talking to other mums who felt and thought the same stuff was a great relief. I'd recommend speaking to someone, as I wished I'd done it sooner so I could enjoy the time with my baby when he was younger.
PM me if you want to chat