I don't think you should be worried - think it all sounds perfectly normal.
Fwiw, I felt soooo much better after DD was born than I did before. (And she's a terrible sleeper, so I'd have expected to feel worse iyswim.)
I was massively worried about everything - in-laws visiting, breast-feeding, being a mother/the responsibility/whether we had made a terrible mistake that was going to ruin our lives, argh, not to mention that actual birth of course - but it's all gone massively better than I thought.
Something that helped me was that I was very harsh (or, perhaps "strong") with my in-laws, and instituted a system of visiting hours, which worked very well. I said (truthfully!) that the midwife had advised this - she advised lots of alone time for me and DD to bond to help bfing (with DH on hand to provide food and drink at my command) - I had no problems breastfeeding at all, no pain or anything, but, heck, it was an excuse to keep the in-laws away. I think that helped enormously.
Your in-laws do not need to spend time alone with the baby, and indeed especially if you are breast-feeding, than your baby will need to be with you - pretty much 24/7 in the early weeks. Babies don't appreciate days out - can you get your DH to talk to his family, and say that when your DC is, say, 2 or 3 and can appreciate zoos or whatever then it would be lovely for them to treat him or her to a day out, but that it just doesn't make sense for a new baby to be able from its mother? Great if they're happy to babysit after a few weeks or months, but it must be on your terms, not theirs.
The reason I was able to be so strict with my in-laws is that I had so many people telling me that their only regret about the early days/weeks/months with their new baby was that they were not more strict with theirs.
I got DH to do all the talking about this... I was "resting" a lot when they phoned/were around! My SIL did it completely differently, and looks back on the first month with her new baby as a dreadful, stressful time, while for me, it as one of the happiest months of my life - sleep deprivation, tears and all. DH was able to have 2 weeks paternity leave, during which I didn't lift a finger around the house - just focused on resting and looking after DD, which was idyllic (well, sleep deprivation aside!)
And if visitors turn up unannounced then your DH is perfectly within his right to tell them that you are sleeping, and that visiting hours are 12-2 or whatever.
I found the last few weeks of work tough too - I just didn't care, it wasn't where my head was at all, which I felt guilty about, but I got through it, and then there was maternity leave, and then there was DD, and then it was all so much better.
I found the threads on here about getting through the first week with a new baby really really helpful - again they helped me to be determined about visitors/in-laws etc. It may be helpful to recognize that maximizing the time that you and your DH have alone, relaxing with your DC and not doing anything but keeping you and him or her warm, clean and fed will be good for your DC as well as for you - it's not as selfish as it may seem, as it will help you all to feel bonded as a new family, and also help maximize the chances of breastfeeding going well.
Sorry, essay. Complete stream of consciousness and no time to go back and edit, as DD will wake from her nap soon - but I hope me wittering about my experience was in some way helpful... I know it's different from everyone, but most people I know seem to have been happier once the baby was actually here!