OP any one moment isn't very hard, its the endless nature of it all. Its like working in a really mundane, repetitive job yearning for the weekend...but the weekend never comes...and you're on call all night.
Agree with pp that it is the initial shock and adjustment that is the hardest and going from 1 to 2 is a breeze in comparison because you have already made that mental leap.
Saying all this, i like you read all the horror storys before giving birth so i was well prepared and so expected to be utterly broken by it all. it turned out to be Ok, knackering yes, boring, painful yes but perfectly doable and although there were set backs each week tends to be better then the last.
Mine are 2 and 4 now and it finally feels like we are coming out the other end.
Be patient with yourself and your partner (easy to be very snappy when tired) prioritise sleep (yours and babys) over all else. Work in shifts if you can, accept help from others, get out of the house if you can, try to talk to an adult about adult things sometimes. Give yourself the first 3-6 months with no expectations beyond looking after yourself and baby, anything else is a bonus.
You will find you will look back after the first 4 weeks or 6 months or so and go fuuuuuuck that was difficult, how on earth did i do that!!!