I went through this not so long ago with my DH. We have two young DC's (3 and 1.75!) and we were both just knackered. Everything he said or did annoyed me, or made me feel like utter shit, and I couldn't bear him to touch me or even joke about sex. And, (so I found out later) vice versa.
One day we got the PILs down to babysit and we went for lunch in the pub. We were hardly on speaking terms.
We played a little game of writing down questions and folding down the paper, then handing it over, and answering the other's question. We sorted through a few things, and both made an effort to be more considerate.
Since then, it's like we've cleared the air, and we are more or less back to our old selves, taking the mick out of each other, even flirting and finally back in the sack together.
We are still knackered, and still co-sleep with DD every night, but it's so much better.
All you need is some time ALONE together to talk. At least that would be a start. Part of the problem would be admitting it to each other that you are both lonely.
I know it's sounds simple, 'just talk' and I'm sure it's much more complex than that, but it's a starting point. If you both still really love each other, it's worth a shot.
Best of luck.