It is easy to say leave it but I know I would feel like you and wouldn't be able to relax and sleep well.
If it's playing on your mind then when the baby is asleep Set yourself one task at a time that will make you feel you be made a start. Say take all of the stock boxes and stack them where they will be accessible but out off the way.
If you have dishes to wash and a dishwasher collect them all up stack and set it off. If no dishwasher fill the sink with hot soapy water and dunk as much as you can in and then leave them to soak.
Collect up all dirty clothes and put them in colour sorted bin bags. Take bin bags to washing machine and put the first load on.
Get bleach/toilet cleaner and cleaning spray. Dump bleach/ cleaner down the toilet and spray sink bath etc and leave to work.
Go back to sink and wash first sink full and leave to drain, refill sink and dunk in next load or dishes and leave to soak.
Go back to the bathroom, clean toilet and flush, wipe down all surfaces and fold towels etc. that room should be done.
If you have lots of miscellaneous stuff out around the house go around with a bag or basket for each room and collect up everything lying around that needs to be returned to another room. Everything in the lounge from the bedroom, put it in the bag and when the bag is full take it to the room the stuff belongs in.
Go back to the washing machine and unload and put to dry and put another load in.
Go back to the kitchen and wash the next load of dishes.
This sounds like a lot but once you get going it won't actually take long and you will see good results quickly. Once you've done that you should be able to relax a bit knowing you've made a huge inroad into sorting things out.