Designate one weekend day, both of you, to tackle it once and for all. Or at least make an inroads.
Get up at a decent time. Throw on crappy comfy clothes (that you don't mind getting dusty), make coffee. Do an hour's sorting before breakfast. Have breakfast and wash it up before going back to it.
I'd suggest having a bin bag ready to chuck things straight into - when it is full, chuck it.
One bag/box for charity (clothes, books, unwanted mugs etc), once full put straight in your car boot.
A box on the stairs for things that belong upstairs. When it's full...er take it upstairs!
Be ruthless with crap such as Ikea pencils -
i recently found 20 bound with an elastic band 
Everything should have a place where it lives, once you get into a routine of putting them back where they belong once you have finished with them, your house should look less cluttered. I have a lot of things, but not a cluttered house, iyswim?
Photos all go in a box which I can go through at a later date.
If i'm not sure about something, i keep it. The next time i sort out, i can look at that thing with fresh eyes and decide again.
You don't have to throw everything away, it can be traumatic and sad.
I'd start with the kitchen: wash up and clean surfaces. Then empty cupboards, clean cupboards, chuck things that are broken/duplicates/you do not use, then put things back in the cupboards. You can also check best before dates too.
It is daunting, but really it is just tidying up. Pile up paperwork. At the end of the day, go through paperwork and order into piles: car, house insurance, receipts for hoover etc. At a later date you could put them in date order, but later not the same day (i could spend JPURS on paperwork!). I have a big book-shaped box on my bookcase where I put receipts when i buy big things: watch, fridge, blender etc. I put each receipt into an envelope and label it with the object and the date.
I am far from perfect and am actually going through a box i haven't opened since we moved 5 years ago
it's an eye-opener!
Once you gave had a big clear out and tidy, hopefully the buzz you get just looking at a tidy house will help you keep on top of it.