Op both my dc have Autism, I know how things can pile up and how difficult it can get.......but I also know how this can make things so much worse in general and how it can affect our dc.
It doesn't sound like he was criticising you, just a general rang. And that's fine. Bring it up with him and how you can both tackle it......I find in my case I have a strict one in one approach. Even if I buy a new tshirt an old one will go in the bin. Junk mail etc is binned straight away.
Washing doesn't build up because it goes straight into the washing machine (( no. I don't separate darks and colours, only whites which tbh I don't really own ))
Clothes for the dc not so much then but I had a very simple and pared back wardrobe so we didn't have mountains of clothes like we did previously.
Toys which weren't played with just went and tried to keep that down to a minimum..
Even the kitchen bin went as that was just another job, instead I had small pedal bin liners in the kitchen, all rubbish and food waste went straight out into the bin.
Small simple tasks which required no effort and yet made a massive difference in stopping my home being a constant shit tip.
As for the cupboard well one of you can take the dc out if thats really needed and one of you can start to tackle it. Be brutal. If it's shoved in a cupboard chances are it's in there because it isn't needed. So if it isn't something you use then get rid of it.