Honestly, my experience would be:
He's doing it to provoke a row.
If you react by picking that row, he will be delighted.
Mine also used to complain that I sent the dc in old, scruffy clothes (BECAUSE otherwise I'd never see the decent stuff again).
I just smilingly pointed out that the best thing would be for him to understand that he was responsible for clothing them when they were with him. I was happy to send clothes to last the stay if I got them back, washed & neatly packed, & if this was inconvenient I'd just send them in what they stood up in & he could go shopping.
I had to do this - once - after he'd absolutely taken the piss over things like brand new trainers, school coats, etc disappearing. Apparently he flounced off to Sainsbury's & bought cheap jeans & tshirts all round.
The sending them back in manky stuff did persist until they started challenging it themselves, but by this point I'd wised up & was making sure I only packed clothes that were on the verge of being discarded anyway.
Ultimately, it impacted more on him - he'd ask the kids why they were rocking up in scruffs & they'd explain that they'd rather not bring their recent purchases, thanks, because they didn't want to have to argue with him to get them back.
As I said upthread this was the point at which he realised he looked like an absolute tosser in his dc's eyes & started casting round for a different way to wind me up 