Words fail.
You had the chance to manage this all by yourself and simply ask them to pick things up, or say to them "kids I'm tired, I need your help clearing up, you clear I'll wash"
But no, you said the words "I'm not happy for them to stay when dh is not there"
And that's all they heard. :(
You have NO right to ban your dh children from their parents house. Not for this, and pretty much not for anything, but by attaching a condition to their stay, you have done precisely this.
...or your dh has bungled the communication of your request and made it look like they are banned.
Get them back and put this right. Do not put conditions on their visits.
Say to them that they need to understand that plates need clearing, stuff needs picking up, washing up needs doing. Say that you are happy washing up, but need them to pick up after themselves and that you will be reminding them.
And their mum is right, why not get them to mind ds while you take a bath?
You blew this good and proper, but you can put this right. The kids are old enough for you to be honest with them, apologise to them for making them feel as though they are not wanted, but explain that you need more help than you did previously. Explain too that regardless of the baby, they are old enough to understand that they need to pick up after themselves.