YANBU. It pisses me off how greedy bitches can steal money off of other peoples kids for their own, my friend was in a similar situation to you. I say WAS because she ended up slitting up with her partner, because of the money grabbing cow ex.
My DF has three children of her own, works full time, minimum wage, so she was in receipt of maximum CTC, about £30 WTC and she also received 80% of her childcare. She had a boyfriend, who lived with his mother, and received income-based JSA, out of which he paid £5 to his ex for his child, she has three children, her two eldest are to someone else. However, his mother threw him out, so my DF said he could move in with her (which myself and the rest of our group of friends were worried about, as unlike your DH, who wants to work, this man is a waste of space, hence his own mother had enough of him).
As a result of him moving in, he lost his entitlement to income-based JSA, due to my DF working, and DF lost her 80% childcare, however her WTC and CTC stayed the same, due to him having no income, as his previous claim to JSA was not counted as income for tax credits, due to it being 'income based'.
So DF's DP looked after her DC's while she worked and the money she saved (the 20% of her previous childcare costs that she had to pay herself) she gave to him as 'pocket money' and he continued to pay his ex the £5 that had previously been taken from his JSA.
Greedy money grabber found out that DF had three DC'S, so asked the CSA to re-calculate her maintenance, and DF'S CTC that she received for HER three DC'S was taking into account, her WTC was not counted as DF was the person working, and greedy cow was awarded £20-odd a week, even though DF'S DP had no income, but because his name was on DF'S TC claim, DF had to take money from her OWN DC'S which was paid into HER bank account to pay this bitch.
And before you all start flaming me for being so nasty about the ex, I would like to point out she is in no way impoverished. She is on income support, gets her FULL RENT and COUNCIL TAX paid, gets the same as my DF in CTC and CB, and also gets a good amount of maintenance from the father of her first two DC'S, which she gets to keep to herself as maintenance is no longer taking into account for IS purposes. Her DC'S have the latest trainers, computer games e.t.c she has spray tans, hair extensions the full works, is in the pub every Fri/Sat night, she is basically your typical 'Jeremy Kyle' type.
As a result of her stealing this money from my DF'S DC'S, my DF ended up in the shit as her budget was affected (and she had to pay FULL RENT and COUNCIL TAX, something the money grabber didn't have to do) and it caused a lot of arguments between DF and her DP, and they ended up splitting (much to myself and the rest of our friends relief, as he was just a waste of space 'cock- lodger').
It is not fair that people like my DF's and the OP'S DC'S should suffer due to money grabbing ex'es.
If your all wondering how I know so much about the ex'es circumstances, my DP is good friends with the father of her first two DC'S, who is a decent hard working man who pays his way, unlike DF'S now ex DP.