Well, I'm no social worker, but if they are informing you that an allegation has been made, I'm guessing those allegations will be easy to refute. Many (adorable!) DC of my acquaintance frequently resemble grubby street urchins, but not a single right thinking soul in the universe would think they are neglected, far from it (and not suggesting that your DC look the same, but ice cream....grass stains....
).
As for your SIL, you were wary of contact with his family all along, so you've been proven right, and I'd like to see her go along to the police with her allegations of 'harassment'. See how much evidence (evidence which will stand up to police scrutiny that is) they can rustle up. Precislely none, I'm guessing.
And sweetheart, I'm going to try and word this carefully for sensible reasons, but if I hit you, your nose is bleeding and you go to the doctors, I can re-write what I like and tell people whatever makes me look better, but all the cold hard evidence points otherwise.
Your children will - sadly - learn the difference between a shouty angry father, and a steadfast, constant, reliable mother. They will even learn the difference between an expensively gifting father with no real time for them, and a mother who perhaps doesn't shower them with gifts but has emotional and psychological supportive input into their lives, if he goes that way.
As Across said earlier, he has most likely descended to a similar level as others of his ilk without you there to keep him up as a moral lifebelt (absolutely not your fault or responsibility btw), and you (and your solicitor... And possibly his solicitor...and no doubt others) are seeing what he's really like. Not nice. Awful, in fact. But him revealing his true colours will help you along the line, not least when he's re-writing history.
Stay strong. Your babies will be ok, and home with you soon, though I dare say not soon enough for your liking. Will be thinking of you (and sending psychic hugs in your direction regularly).