Harman - I suspect he does really hate you - or at least hates the fact that you are in control of the situation not him (even although it may not seem that way to you).
He's a bully and he is trying every game he can think of to get back into the driving seat in your relationship. You've come so far now that I agree you cannot now go backwards!
I would write to the solicitor - if nothing else it will prove that he had full knowledge of the fact that he was making you and the children homeless by not moving out. I would be tempted to get the solicitor to weave the whole story into the letter - i.e that the house is empty most of the day, you are pregnant, you have attempted on several occaisions to get him to amicably agree to move out for the good of all parties etc etc. This gets on record all the efforts you have made to secure things for your children and makes you look like the caring parent rather than him.