Oh Shit Tiny, I'm now 100000 percent on board with you. I just needed the background to get a better idea of how to deal with him. Your solicitor has probably advised not going to court because he will get supervised initially, but if he turns up at a contact centre for a few months, maybe up to six there is no exact length of time these are ball parks only, and he behaves, he can then go to court for unsupervised.
Supervised rarely lasts forever as there simply is not the funding for it. Depending on the judge/cafcass officer on the day he may not even be requested to do drug tests to show if he is clean or not. Even if he gets clean for the test he can start using whatever again straight afterwards and the kids will be at risk.
He may get a couple of hours a week initially but he can build to unsupervised and overnights so her advice to keep it out of court is excellent advice.
I don't think this is a situation where you should be forcing it to court but fighting him if he takes you to court. Without a court order you are in control, you tell him where when and how.
With a court order the courts are in control and you can't trust them. OK you have an honest decent solicitor that's good.
It sounds as if he won't be given the house so do what ever you can to put the landlord off. If there has been no recent violence, harrassment (over 12 months old) it is extremely unlikely that you would have grounds for a non molestation order.
All a non molestation order is, is a promise not to molest harrass you or to get anyone else to do it on his behalf. Without an exclusion order and powers of arrest it doesn't make the police do anything. With those they have to arrest him if he comes inside the excluded area or talks to you etc.
You might want to ask a solicitor about that though, if you applied for one with exclusion zone around your home, he would also get to defend that and say he is getting a home two doors away to be near his kids. Not sure... I'd play dirty first and use the landlord to achieve the same ends.
Remember this: The longer it is before a court gets involved with contact issues, the longer you are in control It shouldn't be like this but it is. Going to court for contact puts the onus on you to have the children to be made available or taken to a certain place it doesn't make him turn up. It's a high risk strategy and not a game I'd play. If you refuse to have the children available you can be threatened with jail, community service etc. even if you are refusing to comply to keep your children safe. 
On the plus side the longer it takes him to take you to court the longer it will be before unsupervised happens. Some cases can go on for a couple of years, hopefully he won't take it that far and you can stay in full control as you are now.