You will need to communicate with him OP, at the end of the day whatever happens between you and him, you are still going to be co-parents from now on, and to do that effectively, you will need an open channel of communication somehow (through you or through a third party) keeping him from his child or his house is dodgy ground legally and you don't want to give him an ammo to use against you.
Also, you are going to be in hospital very soon and unless something is arranged he can legally do whatever he likes in the house when you are out of it.
I'd be inclined to send him a message that doesn't invite a response, to make your position clear, taking any advice on board that you may have got from the solicitor today. He will keep trying to contact you and potentially come round so you may as well tell him exactly what you need from him right now, and if that is to stay away and not contact you then so be it, but you need to do something, even if it's through a third party.
You could tell him if he needs anything he can contact your brother (if he's okay with that) to arrange.
Out of interest who is going to be looking after your 3 year old while you are in hospital?