Hold-you carry on telling who the hell you like about the affair. I'd also be tempted to tell everyone that you have asked him to move out as he refuses to stop seeing the other woman, and he's refusing to go, and you're frightened . Don't be ashamed to tell people the truth, it's not your shame to carry, it's his.
He won't respect your wishes or requests no matter what you do, he has proved this already. However, he sounds as though he cares what other people think of him. While it's just you telling him he's being utterly unreasonable he will trample all over you, but if he knows that all his friends and family know what he's doing then he might feel under more pressure to co-operate.
What's the situation with Women's Aid? Keep trying, they will know how to make things work for you.
And what's the situation with keys now? Are you having to wait in to let him in to the house?
As he won't leave, can you get people round to visit and support you in the evenings? Have, say, your mum round for dinner and to help put your children to bed, and to stay and chat in the front room. Or friends, or siblings. Take charge in your house, you can't make him leave at the moment but you can make him feel bloody uncomfortable and sidelined.