Ex wants to know when I go out - reasonable?

A bit of background - ex is very controlling and intimidating, and can be violent. I left him two years ago, took our two children (9 and 5) to live on the other side of town. We weren't married and have always sorted out the arrangements for the children between us. He sees them every Saturday, and they stay overnight every two weeks. I offered ex parental responsibility when I left but he didn't want to go through any legal process (because it would cost money); he also doesn't pay any maintenance, although I was on benefit at first I was judged as having good cause for them not to go after him. He said he would give me money when he could, but rarely does., although i'm quite happy with that situation, want as little from him as possible.

We've just had a huge row over an issue that has come up before. Basically, ex wants to know every time I go out and leave the children with a babysitter - he feels that he has a right to know who is looking after them and when.

I don't agree; I feel that he is invading my privacy, that he only needs to know that if I go out they will be looked after by someone competent. Although he says that he doesnt want to know where I'm going or who with, simply when I go out, I feel that he is trying to have some control over my life still.

So am I being unreasonable, or is he? Should I phone him every time I go out for a couple of hours, or does he have no right to know? He was extremely threatening in the row today and I feel scared enough at the moment to go along with what he wants in the name of peace, but i need to know if I have any grounds to stick up for what I believe to be right, or whether i really am a selfish unreasonable cow as he says.