Firstly, I have been there. My husband used to accuse me of stuff. I am still with him now and he has changed for the better. He was unhappy, in a depression and just couldn't connect to me. We are much happier now.
So, as I said, he used to do this too. He used to blame me for mood changes (that used to be: say something bad in the morning like 'please put your socks in the basket', get a whole day of non-communication), for not washing up, for doing nothing all day despite not pulling his weight himself. I used to care about it, but some day I decided no longer. One can be angry after a genuine argument but not a whole day. So I left any of those things unanswered, ignored them, shrugged my shoulders. If he used to really shout I used to say, 'I am not impressed, stop acting like a child. If 50% of this family (we have no children yet) does nothing, what do you expect?'
I once read somewhere that grown people are like children: praise them for good behaviour and ignore bad. I tried it for a while and he indeed calmed down. It also created some mirth on my side ;).
After a while things stopped. I believe he had climbed out of his depression then.
However, I think you need to stop and think. Running away is all fine, but do you want that? Is he really the whole time like this or only on working days?
He also clearly has an issue with his view of parents/relationships. You need to talk to him and make him understand that firstly snapping at such a trivial thing is not on, both his and your sake. How can he still be happy in general if he snaps at silly little things? If he understands this, he may start to wonder at the reason why he does this.
He also needs to understand that his children are not going to stop loving him because he is less in their lives than you. If he wants to be more in their daily lives, then that he takes another job. Otherwise, if he is insecure about his relationship with his children (why would he otherwise get upset at his 3-year old?), then organise some quality time with daddy every week. Maybe alternate evening putting DCs to bed? That might be an option. At least they won't get used to you alone.
Oh and quality time together. You can't form a family if you don't talk for 3 years.
I just wanted to offer a possible other explanation for his behaviour. They don't all do it out of maliciousness.