Has he come back OP?
To answer the earlier question, yes, if a woman was regularly walking out, walking back, treating her DH like shit and their relationship and family as only there for when she wanted to play happy families, I'd say not to let her continue to walk back in when it suited her.
Your DH might have depression, which might explain why he's treating you like shit. That doesn't mean you have to put up with being treated like shit. I've been depressed, it does make it hard for you to see other people's problems, that doesn't mean you should be allowed to treat them as crappy as you like because you're mental health is the only thing that matters. What about the OP's mental health? Her DCs? Why does his being ill mean that no one else matters anymore?
Plus, just putting up with the situation is clearly not helping him, if it did, he'd be getting better. He's still doing it regularly, therefore just doing exactly the same isn't making him improve.
I stand by telling him he's not welcome to come back, that he's left you. If he doesn't want to stand by tht decision he made, then he can work at winning you back. That includes counselling for both of you, and possibly him separately, if he is depressed, going to the doctor and getting help, and him proving he's reliable before he comes back.
And I agree, why do you think there's nt another woman? He's going to another town and taking cash out, you might see his phone records, but that's only on the phone he tells you he has. You might have access to one of his e-mail accounts, that doesn't mean it's the only one he has. It's very very easy to do all the practical side of an affair, it's the time and explaining your physical disappearance that's the problem for most men, your DH doesn't have to worry about that, he just goes off for days at a time and there's no concequences for him, well - you shout a bit, then everything continues as normal.
He knows he can continue to do this to you because he knows all you'll do is get a bit upset, he'll say sorry, within 24 hours you'll be back to normal, and if he fancies just buggering off next weekend, he can do it again, and the one after, and the one after. It's completely fine - there are no negative concequences for his actions. He has decided what sort of marriage he wants, and you are letting him have it.