OP - First of all, I'm really sorry that you're going through this. However tough the advice is on here, it's also given from a compassionate viewpoint. It might seem like madness to you to ask him to leave when he hasn't actually done anything yet, but what you're basically describing is that he has fallen in love with someone else - and this is in fact far more dangerous than a lot of affairs.
I was around when you first posted your suspicions about this and urged you to get to the bottom of it. Our instincts about these things are very rarely wrong are they? Please don't blame yourself for opening up the can of worms - not speaking when you did would have just delayed this.
Because I remember your earlier thread, I also remember that the neighbour is in fact his best mate's DW - and that makes this even more problematical. I would think it impossible that she doesn't know how he feels and from your earlier stuff, it sounded to me as though she wasn't discouraging it. You must know her though and it could be you're right that he would have no chance with her - but that doesn't necessarily mean that she hasn't been giving out the opposite signals - some people get quite a kick out of getting people to fall in love with them.
From what I also remember, you've also got some past history of him and affairs, albeit a long time ago, so this further complicates the issue.
I understand completely your desire to keep the family intact and as a general point, hate the thought of children being messed around with parents leaving, coming back, then leaving again. I also understand that it is not that simple for you, as he is their primary carer. In an ideal world, I think it would be the best thing ever if you could tell him to leave. Taking control in these situations is very often the wake-up call people like your P need when they are behaving like lovesick teenagers, even more so when it is all just in their heads. But I do understand that for what ever reason, you can't do that - and so we must help you in the short term.
As a minimum, you do need re-invent yourself and for your own shattered self-esteem, you need to act far tougher than you may feel. You don't need us to tell you that acting like a doormat and a victim is unattractive - so you need to change your response to this.
Can you talk to her husband about this - after all, he's presumably your friend as well? Chances are, he's spotted what you have - even if he's tried to bury it somewhere. I'm sure he's not comfortable about this either, especially as they seem to get together when both you and he are at work. That might seem like a betrayal of your P, but tough measures are needed here.
Secondly, can you tell your P that he is staying because it suits you at the moment, with childcare etc. but that you have decided that until you can come to a workable arrangement where you can live separately, the relationship with him is OVER. That you will actively explore your own other options and will start living your life more independently. Since he appears to think nothing of going out and getting drunk without you, I want you to start booking some nights out of your own. Even if you have nothing in mind, I would even go out on my own if necessary - and tell him you went out with a crowd from work.
These small changes are manageable and allow you to regain some dignity and esteem without rocking the family boat - I sense with you that small steps are all you can manage at the moment.
Get some control back and you will soon get that self-esteem back. I'd love you to get to the point when you wake up and say "I'm worth more than this!" and finally gain the courage to bin this silly, cruel selfish man for good, but baby steps now....