I think whatever was going on with your neighbour was disrespectful to your husband. The grass is not greener, you see this man he is a blank page that your fantasies will let you fill however you like and your mind is projecting qualities in him that he may not even have because you haven't lived or had a child with him. They are delusions and false impressions.
Nobody in the world can give you everything just right, perfect sex, perfect therapist, perfect housekeeper, perfect financial security. It's very unrealistic. You had been suffering with mental illness which can be draining for a partner to live with. If you had very little energy, he may have chose to not burden you with his struggles leading to distance and lack of communication. Men are generally not like women, they generally are not equipped to sit and listen, they want a solution and a resolution. Whether this is is cultural or genetic, it doesn't matter. Most men just aren't going to be like a therapist or a girlfriend.
I think it would be good to take responsibility and acknowledge your part in this marriage, relationship troubles are hardly ever a 100% on one party. If you are financially blessed, get some help with the chores he would have done. Go to a therapist. Go and do some hobbies so that you are not relying on your husband to be your entire world, the therapist, the entertainer, the housekeeper, the provider... It's too much on one person's shoulders.
Your child is very young still and your thinking is clouded by mental ill health, so I wouldn't break up a family over this. When couples celebrate 30+ years of marriage, do you think it was all butterflies, passionate sex and equal housework? Brenee Brown once said that it's not 50-50, it's between him and her they are a 100%. Some days one partner can only give 10% the other picks up 90% for the team. Don't break up your marriage and have your children ferried up back and fourth between 2 lesser off homes for the sake of fanny gallops, laundry and some romcom idea of emotional support. Try to see things from his point of view, working his arse off while his Mrs flirts with the neighbour, comes home after a hard days work to a heavy atmosphere at home, wife is always down, irritable or moody. If your medication or therapy isn't helping, focus on sorting that out. Even with a new man and a new relationship, it will eventually succumb to monotony, there is no magical ever after. Be realistic and appreciative of what he does bring to the table and take responsibility for your part in letting things get stale between you.