Feeling it. I don't know your situation or anything about your relationship, obviously, but you might want to give some thought about why you seem to have taken responsibility for his affair. I think this might need challenging, you see.
I understand why it happens - unfortunately you even see people writing on here with their beliefs that betrayed partners are somehow to blame for adultery and more still that peddle the belief that affairs only happen in unhappy marriages. I think so many people still think this - and they have a huge investment in clinging to those beliefs because then they can also believe infidelity will never happen to them. That they can "control" whether their partners are unfaithful or not. I was that woman, so I understand it.
When it actually happens to us, amidst all the shock and realisation that we weren't in fact able to control what our husbands did, there is a need to find a reason for why this happened. In doing so, we are hoping that if we somehow behaved differently in future, we can control it happening again. This is, I suspect where you're at. You're finding comfort that you didn't respond enthusiastically to "date nights" and so if you put more effort in now, you will somehow be able to stop infidelity happening again.
But the problem with this is that it doesn't address his behaviour and his choices. Adultery is actually a pretty extreme way of expressing dissatisfaction with a relationship. Does it seem to your logical, rational self an adult, balanced way of expressing discontent? Did it occur to you before all this that he might have an affair? I'll bet it didn't.
One of the reasons I believe my H and I have been able to rebuild was that he never once blamed me or problems in our relationship for his infidelity. He saw this as a problem in him and therefore took himself off to counselling to get to the bottom of his behaviour, which was so inconsistent with the values he'd always cherished. In the final analysis, he told me that he would never be able to do it himself again, let alone me. He set about a difficult journey understanding how his previous traits of emotional retardation, low emotional intelligence and selfishness were in fact the perfect breeding ground for infidelity.
In a nutshell, the changes he made to himself were the most effective methods of preventing his further infidelity. If you've read Not Just Friends, you will see that Shirley Glass helpfully challenges the notion that betrayers weren't "getting enough" from their primary partner and asserts that in reality, they weren't "giving enough" to the relationship.
When my husband had his affair, it would never have occurred to him that he wasn't giving enough, but neither did he think he wasn't getting enough from our relationship. His internal "permission-giving" process for having an affair was therefore not that our marriage was lacking - in fact his justifications were pretty puerile - an escape from the stress he was undergoing at work, an ego boost and an adventure he'd never had before. Even then, it took him 10 months to give himself permission to do this and in order for that permission to be given, it was necessary for him to distance himself from me to create anything like a justification. He freely admits that when the friendship started with OW, we were in such a good place in our marriage that infidelity would have been unthinkable. He therefore needed to weaken our connection in order to do this, but he wasn't aware at the time that this was what he was doing.
It is natural for us betrayed to look for faults in ourselves and the marriage - and despite what I've said above, I can see how our marriage is infinitely better and richer than it was pre-affair. But we can both see now with absolute clarity that as soon as my H started giving more - specifically emotional honesty and selflessness, I responded in kind.
You tell us that things were wonderful before he went into therapy and met OW. Trust that memory. Yes, I do think your H is re-writing history and is trying to evade responsibility for what were ultimately his choices. A really good counsellor (Oh, if only you were seeing Shirley Glass or Frank Pittman) would be able to see through what he is doing and confront his behaviour. The counsellor is colluding with this and so yes, I do think you need to ask the counsellor how much experience they have of infidelity therapy and also what their own views are on why affairs happen. If you get any sense that they lack proper experience or have beliefs that affairs are always a sign that a relationship is not working, then find another counsellor.
If you don't believe me, have a look at the thread about women having affairs - notice the justifications they give for why they are being unfaithful. Not one of them is introspective enough to wonder whether they are somehow lacking in themselves to choose behaviour that they fundamentally believe to be wrong and destructive. It's much safer - and less challenging - to believe that they aren't "getting enough" from their primary relationships, when I would rather they question any narcissistic traits they might have, what they haven't been "giving" and indeed, why they haven't left their relationships before embarking on their affairs.
Then consider the people we all know who are repeatedly unfaithful, within the same relationship, or in successive ones. Is it really plausible that in all their relationships, they weren't getting enough from their partner and this led them to seek comfort from elsewhere. Or is it more plausible that there is a weakness in them and that this is, in fact, the common denominator? The problem is with the infidel, not their partners. Until they address this, they will go on repeating the same mistakes.
Your H is the one who has behaved destructively. Ignoring such an extreme behaviour choice as infidelity will not help you move on as a couple.