Ignoring the fine details...
Life is not flowing easily, you have boring adult shit taking up time - work, child, running a home - this makes having the exciting, close, adventurous etc etc lifestyle you had X years ago much much harder to achieve.
He feels like something is missing/lacking - can't be you/him, you're both still there, must be the addition of other people.
It isn't and he is wrong - you need to invest more in your life together, before making time for other people (if you both want that).
The more either of you invest in other people, extra to your relationship, the WORSE real, day to day life, and your intimacy with one another will get.
I see people do this all the time, whether its having affairs, open relationships, someone investing all their time in computer games (fantasy life, so much easier than real life), hobbies ... it all amounts to the same thing, the more you put in to something else, the less there is left for you + partner in real life, the worse real life becomes in comparison.... round and round.
Add in an autistic person who fixates on an idea being THE solution to a problem and it gets so much worse (I have the Tshirt for that one!) as then they get upset and uncomfortable when someone tries to point out it isn't the solution.