OP, I do think that he is having an emotional affair, based on what you have described and agree with PP that what he said in his email 9 years ago, must have been hard to come back from.
My sister asked the other day what the difference between an EA and a friendship is, and I said if it is lied about, hidden, and your DP is seeking emotional comfort secretly elsewhere, then that is the difference.
If they were away, it is possible and likely there has been some physical interactions too.
I am 2 years on from my DHs EA. There were some mitigating circumstances, that were the only reason I stayed. I believe there was one physical interaction but he denies this. I think about it every single day. I had to get him to change all his passwords because I was living my life to check on him and was driving myself mad. There are still a lot of days where I want to leave. There are still a lot of days where I get anxious, and based on nothing, I start to think he is doing it again.
A physical affair is almost easier - it is cut and dry and less open to interpretation. An EA is gut wrenchingly hard, you doubt everything and your mind seeks ways to make sense of something that is out of your control.
I told my DH to leave me, why he with someone he couldn't talk to in the way he was her? He insists it was distraction and basic flattery and no substance, no real understanding. She just made him feel good. But between them, they have made me feel awful, exhausted and insecure and I can't see that changing any time soon.
I stay because we have a good life and my DS is happy and secure, and he takes care of me in a lot of ways, there were some mitigating and unusual circumstances (I posted on MN and most replies were to try and rebuild the marriage), but every day I have to hide how hard it can get at times. Part of me still thinks I will leave, even all this time on.
Please, take care of yourself and your children now. He has behaved appallingly.