OP this is a very sad read. Not because he had 400 or 600 partners in the past, but all of it put together. We all have a past. Lets imagine he has been sexually active for 35 years. 600 women over 35 years, that is around 17 partners a year. In itself I don't feel horrified by that number , because I essentially believe if you have two consenting partners and no one else to hurt, what does it matter if you are sleeping with so many people? But add in times he has spent in relationships, thats a preetty high rate of cheating (and you know he has cheated in the past).
On top of that he keeps some kind of weird log of all the women he has slept with. And he tells you about it. You have to wonder why that is. Then you say you haven't been sexually active for a year (co-incidently must have happened while you were pregnant or had a small baby). When did he start cheating on his ex wife, do you know? Was it while she was pregnant/ had children?
You are not nagging him, you are trying to find out where you stand. I feel for you, I can't imagine what your thought processes are. It sounds to me like he loves the thrill of something new, and perhaps he thought he had found the one person he could be faithful to for the rest of his life. But leopards don't change their spots that much. To go from being a serial shagger to not having sex with his own much younger (and I bet you are gorgeous) wife for a year? Nah. Doesn't add up. He is giving you information so you can't see what he is doing, but I would put money on him not having given up sex all together.
By the way, you are entitled to have insecurities based upon his past, especially as he seems to be showing such little affection for you at the moment, no wonder you are questioning everything. You are very young still, you have your whole life ahead of you and you deserve to be cherished with affection going forward.
I have been in a relationship with a guy who I know has probably had a lot of sexual partners (I have never asked him about his past and vice versa), and never settled down for more than a few years in the past. We are the same age, I was in a relationship from the age of 21-38, and although it did cross my mind at the start that he may have difficulty being with one person, I know he is happy to be with me and not elsewhere. We have a great sex life, I trust him. I know he cares about me deeply, and I know that he feels safe and secure and we have a lovely relationship. If the sex suddenly dried up and stayed dried up, the first thing I think I would think about would be, had he had his head turned elsewhere?
Have a look at the madonna whore complex. en.wikipedia.org/wiki/Madonna%E2%80%93whore_complex