dementiawidow, with every gentleness intended... do you think you might be looking for something to excuse the righteous anger and despair that you feel about your husband's illness? The fact that he has, in effect, left you and you can no longer reach him? Can't talk and have him listen effectively? Forego your 'relationship' for all the things that it was, good and bad?
He hasn't left you, but he's not staying with you either. That would be very tough for me without anything else going on.
With respect, what your husband ex-wife says is really not relevant to you because she is not you and you are not she - and your husband isn't the man he was when he was married to her. He is now married to you and whilst 'dalliances' may not have constituted an affair in his mind when he was with his wife, they certainly meant something different when he met and married you.
What I'm trying (very clumsily) to say is that you are comparing apples and pears.
I mean that, all the mistakes that I've made in my current marriage, if I were to meet and marry somebody else, I would present my best side and I would tell myself that I have learned from the mistakes that I made and I'm not that person anymore - so I wouldn't necessarily tell my 'new husband' that I used to be this or did that, because it's just wouldn't be relevant to me. Your husband may also be of the same view, ie. "Ok, from this point forward this is how it is...". And it was, wasn't it? He was absolutely faithful to you and still is and will always be.
You have enough going on with his dementia. Make no quick decisions because you would dwell on them and may regret them later on. Try to isolate the nub of the matter and forgive him for it, because there really is nothing else you can do. If, as his illness progresses, you want to leave, well, there will be nothing stopping you as he probably won't even register it or be able to.
There is dementia in my family history and it's truly terrifying. I wish you peace and eventual happiness, I truly do.