Lucie, don't allow him to play you for a fool! While you do so, he will continue to treat you as such.
In a healthy relationship, trust and a lack of jealousy are virtues, when your DH has overtly shacked up with an OW, they are the very aspects of your good nature that he will bank upon allowing him to get away with as much as he possibly can.
He is not treating you with any care or respect. Even in the unlikely event that 'there's nothing going on' with this woman, he has moved in with her and given your past concerns this is extremely mean minded at best!- At worst, I don't believe any one would do such a thing if they weren't getting something out of it, IYSWIM?
I fully understand the yearning for a partner when they have left, particularly in such a way but this is exactly where he receives the permission and power to do whatever the hell he wants.
Take the power back! As counting and other have said, garner support from friends and family, get out and about as much as you possibly can. You may not feel like it but in time it will help widen your perspective and independance and in doing that, you will become more desirable to him.
...If that's what you really want!
I chose not to take mine back and I can guarantee it's possible to heal and move on to bigger, brighter things, in time.
Oh, please don't go snooping through their windows! Retain your dignity wherever you possibly can.