Shopaholic, I knew it was you before opening the thread!. I would not condemn you for having an affair with a married man (actually I'm happy you are feeling so happy at the moment and won't say anything about the effects in his family as I'm sure many will comment on that) BUT, even though I have never had an affair (so no real experience), I believe this can only be safely done (apart from safe sex, etc.) IF:
a) If you realise and accept that you will be playing a secondary role in his life. I believe that if you try to change that you may a) be successful or b)you are more likely to get extremely attached to him (not in a nice way) by the frustration that being the second runner up causes you.
b) If you are prepared to be realistic. At the moment, the fact that you are having this affair may make you believe that you were meant for each other, that you should never have parted, etc. but keep in mind that no matter how nice he is to you, this is a man capable to betray his pregnant wife, and may only be taken advantage of the fact that you are not happy in your marriage, you are an old girlfriend who HE left, and may consider you a nice entertainment to give some adventure to his life, the fact that you are married gives him the confidence that you are going to be discrete and you are not going to end up boiling rabitts because you also have a family that can be seriously affected.
on the bright side, happiness now is part of a package which includes sadness, that's what you are accepting, which is fine if you plan to spend years together, but how would he react when his baby's born? would he stop contact with you because he wants to insure the best for this little being? If this happiness is going to last so little, isn't it a hugely expensive price to pay?
Good luck, and even if he was your first love, don't fall in love again as a teenager, however tempting it may be.