It sounds like he's trying to weasel out of his lie, now that he's busted. He told you he lied to you because of his last girlfriend's reaction when she caught him in the same lie? So it's... her fault that he lied about the same thing to you now? I mean, what sort of reaction was he expecting, when she caught him in a significant, ongoing lie?! His trying to play the victim is pretty icky.
Also, he "forgot" he was still married but apparently, he WAS reminded of that "small" fact by his ex-girlfriend. But then he... forgot again? LOL
So, getting a straight answer from him is like trying to nail jello to the wall. Does he often try to play stupid to escape responsibility for his mistreatment of others?
His behavior is highly dishonest and abnormal. How can you trust him with anything, when he's capable of such ongoing, serious deceit and doesn't think it matters? He is disrespecting you hugely, for one thing. You had the right to know if your partner was married to someone else and the right to decide for yourself if you wanted to live with or get deeply involved with someone who didn't feel like getting legally divorced. As we see here, it's really done a number on your trust. There could also have been serious financial repercussions for you, in various ways. He didn't think you had the right to know this and he wasn't even the one who told you about it when he let you post your couple info. on FB. How embarrassing.
Another issue is why he's lying. I'm guessing he is trying to get out of paying child support? Or he wants to pull his marriage license out of his back pocket as an excuse to not get married again, should he get cornered for real rather than just with distant plans?
People generally don't do strange things for no reason. So what's he getting out of this dishonest, "duh, I don't understand the basic rules everyone else goes by and expects" manipulating? And of course, what ELSE is he lying about? You can't trust a liar.
Otherwise, he just has a loose and convenient relationship with telling his partners the truth, or there is something psychologically wrong with him.
Also, not really the point here I guess, but I was a bit confused at your statement that you wouldn't have wanted to marry someone who had been married before. Well, yes, you SHOULD have had the right to decide from early on if you wanted to get involved with someone who was not only divorced but still legally married. But, you do know that his having children is way more involved for you than his having an ex-spouse, right? (Well, in his case, a current spouse, legally speaking).
Anyway, I suggest not letting him off the hook for this enormous, flaming hot red flag lie. I suggest you stop trying to have a baby with him and demand he attend couple counseling with you, if you're even going to stay with him now. Don't accept a guy who thinks it's no big deal to tell you serious, ongoing lies. It's obviously NOT a little nothing to you and I doubt it would be to anyone who had gotten serious about him.
HAS he told you other lies, by the way? You might want to talk to that ex-girlfriend again.
Best wishes