It is such an incredibly horrible feeling - so confusing and scary as well as hurting more than you'd think was possible :(
I am full of empowerment one minute, then full of self-doubt the next
That is completely normal. Exhausting & confusing, but normal.
Of course he wants thing to 'go back to normal' wifey at home looking after the house and the kids, him acting like a Dad when he fancies it and like a single bloke when he fancies that.
Well, Breaking News this is NOT about what he wants, if that is what he wanted so badly he should have thought about that before fucking around with someone else shouldn't he. It is now tough bloody luck what he wants.
How is christs name does he think you can go back to how it was before you had children? Does he not think 'having children changes your life forever' applies to him??
If he wanted to put things right and resolve things - he should have done that before he had an affair - instead of acting like a complete idiot.
He says he has been stupid
... yes, we'd worked that out 
He has not said he is sorry, for anything. He does not acknowledge or ask about how I am feeling
There is no way on this earth that he has what it takes to rebuild your relationship from the ground up. You are right, he wants a couple of conversations about it, to look a bit hangdog and then have it brushed under the rug. Fuck that.
I told him yesterday twice on the phone and again this morning in an email that I needed a few weeks space and that if he had any respect for me and the marriage, he would give me this time. This morning he said he wants to come home. And now again, the same thing
This tells you everything you need to know about how much he would listen & how much effort he would put into rebuilding your marriage. NONE.
I will be living my life obsessing for the next 20 years who he is with; checking his text messages etc. The trust has gone
There is one poster on here who will tell you that her relationship is better now, post affair, because they rebuilt their marriage and 'started again' but her DH was a completely different kind of person to yours & she is the only person I know, IRL & on MN, that has had a happy life post affair, everyone else I know is living the life you describe above - then generally still separating a few years later (which is what I did).
I feel strongly that he needs to leave me be for a few weeks to think. He wants to come straight home. I feel like saying that he needs to give the space I ask for a d that if he forces the issue, I will go straight to divorce
^ Is exactly the right thing to do. Just do it x