Of course its normal to have extremes. I still have the odd day when I get upset for my DC or rant at the unfairness of it. When I think of living as I did with STBXH, it's infinitely better now.
You are at early days.
However bad it is now, keep in mind it will get better. You will come out of this stronger. Learn from your mistakes, why did you pick him as a partner, are you repeating mistakes that you saw your parents made (common), how can you be better at standing up for yourself etc
I doubt he has changed in 2 weeks. What have the problems been? I wouldnt take him back due to loneliness....that will pass. You will find other ways of having friendship, support and companionship. It will get better. Living with an unsupportive partner for the rest of your life? Thats only going to get worse.
Your choice...short term gain and long term pain (having him back) or vice versa?