Ach, x-post again!
I agree to be wary of the apologies.
Maybe he just had a wobble, and everything really is fine.
Or maybe he was just momentarily brave enough to say what he really feels and is now panicking and backtracking, but his doubts are still there.
You need to have a big talk. I would go into it with the approach of: okay, this is our chance to really be honest and share all our problems and concerns, to be brutally honest, and then at the end see whether we can make it work.
Don't let him get away with just 'I'm sorry, I didn't mean it.' Something made him feel that way. Maybe he's frustrated that you don't go out anymore, that you are both too stressed out with things, who knows? But if you are going to get past this, you need to really address what's going on.
And while it's great that you're realising you need to make some changes if you stay together, don't put it all on yourself -- he will need to make some changes too, learn how to communicate better, manage stress better, so he is never this cruel to you again.