Ok your posts are a wee bit convoluted so I'm going to try and break them down.
As I understand it, your real problem is that in your eyes, he overreacted to your mistake of forgetting to replace the card. This is further compounded by the fact that this all could have been avoided had he been responsive to one of your many money discussions, am I correct?
I would say let this go. You are both under a lot of pressure, prone to snap, and it's small in the big scheme of things. If you can, apologise for forgetting the card and tell him how woolly baby brain can be. (I'm on number 3 and barely have any brain function left). That is that solved.
However your posts flag up major issues. You have been together 2 years, are having an unexpected baby, and have just moved in together. You need money off him yet have put your tax credits into his account. Why put the tax credits into his account if you need the equivalent or more back anyway? I would sort this ASAP and get them back into your account.
I assume you made the decisions for you to give up work and move in together as a couple, so why was this not done in full e.g. Housekeeping arrangements. I would also question the wisdom of you giving up work instead of sticking it out until you qualified for maternity. But that is moot now.
You need money to raise your kids, to buy stuff for this baby, to run the household. And if any spare, to treat yourself. Has any of this been discussed? Do you each know how much you need?
You need with some urgency to figure out these sums, subtract what you get from CB, CTC and maintenance, and arrange a standing order from him into your account immediately. If he refuses, it would flag up major issues for me. Also have you done a joint benefits claim in case he can now get WTC or HB?
I am sorry you went into premature labour, this can't be an easy time for you or your family and you don't need the added stresses of money on top. Which is why I would suggest you sort it out with some urgency.
Apologies for the lengthy post.