The thing about marriage is that your relationship has legal status. You have made that relationship legal, which makes you have to unmake it legally as well. That means that if you split up, everything you both own goes into one pot and gets divvied up fairly.
If you aren't married, everything could tick along nicely for years, and then one day it all comes to an end unexpectedly and you have absolutely no entitlement to any savings or assets he's been able to accumulate while you've been doing unpaid labour for the benefit of the whole family such as childcare and housework.
That's why marriage is usually in a woman's financial interests, unless she happens to be much better off than the man.
I would, however, advise that when you're able to (probably not during maternity leave) you keep enough money in the joint account to cover all your family expenses and then divide whatever is left between you for personal spending and savings. Because if you do want to get a divorce at some point and need to pay a lawyer, the fact that you technically own half of whatever he has will not help you exercise your rights if you don't actually have any money of your own, saved in a bank account he doesn't have access to.
This isn't what you want to be thinking of when you're happy together and having a baby, but it's how you prevent yourself from becoming THAT woman who posts on Mumsnet in ten years' time saying her partner is cheating on her and she's been on maternity leave and working part time and the house is in his name and she has no money of her own.
Planning ahead like this does you no harm at all, and hopefully you'll never need to fall back on it.