It's just as much his baby as yours unless it was a virgin birth therefore he should contribute equally to the care of the baby, whether that is via sharing the cost of childcare bills, baby's food, clothing, or caring for baby at home on his own during working hours.
One approach is to work out the cost of caring for the baby and divide it in two, then look at how much each of you is contributing. You can also add in household bills if contributions to household bills are part of the equation. If you are staying at home, you are contributing "in kind." A reasonable way to find a cash value for your contribution is to look at what would be the equivalent cost in nanny care (childcare places are not an equivalent cost as childcare places are caring for your child along with other children so it's not an apples:apples comparison; also you have to transport a child to a childcare business, adding inconvenience to you and might affect your job; a nanny comes to your home and saves you the time of the commute to childcare, as well as you don't have to get the child ready to leave, transport diapers and formula etc, which is actually a HUUUUUUUUUUGE benefit in terms of your ability to get to your own job on time).
If you are working and baby is in childcare, then you are contributing in money given to the childcare place. If you are on maternity leave caring for the baby (without compensation from him), and are ALSO paying half of the other household bills, then you are actually contributing more than your fair share.
Just work it out on paper and see how it looks, then decide how to approach him. Please don't give up your job, this does not sound like someone who will treat you fairly.