You need to have joint finances if one of you has sacrificed income to stay at home with the baby - and then from those joint finances have a household budget which includes all joint costs.
So I'd sit down together, gather all forms of income you have into a spreadsheet/notepad (whatever's easiest) and work out how it all gets allocated in this order:
Anything to do with the household/survival which existed before you had DC comes out of this joint pot first - so rent/mortgage, council tax, utility bills, food shopping, car/transport costs and any bills which are needed for employment - e.g. if he needs to have a mobile phone or internet connection. Nb if it's not needed for work or survival, don't include in this category. (Internet, TV, phone, etc)
Then any debts or other costs that you have to pay e.g. child maintenance if he has other DC or contracts you can't get out of for now - allocate here. Plus anything like a course which is going to benefit all of you later.
Then you want to look at DC-related costs which is anything like childcare (it's a good idea to calculate the cost you're SAVING due to not needing childcare if you're at home, BTW!), any medical expenses, nappies/milk/food if not included with weekly shop, clothing, things like that. I know clothing can be a bit tricky to work out on a monthly basis so try calculating over 6 month period (ie, season) and dividing. Don't forget to later add shoes.
Next you want to decide together on a basic budget for family luxuries - so this is anything like internet, TV, music subscriptions used by all which give a high value-cost ratio or anything which is for DC benefit and you consider the minimum - e.g. swimming lessons, 1x sensory class per week, book/toy fund, whatever. And I'd include in this pot a fund to add up for Christmas/DC birthday - food and celebration costs, rather than gifts as such, but you might want to allocate an amount for the "big present" from mum + dad.
That's the extent of your absolutely needed categories. If there's not much left in the pot, then divide it up and that's your personal spending each for the week/month. If there's still a decent amount left it can be useful to allocate the following:
An amount to transfer into savings to cover emergencies and unexpected costs.
An amount to transfer into savings towards a specific thing e.g. holiday, house move, training course, next stage car seat.
An amount for more frivolous family luxuries e.g. eating out, more baby groups/activities, more expensive subscriptions e.g. TV package.
Then again whatever is left is divided for personal spending. Personal spending includes any subscriptions which are only for one person's benefit e.g. online game subscription, sports TV package, mobile phone contract (unless you have these included in the essentials), clothing for adults, socialising alone, cigarettes, etc, birthday/Christmas gifts for each other and then any extra activities or nice things that you want for yourself, depending on how much there is.
It should be equal - and it shouldn't be "asking him for an allowance" - but simply your own fair allocation of family money.
At the moment it sounds as though you're living as two separate people with two separate incomes and he's for some reason seeing the baby as your little personal project which he doesn't have much to do with - that doesn't really make sense, if you're living as a family. It's not a case of him doing some shopping every so often to take the burden off you, it's a case of combining your finances and working out what you need to cover jointly and what is left over for each of you at the end.