We are the same age. My concerns would be the ones that you are batting away.
What happens if you do want to stay at home? It might be unlikely - To be honest, I love my job, and I can't imagine giving it up - but you'd have no option. That would breed resentment. You need a plan for that.
What if you don't want to stay home, but he doesn't feel able to be a dad? It doesn't sound like that is something he'd be able to get over, so you'd need a new plan. He'd have no way to make enough to either cover your wage so you could be at home, or cover childcare - potentially you could end up with your child in full-time childcare, which neither of you want, and working really hard to pay for that because you've got no options.
What happens when the child starts school? He'll be five years older, and he'll have been out of employment for five years. Just like women who have children to give them purpose, that is a tough time. He might want more children to fill the void, or he might crash into depression, or he might be totally lost.
You obviously need to work within his limitations, and both of your goals, but I think he does need a stable career plan - even if that's a non-standard, not-too-ambitious career. It needs to be something with potential, that he feels he can do.
Could he work in childcare for a bit, starting imminently? Then he'd both learn skills that would help when you have your child, and help boost his own confidence, and be making money.
It's a bad time for you to take a career break, and your career is vital, at the moment. You're also not ready and that automatically vetoes his readiness - he will still be ready, he needs to patiently wait for you with no pressure.
I have a pretty close female friend who had children in her mid-20s because she was broody and wanted a family. She kept having children, and although she's a wonderful mum, she admits now that she didn't have anything else to do or go back too, and children gave her an excuse to socialise and do something - NCT classes, toddler groups, school. She felt worthwhile. She hasn't really managed to work since - she lacks confidence, she's got no real qualifications, her A levels are too long ago to matter. She's beaten to waitressing/shop/bar staff jobs by people younger than her, without childcare responsibilities. Luckily she's still with the dad and he earns very well, so she doesn't need to work, but shes unfulfilled and as her children have become more independent, her confidence has waned even further. She's anxious about everything. She doesn't even really drive anymore, because she doesn't think she does it well enough.
I think you've got a lot of contingency plans to make. I understand that you have a plan, and I hope it works out, but you need plans for unexpected things, too.