OP, your husband doesn't sound very wise.
Unless you have a very safe job (NHS, armed forces, etc) I wouldn't place too much reliance on life insurance through work. What if one of you gets made redundant? Also, have you read the actual policies? Many have ambiguous clauses whereby the insurer may refuse to pay if too many colleagues die at the same time (eg an accident in the building).
You don't have kids now, but, ideally, you should get life insurance BEFORE you try to conceive. Don't wait till after kids are born. What if the husband were to die while you're pregnant, would you be able to cope financially?
Most people size life insurance so as to cover the two key expenses of housing costs and childcare. By 'cover' I mean the amount should let you pay for the house, pay for childcare and get back to work, or should allow you to take 1-2 years off work if that's what you would like to do, etc.
Happy thoughts, I know, but the way I see it is that the risk is very low, but happens and the consequences of that very low risk would be too catastrophic for a family to ignore. That's me - to each their own, of course.
Also be sure to write the policy in trust; it doesn't cost anything and most insurers can send you specific forms to arrange it. It means that, if you die, the payment is not considered part of your estate, but goes directly to a trust which distributes it according to your wishes. This avoids inheritance tax. By the way, it's perfectly legal, not some kind of dodgy trick.
Another thing which doesn't cost too much is to update wills and nominate guardians for your children should both parents die. This is never fully binding (eg if you appoint your sister but she gets arrested for drugs, no judge will give her your children...).
What may cost a bit is to get legal advice on what happens to your assets if you both die while your children are little. In these cases you probably want to set up a trust and appointed a trusted relative or friend to manage it for the benefit of your children, but legal advice on this can be expensive. Worth it if your assets, including insurance payouts, are, say, £1m+, probably not worth it, if they're £100k.