Hi, OP, Congratulations on your pregnancy 
I'm a will writer ( I have a paid -for advert over on the Small Business section of Classifieds).
So far as life cover is concerned, you need to check your current life policies and decide whether you would still be adequately covered; if not, do some shopping around to get the best deal.
You should appoint one another, with your children in default, on the life insurance documents. Life ins. is held under a separate trust from your estate in your will.
If you decide to go ahead and make wills, you don't need anything so far as your guardian has concerned as he's already indicated that he will act.
You do, however, need to decide who to appoint as your executors. You'll probably want to appoint one another on the first death, but you also need to name two other people who would act on the second death. They will usually also act as trustees if a trust arises, eg if you both went before your youngest child was 18.
Ideally, the executors should not be the same people as the guardian.
You may also want to think about whether to leave specific items (eg jewellery) or cash legacies, or just deal with the bulk of your estate, which is called your residuary estate.
If you would like any further info, have a look at my advert, which is titled 5*Will Writing Service Recommended by Mumsnetters, for more details.