Do DH and I need to make wills now we're having a baby?

[PirateWeasel]

This may be a dumb question. We have a joint mortgage and hardly anything in terms of assets, so we've never bothered making wills as we would both automatically get the other's share if one of us died. But now that we're having a baby, is it advisable to make a will in order to 'leave' the baby in the custody of specific people if anything happened to us? Or doesn't it work like that?

Yes, make a will. Always a good idea anyway.
You can name who you would like to be guardians, although this is not legally binding. But will be considered by social services etc. It is also good to put provision in for finances for those you have named as guardians.

Another thing to think about is what should happen to your estate if you were all to die at once.

Solicitors can sometimes advise you to state if one partner dies the other partner only has life interest in the property, so safeguarding the property for your children if the remaining partner remarries.

I don't think you can absolutely dictate who looks after your baby if both you and DH die, you can only put your wishes in the will and these will be taken into account when the authorities decide on the child's future.
One thing to consider which might encourage you to make a will is what will happen to your assets if all three of you die together? As it stands will the law favour that sibling you fell out with years ago and not that lovely supportive cousin for example?