Completely agree that your DH doesn't have to feed to bond.
But if you do want to express - rather than, as you say, do it now and again (which could mess up your supply a bit if you don't express to a set pattern), I would regularly take some off and freeze. It's usually easier to express in the morning than at night.
You would need a pump (there are some good hand pumps or electric pumps) and bottles/cup feeder. Steriliser - microwave or electric steam ones are good. Also some freezer storage bags for breastmilk.
And expressing can be a lot harder than bf - not un-doable - just that the baby efficiently draws milk from the breast whereas the pump pulls it from the breast in a very different way - so don't be upset if you find you get very much milk out with a pump.
Try not to express before 4-6 weeks so as to get an established supply.
If you decide to bottle feed some expressed milk occasionally, I would suggest doing it fairly regularly - I gave my dd a bottle of expressed a few weeks before I needed to go out and have my parents babysit for her. I thought - wow - that was easy but when it came to the night a couple of weeks later, she wouldn't go near the teat! I think it's less confusing if they see the bottle regularly or not at all 
When it comes to it, you may find you don't mind doing the bf yourself and that dh can help in other ways.