If at all possible, keep trying to build your comfort level with breastfeeding and your supply for right now.
You might try feeding him more often, but for a shorter duration.....say every two to three hours at most for about half an hour. Put him on sooner if he shows any hunger cues. The frequency can usually help with supply and also give both of you more practice with feeding. At times, you will feel like you are wearing your baby, but this is what is expected in the early weeks of breastfeeding.
Over the next week, while DH is home, try the more frequent schedule. You will likely find things are more comfortable, you feel more confident, and there is a bit more of a routine.
If you definitely want to supplement some feeds, consider a finger feeder. This will help lessen the possibility of a nipple preference. If you decide to go with a bottle, use rhythm bottling. Hold him a bit more upright and the bottle a bit more horizontal. Let him such 5-6 times and remove the bottle, suck 5-6 times, remove, and repeat the pattern. It can help to pump for a bit after the feeds that he is on the breast, and if possible a few times when you are not nursing.
As redorwhite said, the early introduction may effect supply so it will be something you will just want to keep watch on. If you suspect a diminishing amount, try to up the number of breastfeeds for a few days.
As he gets a little older, he will likely become more efficient at moving the milk and his time at each feed will improve....at that point you may find you start breastfeeding more often, and maybe start doing some expressing for the occasional bottle of EBM for when you are out. There are a variety of ways women manage to continue with some breastfeeding if they choose not to do so exclusively.
Your body may surprise you at what it can adjust to, but you will have to pay attention, and seek support if need be.