You absolutely 100% are not being unreasonable. You do NOT have to supplement, and it may not be in your baby's interests to do so. Before you do, get an opinion from your baby's consultant - my experience is that nurses and health visitors tend to cry formula at the first hint of a problem, and it isn't necessary.
kellymom and la leche league websites may be helpful to you. There is also an online paper called "just one bottle", which explains the changes that take place in baby's gut when supplementation with formula is introduced.
Also, there are usually breastfeeding focused solutions to any breastfeeding issues. Ask to see the infant feeding co-ordinator, who can check your baby's latch, don't be satisfied with a health visitor checking. before resorting to formula, supplementing with expressed breastmilk, or donor milk if available can (arguably should) be tried.
My experiende, and I'm outing myself completely here to anyone who knows me in real life (namechange after this!) - my third child was born with heart defects and "failed to thrive", i.e. she didn't gain the minimum expected amount according to the chart. She was using more calories as her heart was working harder, and was also not feeding as long as she needed because she didn't have the energy to finish a feed. I had a huge amount of pressure to supplement her with high calorie formula. A GP even told me he'd contact social services if he didn't, and the health visitor kept saying she could count on one hand the number of babies she'd been this concerned about.
Making me sound really irresponsible isn't it? Starving my child because of my own wish to breastfeed exclusively? Well, no. Biecause firstl;y, she's my third child. Neither my first or second child gained "enough" weight according to the charts. they dropped down several centile lines. My ds's weight even dropped below the bottom line. This made me suspect that it was nothing to do with her heart, or very little anyway. My other children's weight dropped, then followed one of the bottom centiles slowly, and this continued long after they were weaned and on solids. They're perfectly healthy. My ds eats like a horse!
I saw the infant feeding co-ordinator, who developed a plan - several steps that should be tried before formula - including lending me a hospital grade pump to feed her expressed milk using a supplemental feeding device (google it - it hangs round your neck so baby suckles your boob at the same time as receiving the expressed milk, so stimulating your milk supply, avoiding the problems with reduced supply that are associated with supplementation). next step - expressed milk through a feeding tube if she really didn't have the energy to breastfeed. etc etc. waved this in the face of anyone who mentioned the "f-word", and they did, often.
The cardiac consultant was pretty horrified that the local paeds, health visitors and nurses had put me under so much pressure. He was very supportive of my decision to breastfeed, and said that in his experience, artificially increasing the weight of a heart baby tends to make the heart problem worse, in that it increases the demands on the heart, and babies often go into heart failure when supplementation is started - it should be avoided if baby is otherwise well. Two cardiology consultants from leading hospitals in the UK have said that breastfeeding is probably a huge part of the reason she has not had a single infection, and has avoided major surgery. She is nearly 2, never had a drop of formula, and continues to breastfeed.
If your baby is otherwise well, showing all the signs of getting enough milk (6-8 wet nappies in a 24 hour period, poo should be mustardy yellow and seedy, 2-5 bowel movements in 24 hours. baby should seem satisfied after feeeding, feeding 8-12 X in 24 hours and feeds lasting roughly 20-40 mins)...also, feeding shouldn't be painful for you (if it is, or you have nipple damage, that suggests that baby's latch might need a bit of help which could lead to inadequate milk being taken in...so, if all these other signs are met, and you want to exclusively breastfeed, I wouldn't supplement in your shoes.
I hope some of that long ramble is helpful, but don't let anybody tell you that you have to supplement, because there are other ways.