Not unreasonable at all. It's a common problem, because £100 is below most providers' home-delivery minimums.
Option 1: Let your brother sort it (probably the easiest)
Ask your brother to have some Rand ready when you arrive, and pay him back by bank transfer. There's no travelling, no collecting, and no cash to carry on the flight. If you transfer through Wise or Revolut, you'll get close to the real exchange rate.
Option 2: Use a cash machine in South Africa (usually the best rate)
If you have, or can get, a card with no foreign transaction fees, withdrawing Rand from an ATM on arrival normally beats any UK bureau. Cards to look at include Chase, Starling, Monzo, Wise, and Revolut. Check their current terms, because some cap free overseas withdrawals. There are ATMs at King Shaka Airport in Durban.
Two tips:
• If the machine or card terminal offers to charge you "in pounds" or "in GBP," always choose Rand. The "pounds" option uses a poor exchange rate.
• Use ATMs inside banks, shopping centres, or the airport, ideally in daylight. Don't accept help from anyone offering to assist at the machine. ATM scams are a known issue in South Africa.
Option 3: Click and collect in the UK
The Post Office and some supermarket travel money counters (Sainsbury's, Tesco, M&S) let you order online and collect in store, often at a better rate than walking in. Rand isn't always held in stock, so ordering ahead is worthwhile. If getting to a branch is difficult, it may be worth asking whether someone you trust can collect it for you. Check each provider's ID rules first.
With only £100 needed, Option 1 or 2 will likely save you the most hassle.
Have a wonderful trip to see your family!