South African troops didn't serve as part of the British Army. They served as the South African army and indeed navy and air force.
Of course individuals also volunteered to serve in British forces – including the RAF during the Battle of Britain, as posters here know full well from their frantic googling.
South Africa was self-governing in 1939 and declared war on Germany on its own behalf on 6 Sep 1939, three days after the UK did. South African troops were all volunteers, not "called up" even by South Africa, never mind by the UK.
en.wikipedia.org/wiki/South_African_Army
The South African forces were mobilised in mid-1940, ie before and during the Battle of Britain. In practice, their engagements were against German-allied Italian troops, because that's who was in East and North Africa at the time (first action I can see was 11 Jun 1940, bombing raid against Italian troops). The German Afrika Korps arrived in Feb 1941.
So if you want to make some incredibly narrow claim that "the only troops who fought against troops who were under the Nazi swastika during the British version of the dates for the Battle of Britain, 10 July – 31 October 1940, were fighting under the Union Flag," then it's possible you may be able to do so (I don't know about other forces and engagements).
If you're trying to claim "Britain stood alone"... Nope, and seriously insulting to those who volunteered.