Sorry op, not sure where i got 9 years from! But I think it still holds true - that at this age they are not professionals who should have to put up with pain for the sake of the game.
If it was the ref that insisted on having the shirt then that is absolutely dreadful. I'd be putting in a formal complaint very rapidly because if he demanded a boy with a broken wrist have his own shirt taken off for somebody else in the game to wear despite causing agony and potentially making a bad injury even worse, then he is not fit to be refereeing. And the rules should allow for this sort of situation - that a normal shirt can be worn. I'd also be upset with the coaches for not standing up to the ref.
What if next week it's somebody with a neck injury that he demands a shirt from - his actions could cause a kid to be paralysed.
Just because your ds didn't need much manipulation before having a cast on - the ref didn't know that at the time! He could see he was causing intense agony. Depending on how bad the break was, he could have been causing damage to blood vessels or muscles internally - he couldn't tell in advance that he wouldn't.
Although your dh was there, he may well have been in shock too as others have said. But if it was the ref that was demanding the shirt or the game be stopped and there were the other coaches and parents around he may have felt intimidated or scared to say no because he didn't want o be the one to cause the match to be forfeited and face the wrath of the other parents, players, coaches and ref. if there are several people who normally have authority in a situation (ie coaches and the ref) and they carry on barking out orders, if you're a parent/player who is used to doing what you are told by these people, then it is very difficult to step up and say 'stop', especially when you think these people should be putting your son's welfare first, you don't expect hem to put the game above a child's injury!
Hope your son's wrist heal's well and doesn't give him too much grief!