Well written Bavarian
There seems to be a gap between the Op's expectations of herself and her ds, and what she expects of other people.
When I heard him crying for the first time, I disregarded it as I thought that he fell in the grass by accident:
Surely 1. you were watching a 3yo enough to see whether he fell over. 2. I've never known a parent just assume a child who is crying has fallen over and doesn't need to be comforted. Surely if you see your dc crying you go and check they're okay the first time?
I told him that we should go somewhere else, but he ran after them only to hear him cry hysterically again and again. Surely if your dc has been told they're not wanted the last thing you let them do is let them follow again and again crying. I can't imagine any child is going to turn round and say "actually he's bothering us so much we better let him play"-or adult for that matter.
You had a choice at that point, take him away and distract him, or talk to the children nicely, make friends with them and then maybe ask if your ds can join in for a short time. And then you stay and encourage their game.
You were setting him up for a fall, if you thought him continuing to follow was going to do anything other than upset him further.
You didn't get your 3yo to do as you wanted-but you expected other children to behave as you wanted.
Other parents did not react at all well you didn't react to your own dc crying "hysterically again and again", so why should other parents.
I tried to explain to the girl that maybe he didn't understand them because he was smaller and perhaps had some language issues why didn't you explain to him then? If language is the issue (although he obviously understood enough that they didn't want to play) why didn't you explain in his first language rather than leave it for them to explain. 4yos are not subtle, they won't sugar coat that they don't want to play.
I don't know if this is normal and if children can be so malicious as had been said the most malicious one here was the OP. As I said before 4yos are not subtle and can be nasty, but it's not malicious particularly not letting an unknown child play.
If it was my boy, I would clearly instruct him to let other children play with him as well, regardless if they are part of his group or not. I take it you don't have older ones? So if your ds when he's older is doing a jigsaw with a friend, and a toddler comes up and wants to play by pulling the pieces apart, you will tell him to play? Or when he's 6 or 7 playing football with friends and a toddler comes up, you will expect them to stop kicking it hard and pass gently to the toddler? Or when he's showing a friend that he can play "twinkle twinkle" on the piano and the toddler joins in by bashing the keys so hard your ds can't be heard-all fine? If your ds is drawing a picture, you're happy for a younger one to scribble over it because they want to join in?
The thing is 4-5yos are beginning to get more complicated games. A 3yo boy, with some language barriers, is probably not going to get their game entirely, and may well spoil it. I was always very grateful to an older child who allowed by toddler to play, but certainly wouldn't have expected it.
I remember dd1 and dd2 at that age had very complicated games that only they and the friend(s) they were playing with understood. They couldn't have explained the rules at all because it was something that had just evolved into (to them) a hugely entertaining and hilarious game. (but totally baffing to me)