In my case the girls just needed moving elsewhere within the class - they'd existed happily in the class room previous years.
It was part of a larger dismissive attitude towards my DD1 issues - sensory issues, reading and spelling issues all flagged up by pervious teachers being dismissed. Teacher tried to blame DD1 stress on our parenting - despite no changes and no previous issues.
Everything was dismissed till she lost it in school and it suddenly became an issue that couldn't be dismissed.
It is happening with DS now as well as my daughters but it took him maturing for it to be a more permeant thing - previously it got an instant negative reaction.
it seems to have had no ill effects on DD socially or educationally and I wonder whether that's because I shrugged it off as just being the way things work
I couldn’t shrug it off 
DD1 was having melt downs as soon as she step in the front door with two younger children it was extemelt difficult to deal with but – then it got so bad it was happen as soon as she left school and refusing to go into school and leave me.
That was first time I'd ever had to complain for DD1 - she mght not have been happy but she'd coped.
I didn't have to do anything with DS as his behaviour deteriorated so rapidly the teachers always sorted it immediately.
I suspect DD1 and DS cope with disruption now because they are older and, with help, are struggling a lot less with their own issues. They have more tolerance capacity - possible why DD2 has always been less bothered as she's not had her siblings issues.
If Op DS is getting to stage he doesn't want to go to school then I think it needs acting on now at very least approaching the teacher to raise concerns.