My sister's literacy levels were above average at primary school. Her spelling was good and she read all the time, she even wrote girly stories about air hostesses and princesses. On arriving at secondary school, she was above average in literacy. But she was denied music lessons which she was very keen on, instead she was herded into a remedial group for extra literacy lessons with kids with low literacy levels. She couldn't understand why .
In this remedial class, despite her upset at being there, she excelled getting full marks or thereabouts on spelling tests. The teacher even excluded her from the tests as she found them too easy. He'd let her read, even letting her help the other kids spell. These lessons took place in a room with big windows so all the kids could see them and the teacher let this class run over into the morning break, which gave the other kids a chance to gather around and take the piss. The teacher often told her he couldn't understand why she'd been put in this class. But nothing happened 'till she told me about it.
I told Mum that she was really unhappy, Mum went to the school, saw the head who summoned the teacher who told them it's al been a mistake, that her name should never have been put on a list of kids that needed extra learning in the first place, as if anything, she was much brighter than the average. She was taken out of this class. But she carried the stigma, it damaged her. When she'd do well in the other classes the kids would enjoy taking her down a peg by taking the piss out of her being in this remedial group. She felt from then on, it didn't matter how well you did, or how bright you were, things like this happened. She felt unlucky, I suppose semi-depressed and became disruptive. She pulled it around and did well academically she even went on to uni'.
So, yes do something OP, intervene the quicker the better IMO.