Something similar happened to me when DS was in Nursery. The system was that the children lined up, and then the classroom assistant would call each child out when she saw their parent/whoever was picking them up.
One day I arrived and DS was standing outside crying. Another mum was standing nearby; she had tried to comfort him but it was scaring him more, so she stepped back but watched to make sure he didn't run into the road or anything.
I took him back inside, calmed him down and then told the CA that I'd found him outside. She was horrified, apologised, checked he was ok etc. The Principal happened to be off that day, but the CA clearly told her what had happened, and she took me aside the next day, apologised again, and assured me that they would be reviewing the arrangements for home time to see if they needed to make changes.
So if I were you, I'd talk to the teacher, just calmly explain what happened, and see what they say. If you don't get a satisfactory answer, talk to the principal. Don't go in all guns blazing as you won't achieve anything extra and will just get off to a bad start. If they don't respond well, then you can eventually escalate to the board of governors or whatever, but start with sensible discussion.