OP, I took my quiet boy out when after a month he was still crying and didn’t settle the whole time he was there. DS2 was only a few months old and I just think it was all too much for Ds1 as it was a church playgroup type of environment where it was mostly free play. I think it was too unstructured for him and he just felt lost. He likes routine and the reassurance of a leader saying “ok, children, today we’re going to learn about ”, not someone saying “what would you like to play with?” And leading him over to the cars or whatever. He was a similar age to your son.
I left it till he was 3 and switched him to a more structured nursery, with more educational activities and it suited him so much more. People go on about kids learning through play but with the first playgroup, different play areas set out the same every day in a big church hall, with some paints in another room, just didn’t suit my son at that age.
The new nursery had a curriculum. They did topic stuff, so when it was autumn, say, they would do play-based activities based around that for a few weeks. So collecting leaves, doing leaf prints, visiting the greengrocer to look for a pumpkin, singing songs about fireworks and bonfire night, doing little cookery classes based around the theme. It was so good and I still miss it. God, they did all sorts. Snail racing was memorable! Making and eating different types of bread. Learning about Divali and making little candle holders out of clay and decorating them for that. They generally did more free playbstuff in the afternoon but the mornings were quite guided and he thrived in that kind of “school” environment. He was older of course, but I just think the structured environment suited him better. He is quite intelligent so maybe somewhere with more stimulation was better for him (current excelling at grammar school so definitely didn’t do him any harm anyway!)
So check that it’s the right set up for him, and if your instinct says take him out then do it. It was the right thing to do for DS. He just wasn’t quite ready for it when I tried him. At 3 he was much better. Took him till the second week to not get upset when I left him though.