I don't think you are being unreasonable, but it may be worth chatting with the staff to form a plan for how best to settle him.
Could there be particular trigger points that he finds more difficult than others?
I found with DD that she found the time when all the kids were arriving and being left by their parents was not the best time for me to leave her.
For a good few weeks we used to turn up intentionally late so the staff were more able to focus on her when she needed them to.
I was also lucky as there was a kitchen with a hatch so I used to stand in there. When she wanted things (e.g. an apron for painting) she would come and ask me and I would encourage her to ask the staff instead. Things improved a lot once she realised that they would help her in the same way I did.
I also found that both DS and DD found it easier to be left if they were settled in an activity before I went. So I will even now (six months after starting) often hold DD's hand until she finds a table she want to be at. DS didn't need that for nearly as long, he soon twigged that there was always an adult-led craft table and used to always start his mornings there.
Do you trust the staff to call you? If you don't trust them then that is going to make it harder for you to leave. In the early days I used to stress that I was not going home but just to the shops next door so I could be back as soon as needed if they called.
Independence is a great skill for kids to learn. But I think learning it can be hindered in two ways - one by forcing it too early, and one by not allowing it to develop.
Good luck, sometimes parenting is hard.