kirsty is it a mainstream nursery attached to a school? Or a private nursery?
I am glad I sent my ds to nursery, it gave him 2 years (we are in Scotland) to adjust to school, and two years for them to get to know him and get him the support he needed. As well as letting him socialise with other children (okay it was mainly parallel play, but the children got to know ds too), but he did interact eventually.
My ds got really great support, school applied for a one to one assistant, he also saw early years support teacher twice a week. I got a break, two hours but it helped. Ds has just started at the school his nursery was attached to - the transition to school has gone well.
If you are thinking of HE then you don't need to take your ds to nursery, but if you want your ds to go to school then it is worth persevering with nursery... ds only went 3 times a week to start with.
My ds was a screamer, it was very hard when he was three, to leave him, he was vulnerable, but I think I did the right thing. He is now so different, it has taken two (long) years but he is a very happy boy most of the time.