Hi thisisyesterday.
Dh and I went through all the feelings/thoughs you describe whilst trying to decide whether or not to go for assessment with ds1.
I was exactly the same, as although I 'knew' there was something wrong, I was also sure that whatever it was, it wasn't severe and maybe we should just try our best to support him rather than put him through an assessment process.
He too coped well in infants and academically he was fine, with just the odd wobble in September when he had to settle into a new classroom and get used to a new teacher etc.
Like you, we were worried about him being 'labelled', and wondered if he was better off just being thought of as quirky, especially as he seemed to be coping well at school.
Eventually I posted about our dilemma and got some wonderful advice from the lovely people on here and in the end decided to go for the assessment.
What clinched it for us was that he started coping less and less as his class was being prepared for going into Juniors (so at the end of year 2) and then really struggled when he went into year 3 and actually entered the Juniors . At that point it became obvious that he was going to need extra help to cope with the level of independence that is expected of them in Junior school compared to the infants. He also found it hard to cope with going from one dedicated class teacher to having 8 different members of staff teaching him at different times over the course of a week. (I have since found out that a high proportion of assessments are done on children when they go into Juniors at around 7/8 years old.)
He doesn't need a lot of extra help, but definitely needs support around organising himself, his books, sports kit and knowing where he should be and when etc. We did think that he might get used to all that eventually and settle into it, but realised that if he didn't/couldn't there was absolutely no was he would cope in senior school without a formal diagnosis and proper support plan.
Once we had decided to go for the assessment, we asked to see the gp at our surgery who specialises in paediatrics and went to talk to her without ds to discuss our concerns. She was lovely and agreed to refer him to a paediatrician. She asked us to prepare a briefing letter for her to send along with the referral request, detailing a bit about ds's history and what our main concerns were. We were seen less than 6 weeks later.
We saw the paediatrician with ds in January, he spent half an hour reading our briefing notes, then sat and chatted with us for about two hours, at the end of which he gave us a verbal dx of Aspergers (basically he said that he felt a dignosis of Aspergers would be appropriate, but that his feeling was that ds is extremely high functioning and only just on the spectrum, if that makes sense) and referred us on to the local multi-disciplinary ASD assessment team for formal assessment. We have just received a letter telling us he will be seen in 12 months time. He also had us do the CONNERS questionnaire to rule out ADHD (although he said he was pretty much 100% sure ds doesn't have it) and raised the question of dyspraxia, which is something we hadn't considered and didn't fully understand. Since researching it further though, we are now aware that the end result of the assessment could even be dyspraxia rather than ASD or dyspraxia with ASD as the two have a lot of similar traits.
Anyway, I digress. I think we really resisted going for diagnosis as we felt we would be labelling him and making his school years harder. Dh was particularly anti and it took me a long time to talk him round.
In reality though, it has been the opposite. Since getting the verbal dx, the school has tried very hard to help ds (they haven't always got it right - but they do try and they WANT to help) and we now have good lines of communication with the head, the SENCO and his teachers, which has made all the difference to him on a day to day basis.
We do feel better now that we are being proactive in doing something to help him and that alone is better than spending sleepless nights worrying about what the problem is and how we can help him.
Whether or not he ends up with a formal diagnosis is yet to be seen though.
At the end of the day its your choice to make and I don't think there is any right or wrong answer, as each case is unique as is their family situation etc.
Sorry to ramble on. (Tis my way I'm afraid. ) I hope I have said something helpful/useful in there somewhere and wish you good luck with making your decision.