Sounds very familiar Mrbrowncanmoo. Ds' school insisted it was all in my head right through the infants, then suddenly called me in the summer before he moved into year 3 and said they thought I was right, but as it would be the juniors' remit come September, they would leave it to them. 
We started to think ASD when he went into nursery and I realised how different he was to his peers. Obviously we'd noticed things before (he had his own language and was obsessive about the Argos catalogue 
vacuum cleaners and washing machines - for starters) but with him being our first dc, didn't know what was and wasn't typical developmentally. Most people just complimented us on having such an intelligent, articulate lad, as it wasn't and still isn't, apparent immediately that he has ASD, unless you know what to look for.
He spoke and walked early, had a massively advanced vocabulary and seemed to meet his milestones, so didn't raise any red flags with health visitors etc, but then when he failed his hearing test and they just put it down to him being tired, whereas we knew it was because he simply wasn't interested in the person making the noises, if it wasn't on his agenda he wouldn't respond - that was just the way he was. Now I would see that as a red-flag, but back then I just thought he was a bit stubborn and independent. He would also happily go to/go off with anyone without so much as looking back at myself and dh - I was told at the time this was because he was securely attached - it wasn't.
In nursery he just couldn't interact typically with his peers. He was convinced one particular boy was his best friend, but the lad concerned didn't even know who ds was. He liked the routine of the nursery schedule, but couldn't cope with dress-up days, inset days etc and things like the nativity were really hard for him to handle. He went from being an easy child to angry and agitated at home and started having meltdowns, but when we raised it they blamed the birth of his brother - which was a year earlier.
By reception he was isolated at playtime, spending each break sat on a bench on his own. 
Then in Y1 he butted heads with his teacher over things that he felt were black and white and refused to back down when he thought she was wrong. She then spoke to me about him and said she was shocked, from speaking to me and seeing how I interacted with him (when helping out in class one day a week) to see that I wasn't an over-liberal parent that let him get away with doing whatever he wanted, but had strict rules in place and made sure he knew his boundaries. She said she honestly thought he must be completely indulged because he spoke to adults as equals and expected them to interact with him on the same level. She wouldn't be drawn on whether or not there was a real issue though and when we suggested ASD we were told we were talking nonsense.
For Y2 he had the head of KS1 for his teacher and she hinted a few times that she felt there was something going on, but refused to be drawn further until the end of the summer term and despite repeated requests for a meeting with the SENCo nothing ever happened.
He basically went into Y3 (different building, different daily schedule/routine, loss of a playtime, less play-more focussed work) his teacher went on long-term sick and just started to implode. He became physically ill with migraines and digestive issues, stopped sleeping and had horrendous meltdowns at home every evening. Essentially he was having a breakdown. I ended up calling the Educational Psychology's emergency/crisis line and sobbed down the phone to them, they came out the following week to see him in school and almost straight away suggested ASD. He was 7 then and got his diagnosis when he was a couple of months off 9, but in the meantime the ASD outreach team and Ed Psychs started working with him and advising the school and things settled a little, so he could at least cope with being at school.
He's 13 now and doing really well. He's really matured in the last 12 months and is finally happy at school, since his bully was excluded last summer. He is still hopelessly disorganised and needs a lot of help with organisation/planning, homework etc, but we have far fewer meltdowns and he is generally calmer and more able to cope with his emotions (we've done lots and lots of work around emotional literacy). He's come into his own at secondary, because he's a bit of a computer whizz and of course that's socially acceptable/desirable at this age, so he has both SEN and NT friends in his group as a result, because they all come to him for advice/info re gaming.
When I was where you are now, I couldn't see forwards. It all felt really big and scary and I didn't know where we would end up, but I have learned to just put one foot in front of the other, deal with what needs doing now and accept that things will come together along the way - there really is no deadline or massive rush to have him fully educated and ready to work by the age of 21. Currently we're looking at GCSE options, which of course leads to A Level decisions and career choices and instead of panicking, dh and I just think we'll do what feels right for him at the time and help him get where he wants and needs to be, even if that might take a lot longer and a more circuitous route for him that it would for someone the same age but NT.
MNSN is a fantastic place to get help, advice and support. I'd never have got this far without it.
Have you popped your head around The Goose and Carrot pub thread over on SN Chat? It's a great place for getting general support and sharing either successes/achievements that other parents might not get/appreciate or equally having a good old rant and letting out all your frustration among people that understand what it feels like and can offer virtual wine and cake. You don't have to keep up with the thread, just dip in and out as you feel you need/want to. It might sometimes feel like we all know each other over there, but there's new people coming in and out all the time and it really is open to all.
