I can relate to EXACTLY where you are. The same thing happened to us. I was so angry with everyone, because my boy was happy, he was clever, he was/is perfect. He walked at 22 months. He barely spoke and there was lots of noises/jargon rather than words, potty trained at 3.5. Lots of people hinted at stuff but no one would say anything, if that makes sense.
And then someone did. That person was my sister (who is a teacher). And so I went through the diagnosis testing to shut everyone else up. I remember describing to DH how I wanted to run to the hills with DS and tell everyone else to FO.
Turns out DS is autistic. And that early therapies we engaged in were actually brilliant.
I am not saying that this will be your outcome. What I am saying is listen to them, take the help. It is absolutely no reflection on you, or your boy. They will see how delightful he is and just want to help him because his struggles will get harder and harder without the additional support. And actually, the support staff and therapies (esp occupational!) are FUN! DS has a BLAST!
Go with it, I know it's hard. I cried A LOT. He will be ok, he has a wonderful mum who is protecting him. Xx