Unfortunately, autism is one of those things that isn't fixed. The 'severity' of autism changes throughout life, and early support allows for more successful outcomes. Your DS will be who he is regardless of whether you get a diagnosis of ASD on his file. But I can guarantee that he will fare better in life if his needs are recognised and he is given support at an early stage.
I'll give you 3 examples:
DD1 is now 20. Her ASD wasn't diagnosed until 15, although she was in special school from age 4. Because her ASD wasn't recognised, she didn't have ASD specific support. Some of her provision was actively harmful because her autistic overwhelm was seen as 'behaviour'. When she was in primary school, she was identified as one of the students that would be likely to go to the 'independent living hub' once she was 16 - they thought she had to potential to live a semi-independent life, have a job, etc. Because she was treated so badly, she developed an eating disorder at 14 (took until 15 for them to accept it and hospitalise her because of lockdown), now has severe mental health difficulties, and is very likely to need 24/7 support for the rest of her life.
DD2 is now 18. She showed signs of ASD at a very early age, but I was told she was anxious, copying her sister, etc. She was 11 when diagnosed with ASD. No support given in secondary until she fell apart in year 9. By year 10 she was out of school completely. Year 11 she started a special school and was put back to year 10. 2/3 of the way through that year, the special school hadn't listened when I told them that certain things they were doing were causing issues. She was out of school completely again by the March. She had a year on EOTAS, then another specialist school who have worked incredibly hard with her. She's now going to try college, 3 years behind where she should be, but will need extensive support in adulthood.
DD3 is now 17. She was very academic and masked very well. Sensory issues and OCD traits from early childhood, but not picked up or acknowledged at school. Fell apart in year 9. Diagnosed with OCD, ASD, ADHD. Then events happened that gave her PTSD. Now in a special school, 2 years behind.
This is why people are being so abrupt. You have a wonderful little boy who is just starting out in life. You are being told he is showing signs that he will struggle. You have the chance to get him the help he needs to be able to make the most of his skills and build strengths, overcoming weaknesses.
Please, please, don't throw that away.