Stick to your instincts. You sound fairly balanced from your post and the things that you have admitted you are thinking through.
The only thing tests assess is ability to perform that test. It has some validity in terms of generalising ability in other things as a standard across a population, but individually, not so much. It provides a best guess at most.
There are all kinds of things that affect test performance. Processing speed, understanding of the question, individual motivation and concentration levels, anxiety, test duration, sensitivity and intuition of the person delivering the test, and, IMO for children with some SEN, how much they have practised such tests before.
I know that practice can affect my son's non-verbal reasoning scores, though not his verbal reasoning scores. This is because there is some assumption that the children will automatically understand what is expected of them, and process things in a way that typical children do. Practising gives the training on HOW to perform in those test situations, which can improve the score.
You can suggest that the test does not reflect your real life lived experience of your son and ask if there is another different test that will give a more refined/accurate picture that will explain the results in the first. EPs tend to pick the most crude, cheap, easy-to-deliver tests but there are more sensitive tests available. WISC 5 is probably a way better one.
Hope that helps.