As a scientist (lapsed) I explain the use of short nonsense words as a decoding test to myself thus:
The test is designed to be a test of decoding ONLY. It is not a test of comprehension, and it is not a test of working memory.
Therefore, the test needs to be designed as far as possible not to be affected by a child's comprehension skills or their working memory - ie to be a 'fair test' unaffected by other variables.
If a child decodes a long 'unfamiliar' word incorrectly, then there are two possible explanations: they may have a problem with decoding, or they may have a problem with working memory [I have in my class an SEN child with extremely poor working memory - he can decode almost all graphemes in isolation and in short words, but he cannot decode all through a long word because of his working memory issues. He would - correctly - pass the phonics screening test, but the working memory tests he has had show that that is where the problem lies. We are addressing the problem of him decoding longer words not by working on phonics - he has that knowledge - but by focusing his working memory issues. A test where the two issues are not distinguishable would not allow us to intervene appropriately].
If a child is faced only with 'known' words then we do not know whether they are using decoding skills or word recognition skills - again, that is not a fair test of decoding.
If the child is given short, phonically regular nonsense words then the decoding can be tested in isolation - which makes it a good test of decoding.
It is NOT a test of reading. It is NOT a test of working memory. it is NOT a test of comprehension. It is NOT a test of social background, how much children have been read to etc. It is designed - and as a scientist I would suggest reasonably designed, with other variables as far as possible controlled - to test decoding in isolation. Another variable - behaviour of teachers administering the check, especially where it challenges existing practice - should perhaps be a focus for future control ....