I have no issues with tests per se.
I think there are some genuine issues with SATs. They are narrow in focus (maths and English only).
The thing is that England, unlike some education systems, does not have a closely prescribed curriculum in general.
(In some countries, primary schools have to cover a fairly closely prescribed set scheme of work, year by year, term by term, based on state-mandated textbooks, and are required to spend a certain number of hours per week on each subject.)
When a country does not have a closely prescribed national curriculum and then introduces high-stakes tests (punishing schools who do not pass at a certain level), then what tends to happen is that "the tests themselves start to become the curriculum." So schools increasingly dump anything that they think will not appear on the tests. And when tests consist solely of maths and English, well, you can see what is going to happen to other subjects like history, geography, music, art, science and RE.
You could get round this problem by having a much more detailed and prescriptive curriculum including textbooks a la Finland/Japan/Singapore, and having a wider range of SATS that cover all major subjects. And if you did that, you could probably afford to stop making the tests so high-stakes and punitive for schools, because you could feel a bit more confident that teachers are (roughly) following the curriculum that you want them to, so there would be less need to threaten and punish schools and teachers.
Honestly though, can you imagine the reactions of most English/Scottish teaching unions if governments proposed any of this (textbooks, more govt control over curriculum, a larger number of SAT tests that kids have to sit)? I think it would be wildly unpopular.
Just "remove the tests and do nothing else" is not working in Scotland. But if an education system does not want a high-stakes test-and-punish system, then schools need to be OK with having alternative kinds of government controls instead, like Japan and Finland etc. do. I see no political appetite for this.