^^exactly this.
I have twin DC, one in specialist provision, one in mainstream. Both started in mainstream.
My DS had a statement while still in nursery but I was pushed VERY hard by the local Advisory Teaching Service to "try" mainstream because "do you want him to fail in life?" I didn't know better at the time and just accepted their "expert" opinion even though it went against every instinct.
HIs experiences in mainstream seriously scarred him, even though we were at a lovely school.
However, the support typically isn't an expert in autism - due to resources, any old TA can get assigned as their 1:1. The 1:1 can be often diverted to do classroom work because children in mainstream don't get completely F/T hours.
In a class of 30 (or more) young children who are all trying to navigate learning and staying in their seats and being quiet, there's just not the capacity to give your child what they need to flourish if it's vastly different from the norm, even with a 1:1.
As the academic demands grow, the gap often gets wider and this all takes a toll on your child's confidence.
So you decide that mainstream doesn't work after all, but what's that? The LA tell you that you can't just decide that it's not best. You need documented proof for a change of provision. That's emergency meetings, and evidence from professionals, plus the school needs to agree they can't meet his needs. If your child is just quietly shutting down rather than aggressive and outwardly exploding, many schools won't recognise just how bad things are. And that's because they don't have the in-depth knowledge of your child because there's 30 others in the class. It's a lottery whether the Ed Psych will support the move - some seem to think that an autistic child stick with mainstream no matter what.
And also, if you chose mainstream the LA will often expect you to demonstrate why things have changed - especially if you stuck mainstream out for some time.
Even if - perhaps with the help of you fab lawyer again - you manage to get sufficient evidence to persuade the LA's faceless board, you then face the challenge of finding a placement. Your child is in school, they aren't a priority.
As well as being a parent of two autistic children, I was also the SEN governor for our school for 5+ years. I could tell you so many stories of parents whose child desperately needed a specialist provision but getting the move accomplished was an uphill battle, and it was getting even harder when I left a couple of years ago.
In other countries, children don't start academic learning until the age of 7 - we start very early and our academic results aren't any better as a result. You have time to make a decision. If there's an autism-specific placement near you which has the same small class sizes as a regular special school, plus the same access to expert TAs and teachers, then I'd snap that up.
Like @saraclara says, it's easy to switch to mainstream from specialist provision but I really can't emphasise enough how hard it is to go the other way - regardless of what the recommendations are now for your child. LAs are squeezed financially and there's just not enough places to go round. You will have to - and I quote - be able to prove that your child is failing in mainstream to be able to be eligible for a move. It doesn't matter if they're barely clinging on; if they're coping, then you won't get a move.
I know it's crap and I'm sorry. But you won't be dooming your DS just by starting out in special school. How about supplementing it with mainstream clubs so he gets a mix of social interaction?