By accommodation in school I mean an individual education plan, a TA in the classroom, involvement of SLT, OT, or any other services geared to help him succeed in school to the best of his ability.
I assume your ex has no contact with the school, no contact with the doctor who made the diagnosis, basically has nothing to do with the serious parenting of his children.
You need to put an end to this contact nonsense.
Calling his own son that awful name
. I have no words...
Take him to court.
Insist that if he wants visitation he needs to parent, not just splash money around (using the children to show off his money, basically) and disregard his responsibilities.
He needs to start engaging with the professionals who are part of your children's lives - going to parent night, attending doctor appointments, co-operating with advice from professionals when it comes to dealing with the autism traits.
You should not be willing to let your children be badly treated by this man for the sake of keeping relations between the two of you amicable.
If the consequence of trying to secure appropriate treatment of the children is that he turns unfriendly, then he has all the power here, and you have none. The losers in this situation are the children who are being neglected (allowed to roam unsupervised) and mistreated (exposed to unsafe conditions like swimming venue) and called ugly, awful names.
He basically does what he likes here - has them if it's not inconvenient to him and if it fits in with his older family's schedule, and disregards their needs. None of this is in the best interests of the children.
He needs to agree to a proper visitation schedule and arrange his work around that.
Above all, he needs to engage and co-operate with all the professionals involved with autistic DS. This is what your deal breaker should be.
If he won't do that, then you need to fold your arms and end the contact.
I'm guessing he pays no child support. Hope I'm wrong.