Lots, but not all, anti-vaxxers do seriously dislike people with autism. This is the niche of anti-vaxxers I am talking about.
"In modern society, a potentially serious adverse event attributed to a vaccination is likely to be snapped up by the media, particularly newspapers and television, as it appeals to the emotions of the public. The widespread news of the alleged adverse events of vaccination has helped to create the "urban myth" that vaccines cause serious neurological disorders and has boosted antivaccination associations. This speculation is linked to the fact that the true causes of many neurological diseases are largely unknown. The relationship between vaccinations and the onset of serious neuropsychiatric diseases is certainly one of coincidence rather than causality. This claim results from controlled studies that have excluded the association between vaccines and severe neurological diseases, therefore it can be said, with little risk of error, that the association between modern vaccinations and serious neurological disorders is a true "urban myth"."
"Many severe neuropsychiatric diseases, such as Alzheimer's disease, multiple sclerosis, autism, epilepsy, schizophrenia, encephalomyelitis, encephalopathies, transverse myelitis and optic neuritis, do not yet have a well-defined etiopathogenesis, although important progress has been made on their causes. Several studies have shown that these diseases are due both to genetic factors (intrinsic factors) and environmental factors (extrinsic factors). With regard to autism spectrum disorders, for example, as early as 1977 Folstein and Rutter published the first study of twins and autism focusing on genetic aspects, which showed that the concordance rate in monozygotic twins was much higher than in fraternal twins [1]. Incomplete understanding of the causes of the above diseases has sometimes led to the belief that they are caused by vaccinations; in reality, however, the relationship between vaccinations and the onset of serious neuropsychiatric diseases is certainly one of coincidence rather than causality."
All of this is from "The "urban myth" of the association between neurological disorders and vaccinations" which I found very interesting to read. (found here: www.ncbi.nlm.nih.gov/pmc/articles/PMC4718347/)