If concerns are about the children being socially well-adjusted, I used to mention the many children I had met through a local HE group and HE camps, how well they play together and how well-adjusted they are. I suppose one could also mention some well-adjusted famous people who have been home educated, and ask, are those are the kinds of social problems you are talking about? If they are still unconvinced, ask how many HE children they have personally met. If the answer is one or two, well, it's silly to generalise from one or two people - if you met one or two schoolchildren who had social problems you wouldn't just conclude their problems were the fault of school, would you?
So really, if you are able to tell the person that you yourself know a number of HE children who have good social skills, and if the person has herself only ever met one or two HE children or none at all, they'll have to accept that you're in a better position to know. You could invite them along to meet some HE children if they are curious.
Fortunately, people come out with this sort of thing less often as your children get older. Most realise that it would be incredibly rude to say that HE children have poor social skills when you have just told them that your older children are HE! Some do express surprise to see such confident HE children, and I say yes of course, they have many opportunities to mix with different people every day.
It's trickier to counter this worry if your children are not exhibiting "average" behaviour though, because you'll get told that school would "fix" your special child. Then I guess you have to point out that chucking children into an environment that they find overwhelming does not miraculously turn them into average children, it just makes them stressed and unhappy.