I'd say, in answer to your title, that it's because they don't have to live with the situation themselves and really can't be arsed to think about it. It's suddenly "too difficult" to have people to dinner, because "you can't really eat anything, can you?" (this has been said to me) Er yes, there are still loads of things I can eat, not every meal has to involve tomatoes or pasta, you know 
It's causing them trouble - they have to think outside of their narrow range of experience and, oh my God, possibly actually work out what is in the food they're cooking; which is often just far too much like hard work.
People who don't have to live with allergy/intolerance mostly do not understand what it's like and quite frankly, most don't want to have to. Which is often why, IME, they will belittle the experience of others.
Added to that, there is a lot of misinformation and misunderstanding out there, including the idea that for an "allergy" to be serious it has to involve an epipen. No, it doesn't. Epipens are only effective for anaphylaxis - they won't do anything against sudden violent stomach cramps/explosive vomiting or diarrhoea.
Equally, the idea that an "intolerance" is just some kind of lentil-weavery MC sappiness (which I've seen it referred to as, on MN as well as elsewhere) - well no again. Gluten intolerance (coeliac disease in its severest form) can cause malnutrition and even death over time; but has a range of unpleasant symptoms up to that. Lactose intolerance can also cause death - severe dehydration due to diarrhoea etc. Often they go together, because the destruction of the villi in the small intestine reduces the production of the lactase, which creates lactose intolerance.
The rate of allergy/intolerance is ever increasing, no one really knows why, the hygiene hypothesis is just one idea but some don't give it much credibility - and just because "no one ever had it when I was at school", this is not a valid argument to use as to why "people must be making it up now."
(none of the " " bits are quotes, they are all there to show my feelings about the comments)