I think it's 2/3rds of adults who are lactose intolerant in this country, so definitely worth a try. Turns out I'm in the 1/3rd who aren't lactose intolerant, but I'm gave up dairy for a year in the hope I would have some positive effects. Just didn't work for me. I would always recommend everyone try giving it up at least once, and always if they have any kind of digestive issues. It's worth looking at the milk ladder, because some people are ok up to a certain point with lactose, there are also lots of lactose free versions and you could try taking a lactase enzyme.
Gluten I would only eliminate if you have celiac disease, reducing it can help with other digestive problems. But eliminating it can trigger celiac disease if/ when you return to eating it, and that is not an easy thing to live with as it means no cross contamination.
The thing I have noticed a lot of advantages from is reducing sugar. Personally I have a lot of sweeteners with no ill effects, but one of my children has an intolerance to some sweeteners so they aren't ok for everyone.
I know somebody else who cured their IBS through giving up products with yeast in. But yeast has no negative effects for me, and again cutting out bread I just feel hungry a lot more often. So it really does depend a lot on the person.
One of my kids also had fructose intolerance as a young child and could only eat a very small amount of fruit or would get a major stomach upset. There really is nothing that works for everyone, so it is worth experimenting with what works for you, through doing an elimination diet or similar.
Another friend has an intolerance to things in the nightshade family so cannot eat tomatoes or peppers and some other things. So random how different we all are!