So the operation next to us was aquaculture. When we moved in it was a quiet little operation with a single large tank (circular 50' diameter). About a year after we moved in the business was sold and with the new proprietor it became far more intensive with 8 large outside tanks. The tanks had heaters, large pumps and the site generated quite a bit of white noise. You didn't notice it during the day or if the wind was blowing the leaves on the trees but on still nights the noise was sufficient for use to close the windows in the summer on that side of the house.
At no point did we think of selling out house and moving. However, it did postpone us from extending the house and investing in our home until the certainty of the site next door was settled. We knew that if we kept the value of our house about the same level, we'd always be able to sell our house to someone. If we invested £200K in further developing our property, the site next door might become an issue for potential purchasers.
All the time, we knew the proprietor next door was very keen to sell the site for development. However, he was refused by the district council and at appeal every time he put in for planning. Nearly all the neighbours wrote in support of the applications on each occasion but this didn't influence the district council. They wanted the site to remain agricultural as part of the business development strategy for their local plan and refused to change the site's designation. After 20 years, planning for 2 houses was finally approved after yet further appeals.
I think our worry was that the proprietor was getting increasingly keen to sell. He might have gone down a change to light industrial designation which the council might well have approved. We could be living next to an agricultural contractor depot with tractors, trailers coming and going at 3am, or had pneumatic air tools running all day which would have been very intrusive.
As it turns out, all we had was a slightly noisy (on a windless nights) eyesore to one side of our property for 20 years. I think if your site isn't in a rural area or AONB and is likely to have housing development fairly readily approved then you may well be fine.
The new houses are better neighbours. In our instance the site is large enough to have had 16 - 20 houses, so having just 2 large detached houses is more than OK. They probably look in our direction us as we did on the previous site and think what a shabby little place, wish they would do something about it!