I think a permanent encampment wouldn't see too much trouble - most issues tend to arise from the transient groups that illegally set up camp on private or public property.
I honestly used to be very much of the "live and let live, don't be prejudiced" mindset when I was younger, but every single large* illegal traveller encampment that's been set up in my area for the last ten years has brought inconvenience to local residents and businesses, a massive uptick in anti-social and criminal behaviours (ranging from theft, dine-and-dashing, harassment, vandalism, human faces), and a massive cost to the local council to clear the sites after the travellers have left (including having to bring in environmental health due to the human faces left everywhere).
My mate's business, a motorcycle training and company, had to close for nearly two weeks due to their paddock being broken into and the travellers setting up on it. Took the best part of a week to get them evicted, and another to get the site cleaned. The fuckers tried to break into the containers where the training bikes were stored as well, but fortunately failed (but it cost a bit to repair the damage).
So yeah, those travellers get fuck all sympathy from me - they revel in their lawbreaking and disrespect of the communities they inflict themselves upon, and seemingly don't give a shit about the reputation they give their entire community (because, like it or not, if people keep having the same experiences every time travellers come to town, they are going to draw their own conclusions).
I'd like to think that the travellers who stick to authorised sites are completely different, but I'd definitely think twice about buying a property near one though.
- I will say that the much smaller Romany encampments (in the traditional caravans) that occasionally pop up on the roadside never seem to cause any trouble.