YANBU I would feel the same. Apart from all the contentious views the children would be leaving fairly often, does not make a stable environment.
The only experience I have of travellers is a bad one. Two girls moved into our block of flats (was in a u shape). They were travellers, they had their family over, I stupidly invited them in. It was like a fucking plague of locusts, can I have this, etc. I soon booted them out.
Also one of the kids made off with dd's bike, he got as far as the green when dp caught up, he was like "It's my bike mister" fucks sake, even though it was pink, and for 3yo. Dp went to his Dad and he twatted him round the head
.
Apparently they were so bad on the green that night a letter got sent out by the council advising all that the green was for all to enjoy and intimidating behaviour would not be allowed.
The 2 women who actually lived there nicked my buggy which was stored outside my house. They had the brass neck to actually walk past me with it,the cunts. They never used the doors to the flat they had, would go in through the window
.
They lived there for about 6 months and had "pikeys out" graffitti emblazoned quite often.
They fucked off quite soon after. Good.
I'm not judging all travellers by this, but meh they seem to not give a fuck about the area they invade.