I replaced about 20m of 6ft fence 18 months ago which had been weakened by my neighbour fixing trellis and growing fruit and vegetables on it and it then fell down after a storm.
When it was replaced at my cost, I asked her not to attach anything to it or paint it. Within weeks, trellis was back up, plants attached on her side, flower pots hanging off the top of the fence. I'd draped some lightweight bunting and LED solar fairy lights around the solid posts my side - they were destroyed by her planting, hooking, using bamboo sticks, trellis, etc and it all broke, ripped and didn't work.
She wants me to replace a 3ft fence which runs the rest of the garden (it's stupidly long so about another 20m), in order that she can fix up sweetpeas, beans, etc, onto trellis attached to MY fence. She can jog on.
You must be heartbroken about the paint OP. All I can suggest is you either ask them to spray your side to match as they have damaged your property, ask them to put right or replace, or get some fuck off bright paint, neon maybe, and spray your side and accidentally spray their garden at the same time.
Sorry for hijacking your thread with a rant.