Trying to keep it short. I live down a very weird road- very few houses have driveways. Some of us have allocated bays in the street. Other people- generally the owners of newer houses- must park in a car park about a minutes walk away created especially for our road. It's literally at the end.
Neighbours have a weird upside down U shaped house, facing the road. They have decided to put a very ugly, gravel driveway on the reminder of their front garden.
However, MY allocated council parking space that belongs to my house is where the exit to their 'driveway' is. Their DD just passed her driving test- hence driveway- and I've had to move my car from my space SIX times since Tuesday.
Neighbours have now sent letter asking me to park elsewhere so their DD can use 'driveway.'
AIBU to have posted a polite letter back basically saying no and that I will not be moving car multiple times a day in future? 
Car park for our houses requires a permit. As I have the allocated space, I cannot apply for a permit as I already have a space. Neighbours have one space in car park provided for those without allocated spaces, however can apply for space for their DD as car park has spare permits available. I can't park anywhere else as you can only park in allocated bays.
AIBU to think my allocated space is mine, it's not my fault it's in front of their house and that I shall not be moving my car from now on?
Diagram provided. 