Firstly, the dropped kerb may have may been paid for by the neighbour, but it does not belong to him. It is part of the highway so pedestrians and road users could use it.
The problem you have is the dropped kerb does not extend across your 'drive' so you do not have off-street parking. You have a paved front garden. If you want to have a proper driveway you need to get the dropped kerb extended (speak to the Council - some will let you use your own contractors or pick from a list so you may be able to shop around a little).
You should not have been sold a house with off-street parking as you do not have it.
You need to check the deeds for your house. From the picture, there is a dwarf wall and post but the paving extends beyond that. I would expect that you own up to the end where the paving meets the pavement. If so, you are driving across the neighbour's property.
The pavement is not wide enough for you to get onto your driveway from the dropped kerb without driving over the paving. Just because there is no wall separating it does not mean it is not the neighbour's driveway. The NDN is fed up of you driving on part of his driveway and trying to stop you doing it.
Unless your deeds say you have a right of way over the end portion of the driveway (i.e. before the pillar) then you cannot drive on that area which from the picture looks as though it belongs to the neighbour.
It does not matter if the previous owner told you to. You have to be driving across part of the NDN's driveway to get onto yours as there is no other way to get in at that angle (unless you drive a Twizzy when maybe).
YABU.