I agree, one more "I'm not interested, now leave me alone" and then from then on, just ignore him completely. It's a good idea to do the double back thing, or go into a shop and tell them that this man is bothering you and needs to leave you alone.\
It is good that your ds dad is back next week, but as you know, you can't be bodyguarded all the time, and the bloke might even think that it's not you that wants him to leave you alone, it's only because you are with your ex.
Stalkers will do whatever it takes to get the feed they need from you, contact, a reaction of any kind. It doesn't matter if it's positive or negative, it's the reaction they crave. The rest they will twist to the warped dialogue in their head.
If you are bothered with a stalker calling you and only answer after the 236th time they ring, they won't get the hint not to call you, all they take on board is that they have to ring your AT LEAST 237 times to be sure you will respond at some point.
There may be online sites based here for tips on how to manage stalkers, and the thought process behind it all. That might help you understand the situation you are in.
My advice is to tell him clearly next time that you are not interested in a friendship with him, nor knowing him at all and that you want to be left alone by him. No contact, no nothing. You are strangers and don't know each other and that is how you want to keep it. Full stop.
From that point on don't even acknowledge him, if he is on a street, cross it to avoid him or double back. If he crosses too, go into a shop or knock on a door and tell them what is happening and ask for help. Do not at any time engage with him or address him directly.
I had this mildly with a guy I went out with a couple of times, he tried emotional blackmail, I binned him. then he texted me and texted me, same message "thinking of you' every couple of days. When that didnt' work I got another message late on a Sunday "HELP"
I knew he had health issues of some kind so I rang the non emergency number and told them that I was fairly confident that this was a ruse but due to the health problems I felt it best to alert someone for them to evaluate if he needed help. I told them the situation that he was an unwelcome person in my life and that I had no desire to have anything to do with him, as I thought this was a ruse to get me (and my young son) out in a car late at night to go and see if he was ok... Yeah.. Right...
I gave them the guys number, his address and all the details I had for him, they said they would call him first and if they didn't get an answer to go round and see him at his house.
25 mins later I got a text from him to say that they police had been and sorry to have bothered me. I didn't reply to that, or any other message that he'd sent.
I heard nothing for 18months or so, then a message telling me how he knew I'd be relieved to hear that he was jacking in the job in my vilage. (Didn't bother me, I was working outside the village during the day anyway) and emigrating to Australia...
I still did not reply, just blocked him.
People like this are sick. They don't get hints or think about how they will affect others, even telling some of them has no effect. Be strong and keep posting.
You aren't alone in this lovely, we're all worried for you and thinking of you.