You are now at a point where you have made it very clear you do not want further contact with him.
After this, if he does try to in anyway, you don't respond to him - he's looking for your attention.
If he contacts you again, you treat it as harassment. There are two options for you to go down in this situation (this is via citizens advice):
Reporting harassment to the police
You can report harassment to the police. They can charge someone with criminal harassment if:
the person has harassed you more than once
the harassment made you feel distressed or alarmed
If the police decide to charge someone, they’ll send the case to the Crown Prosecution Service (CPS). The CPS is an organisation that can take people to court - this is called ‘prosecution’. If the CPS decide not to prosecute the person who harassed you, they must let you know.
You can report harassment to the police on their website.
Taking someone to court for harassment
You can take someone to civil court if:
they’ve harassed you more than once - this includes stalking
the harassment made you feel distressed or alarmed
The court can order the person harassing you to stay away from you - this is called getting an ‘injunction’. The court can also award you compensation.
If the person keeps harassing you after you get an injunction, they’ve broken the law - this means they could go to prison.
You can go to civil court even if:
you haven’t reported it to the police
you reported it to the police, but the the CPS decided not to prosecute the person who harassed you
the CPS prosecuted the person who harassed you and the court decided they weren’t guilty
If you’re thinking of applying to court, you should get legal advice.
If you go to the police you DON'T have to push for him to be done for sexual assault. It'd be difficult for you to prove anyway and they will probably acknowledge that.
The action of repeatedly trying to contact you against your wishes is enough of a problem altogether and much easier to prove.
For this reason I recommend you keep the note as potential evidence.
If you go down the civil route this then would make it easier for the police to act if he tries to contact you again.
In reality knowing both of these routes is useful because you can use them in combination.
The point is, you have now made it absolutely clear beyond any doubt that you do not want further contact.