What a shit situation. I know how this feels, my former landlord did the same thing. I kept wondering why my stuff was not where I thought I'd left it - so I set up a webcam to record during the day. Sure enough, the landlord (who turned out to be selling) was coming in, moving my furniture to make the place look bigger, taking photographs, then putting things back to how (he thought) they were.
I complained to the agency (not an option for you) with the evidence and they told me they would deal with him. So guess what? A couple of weeks later I got home to find an estate agent showing people around my home!
The trouble is there is fuck all you can really do. He has no right to come in, he is in breach of the tenancy agreement. In theory you could take him to court - but what would you gain? How can you quantify the damage he is causing when it is psychological?
I had the exact same experience as you on another occasion - I was in, the key was in the door, the landlord pushed his key in knocking mine out. Assuming it's a traditional deadlock-type door, if you lock it and then slightly turn the key (there's usually a bit of give before it begins to actually unlock), it is impossible to push or pull out until being turned back to the proper locked/unlocked position.
You could change the locks but this can be expensive unless you know how to do it yourself. The landlord would be livid but he's not likely to take you to court within three weeks (even if the tenancy agreement says you are not allowed to do it).
You could write a letter to your landlord instructing him not to enter your property without your prior written consent. Send it recorded so you can prove he received it. Tell him that any further incident will be treated as trespassing (because it is) and that you will contact the police and/or take legal action. It might make him think it is better to wait it out rather than force your hand.
If you find him in your home again, call 999 and say there's an intruder. If you get home and find he has been in but now left, phone the police and report that someone has been in your home unlawfully. If possible, set up a webcam to record evidence.
Rogue landlords like this are the scum of the earth (worse than an RMT official). Try to hold on to the fact that in 18 days this landlord won't be a worry to you any longer.