I came to this forum looking for advice, to make sure that I am not the one acting unreasonably here, in order to know what to do when i have to face him later.
You are getting what you came for. Seems it wasn't what you expected, but I assure you it's what you need.
Loads of us have been through awful, abusive relationships, and experience and hindsight have given us the knowledge we wish we'd had then. We're trying to help you now.
Advice: Leave him! He's a controlling, possessive, and jealous user. He is raising several red flags, ones we've all seen before, even from the limited info you've given us. It will not get better; it will get worse. You cannot reason with him or make him see the error of his ways. You will only hurt yourself. Don't waste any more time or stress on him.
And honestly, you're not married, you're not dependent on him, you don't have children together, and you've only known him a short while. It's not worth the risk!!
Boot him from your flat and move on.
AYBU?: No. He has no right to be angry about this. You made it clear from the outset that it was happening. There is nothing dodgy on your end. He is trying to coerce (control! possess!) you into not going using unreasonable childish behaviour. All of his reactions to this - something he should be happy for you about - are red flags of an abuser.
What to do when you face him: Tell him, in no uncertain terms, that he must go immediately. Do not let him stay until he finds a place. Do not let him talk you into feeling guilty, about anything. Be prepared for him to try; have a friend with you or talk in a public place.
We will be here to help if it goes badly.
Please, PLEASE remember that MANY of us here are survivors of abuse, of all sorts, and we've seen such behaviour before and know what we're on about.
Please remember that we genuinely care. Nobody wants to be right about this; nobody wants that for you! We're not here to disrupt your life, nor to say I told you so. We want you to have what we wish we'd had. We want to help you protect yourself!