He doesn't care about what happened - he cares about having a stick to beat you with.
He wants to 'have something on you', something that he can use against you, to constantly express disappointment about you and have you dancing a merry jig to placate, to reassure, to be his fucking inferior. He's an inadequate man who wants to feel Big and the only way he knows is to make other people feel Small. If you'd have mentioned shoplifting when you were twelve he'd have happily used that, but it's easier to use your 'past', because there's very few people who don't have one!
So, have a word with yourself - why do you feel you have to accept his behaviour? Because although you're asking here about it, to date you have accepted it, haven't you? My guess would be is that he's found your vulnerability (we all have one) and is playing on that. Calling you promiscuous and making you feel dirty? He's a nasty one.
"I feel fucked off to be honest, no answer will be enough for him he will just accuse me of lying"
Absolutely, no answer will be enough. Why would he give up the stick to beat you with? He won't. Ever. So get rid of him now before he turns you into a shell of yourself. I'd probably turn his own words back on him. He said "he thought I was someone else and if he had known what I was like he wouldn’t have dated me". Well those words apply so much more to him, so that's what I'd say - 'I thought you were a decent man and if I'd known you were a prick with a taste for coercive control I would never have dated you. Bye.'