Of course, he could be entirely innocent here but I know what my instincts would be telling me...
Be honest with yourself: you had an argument with him and he said hurtful things; did he instigate the argument? Did he overreact? Does he say hurtful things often or is it just when he is usually in the wrong? Often, when someone has done something wrong they defend themselves by attacking and turning things around to make you feel guilty. The next time it happens you have already convinced yourself that it was your fault the last time so you won't be accusing/upsetting/disagreeing again. In other words, emotional abuse. I realise your post was about the phonecall but this sounds like a familiar situation that I had for about 10 years of an 11 year relationship. It was your reference to the hurtful things he said that got me thinking.
Yes, a phonecall from a mobile number at that time of night is unusual unless you know the caller. But it's his behaviour which is ringing alarm bells for me. Your course of action depends entirely on the type of relationship you have. If he lies to you, he'll lie about this and (despite what most people think) will probably never admit to doing anything wrong. On the other hand, you could check his phone- but I truly believe that if you feel the need to do this to find the truth then you know in your heart that the relationship is in trouble. Once you've done this, I don't think you can go back. Checking the phone is confirmation, not the deal breaker.
Perhaps it's nothing, but perhaps it is. If he's acting shady about it then he's most likely hiding something. You seem to think this is a paranoia trip but why would you think that? When you react to suspicious behaviour, it's not paranoia but you've obviously been taught that it is, somewhere along the line.
FYI - if you go through his phone, check the contacts for anyone saved as 'withheld' or 'no number'. That gave me all the info I needed! But once you decide to go through the phone you decide that the relationship is finished, because trust has been broken.
Seriously OP, I hope this is just a misunderstanding, but you sound like you've had your self esteem battered.