OP did he do any of this in writing? Text message, WhatsApp, fb messenger etc?
Honestly, his sort are utterly universal. Bluster through life, thick as shit, behaving as though they're the bloody Krays, violent and aggressive. Like they're something special and above being arrested or having any consequences at all.
They surround themselves with people who can't / won't report them because they're up to their necks in something illegal themselves.
That's not you. You're just a person, with a job and a child, and you can report him, and anything he does to try to intimidate you, you can report that too.
Imagine your bank manager or the GP or the head teacher at a school having this lumbering idiot of a gorilla threatening them with violence. Would it work, would they do what he says? Of course not - they'd call the police, ban him from the premesis, and he'd get charged with a crime.
They wouldn't say 'ooh, he told me not to, and he's always in the pub getting in fights and dealing drugs, everyone knows who he is, I'd better listen to him'. They'd think 'who is this fucking idiot, what a load of shite'.
He literally only has the power that you give him. Give him none. Take it back.