Stay calm!
The lies and aggression are designed to rattle you, so don't reward him with being rattled. Unfortunately, I've become accustomed to this tactic and in the beginning I was a gibbering wreck when court was mentioned. I soon realized that proactivity was better for my mental health than being reactionary, I became my daughter's advocate and fought for contact on her behalf.
All contact correspondence must now be in writing. Don't get sucked into his lies and waste time refuting his unfounded accusations, be factual and concise.
Do not go to mediation, it will be a waste of time because he will be too busy playing the victim.
Show initiative by drawing up a Parenting Plan, ask him what contact he wants and if it is impractical, counter with something more practical. I decided on a Parallel Parenting Plan because my DD is older and I wanted a rigid plan because Ex is a wriggler.
Accept that he isn't you and he will parent differently, children adapt to different routines as long as their consistent and on the off chance that he genuinely wants contact, it will be in your child's best interests to facilitate contact.
Do you know his family and do you trust them with your child? Contact with his family could be a good thing and you may want to think about initiating contact directly with them.
There's a good chance that as soon as he sees you being calm (not being frightened by his lies and aggressiveness) and reasonable (asking for his input, whilst being practical) he will disappear again because this could be about getting at you, rather than seeing his child.
Threatening court for child contact is one of the first chapters in the twats playbook, it may not feel like it now but this predictability is strangely comforting in the long term because you kinda know what's coming next.
Ex has scuttled back to the rock from whence he came but it took him a long time before he realised that his threats and mediation only made me stronger and more determined.
Stay calm!