You'll have to keep tugging on different threads until something gives and it all starts to unravel.
My grandfather's story was similar. I knew his name and his birthday and a vague story about where he was born (he thought in a particular workhouse). I also knew his history after he married my grandmother and had my father and uncles. Most of what he thought he knew was wrong, except his birthday, but I did find out about him in the end though it took bloody years.
I started with his marriage certificate, which had his father's name and occupation on and his age (though that was a lie as he was under 21, which was the legal minimum without parental agreement, which he couldn't get, being an orphan). I also looked at his death cert, which actually did have his correct YOB, so I knew there was a discrepancy and to look across a range of years for his birth cert. There's also the 1939 register, which could have filled in some gaps if his name had been transcribed correctly.
I couldn't find him on any censuses, so I focused on birth certificates and ordered a shedload of incorrect ones based on name, surname and presumed place of birth across a range of about 10 years, because I knew his birthday and needed an actual copy to cross-check that. Once I found one that fitted, I realised that the middle name he was accustomed to using was actually his mother's maiden name. I also discovered who his father was and (later) that he had been born in the workhouse, so figured I could let that info go as a misremembered family story. I confirmed his father's occupation and by coincidence Who Do You Think You Are did a thing about apprenticeship registers for that occupation right at that time, and I found his entry, which led to a whole other story I won't bore you with now.
Then I researched the address he was born at (on the birth certificate) via the 1891 and 1901 censuses (he was born between them), which turned out to be his mother's brother's house, so I knew his name. I was a bit stuck then so I started looking at their family, and found her parents fairly easily because it turned out that her mother's maiden name had also been used as her brother's middle name - it turns out this is ever so common and is really useful as a confirmatory piece of data. Once I started looking at the parents on censuses, I found that my grandfather had been brought up by his maternal grandparents after his mother's death, so despite thinking he was on no censuses, it turned out he was actually on the 1901 and the 1911 but under their surname, not his own, which he reverted to as an adult (but spelt wrong...).
All of this tedium about my family tree is just to illustrate that you need to use a variety of sources and keep moving back and forth between them while you fill in the gaps around your relative. It's like a missing persons enquiry and you just need to keep gathering little snippets of data that will lead you to other little snippets. Even with orphans there will be bits and pieces you can use to build up the whole picture. Orphanages had to list all the children in their care on the census too, and they will most likely be listed as 'inmate', so you can use that as a search term in conjunction with whatever else you know to hopefully find his entry/ies. Orphanages usually cut kids loose at 14 to be apprenticed in a trade. If you know the orphanage, there may be a particular link with a particular set of trades or employers.
HTH, good luck!