There is a big difference between 'knowing the alphabet' (memorising the song) and actually knowing the letter sounds, recognising the symbols (then eventually matching each symbol with its letter sound) then also understanding the concept that those letters can convey meaning, when written in words.
Knowing the alphabet & numbers as you have described is rote learning. It is no different than learning a song. The deeper level learning required to become functionally literate, takes being exposed to language & letters in a variety of contexts, over a long period of time. Some children pick up the skills quicker. Over the last 14 years I have been some children who begin to understand these concepts at age 2, others who only just start grasping the basics at age 5. Most are somewhere in between. There is a massive range.
As a teacher- my preference would be to have children who are read, sung & spoken to often, rather than a child who has been sat down & taught their numbers & letters formally by their parent.
The easiest children to teach are those who enjoy books, who have been encouraged to experiment with mark making, who have parents who focus more on 'learning dispositions' (Have a go, try your best, mistakes are an important part of learning- everyone makes them! Keep trying...) These parents focus on participation, effort & process rather than academics & 'doing it right.
So relax- learning about letters & numbers can happen organically, naturally during your normal schedule & day. We are surrounded by numbers & letters as part of life, so it can be easy to subtly & occasionally add them into conversation. Waiting for a bus, texting, seeing a sign. Point these situations out to your child. Quick, meaningful & with context :)
Activities like duplo, playdough, threading, and mark making are all the practice he needs for handwriting right now.