OP - My DD was very behind in reading and writing (and other cognitive tasks) when she was 4.
Now she's 9 and she's almost caught up with reading, and is catching up with writing.
So, you're right to be concerned and worried. But what you really need to be is proactive.
My DD has caught up because I have done an enormous amount of work with her. School also did - but she never would have caught up without the stuff I did with her as well.
So you need to try and understand what his problems are, and then decide how to attack them.
Most of it is 'trial and error'. Try something, if it doesn't work, try the next thing.
He's only 4. You have loads and loads and loads of time to find out what works and help him catch up.
So, I think the first stage for you is to find out what he can do, and what he can't do. Then you can make a plan.
Never listen to anyone who tells you he's just young, or just a boy, or they all catch up in the end, or any other platitudes. If his making normal progress school wouldn't have sent the note home.