I agree with fluffybunny. It is very easy to stimulate in areas where you know he is gifted, and very often this means that other areas are left out. A truly gifted child does not need to know about photosyntesis at four, it might be more useful for him to teach him social skills, let him play, learn through play and interact. If you can, try let him take up a sport, or playing an instrument. Music is especially useful for a mathematically gifted child.
We left all these areas out, and our son was doing negative numbers and currency exchange and countries when he was 4. We had microscopes and telescopes, and spent time on his maths and science interest, because it was easy.
Now, age 7, we have relaxed all that, he is doing brilliantly in school, but he has social problems, and problems with empathy. We have focused on sports, he plays football (team sports), martial arts (self control), and skiing (for the adrenalin junkie in him). He is currently having a break from his violin, so he can spend time interacting with friends, and learn how to behave and how not to behave. His violin teacher said he was very gifted, he could easily write music just from hearing it, and it was enough to hear something played once, and he could play it.
Good luck!
Dont be too hung up in his giftedness. There is a little boy in there, too!