They will pick up from wherever he is so it means if I take him off leveled books he'll stay off them
any school worth their salt will test him to see what reading level/age he is. Don't worry about 'taking him off' the reading scheme.
Reading schemes are just books. Buy and read a wide variety of books, as wide as possible, include funny poem, books in rhyme, picture books with beautiful art work for older children as well as children's novels.
Find books which expand his horizons. There is a veyr simple set of books about a girl who goes to the National Gallery, use that as a basis for looking at different styles of pictures and places and so on.
Someone up thread said - well done you have given him a huge start in life, well, I am sorry to say that isn't exactly true.
Early readers often balance out by about year 3, and if you want to find the children who can write the best creative writing aged 7, then it is linked to the children who were doing the most imaginative play aged 4, not the ones who were doing the most reading, so while it is great he is reading, I would concentrate on making sure he has a broad and wide experience base, including things like playing in muddy puddles, building dens in the wood, museums, climbing on a Roman wall, cooking and so on.
As to maths, counting things is the starting point, then addition, subtraction, making things into sets, time, weight, measurement and so on.
It is essential that he grasps them well with objects, before doign anythign written. Doing it aroudn the house is the best way - ds please can you get out 4 forks for dinner. If Granny and Grandpa had dinner with us, how many more forks would we need? And how many is that altogether? Start with up to 10, then 20, then 100.