It is good to hear that you seem to be doing the right things for him, if in slightly difficult circumstances.
As he is so young, I wouldn't worry too much at this stage. Just involve him in a wide range of reading matter, including non-fiction books, magazines, adverts, leaflets etc.
Usually, the accepted 'yardstick' for whether or not a child can read a book correctly, is that they get 90% of it correct. There need to be a few challenging words, but not too many, if they are reading a book on their own.
HOWEVER, a technique I often used at school, as a Teaching Assistant, was to share a more difficult book with a child, with him pointing to words as we went along. He would read the words he knew, but would pause his finger on words he didn't know, and I would read that word for him. As the child is doing the pointing, stopping when he needs help (or maybe just reassurance on a certain word) he is in control of the situation, but doesn't need to worry that he will struggle on hard words.
That way, working together, you can tackle quite difficult books, and it can be more entertaining for child and adult. But don't leave him to struggle (unless he asks that you don't help TOO much, and some children will say that) but let him gain confidence on the words he can read.
A useful book that you can get from Amazon (or maybe order from a local bookshop, if you have one) is reviewed on the MN book reviews section, under 'children's educational books and courses', the Oxford Phonics Spelling Dictionary; that explains a lot about phonics, and will support children (and parents!) right through primary school.