The standard advice is not to retract the foreskin at all. However, If you don't do so, then problems can arise. Many boys reach puberty and adulthood unable to retract and it does ruin the sexual experience for them and their partners.
The foreskin is a 'sealed unit' which is fused to the glans to protect it. It normally separates anywhere between birth and age four, though it often takes much longer than that. The question is at what age do you check it out? You can be wrong if you do and equally wrong if you don't. As you (or he) gains the ability to retract, you also open the foreskin cavity where bacteria and foreign matter can enter and collect. This can lead to infection. Once you've begun this process you need to continue to wash everything away every day to prevent infection and keep everything under the foreskin clean and healthy.
Sensibly, you sound to have steered a compromise course by gently retracting your son's foreskin only as far as it will easily go. Sometimes as the penis grows, the foreskin fails to grow enough to accommodate the larger glans. This might have happened in your case and it may adjust itself in time...or it might not, and become worse. When this happens, or if your son gets persistent infections, he will need circumcising.
UK doctors are quite reluctant to refer boys for this simple procedure, so it becomes a gamble of 'wait and see'. The longer you wait, the more painful and embarrassing it will be for a boy if his problem persists and he needs to be cut. Having seen my nephew suffer for years off and on before his eventual and successful circumcision aged seven, I researched this subject. Result was I decided to follow the practice in the US and elsewhere, where the majority of boys are circumcised soon after birth.
And I've no regrets. Defective foreskins are not rare, they are a regular concern amongst the posts on this and many other parenting boards. Both my boys are doing fine without their foreskins and for me, it has taken away all the uncertainty which attends this aspect of raising boys.