See a solicitor a and get proper legal advice. Some solicitors have a family law surgery which will give a free initial consultation.
I am not sure what the exact situation will be in the case of a stepfather without PR who has been divorced 3 years but in the case of my parents they had to apply for permission to make an application. Only a parent with PR can go directly to court.
Anyone making a court application will need to attempt mediation first. You could refuse mediation in certain circumstance, if there was domestic abuse, or on the grounds the child does not wish to have contact. If there are concerns for your or the children's welfare, then the mediator could deem mediation to be unsuitable. (I personally don't think there'd be anything to lose by attending an initial mediation assessment, known as a miam, to discuss the situation with the mediator and see what happens. If you go ahead with mediation, for example you could just state that the child does not want contact and offer indirect contact (letters, cards) to pass on to the child, with it being left to the child to respond if they chose to do so in the future.
Once mediation has been attempted, or deemed to be unsuitable, then he could apply to the court for permission (please check this part with a solictor as I may be wrong) . There will be a directions hearing and you will be sent details of this. You could turn up and oppose it, on the grounds the child does not want it to happen, but bear in mind in most cases in the interests of giving a fair hearing, the vast majority of applications are granted. This bears no relation to the merits of the case, whether they have a strong or no chance, so dont worry if it's granted. You will then need to attend a hearing called a FHDRA, which is a bit like mediation to see if it can be resolved without going to court. It's very likely that your dc's wishes will be taken into account. I think your ex may have a lot of hurdles to cross to get contact, but do get some proper legal advice. If you can't afford it there may be volunteers at the court that can help explain the process and help with paperwork, or there may be volunteer solicitors there who can give a bit of advice, and explain a bit more about the legalities. Ask at Citizen's advice to find out if there is anywhere else you could get help x. Good luck