I had a good old google about this, and it was Aspinal's secretary who came forward fairly recently to say that she had organised the trips for the children. She'd had to do it on either fake or duplicate passports for them, and it was so their Dad could see them from afar, so they wouldn't have come face to face with him. The children have claimed that the trips never took place.
It's all deeply sad, not only about the Nanny's death but that the children effectively lost their father over this. I really feel for his wife as well, it seems as though she just didn't stand a chance against Lucan and his support network, even if they supported him firmly believing him to be innocent.