Writing here as an adopted child.
Of course you are not his 'babysitter' - and you know that, deep down - but it's no good just dismissing that feeling.
I don't think it's a given at all that he will want to find her or that if he does it will end up that they having a relationship where they see each other regularly etc.
It's hard to get a real sense of how many adoptees want to find their birth families. I have searched and have known WHO they are for over ten years - I got my adoption records 17 years ago. The agency sent some letters out which didn't go anywhere. I've had a choice to pursue things more aggressively and haven't for various reasons.
For me - I think it was the knowing and the being able to know which was most important though. My adoption was closed. It drove me nuts that other people with no connection to me knew more than I did about my circumstances. It is very different satisfying that need than actually taking things forward to reunion.
I have known other adoptees who have searched and met birth relatives from closed options and other who have not. US Facebook sites make it seem like EVERY adult ALWAYS searches but this really distorted. It might be too early to say how often letterbox contact adoptees end up in face to face reunion.
Throughout my life I have heard both: "I can't see how you could EVER do that to your (adoptive) parents." and "I would NEED to search. I can't understand why you wouldn't." The key will be supporting your DCs choice in a neutral way.
I am so very sympathetic to you - I can understand it would be hard to draft that letterbox contact when you are feeling anxious. I am not sure how my parents would have coped with having to agree to something like that, especially my mum.
The birth mum's life is going to move on too - I think it's really key to remember that. Her life's not frozen in time. She may be keen on letterbox contact and then not. She may have all sorts of reasons for backing away and when the time comes not be at all or very interested. It's a two way street, it's not just your DC and this woman waiting, indefinitely, with open arms to take him back.
Your the one building memories and a life with him. 