MrsMcAvoy it is totally not daft! It is totally fine to not want other people to look after your baby.
If you decide to let her look after baby at some point in the future, make sure you feel comfortable with her ability. Despite what others might think it is not always easy to look after a baby well. Some might do a brilliant job, some not. The key thing is, do you think she is a safe person to look after your baby, alone. If you think she is, then go ahead, when ready (as others suggest leave him for a set time with her in your own home and do whatever you wish with that time). If you do not think she is a safe person to look after baby, don't leave him with her.
Children are precious, and there is nothing wrong in caring about who gets to look after them, especially at the stage where they cannot talk or tell you what they need.
If your friend keeps on nagging/offering, just say you will ask her when you are ready. Do not be pushed into something because another adult wants it, that isn't fair on the baby.
But I agree with others there can be a time when a baby needs to be looked after, emergency, health issues for parents, funerals, whatever, and then it is good if baby can be with someone else, but only if that person can look after them well.
And a reminder, you are not being silly, don't let anyone tell you that you are. 