OP, YANBU and on the whole I agree with you, but I do know quite a lot of otherwise sensible people, including my own DH, who are more protective. When DS1 was about 11 we had arguments (not in front of him!) about whether he should be allowed to walk to some places on his own, and DH actually asked me to follow him at a distance once or twice to make sure he was coping OK before he was happy to let him.
I personally think it depends a lot on where you live and what the traffic is like there, but most children should be getting some degree of independence by the time they're in year 6, including learning to cope with traffic on their own (perhaps not on a busy dual carriageway straight away, but no reason why most of them couldn't handle ordinary suburban roads).
If anything I am more concerned about traffic than being attacked, which I suspect is rare enough not to be worth letting children miss out on vital life skills. If there was absolutely nowhere my children could walk alone close to home because of unusually heavy or dangerous traffic, I would positively welcome them having a chance to go to a friend's house in a quieter area where they could get that practice. I am wondering if the parent in this case actually knows how safe your area is, traffic-wise.
I don't think I would try to discourage the friendship, but maybe only invite the other girl round at times when it's convenient for you to do something with them, and if the parents ask why she's not being invited so often, make the point that it is their restrictions that make it difficult for you.