living in a place where kids still call for each other and have local friends within safe walking distance - this is something most people actually aspire to! 2-3 hours is a decent length of time for play, and the 'walking home in the dark' - it's January, and dark by about 5pm, unless it's a weekend you'd be walking home from nearly anywhere in the dark!
Most people on this site will have grown up doing the same thing! No harm in seeing your friends and playing most days, if you're sick of it being at your house, then tell your son that he won't be able to have anyone round to play today, and then have him answer the door and tell his friend he's not allowed anyone to play today. Or do what was done back in the 90s/00s and say 'you can go and play out, but you can't have anyone in today', and send them into the garden, or the local green etc with a football.
Alternatively, you could also encourage your son to go and call at his house too - if you're nervous about the walk, 9 is a fab age to be building skills and practicing - do the walk with him a few times right up to the door, then you go most of the way, but wait at the end of the street, then walk him halfway, then he can go himself. Same with coming home - agree a time you'll pick him up if he's allowed to stay and play the first time, then meet him halfway at a set time (and discuss when he'd need to leave his friends to get there on time), then he can come home on his own for a set time.
Absolutely hate this modern idea that your parents should be in entirely in charge of your social lift as a pre-teen, teenager etc! 9 is a great age to be taking a bit of freedom to walk to a friends house and play for a few hours. Plus it puts a huge admin burden on parents (mostly mothers) to ensure their child get plenty of playdates, and to control their kids social lives!
If you're mega nervous, take your son to his house, and speak to his mum, and just agree a time where you'll boot him out of yours when she wants him home by, so she knows when to worry if he's not home by. (Eg if she says he has to be home by 7:30pm, then you say, okay, that works for us, I'll make sure he has his marching orders at 7:20pm). And get her to agree to do the same for your son.
You can also mention that you don't mind him calling around, but to please not be offended if you say little Jimmy can't play today.