jaybee, I've had similar problems in the past with my 9 year old's playing out friends, and friends of friends.
You and your dh are wise to keep a close eye on things, but I know how difficult it is to avoid coming over all heavy handed.
I would do just as you are doing. Keep your ds busy.
Going on my own experreince of this, I'd limit the time he has to play with these two boys. Trouble is less likely to rear its ugly head if your son is out with them for 30 minutes rather than 2 hours.
Also be very firm about rules. Don't feel you have to be mega hospitable every time now that these boys have had an invite in and have been taken to the park. Don't let them assume that's what will happen every time they call on your son. It sounds like they are bored so they may well leap on any invite that comes their way. If this ends up making you feel cornered and put upon, if you are anything like me, you run the risk of losing your temper (a bit) in front of them, so best to cool things, even if you appear a bit unfriendly, and keep a polite distance.
If your ds is upset at your apparent lack of hospitalty, talk to him about the risks you feel he runs here, without saying anything nasty that could get repeated.
Best of all, imo, could say to your ds that it's the ages of these two boys that causes you to worry. You don't want him playing out for long periods of time with near teenagers. They are just too old for him. I say this to my son and he seems to accept this.
HTH