Are you sure the children didn't ask to go in? I think you'd be harsh to go telling the couple off. The vast majority of people mean your children no harm - let's remember the percentage of the population that would ever, ever harm a child are miniscule. What does telling off an innocent couple do? If they were child-harmers, telling them off isn't going to stop them, is it. So all you are doing is chastising someone who means your child no harm. It's not protecting them from child predators, is it?
Much better to teach your children (as you have been trying) not to go with strangers. If that couple had meant them harm, you wouldn't have had the opportunity to tell them off for letting your child in their house to pet their shy cat.. the damage would have been done already.
On the other side of the coin, I rehomed a pet from a neighbour I didn't know (not cat or dog, think small pet). For months later, I had the child turning up at my door both alone, and with several friends, asking to see the pet as they missed the pet. Even as a (childless) woman, I had to turn her down again and again, and advise her to have her mother contact me to arrange a day/time. Sometimes, it would be my childless DH, home alone, when she knocked the door.
Girl turned up again advising her mum knew she had come over. I told her I hadn't heard this from her mum so would contact her. When I contacted the mum, she had no idea the girl had been knocking at all, and bringing several friends with her, at times. I told her girl was welcome to see pet, but that I felt it was only appropriate at a prearranged time when mum knew exactly where she was, and not with a big group of friends as I didn't know their parents/they wouldn't know where their kids were. Girl was likely bringing friends for comfort! Mum was very grateful for the offer, and we tried to facilitate but schedules clashed. She told me it was absolutely fine for us to let her in if we wanted, but I genuinely was too uncomfortable to do it off the cuff when mum didn't know she was in someones home at that exact time. I dread to think I'd have angry mums knocking on my door making out I'm a weirdo for giving in and letting them into living room to say hello to pet, I'd be so upset, for trying to be kind and patient with children.
Basically, wee girl missed out on a couple visits to her old pet, which was 20 steps into my living room and would have been 5 minutes, because people are terrified to be kind to children that aren't theirs, because of reactions like yours OP.
You're not unreasonable to be uncomfortable they were in an adults home. You are perfectly reasonable to not allow your child into peoples homes. You are unreasonable to chastise them I feel, in this case. And you are definitely unreasonable to judge them especially 'because they have no children'.
You won't the chance to chastise someone who actually means your child harm, I wouldn't go round telling innocent people off, it's much more important to teach children not to do it. All this just creates a culture of people who don't want to interact with children at all, for fear of being labelled something unfair.