I grow my potatoes in bags on the patio too. I think you were just unlucky, that they found a nice place to live. Doesn't mean any other bags will have them. Being popular with slugs is worse, they eat all your spuds too!
SmallBoy and I just harvested one of our two grow-bags of potatoes. Not a bad little crop and it provided a springtime full of fun (planting, regular watering, checking the top of the plants, more watering, etc.). However, the soil in the grow bag was full of ants and what I think are ant larva. Very little freaks me out but the quantity of ants gave me the jeebies and every time I think about it I itch. I stopped SmallBoy (aged 3.7) from digging in the bag -- to stop the spread of ants, so they didn't crawl up him, etc. -- and put all of the used soil and tops of the plants in the brown (food & garden waste) bin.
Did I do something wrong with the potatoes or is it the nature of the beast to get lots of ants like that? If it makes a difference the bags were on a concrete path that had grass on one side and my house on the other (though there's a small space with rocks between the path and my house).
We still have one more bag to do but now I'm worried about letting SmallBoy dig around again.