You're worried about the air being full of feces, the toilet roll is right next to the loo and it is sitting there soaking up fecal matter, the whole roll will have absorbed fecal matter. Every time you handle it you are handling fecal matter that has had time to multiply bacteria.
Old fecal matter is worse than new fecal mater because of increased pathogen load. In suitable conditions like a moist toilet environment, opportunistic pathogens can multiply, leading to a higher concentration of infectious agents compared to fresh feces.
As the fecal matter degrades and dries, particles and pathogens can be dispersed into the environment more easily, potentially contaminating surfaces, water sources, hands and making their way into the air.
If you have any skin cracks at all, around your bottom area or on your fingers, you are therefore rubbing pathogens from fecal matter into your skin when you use toilet roll, and rubbing it on your hands.
What's that you say - you wash your hands with hot water? Yes, just like you can do with your toothbrush.
And if you are really really bothered about this, you should change the towel every time you flush, because the fecal matter will be happily breeding pathogens and bacteria all over the towel each time you flush.
It's all or nothing. Either toilet roll and towels (which are harder to clean than a toothbrush) are also a health hazard in a toilet environment, or none of them are.
(Hint, none of them are).