Had guinea pigs growing up and just used hay and straw from memory.
We had some about five years ago and did the same.
We adopted a Mum and two babies from someone who had been sold two 'females' and they are filthy!
No matter what I use it's filthy and stinks by bed time. I'm changing them and scrubbing it constantly because otherwise they would be sat in their own filth.
Seriously obviously straw and hay aren't working. I've tried that paper bedding too. Help!