Same. Sometimes they kill them, or they die of shock, then we get either a whole mouse, or bits of one, to pick up.
One of our cats is an excellent mouser. The other is not.
The excellent mouser brings in live mice to teach the shit mouser to catch them.
Shit mouser is not learning! She plays with it for a bit, then gets distracted and looks away, and the mouse runs off! Unfortunately at this point, decent mouser has normally wandered off, so the mouse ends up just running around the house looking for a place to hide, only to be discovered again later.
Thankfully we now know the telltale suspicious activity signs in our cats to know we've got a live one knocking around.
If you're trying to catch a live one, I can recommend long handled dustpan and brushes, particularly the ones with the pan that swings up when you lift it up. We had a dedicated mouse catching one and can usually catch and release a mouse within a couple of minutes of being aware of its presence.
I close off as many doors as possible to try and contain it in one room, preferably one with an exterior door. Open the exterior door, so that it has the option to take itself out, or you have a quick route out yourself if you do catch it!
If you have a cat, take it to a bush somewhere and drop it in there to give it chance to recover and scurry away. If you don't have a cat, just chuck it out the door and shut the door quickly!
The worst one I had was the catch and release of a traumatised mouse that had been stuck under our bin all night. Then only to have to rescue it for a second time when the cat had sneakily been watching me and went and caught the poor thing again!