I have a few stories from over the years, so brace yourself 
Our first kitty was from a shelter. As soon as we walked in, she climbed up and started hanging off the ceiling of her cage trying to get out. When they opened the cage door she jumped on my shoulder and wouldn't move.
She's a princess. She had to have an operation on her eye a while back, and had to stay there for 1 week to recover. The vets told us she wouldn't touch her food unles someone was sat with her, so the vet nurses took turns eating their lunch with her.
Our second, a sweet skinny tom who didn't leave the garden for 2 weeks. It was winter and he was too scared to get into any baskets we left out, so we let him sleep in our kitchen eventually. The first night we let him in, he slept so deeply and peacefully, I kept checking he was still breathing. He clung to me whenever I went in to see him,
Like yours, he was terrified of men, just hearing a male voice would scare him off. He was also terrified if you walked towards him with shoes on, so I hate to think what he went through 
Our third (!), we were 100% sure we would take her into a rescue. I saw her peaking from behind the gate once, and she ran in, nicked some food and ran back out, but didn't leave the garden.
Her fur was falling out constantly, and she had a bald bleeding patch on her back. I also suspected she was pregnant, so we took her to vet who said she was just chunky.
4 days later, we found 4 kittens under our bed. 3 dead, the 4th lived for a day
After that, we couldn't bear to send her away to a shelter.
All three get a long, but the third treats me like a fellow cat, wrestling, pouncing, kangaroo kicking.
Lastly, no4! We didn't keep him, but rehomed him with a relative. He was a long hair silver tabby, who was completely matted all over. To the point that his skin was tearing and looked burnt. His bum area was matted with poo. When we got the matts shaved off, he looked like the gremlin with a grey tuft of hair 
None of the strays were chipped, neutered and we searched for their owners for about 2 months each. Sadly no one came forward.
I still check the lost and found cat sites, just in case they did have loving owners who are missing them but no one is asking about them sadly (luckily for us though!)
So, we're a bit cat crazy!