OP I think you handled it very well. I definitely think you are correct to go and speak to her about this - she clearly needs to learn about boundaries and what is/isn’t acceptable.
To those saying she should have just helped, I can only assume you are the sort of people who don’t have any sort of sleep issues. Not everyone falls asleep easily, and some once people struggle to fall back to sleep again. Or maybe the OP has a job that means her sleep is crucial for other people’s safety. I’m curious, those saying she should have helped and this was all ok, if OP had said she’s a surgeon and you found out she was due to be operating on you or a loved one that day, would you still have felt so strongly that she should have helped or would you think her being well rested is more important? Or what about if she was a bus/taxi/train driver or a pilot? And you were due to be getting into a vehicle that she was due to drive/pilot and you found she’d only had a few hours sleep after being woken up for a missing cat - are you still saying she should have helped? And if not, why not? Why does it change just because it could impact you?
I love cats. I have a cat, have had multiple cats in the past, grew up with cats etc. I’ve had cats go missing and have searched round neighbourhoods for my missing cats and for neighbours too. I have never once thought it acceptable to go and knock on a neighbours door in the middle of the night. The OP mentioned there’s an ally between them (the neighbour is behind her house) - so presumably it’s house - garden - alley - garden - house, could the neighbour even see for certain the cat was in the garden (given when they came back at 5am saying the cat was in the garden but it wasn’t I think it’s unlikely they knew it was there)? Why wasn’t this neighbour looking hours earlier for the cat? Because knocking at 9/10pm, whilst I wouldn’t do it myself, would have been a more acceptable time to ask for help.
I’m someone that struggles with sleep. I often don’t fall asleep until 1-2am, and then wake up roughly every hour needing the toilet. It can take me 20 mins to get back to sleep on a good night, on a bad night I can lie there for a couple of hours. My alarm is set for anytime between 6 and 7am depending on if I am WFH or driving to the office. If my neighbour woke me up for a healthy, adult, outdoor cat possibly being in my garden in the middle of the night I would be angry. I am constantly exhausted as it is - a wake up like that would scare me (I’d assume something had happened to my son at university or my parents and it was the police knocking) and even if she left immediately, I’d still struggle to fall back to sleep with my heart racing. If I got up to help her there’s no way I’d be getting back to sleep after, and with a number of chronic health conditions plus severe pain levels, I know this would make everything worse for me as I always struggle more if I haven’t had enough sleep. Why is it ok for someone to do that to me for a non-emergency situation? Why should I put my neighbours unreasonable anxiety over my own physical health and to choose my health would make me unkind?
I wonder, if someone had posted on here saying “my cat has been outside all evening and I’m getting worried. He is an outdoor cat, but it’s 2:30am and he hasn’t come in yet. I think I can see him in a nearby garden (the house behind me). Would I be unreasonable to go and ring their bell/knock on the door to ask them to get the cat out of the garden for me?” - I suspect they’d have been overwhelmingly told they would be unreasonable to go and do that. I think even posters telling the OP here that she should have helped would tell this person not to do it. I imagine they’d tell her that she needs to get help for her anxiety, and look at ways to keep her cat in her garden/get a cat flap/make it an indoor cat. I cannot imagine anyone telling her to go and knock on the neighbours door.
Sorry for the long reply!