It is constantly listening, but it doesn't start recording until you say the trigger word.
If you believe what Amazon tell you, sure.
"In one highly publicized incident, a Portland family’s Alexa captured a private conversation after the voice-controlled device misheard what it thought was the wake word. It later sent the audio recording to someone in Seattle whose number was stored in the family’s contact list."
www.politifact.com/truth-o-meter/statements/2018/may/31/ro-khanna/your-amazon-alexa-spying-you/
Personally I don't believe a corporate like Amazon have my best interests at heart. There is a reason they sell these things at such ridiculously low prices and I don't believe it's just to encourage more people to use Amazon Prime or to sell them more gadgets.
People can have them if they want, that's their business but others are well within their rights to be wary of it. The idea of a device in my home that's ultimately controlled by Amazon (yes, you can switch it off, but really, how many people do? And if it's linked to your lights, kettle, heating and goodness knows what else, you can't really do that without a lot of hassle).
I don't believe Amazon are just selling cool little voice-activated gadgets to make my life easier. There is some ulterior motive there somewhere. Personally I don't want to buy a device that records my conversations and sends it to goodness knows where. How do you know Amazon aren't paid to send certain conversations (which contain certain trigger words) to the government or the police?
It's already happening in China, I don't think it's massively out-there to say it could be happening here too.