I agree..
Firstly because you can have a 'crisis' at any age, not just in mid-life.
Also it's a very dismissive phrase to use for people who are actually going through a difficult time in their lives, whether they are middle-aged or not.
The phrase trivialises difficulties some people have.
Personally I think you can go through a 'crisis' at any age when you haven't met your aims or goals, or when your expectations don't meet reality.
For example I wanted to be married with kids by now and I haven't even got a boyfriend let alone children. So I am having a bit of a crisis about that. And also about my career.