I have just made this switch. Gone for a job paying 30% less FTE than my previous one (actually even a bigger wage gap - as I am now also going part-time, so 40% less gross). Nevertheless, it has a much better work-life balance, and its pension is double as good as my previous one. Even though I earn considerably less, I am therefore building up more pension, and it is all DB, contrary to my previous one.
I am 40, and prioritising building up a decent pension was a key consideration. Having a young child, I also wanted to go part-time, and have a job within easy commuting distance, as well as WFH for the vast majority of time.
I will miss my old colleagues and the passion I had for my previous job, but if I take the right decisions now from a long-term perspective, I will set myself up much better for my financial future. Loving your job is great, but once I am old and cold and can't afford to put the heating on more than two hours a day, I wish I had taken a different decision, and been a bit more sensible.
I would make the switch if the job causes you stress, due to a toxic work environment, or if there is literally zero chance of career progression. If it just isn't terribly inspiring, I would stay put.