Really upset and angry as clearly they don’t value me enough to be head of department
You can't be an experienced manager and be this naive?
Of course you will feel upset a d disappointed not to have been successful.
However 'angry' is inappropriate. The other candidate met their expectations better, and on the day, performed to a higher standard.
In general that's the case - of course sometimes it is a poor appointment or other factors are at play.
In terms of moving on, yes, you probably should consider this. Not in response to failing to get the promotion but practically speaking, it's unlikely a new promotion opportunity will arise soon, and it might be time for a new environment / role anyway.
Definitely get feedback. Though sometimes there isn't much they can say - I was in a somewhat similar disappointing position (tho for a temporary role, and I still enjoy my current one), didn't get job despite more experience. I was disappointed; I knew I'd done a good interview. I didn't get useful feedback - ultimately I had to accept they preferred the other candidate to me, which was hard but is just one of those things.