I had a job interview a few weeks ago. I wore a mid blue spotty wrap dress and a short blue jacket from Fenn Wright Manson. I had bright red patent shoes on and a big red bag to match. I got offered the job.
Wasn't traditional but was smart.
It was public sector too. I think if you feel comfortable you will feel more confident. I don't feel comfortable in suits as I'm quite petite and the tailoring is never quite right.
Good luck, and don't worry about standing out from the crowd (as long as it's for the right reasons!)