My Mum was a school secretary for more than 35 years - in both infant and junior schools. So I can tell you what she wore, if that helps.
In winter she tended to wear a skirt (cord or similar) - below the knee to mid calf length. Thick tights underneath. Calf / knee length boots with flat / low heel. Plus a light to mid weight knit. She looked a lot like this - from here.
She also wore dresses - below the knee but not too long - jersey, cord, wool or similar. Or she wore smart cord, jersey or other warm trousers with a light weight knit like this, and ankle boots.
I think her choices were influenced by the school sites and our location - up north, plus schools had numerous buildings - this meant she was frequently walking between buildings in the cold, rain or snow, so she wanted to be warm and have dry feet.
In summer she tended to wear a lightweight below the knee skirt, v-neck t-shirt and sandals. Or lightweight trousers, a top and sandals.
She never worse a blazer. Ever.
It was basically, clean, smart, practical and 'respectable', but not overdressed. Definitely no short skirts, leggings or higher heels - skirts and dresses need to come to your knees (at least) when you sit down.
If you're used to leggings and a jumper, I'd keep it simple with a smart dark pair of trousers, plus a lightweight knit like this. You want to be comfortable so you can concentrate on what you and the panel are saying, rather than being distracted by your clothes.