I have one of these to carry stuff with me. But I'm strict about what I have in it (ie - as little as possible) - hat, gloves (I ski in thick mittens) and sunglasses for when we stop, sugary snacks (DH is diabetic, so I carry some as a back up), small tube of sunblock and occasionally a small bottle of water.
I've never taken it off at a lift (everything is squashy, so it doesn't prevent me sitting on a chairlift properly) and it doesn't affect my balance/ability to ski.
My trousers don't have pockets, and my coat pocket has the piste map in one side. I could possibly put gloves in the other, but I still want hat and sunglasses, so I may as well have a small bag.
Whatever you do, put your lift pass in a pocket (the one on the sleeve if you have one) and DO NOT OPEN THAT POCKET AGAIN. Losing your lift pass is something you don't want to do, and if you have anything else in the same pocket the danger of it falling out is high. Drum that into the kids too!
Add a couple of buffs to your ski gear list - invaluable for keeping warm.
For evening wear - I travel in one pair of jeans and take another - and that's it for the week. I probably have 5 tops max - all casual. You could easily get away with 1 or 2 skirts instead if that's all you have - but definitely ones you can wear with trainers/boots - nowhere is very formal! I don't think I saw a skirt/dress anywhere last week.