I skied at 20 weeks pg in Les Deux Alpes (quite high alt). It was fine. My major asset was being very fit which helped with the whole pregnancy, tbh. I did get tired more easily and so I skied an hour or so less than I might otherwise have done. I loved choc chaud as much as I would have loved vin chaud. Bump also loved choc chaud, and really let me know about it!
Yes, there are risks in skiing, including being hit by someone else. I judged that this was no greater than the risks I was encountering in my daily life at home (driving, my work environment, crossing roads...) Also, a sharp landing on your bum does not immediately translate into the worst outcome for your baby, just as it's perfectly possible to have disasters on a beach holiday. My philosophy was centred around not putting my life on hold. Once you've had your baby, you have years ahead of you to not be able to enjoy the full range of the pursuits you formerly loved.
Only you can decide about this - and if you are feeling at all nervous you probably should not do it. If you are a confident skier at your level, and you have someone with you all the time, you may well be fine. If you are going to rely on someone to chaperone you around, they need to be comfortable with your decision as well.
Even if you go on the holiday and stay off the slopes, inform your travel insurers. They shouldn't refuse to insure or change the price (discriminatory etc) but if you claim for even a totally unrelated injury (one of my party dislocated a thumb having over-planted a pole, for example) and you have a 'condition' you haven't told them about, you might be somewhat stuffed.