I'm not keen on either tbh, but Violet is marginally better.
Violet sounds like violent and the 'shrinking violet' shy, wet persona that summons up isn't appealing. But it is also a fabulous vibrant colour, and a gorgeous scented flower, so has that in its favour.
Olive is a drab colour or a salty, oily snack.
Don't really get the appeal of either of them.