I quite like it, probably because of Katy's sister Clover in 'What Katy Did' but a lot of people don't seem to be aware of that and jump straight to butter and cows. If you can have Heather, Ivy or Willow then why not Clover though? Just another plant/flower name. Sounds nicer than, say, Ivy if you ask me.
But I probably wouldn't use it sadly, due to those people's reactions. There are lovely names with a similar sound - Chloe or Clodagh.