Not knowing your name, I can't comment on your situation but; DH's Nanny was christened Clipsy with no middle name. She hates it. As soon as she was old enough to have a say in these things she introduced herself as Peggy ... She was quite cross with us for not giving DD a middle name.
By the same token, I also know someone called Elizabeth Anne. She hates Elizabeth and has always been known as Anne ... my point being, it's not only 'unusual' names that can cause upset. You just have to go with a name that you like and hope that the child likes it too.