People might not care, but they do judge or make assumptions. Conscious or unconscious. Because certain names conjure up an image, an expectation or whatever. Might be cultural, social class, something else.
Not helped by names coming and going in fashion. Celebrities etc.
Names are really important. If you're worried about a name, maybe think how you feel using it. If people did judge.
You should love the name, your child has to live with it. Personally I wouldn't use a name with strong negative connotations. Difficult to know OP when you wont tell us what it is.