Of course some people who say someone with a penis can't be a woman may be bigots. Others, though, may simply disagree about the meanings of words; to so disagree simpliciter could not be called bigoted by any stretch of anyone's imagination.
I think the word 'woman' means 'adult human female'; it follows from this that I think the set of women and the set of people with penises are disjoint.
Those who, in support of marginalised trans people, or for whatever reason, think someone with a penis may be a woman, obviously think 'woman' means something different to what I think it does. They must do, if they are sincere.
You, for instance, @Girlboss1989: can you help us out? What do you think 'woman' means?
[I take it that "'woman' means 'whatever someone who claims to be a woman is'" won't do; it's clear enough that goes nowhere towards explaining what 'woman' means. (How strange this needs saying! O tempora! ... )]