If you don't identify as either male nor female, or if you identify as trans then what do you do? If you can't compete as the gender that you choose then the only other option is to compete according to your birth sex, if that is what society forces you to do.
There are plenty of literary prizes that aren't segregated by sex. This prize is specifically for women because women are underrepresented in literature. You might think it's all bullshit and that there is no reason to have a special literature prize for women, but the only thing that women share is a sex. There is no female identity.
I think this writer is perfectly entitled to take part in this competition because of their sex, and it's also obvious that their work explores specifically female experiences.
The only problem is the assumption that this writer is unusual and ground breaking because, unlike all the other female writers, they don't have a specifically female identity. That is insulting.