That depends on the set up - I think ideally, that yes, the specific representation officers (Women’s, BAME, LGBT, Disabilities) should be voted on only by those registered as belonging to that group.
However, if a CLP follows the ‘All members’ format, and doesn’t have specific forums, then all members vote on all positions.
Of course, the smaller CLPs that are more likely to be all-member, are also less likely to have contested positions (I think my CLP has had only one applicant for BAME and Disabilities for several years running, the LGBT position hasn’t be filled in yonks and the WO had two applicants a couple of years ago, one of whom pulled out on the night of the vote in support of the other one, who has since been re-elected unnopposed).
Before the membership increase of recent years, many of these positions (along with the chairs, vice chairs and secretary roles) were decided by the same 10 or 20 people, who would just occasionally swap roles!