No one has a neutral sex and I am very pleased the French judges were sufficiently in touch with reality to recognise this.
Humans are a sexed species; everyone is sexed, including intersex people. People born with disorders of sexual development are not a third sex (they don't make a third type of gamete) nor are they a sort of blank non-sexed person ('gender neutral').
As others have pointed out, the push to create a fictional legal category of a 'gender neutral' or non-sexed person is largely coming from trans activists, and lots of judges and politicians are going along with this without carefully considering the repercussions, as the French judges did in this case.
The OP was perfectly justified in describing what is happening to the legal category of sex in many jurisdictions as 'mucking about', for that is indeed what lawmakers are doing. They are acting as if it is NBD to sever legal sex from all objective reality, which begs the question - why do we have it as a category to begin with? If sex is purely a matter for the individual to decide, and members of each sex category have no shared objective characteristics, why have it as a category in law at all?
Thus I was pleased to read the judges' reasoning that:
The distinction between male and female was “necessary to the social and legal organization, of which it is a cornerstone"
And that: any judicial recognition of a sex outside male/female categories would have “profound repercussions on rules of French law,” and would entail “numerous legislative changes.”
Well, of course it would. That much should be obvious to anyone who works in law or policy. But somehow, lawmakers around the world have failed to grasp this as they blithely create 'non-binary' categories for anyone to opt into on passports and driver's licences.
Having said all that, I agree with ItalianGreyhound that there is a case for creating the legal sex designation 'intersex'. Not 'gender neutral', which is something nobody is. AFAIK, most intersex people are happy to be recognised as male or female, but obviously some, like the man in this case, would prefer to have legal recognition of the specific nature of their sexed bodies (intersex literally means 'between two sexes'). Of course this category should only be open to people with recognised intersex conditions, not people who claim their personalities and/or feelings make them something other than the sex they are.