Friend shocks me by suddenly saying he's female. How to handle this?

If it's a project that means a lot to you, I wouldn't risk saying the above - it seems like cutting off your nose to spite your face, not to mention that it sounds as though he could take it badly and possibly use it against you. Maybe the first bit, that you've prepared, would go ok but then if he argues back who knows where the discussion might go and how he might interpret what you say?

What about just being non committal about the subject and using his name all the time?

(Because you seem to have as much or more to lose from backing out than he does?)

No, you can't do that.

This

I think I’d be really pragmatic with him.
I invited Steve to work with me on this project and I’d like steve to turn up. Is that going to be an issue? I don’t want to know about your personal life, but to focus on the task.

As pp have said, It depends a lot on your work set up OP. I wouldn't speak with him about it until you know where you stand legally.

I don't see why this should make you give up the project if it's a project you'd like to do for whatever reason. What you want matters more to you than this person changing gender.

I think I'd say something like 'oh ok, thanks for the update - I've known you for ages as a man so apologies in advance if a 'he' or the name I knew you as slips out. It won't make any difference to how we work on the project will it? '

I mean just to be polite and show you are not expecting it to affect working on the project and covering yourself for any mistakes. I would find it really hard to avoid slips after knowing someone so long as a man - maybe when the person is in the room you'll remember because the person will presumably be dressing as how the person views as a woman.

I do agree it's mind bogglingly confusing and it's amazing how many people are changing their gender. But it sounds like you got on well up to now, so hopefully that will help make working on the project together go ok even if the friendship is no longer the same.

Sounds like the friendship is lost forever anyway, if you can't accept that he is now a trans female. I'm not saying that you are wrong - in your shoes I would struggle tbh. Your feelings are valid, and just because they have taken this step it doesn't mean that you have to be on board.

I'd try any way I could to cancel or delay the project

Just call him by his name, instead of a pronoun. I did this with a colleague. It’s a pain in the arse to do tbh, but the pragmatic solution. If he tries a conversation about being trans just don’t engage and changes the subject. If he forces it, do what pp says and say ‘your private life is not my concern’ and talk about work again. Then he can flounce if he wants to.

It depends how important the project is to you, I suppose. If it’s not the important just call it off, but if it is important to you, I would do the above.

I'm very sorry you're faced with having to deal with this OP.
The only tiny ray of positiveness I see in it is that it is only starting out, you are in at the beginning of his...what shall I call it politely? 'action'.

It's not like having to deal with someone who is already trans-IDing.
You already have a relationship with this person, on the basis that he is a man.
A friendship is a 50-50 thing. He has made a decision, he wants your validation, but he can't demand it from you.

I agree with MyThreeWords about establishing boundaries - this is where being 'in at the beginning' is useful.

He undoubtedly feels it is important for him to be 100% honest with you about who he is [now], and you are equally entitled to be 100% honest with him about how that changes the basis for your friendship with him, changes his standing as your long-established colleague, and causes profound issues for you, which are as genuinely important to you as his are to him.

As far as the work project is concerned - I just don't know what to say about that, except that it sounds very tricky to avoid it without making it appear that it's all about the trans thing, but months of working closely with him and having to suppress your feelings would be awful.

I hope you get useful advice here - at least you can come on here and tell us how you are feeling about the situation

Sorry if I missed you saying, but it's not clear if the project is in work time and you're an employee, or you're a self employed contractor, or it's something like the parent council at school, or if it's a completely voluntary/social hobby thing.
I think it'll change my answer slightly depending on the exact context.

From a legal point of view, it's best just to not work together again.

This is poor legal advice. Gender critical employees have tended to lose legal cases where they have explicitly refused to use a trans person's pronouns, and nothing in the law has changed that makes this any less likely to happen in future.

OP, it's not clear what capacity this research is taking place in - is this a work thing, a voluntary thing, an academic thing? If your relationship could be interpreted as employer/employee you have to be very careful about any changes of plans, because I'm afraid this person would have a case for discrimination if you suddenly drop them as soon as they come out as trans and can't come up with a very convincing alternative reason why this happened at just the same time. The same might be true if it's a voluntary or academic position, depending on the set up and the dynamic.

There is never any compromise here. You can't just say 'ok well can you believe what you want and let me believe what I want'.

That's the problem, there's no middle ground, you are expected to adopt his language and beliefs and that's not ok.

You are absolutely within your rights to say I don't want any part of this. You do you and let me make my own choices too.

You ask her/him how they wish to handle it.

They have been upfront with you and clearly feels able to talk to you. So feel able to talk back and ask.
I worked alongside trans people many moons ago. It takes a great deal for a person to have such a conversation. They know what is said about them and are at huge risk of personal attacks, bullying and ridicule.

Talk to her/him.

I really feel for you OP, what a hideous position to be in. If it was me I’d speak to him in advance of the project start date and say while I’m happy to respect a change of name I won’t call him she/her because as we all know it’s impossible to change sex (and very insulting to women to suggest that earrings and eyeliner makes him a woman). That way if he’s going to have mantrum he’ll not do it part way through and you have time to find someone else. The tide is turning on this nonsense but there are still some zealots who will affirm no matter what so it’s still a tricky situation.

I think you might have to do this unfortunately.

This approach is good. See how it goes.

I would be clear to both him and my company on my position from the outset ‘I’m not calling you she because you’re male, and thus ‘he’. It is degrading to pretend males are females. Do you want this project to go ahead.’
so many events have happened recently that we’re allowed to tell the truth again

Even if I can manage to avoid the pronoun thing, how can I stay silent or dodge the subject if he looks me right in the eye and tells me he's now female?
You could say "Your private life is not my concern, we are here to work," and then immediately ask him a question about the work.
Alternatively say something like "pull the other one" or "I don't see it the way you do" and again immediately change the subject to work. If he refused to let it pass and tries to engage you in arguing, then it is option one above: you are here to work not talk about his private life.

This frames his "trans-ness" as something private to him, nothing to do with you or work.

He hasn't yet changed his name but if he does I just don't think I can bring myself to call him by a female name.
That wouldn't bother me. Names are just names.