Some people are born infertile: they make no gametes, however still have a gender! So although it's correlated with gametes, it's not caused by them!
You mean "sex" all the way through this?
And the causation is the other way around - the gametes are caused by the sex. Therefore if you're producing sperm, you must be male.
There are two forms of an organism, one produces the small motile gametes, the other the large immobile ones, and those are designated the male and female forms respectively.
This one is a thought experiment, so it's obviously not real (yet) but the argument is still valid in principle: imagine the transplant of a female reproductive system into a man. His gender wouldn't change just because of this, in fact in this hypothetical scenario he may even retain the ability to make sperm.
I assume you mean "sex" here too? Reproductively he would be both male and female.
Although at that technological point you'd have to draw the distinction between natal sex and current reproductive sex.
What we normally care about is phenotype, not reproductive ability. It's the actual physical and behavioural characteristics than normally matter, not the gametes that were used to identify the two phenotypes as male and female.
An inserted uterus (from some poor woman) isn't going to make a man eligible for women's prison or women's sport, any more than any other body parts in his pocket.
A man with surgically removed male bits or inserted female bits is still a modified male type of human.
We don't have the ability to turn a man into a woman any more than we can turn a cat into a dog.