Depends hugely on the child!
I suspect with a lot of children with similar difficulties, it is not the presence of the TA that makes them feel different or that alerts other children to the fact that they are different - their actual differences do this. Without the TA, they will still be different, and potentially even more subject to being ostracised or bullied. The TA can hopefully prevent some of this, and allow them to develop confidence and ways of relating to other children.
The presence of a TA might allow the child to perform at his or her best. It doesn't matter if other children in the class are academically weaker and need more one-to-one help than they are getting; that's a separate issue.
There's no way to tell whether a child at secondary who finds it difficult without a TA would or wouldn't have got on better as a result of having one or not in primary - there is no control to this experiment. It could well be that the child who needed intensive support in primary woudn't have made it to mainstream secondary without one, and so the fact that they are there, even if struggling, may mean that having a TA was just what they needed in primary. That doesn't mean primary was 'sterile' or 'managed' in some negative way, but might have been the way that allowed that child to progress to their current point.
Of course the social experience may be different, but in many cases, it's not like the child would necessarily have a normal social experience anyway. They might not be able to socialise freely because of their difficulties. If they are able to, a good TA is not going to interfere in that way.
Yes there might be situations where a child doesn't need a TA all the time, but a good TA can foster independence as needed. There are times when children may depend too much on the TA for things, or when a TA provides unhelpful support that stops a child developing, but those are problems with the TA support, not from the fact that the child has a TA.
I think it's totally individual to the child, the TA, the type of difficulties, the classroom, and it would be extremely difficult for an outsider to make an informed judgement.