Repeating a year isn't a big deal when lots of people do it.
While there are countries where it is more common, there aren't really schools where 'a lot of people' are held back and even with several, they will stand out. 'Super seniors', 'doing a victory lap' and other sarcastic labels are generally applied to Americans kids held back or otherwise end up being 19 when they start their final year (with the youngest being 16 when that year begins).
Young kids who are pushed forward also stand out, though less so in schools with mixed age classes, but those in US systems which strongly encourage high school kids who can test in to take university classes, there is a big issue of burn out, emotional and social isolation, and the youngest who do start anti-social 'adult' behaviours tend to start doing them at a younger age compared to their older peers (as they're likely doing it alongside older classmates). A 16 year old might have the academic ability for University English, but that doesn't mean they have the ability to handle all the adult situations that go with University. There is a reason why even in parts of the UK which have college and university programmes for pre-16s, they usually mark those students out so staff can keep an eye on them and often keep them mostly separate from older students.
There just isn't much evidence that repeating a year helps anything - in most studies I've seen, there is a strong correlation with grade retention and dropping out of school entirely and making less progress both within the repeated year and over the long haul compared to those who haven't ever been held back. Targetted, additional remedial classes work better than just doing everything all over again (if it didn't work the first time, why assume it will the second just because they're older?) and would also be better to support those students who could do extra without putting them at greater risk.
No system is perfect, and age year groups certainly aren't, but I'd rather push for better support within age groups with maybe more flexibility around the edges for the oldest and youngest than make an ability-based system which I'm not sure how it could ever be holistic. If someone is Y8 level for English but Y5 for maths and Y3 for PE, how would that work and what would happen to them socially?