Benjamin was the 40th most popular name in England and Wales in 2020, so far from rare.
And I totally agree with @pinklavenders that’s it’s not only your baby’s birth year to consider when thinking about popularity. Ben feels dull to me because there was at least one Ben in every classroom, more Bens at uni, more Bens at work. My husband had two Bens on his stag! After education age they’ll be in mixed age groups, so it doesn’t matter that they were the only Ben (for example) in their year group, because as soon as they hit work, they’ll be in meetings with two other Bens from the time Ben was very popular.