It may seem a crude analogy (although the science supports the similarity in effect) but think of porn like a drug.
You’ll find people who can take drugs very occasionally, but drugs are addictive and those who use them regularly are highly likely to become physically dependent on them at best, addicted at worst.
Just as with drugs, where you build tolerance over time, heavy porn users need increasingly novel / extreme content to get the same “hit”. There are plenty of studies into the effect of porn on the brain.
Porn is designed to be addictive, to be more appealing than reality. And those who use porn regularly can find that the actual physical act of sex is not as appealing as porn. Erectile dysfunction may occur, but delayed ejaculation / “death grip” is likely to be more of an issue.
These are commonly recounted issues - not unusual.
Then add in the ethical issues, the gradual coercion into acts you’re not comfortable with, the impact of porn on a person’s perception of sex, and it can do significant damage to a relationship.
So maybe you’ve got a partner who’s the equivalent of someone who takes cocaine a couple of times a year and never thinks about it in between times. Or maybe you’ve got a partner who you think takes cocaine once a week and is actually taking it every day. Not a risk I take personally after my own experiences of physical, sexual and mental abuse but you seem comfortable with it so that’s your choice. Just don’t be so quick to assume it’s harmless.