There's not exactly a direct link between those two things for most men, otherwise we'd spend most of our teenage years wandering around with an erection.
It's not like we can get it just to spring to attention on command, and yes, with age it can take a bit more coaxing. And to be honest, familiarity also means it's less likely to appear spontaneously as well. If it's the first time I've ever seen a woman naked, then yes, I'm probably going to get an erection without even being touched, because if its gotten to this point, then there's a fairly good chance I'm going to have sex.
But I'm not going to get an erection just because DP of 20 years is walking around the bedroom naked. Yeah, it turns me on, but it's not a reliable precursor to sex, because it's most likely she's just got out of the shower and is getting ready for work.
And there's a million reasons why a man might really want sex but not be able to get or maintain an erection. Tiredness, stress, nervousness, high blood pressure, diabetes. Hell, for some reason I always get horny when I've got a stinking cold, but there's no way I could do anything about it even if I could convince DP to come within 10 foot of me.
And then there's the really big reason. It happened before, so now I'm worried it'll happen again. There's no more self fulfilling prophecy than "I hope I don't lose my erection"