I work in recruitment, and if you're posting a popular job, you can end up with literally hundreds of applications. I'm talking 500 or 600 in the space of two days.
We physically can't write a genuine reply to every single one of those, so all the unsuccessful people would get would be a copied and pasted rejection email, if we sent emails to everyone. To those people, the job adverts really SHOULD say (and mine do), 'if you don't hear from us in X amount of time' then you're unsuccessful, just so people aren't hanging on for ages.
As a recruiter, I will contact people that 'got close' but didn't quite make it, so they know they're on the right track (useful for them and their future applications), and also those who wrote very sincere applications or appeared to have taken a good deal of time over them. You sound like someone who does care and who wouldn't send out hundreds of generic applications, so you're the kind of person who would hopefully get a reply from a decent recruiter.
But to be honest, I REFUSE to send a rejection email to someone whose cover letter reads, "I want to work for X Company because of my passion for engineering." We are not called X Company, they have forgotten to copy/paste our name in, and the job has nothing to do with engineering. I also don't reply to those who have CLEARLY not read the advert.
I take your point, and agree that where the applicant has made an effort, a bit more effort from recruiters is polite and fair.