Why can't people just say "I didn't go to university, I chose to start work/concentrate on my home life/whatever" then if they don't resent students and graduates? IME, it is used by people who don't see the point in learning for it's own sake, and only see it in the context of money/work.
I used to hear it a lot more when I was at uni myself (I dropped out, and yes my life experiences since then have turned out to be more valuable to me, but they are not the same in any way) as a way of taxi drivers, managers at work etc trying to assert their supeirioity and demean what I was doing.
"Ooh, I don't need all those books telling me what to think, I went to the University of Life, me." and 9 times out of 10, that means they don't bother to read up on history, politics, art and so on, because they don't think it relevant to them. Not only that, but they look down on people who do. Imagine a world where everyone was like that. Or worse, imagine a world where just everyone working class was like that - if they didn't happen to go to uni, they woyld then decide that "real life" was better than reading or studying.
I read and study every day, I see it as my responsibility as a member of society and of the working class and as a parent - if everyone like me gave up and just claimed to have been educated in "life" then we would be in a sorry mess.
I know some people might use the phrase to mean something else, but every single time I have heard it IRL it is from people belittling learning and education, because they don't happen to like it for themselves.