What business are you in?
From your last post it sounds as though the job two levels above you is solely about managing files. Is it insurance? If this young man isn't very good at the files side, is there something else he does do? I don't mean to sound critical of you, but I think you may be missing some element of what these managers do.
I still think you are focusing on efficiency, but if they repeatedly promote people who are less efficient, it would seem that efficiency is not what they are looking for.
This is something the book is great at. I have read it many times thinking "unfair bds" and come away with "OK I'll try that". For example, the promoted guy has a good personal relationship with someone important, so it may be that they are looking at some type of networking ability. Without knowing what business it is I couldn't hazard a guess what they may be after.
In terms of the opportunites you are not considered for, what exactly happens when such an opportunity arises? Is there any way you can be more active about actually doing this? EG do the opportunities go on a website or anywhere where you find out about them and can officially apply? It sounds as though you have said to someone "I'd like to be considered" and that isn't working. (Last December I told my boss I would like to be considered for official homeworking....I'm still waiting to hear whether it's even possible....) If you have sent an official application in, it's harder for them to ignore you, and they will have to prepare a story as to why they don't want to promote you. Often I think these things aren't sinister, they may just vaguely feel your face doesn't fit the next level up, and you will have to push them slightly so they realise that you can actually do the job and are a good promotion candidate.
I don't think there's anything you can do about your boss not being promoted - they can't discuss with you why someone else didn't get promoted. What they can do is tell you why you don't get considered. They should also be able to say why someone else is considered over you.
In your performance appraisal I think you should just write it all out and ask them all these questions, as they relate to yourself. Practicing in front of a mirror works for me if I want to stay calm and not get emotional - when something makes you sad, saying the words can bring the tears out, whereas if you practice the words will become more familiar.
EG
"I have expressed a desire to be considered for promotion opportunities, however x opportunity and y opportunity came up and A and B were put forward instead. I am keen to advance in my career and if you can give me any feedback about how I can improve so I do get put forward, I would appreciate it." Look them straight in the eye and brace yourself to hear things you might not like hearing. This would be a good result, because they are showing you enough respect to communicate honestly with you. They might talk about something like a positive attitude for example. It sounds fluffy, but it gives you a clue as to what you could change.
If what they say is unclear, ask for clarification.
A great one from my previous (sexist IMO) employer was "Well...you don't really have gravitas."
Me: What do you mean by gravitas?
ExBoss: Er...well...I think of gravitas meaning I believe what someone is saying when they present.
Me: Can you give me any examples of colleagues who have gravitas?
ExBoss: Dave, Jim, me haha...
Me: Female colleagues?
ExBoss: Er...I didn't like Thatcher but she did have gravitas.
I am not an MP btw 
That dialog gives me a clue (no sh*t Sherlock!) that the problem is not something I can fix. Hence it was his problem and not mine. If they can't tell you what the problem is, it's theirs not yours. Although, if the meeting gets very awkward, they may not tell you because they feel you won't be able to handle it. This is why it's worth practicing staying calm.
The book has some examples of similar dialogs where the boss does have a proper reason why someone is not seen as suitable for the next level up.
This is very long but hopefully it is of some use.