OP, I have also been in your old boss’s position many times, and been on hiring committees where my friends/colleagues/people I supervised were candidates (all disclosed to HR).
Jobs varied between working directly for/with me, or on my team, or with me as an independent and the job for another team. Each case was difficult, but I was required to - and did - put my personal relationship to one side and judge the person and the fit for the job. I always expected/hoped that the person knew I would be doing this, and whatever private disappointment they might feel, would understand it was professional and keep it professional.
Like the other poster, some rejected candidates impressed me with their professionalism in the face of disappointment, and their redoubling down of efforts. This all helped them next time. Other friends/candidates disappointed me by making it personal. For most of them, that behaviour reaffirmed to me that my decision was correct.
The comments in your posts about “ruining us”, wanting to message him outside work lines for feedback and crying if he called you to say you didn’t have the job etc all make me think the lines between professional and friendship have blurred too much. I would not want to hire someone who might cry if I have to reject their leave or couldn’t respect working time boundaries. When I did work closely with friends, they always knew business decisions would come first.
I appreciate you haven’t done these things, just thinking of them, but that you did think them does show the relationship has evolved to one where you are too comfortable as friends. Whatever his motivation in not choosing you, perhaps you could see it as a positive thing, that had you gotten the job, it may have been the end of your friendship anyway, in a potentially messy and difficult way over an issue where you thought he should understand or ‘factor in’ that you are friends, but he needed a PA, not a friend at that time.
If I were you, I would grieve in private. Don’t ask for feedback from him or HR - as you say, you won’t get the truth from him and it would be awkward. Then, if you can, contact him as normal, head held high and onwards. Don’t mention the job or interview at all. Think of it like seeing an ex-boyfriend after a break up. Be your normal positive, efficient self. Make him see your behaviour as professional and positive after this. It may get you a job from him in future, or it may mean he is a positive reference for a different job for you in future. Or it may come to nothing. But professionally you will look good, and I think that’s important.
As to the friendship - after your initial contact as normal, play it by ear. You may want to cool it off, you may want to continue, embracing the fact you are only friends now. No need to decide that now.