I think the reason this is raising hackles is that - as with so much on Mn! - wearing pyjamas in a public place is partly perceived as a social class marker.
Though I think it's also cultural and generational. Long before you ever saw people outdoors in pyjamas in the UK (and I have to say that I personally have seen this with my own eyes just once, and that was in Ireland), I taught briefly at a major US university (late 90s, I think), and was a bit taken aback on female students attending morning lectures in pyjamas, having clearly just literally rolled out of bed and stuck their feet into flip flops. (Although a lot of the time 'pyjamas' meant short athletics-style shorts with side vents and a tee shirt or vest.)
When US students came on JYA schemes to my home university, they sometimes did this initially at the start of their first term, but soon stopped because it caused so much comment/laughter among the home students, who were always fully dressed for 9am lectures. (Also because it was cold and wet.)
From talking to the U.S. women students, they regarded being fully dressed, especially wearing actual boots/shoes rather than flipflops, and being made up and having your hair styled 'just to go to class!' as weirdly 'formal'.
But to go back to the original question, I think that, apart from spas, where the clothes dynamic is different, 5 star hotels wouldn't allow guests to breakfast in night clothes because their ethos is more formal. I stay at the St Pancras in London reasonably often, and the only time I saw a couple coming into the Chambers Club at breakfast in the hotel robes, they were intercepted by a staff member at the door. Don't know what nationality.