I'm in my early 60s. Yes there were fewer coffee shops but there were still cafes, bakeries and what we called 'confectioners'. To be honest they were still bakeries but erred more towards cakes and lunchtime pies, pasties, sandwiches, etc. We had two or three confectioners within walking distance of our house on the edge of town - although to be fair there was also a large employer nearby and nice picnic spot.
My Mum baked but we had fresh cream cakes from the confectioners as a regular treat, particularly if we had a day out. If we went to the next town shopping we would always get lunch and a cream cake. Confectioners also tended to be highly skilled, making their products on the premises rather than buying in.
Despite home made cake at home and the occasional fresh cream cake, I yearned for 'shop bought cake' and there was plenty around in the shops. I still love a really good synthetic cake - ideally in an alarming shade of pink or yellow.
Cake has always been around, albeit with different trends, but I just think there was less disposable income then. A lot of things were more expensive, including food, despite so many people thinking it was the land of milk and honey in those days. Chicken, for example, was as Sunday roast treat and not a ubiquitous cheap food.