Female humans are also one of the only mammals to menstruating. And it's not because it's attractive to men.
It's because the physical resource drain of reproduction is far greater on female humans, than other mammals. Women menstruate as self-protection, because the egg burrows so deep into the lining that a thicker, shedding endometrium allows for survival.
I would say that logically speaking, the reason for fatty deposits at the breasts, stomach, and hips, would be because producing milk to sustain human infants is such a physical drain on resources, and having stored fat increases survival. Also, with human infants unable to cling onto the mother, and the infant breastfeeding regularly for several years, the pendulous shape of the breast would allow the infant to feed while the mother engaged in other activities, increasing the survival rate.
Also, as others have mentioned, seeing as breasts are composed of fat, in our ancestral past where humans would have been on the verge of starvation a good deal of the time, it seems likely that very large breasts are peculiar to modern times.
Some people also seem to have the idea that evolution actively selects for things. It doesn't. All evolution does, is mean that traits that don't actively kill you or your infant early on, or affect your reproductive success, are continued, and traits that do, are not continued by virtue of the carriers dying out.
So human males finding breasts attractive making breasts bigger, is an entirely illogical way of looking at things. If anything, the larger breasts would have to have existed before males found them attractive. And then one could argue that they meant those females were mated with more often, and thus it became more wide-spread - except I haven't noticed that flat-chested females are less likely to have children, so that explanation doesn't hold up.