Very useful tweet explaining the mechanisms: I hope this Twitterer doesn't mind me copypasting his entire tweet:
'Why can't a male develop identical breasts to a female, since breast tissue growth is sensitive to high estrogen exposure in both sexes?
This has to do with what's called the hypothalamic-pituitary-gonadal axis, or HPG. This axis forms an important brain-to-gonad feedback loop at puberty that causes the masculinization or feminization of the body, depending on the gonad type you have.
Many of the effects will be permanent (so called organizational effects).
For females, the HPG axis is specifically the HPO axis (O for ovarian). At puberty, the cyclical production of estrogen from the ovaries and growth hormone from the pituitary creates a cascade of events that "transform the rudimentary mammary structure into the mammary gland."
Due to progressive elongation and branching of the ducts through puberty, thanks to the HPO axis, an extensive duct network of branches is formed. These branches will lead back to what's called the terminal duct lobular units (TDLUs), the structures that produce milk during lactation. Such structures are ultimately built thanks to the HPO axis, but they remain more dormant until pregnancy and childbirth.
For males, on the other hand, the activation of the HPT (testes) axis and the high amount of testosterone from the testes inhibits any major duct development. Males can develop extra, dense breast tissue (known as gynecomastia) through excess estrogen or other hormonal dysregulation such as cross-sex hormone use. But this is as far as the development can go. Males will not develop extensive lobules for milk production nor extensive duct branching.
The extensive and complex duct structures seen in the female breast cannot be formed in the male due to the profound impact of the two different sex-specific HPG axes across development (activation of HPT in the male and absence of the HPO). For more, see:
https://mdpi.com/1422-0067/23/7/3883
https://ncbi.nlm.nih.gov/pmc/articles/P'
https://twitter.com/zaelefty/status/1698588982148251770