So, about why non-binary seems to mean no boobs.
There are some obvious issues around this as others have brought up, but I suspect that part of the reason people are able to convince themselves of this is because children of neither sex have boobs. So that's seen as a sort of neutral state. In a way, you could say men's lack is neutral as well, they are an undeveloped organ. Breasts are very much an overt secondary sex characteristic, and directly connected to motherhood and reproductive role.
If you examine it for a moment, it shows up the lie of non-binary of course, because there is actually no neutral body, we are all sexed.
But lots of people have an image of themselves as children as more or less non-sexual.