It's confusing isn't it!!
When gardeners talk about "drainage" they're really talking about two interconnected things (possibly more, but I can only think of two right now). The first is oxygen exchange (which is needed at the root as well as the leaf), the second is moisture, i.e. the ability of the soil to hang onto water.
If you have a really heavy clay, it can be like the stuff you made pottery out of at school and this is a problem in two ways. Firstly, it's really dense so oxygen can't get in and out easily. That leads to a build up of CO2, which leads to a build up of toxic chemicals/denitrification. Secondly, it's really damp. Little seedlings may well struggle to force their roots into it, helpful bacterial and mycorrhizal cultures may struggle to establish while unhelpful ones are encouraged (damping off), and it's likely to be too waterlogged for anything that likes normal to dry conditions.
A heavy potting mix can pose similar problems - so you add perlite to encourage oxygenation and a bit of drainage, without making it too freely draining. It gets all in between the big lumps and keeps it loose and friable, which helps both air and water get in and out. That stops the mix getting all heavy and claggy and 'orrible.
You still want to hang on to a bit of moisture for some plants, though, so the fact that it does that a little can be helpful. The fact that it takes on water quickly helps plants to hold onto the water as well - I don't know if you've ever tried to water a very dry pot, but it can be that the stuff just bounces off the top or runs straight through rather than staying in the soil. Perlite helps with that by grabbing and holding onto the water. For other things where you really want free drainage, grit is better.