yep, assertive is about being in control of how you put across your thoughts and feelings. Neither passive or aggressive (or passive/aggressive) are being in control - one is suppressing, and the other is exploding.
However, often, people do not realise they are being aggressive versus assertive, as I said it is a fine line, and not always clear to us as individuals in the midst of expressing emotional views that we are being aggressive rather than assertive. Leaving things to build up means that even if we think are 'in control' of how we are objecting to a situation or opinion, we may in fact not be, as our resentments project through and come out in what we are saying, in tone, body language, as well as the actual words.
Something like 7% (can't remember the stats exactly) of communication is verbal through words, the rest is body language, tone, pitch and if we are not aware of how those aspects of how we communicate appear to others we are not in control.
For example, an opinion that is assertive can come across as aggressive if your arms are folded (if you have already been feeling annoyed, this defensive stance is one of hostility at the potential aggression or hostility expected in return, it sets it up for negative response).