It must surely depend on the child. My eldest was a joy as a toddler. In fact, all of them were pretty easy, dd was the only one slightly prone to stamping her foot, but none of mine had tantrums. They slept, they were happy, they garnered admiring looks wherever they went. They were fussy eaters but not as bad as many children. Oh and they played and got on with each other.
Now my happy eldest child is 13, nearly 14, and he is as though he has been swapped for another child. He is frequently morose, he rarely laughs except at other's misfortune, I can never get a word out of him, he can get depressed, he seems totally lacking in motivation and by golly he can wind his siblings up if he wants to. The best I can say of him is that he is well behaved in school and his temper is very short lived so one minute he can be as uncooperative as it is possible to be, banging doors, shouting etc, and the next he is laughing and amenable. And all the time, there is the potential for much worse: drugs, drink, sex, etc.
So for me, based on my experiences so far, teens is harder than toddlerhood, but I would say toddlerhood was quite pleasant for me.
That said, I agree with Getorf, I much prefer teens because they are at least becoming adults and you can sometimes talk about interesting things with them.