Babies are usually difficult because they have an as yet unmet need - which is not usually the fault of the parent, I hasten to add! The baby who wants holding 24/7 or cries a lot has a reason - reflux, colic, cow's milk protein allergy, tongue tie causing difficulty feeding/ gulping air and resulting discomfort, sensory issues which cannot be diagnosed in babyhood but cause sensory seeking or sensory avoidance, and of course teething is more painful for some babies than others just as wisdom teeth cause more problems for some young adults than others.
Those things are in most cases not predictors of the mostly different issues which cause toddlers, children and teens to present with challenging behaviour.
Also parental personality plays a massive role in what's perceived as "difficult" - for some people holding a baby all day is unbearable - they have to put the baby down, the baby cries, the baby is "difficult". For other people carrying a newborn/ baby under 6 months all the time and safe cosleeping is instinctive, natural, what both mother and baby want - baby is content, baby is "easy".
The parent who finds the baby phase easy might find the toddler or child or teen phase hard though, and vice versa.
A lot of parents find certain phases very challenging or very natural, and this isn't necessarily a true reflection of whether the child is "easy" or "difficult" but of the parent's natural comfort zone.