We toddlers sleep rubbish, just the same as some newborns. I don't think there's any more to it than that.
My first was bottle fed from birth, slept decent stretches until the 4m sleep regression, then got worse. She didn't sleep through at all until gone 3, and now it's about 2/3rds of night that she does at 4. Nothing bf related at all, she just struggles to sleep through.
Until gone 2, if during a wake up (or her long bedtime - it took 1-3 hours!) we needed to pop to the loo, or try a drink, she'd often be hysterical and retching before we even reached the loo. Not clingy in the day, but very clingy at night (but refused to come in with us).
My youngest is breastfed, and she slept far worse - she was the double digits one. She also liked a wake up for a 3 hour block overnight, most nights, for a period of a few months, when aged around a year (so generally wake ups anywhere from every 30-60m with being awake say from 1-4am, and then maybe a 2-3hr block if lucky. She is now (generally) much better than the 4 year old, despite being younger. Thank goodness.
It's pretty normal for toddlers to wake up at night though - we certainly aren't the exception. Certainly from talking to friends, a child that sleeps through before a year is extremely lucky, as most are still struggling at 2 and 3.
There are enough difficult stages and challenges with parenting that I think it's impossible to pinpoint an overall most difficult part for everyone. Everyone's experiences differ so much.