I found 0-1 extremely tough.
0-1 is a shock to the system in terms of lifestyle (always having a child in tow, needing to think about naps/meals/bedtime etc), whereas you've already adapted to that when you have additional children, so there's less change in that respect.
0-1 was additionally hard for me because my oldest child was a demanding, high maintenance baby (everything was harder than with your average baby... sleeping, colic, feeding, weaning, etc.) and I struggled. Most people would have stuck at just the one baby after the experience I had, but I knew I wanted at least 2 children and just prayed for a different experience with the next baby... It did take me much longer than I'd anticipated to feel ready to try for number 2, though... I'd always imagined a small age gap (18 months - 2 years), but the gap is 3 years, for good reason!
1-2 (with a 3 year gap) was much easier than 0-1. My second child was an "average" baby compared to my first, so I adapted to having two in tow relatively quickly... It was a little tricky juggling the needs of two at first, but easier than I'd anticipated after experiencing my first baby.
2-3 (2.5 year gap) was the easiest change of the lot. I'm already used to juggling more than one and know all the tactics, so she's just slotted right in and it's been a very easy adjustment. I will add, though, that she's a very "easy" content baby... and (no matter what people say) she's not just super content because "I'm more relaxed due to experience", it's simply because my babies personalities are different. (There's no doubt, that I'm a more relaxed parent and I don't second guess myself like I used to, but that doesn't influence a babies personality or whether or not you can put them down "drowsy but awake", whether they're colicky, or take to food well, etc). What I'm trying to say, I guess, is that I know I've been lucky with my transition from 2-3 and it would have been a very different story if my children had been born in reverse (no doubt, in that scenario, my third would potentially have felt like a really bad decision for the first year or two, because I would have struggled to juggle his needs with caring for the others)!
I think, at the end of the day, it's about the long term... having another baby is always going to feel tricky and it's luck of the draw with personality and how "easy" your baby will be... but long term, your second won't always be a baby and it will likely get easier to manage your children as they grow up (the constant demands of babies and toddlers are very different to those of older children, although there are certainly still challenges, they're different/less full on and I find them easier to navigate).