"Is painting really that hard??"
No. You just need time and patience.
"We both work full time and have two toddlers so I would do it on evenings or the weekend."
I have two young children and if DH and I both worked full time, I would hate to spend my evenings and weekends decorating. Obviously one of you will have to look after the kids, cooking and clearing up, etc, while the other decorates. I think it could work if you're happy to take your time and just do it room by room over a long period of time. But if you want it done quickly and well, it would be better to pay someone.
DH and I are crap at DIY (we're also not into cleaning or crafting, weird comment!) but we enjoyed decorating together... pre-kids
I have fond memories of decorating our first flat and DC1's bedroom together. We'd put the radio on and treat ourselves to a takeaway afterwards.
Since having the DCs we haven't done any decorating, though, we've paid people to do it... but circumstances were a factor (I was heavily pregnant, had PGP, we had a demanding 3yo to look after, and several rooms that needed painting before baby's arrival). Perhaps when the DCs are older if we just want to redecorate one or two rooms we'll do it ourselves.
I think it makes sense to buy good quality paint, not the expensive "premium" stuff (F&B etc) but trade paint, we found Brewers had a good selection for decent prices.