I found potty training far and away the most difficult and stressful thing I have had to do as a parent. The only time I have ever hit one of my children has been over struggling to get him to use the toilet.
I don't think that I'm Mother of the Year or anything I do really, but false modesty forces me to declare otherwise, but I think I'm basically a pretty good parent. When it came to potty training though, I was a total disaster - I should have reported myself to Social Services.
Yet I've had friends who have breezed through it the bastards. However, they have really struggled with, say, breastfeeding which I loved.
Obviously this is so far - I am aware that there is probably many a nightmare scenario just waiting for me. 
So what about you? Which stage or process did you find most unbearably awful challenging?