I've been teaching 11 years now. In my experience, if you're in teaching because it is your vocation, then it is the best job in the world, but if it is just a job, then I think it is very easy to find the whole thing stressful and a pain tbh.
The downsides of teaching is that there is always more to be done, and if you are a bit of a perfectionist, then you can end up putting in ridiculous hours. I know I make a worksheet, the come back to it next year and think I don't like x,y,z and so need to improve it... and so on.
However, the plus side is that I am able to fit my working hours around my children. I work mornings, so am always able to pick up my daughter from the school gate, spend time with her until bedtime and then I start work again at 7pm, once the children are in bed.
But, teaching is not always flexible about time off - for example, if my daughter is doing a school assembly then I am not entitled to time off to go and see her.
Despite this, I love my job and enjoy working with the students. The hours are quite long - despite being employed only mornings (0.6 of a timetable), I also work every evening (usually 7-10) planning and marking and Sundays from about 4pm, onwards. But I think I am a bit of a perfectionist, so may do more than others!
If you love working with the students, then you will love it and it is worth all the hassles that go with it.
HTH