Yes. The costs are huge. As an owner of a let holiday home (in the UK), you pay the council tax and all the utility bills. If holidaymakers put the heating on in July and leave all the windows open, you pay. You have to pay for turnaround cleaning every week the house is let. In a remote area, cleaners have you "over a barrel" because they're thin on the ground and can charge you £100-120+ which doesn't include a charge for laundry every time (another £30+). If you let your property through a holiday cottage agency, they will take 20-25% commission, but they won't fix things for you: you find the plumber, you find the electrician. Breakages will cost you too, from a table lamp to a new oven or new dishwasher, stained carpets all over the place. People I know with large houses have had them trashed by stag parties.
You need to be willing to fit your holidays (in your own home) into February half term, May half term (if you're lucky), possibly a week in summer, October half term, New Year.
You most likely wouldn't break even unless you're buying with cash, no loans. But on the other hand, OP, you may have enough money that none the things I've said matter. Some people actually do have the money to keep quite large holiday homes for generations, with no worries at all about outgoings.
A second home is wonderful when your children are toddlers to teens. It's so lovely, that feeling of going to that same familiar place. Again, OP, you may have the kind of income that gives you freedom to take wonderful holidays all over the world. But if you don't, you would have to take on board that your holiday home is your holiday, every time.