I passed the 50 mark last year and still am a size 6 (5 foot 2, so proportionately slim, not skinny). I have never counted calories or dieted, so it can be done.
Genetics play a large part (everyone on my family is easily slim), but also diet and excel use. When I say diet, I mean how you eat. I grew up eating majority of home cooked, from scratch food of my parents’ East Asian culture (mostly veg, rice, loads of fermented foods, and more fish than meat) with a minority of typical American junk, probably 20% or less. I continue this way of eating today, but eat a larger variety of cuisines than I grew up with. Sugary treats exist in my diet, but they are home made so I know what is in them. My gut microbiome is very healthy with all of the fermented foods I grew up eating, and continue to eat daily (kimchi, miso, yoghurt, cheese).
I have also loved exercise and switched from swimming to running when I hit uni. I still am running, but added weights about 15 years ago. If anything, running does not affect my body (just makes me have a larger appetite), but weights really make a noticeable difference. We lose muscle mass steadily after our 20s, and muscle mass is directly linked to metabolism, so weight training is essential to keeping weight off as we get older.
If I were in your shoes, I’d start an exercise programme focusing on weights with some cardio. Diet wise, I’d focus on whole foods and adding gut friendly things (whole grain, fermented food). Ease up on calorie counting, and make your own sweet treats. Also, accept that change won’t be dramatic, but slow changes are usually permanent ones.