I gave up refined sugar ages ago and although the first few days were hard it never felt impossible and now I just don't want sugary "treats".
Dairy is another thing altogether. I drink my tea black but cheese and plain yogurt with fruit are my go to treats, especially since I weaned myself off the sugar. I don't know what to have instead and find if I don't include them in a meal, especially a packed lunch type meal, I'm still hungry.
I'm wanting to give it up mostly because of the environmental impact. I've tried some of the substitute milks and "cheese" but they're not nice and seem over processed to me, quite the opposite of the healthy whole food diet I'm aiming for.
I've also really reduced the amount of meat I eat, that was easy in comparison to cutting back dairy.
How is it done?