I adore flats so much, the way they look, how comfy they seem, etc (esp with slim leg jeans). But I had to give it up. I have basically worn a small heel since my teens (mostly wedges, sandals, stuff like that) and never experienced foot pain like I did when trying to migrate to flats!
You'd think they would be lots comfier, but it didn't work out that way for me. I guess it has something to do with your foot shape or arch, I dunno, but it was such a relief to return to a small heel.
I am good with 1-3 inches, but best midway at about 2inch for most comfort. Not sure if genetics or habit contributes to this kind of thing but I was really miffed at first when I couldn't wear flats.
Interesting example: last year went for walk in a hilly region, many miles. Had a hunch to wear my cork heeled sandals with super soft sole (2.5inch heel) and was comfy all day. If I even dare to walk around city center for 30 mins in flats I am in agony.
Not sure why, but that's that!