YANBU. I envy them as well.
But, then again, I also envy people who grew up with parents who loved them, who had a nice, warm, clean home, who had friends and then went on to do well in education, meet somebody who loved them and they loved, are employed and in a position to buy a home, then have the children they wanted, so that they simply don't have a concept of pain, abuse, fear, poverty or loneliness other than in the abstract.
I've met a few at a hobby group - much, much older than me, and they are absolutely lovely people. I wouldn't want them to know what it feels like. But I still feel that pang of 'I wish I could have that innocence'. They're still very aware of how fortunate they have been, they are compassionate towards anybody who has suffered - but they don't 'know' what it feels like.
It's a natural feeling, I think.