yesterday's thread - sorry (was looking for something else)
i'm also your ageish OP with young skin etc and also have white hair - started going grey in my early 20s, battled with it until late 30s then gave up. I love my hair and people do rave about it (though a male friend once took me aside and carefully, kindly told me I would look younger if I dyed my hair).
I recently visited london and the thought occurred to me when I was on the tube that someone might offer me their seat. so I didn't make eye-contact with anyone at all and tried to look cosmopolitan and chic.
I find the same as you OP - people often see the hair first (and it comes out badly on photos, a block of white, no definition 
) and make assumptions. when I talk to them - particularly men - I can sometimes see the realisation dawning and feel them fidgeting with embarrassment - almost see the thought bubble coming out of their heads along the lines of "oh! i thought she was old! but she isn't!".
years ago I was on a camping holiday (hair a mess) and went to an outdoor theatre. the girl on the 'door' (hedge) asked "pensioner?" when I went to buy my ticket. i was 42. I told the women on the tea stand - they were sympathetic.
a friend of mine, similar age, has given up dying her hair and wears it very short, like a boy. she visited the states recently and people were being very kind to her, looking at her with moony eyes. they thought she'd had chemo.
so you have my sympathy OP. I'll think of you if you think of me 