Hi there, my DD wasn't premature, in fact she was a week late, but she was very sick and spent a fair bit of time in the Neonatal unit. She's always been a sensitive, touchy child who even now, a 2.9 years, is so sensitive to noise, new situations and people.
I chatted to her playleader from 2s group about her sensitivity. She told me she was actually interested in researching the effects of neonatal units on babes as many "sensitive" children that have passed through her care has been in the neonatal unit. She wonders if the noise in the units from the alarms, and the constant to-ing and fro-ing of staff, and all the nasty procedures preconditions the kiddies to be sensitive to noise and the unfamiliar.
We did avoid stressful situations and known triggers when DD was young as the hassle wasn't worth it. All my mum friends had easy going, healthy babies so didn't understand what it was like. Now that DD is older, we do take her into different situations and in the main, she's getting better. However, we do have occasional times when the noise and chaos of her groups just gets too much and she crumbles.
I would say, just go with what works for babe just now. He's only little, and getting used to life. Your HV should be there to support you, so if you are feeling low and worried about how to cope, give her a call.
Sorry for the long post, just wanted to say that we'd been there too and whilst DD is still a sensitive child, she's learning to cope and so are we!