It's really tricky as according to bloods you are just about ok; your thyroid has enlarged to accommodate probably antibody activity, and keep your t4 roughly ok, but it doesn't mean there isn't an issue which could get worse.
I'd ask if any antibodies have been taken - I wonder if this is the other test they mentioned?
Have a look at the nice guidelines for borderline results, WITH antibodies; I think if you have symptoms you can be treated. Borderline is usually 4.5-10 though.
I'd ask to be monitored every three - 6 months as as result could come back higher.
Another option is to be referred to an endo to get a really clear explanation and plan of action or as said above go to another GP.
With family history It's a v strong indicator.
A low dose of only 25 or even 4 doses of 25 a week would be enough to ease symptoms, but you're likely to need more eventually.
By the way, if you wanted to conceive, tsh 3 is way too high and you'd be put on thyroxine according to nice.