Your doctor was negligent. I would actually make a formal complaint about his advice because it was utterly wrong, and made your test a waste of money and time, and has potentially affected your health by delaying your diagnosis.
How long did you stop eating gluten for? If you stopped eating gluten for several weeks before the test, it will invalidate the test. The coeliac antibodies are made in response to gluten in the gut, so once you stop eating gluten, the antibodies stop being made and aren't there to detect. Eating gluten for 3 days before the test is not enough at all. It takes time for the body to produce antibodies in significant enough amounts to register on the test. You have to eat gluten for at least six weeks, and be eating significant amounts in at least two of your daily meals for the whole of that time. Look at the Coeliac UK website for definitive advice on getting an accurate diagnosis.
It sounds to me like you might well be coeliac, but your antibody levels had dropped due to you not eating gluten. What a waste of time!
I would do one of two things:-
a) see a different GP and, armed with information from Coeliac UK, have a blood test again, after 6-8 weeks of eating significant amounts of gluten daily.
b) ask to be referred to a gastroenterologist, who will look at you blood test results, (make sure they know that you weren't eating gluten and for how long) and decide whether to biopsy. Three days of gluten is absolutely not enough, and no doctor should suggest it is.
In your circumstances I would consider seeing a gastroenterologist privately if you are having trouble getting to this stage.
FYI - the same rules apply for the endoscopy/biopsy as for the blood test. You MUST be eating significant daily amounts of gluten for AT LEAST six weeks before the test. Otherwise a false negative may result.
Good luck. I have two coeliac DC. It took nearly two years for me to get a diagnosis for my eldest, due to similar incompetence from ill-informed doctor - but when we finally got to a clued-up doctor, the tests were rushed through, and he turned out to be very severely coeliac with a lot of damage. 3 years later and he is thriving, healthy, a different child. It is so worth getting the diagnosis, and it shouldn't be so bloody hard to do it!