essbee, I hope the doctors appointment leads to more support for you, and not a 'come back in 6 months' answer.
I think roisin has a good point about how it could be a good idea to leave your son behind, as it could inhibit you talking about him if he is within earshot. But on the other hand as you've pointed out, it could be easier for the doctor to understand what you are saying if your son is there with you. Any good phoning the doctor in advance to ask them what they think would be best? Or possibly consider making 2 appointments - one just you and the doctor and one with your son also?
I don't know if this is a possiblity, but when the nursery had concerns about my youngest son ( he was 3 years old at the time), the health visitor arranged to come round to our house, did some tests with ds then deliberately stayed and chatted to me for an hour, just observing how ds behaved over that time.
Good luck anyway with whatever you decide.