I think YANBU, OP. I can sympathise, as I have the same issue with dd1, now P3. She is a dreamer, but still does really well. I do her homework etc with her, and try to help her to improve her concentration, but it's not easy!
In P1, I went to her first PE, happy that she was doing fantastically with her reading and really coming on with her maths. But her teacher kept going on about how she was always the last to get her coat at hometime, forgetting when she was to put her plimsoles on etc. I wanted to say "She's 4! Is her dithering a bit at hometime really a huge problem??" I was a bit disappointed, tbh, as she seemed to gloss over all the good things about my smart, kind, keen to please wee girl and focus on the negatives.
But then I thought, I don't actually need that teacher to tell me how well she is doing. I talk to her, I help with/ check her homework. If I'm honest, it would have been nice to hear nice things about my daughter BUT hearing it from a teacher wouldn't make it any more or less true. It's also true she can be a bit of a dreamer. I already knew that too!
Now when I go to parents' evenings, I realise I am unlikely to hear anything I don't already know. Sometimes I even tell the teachers things about my dds that they don't know (eg that they seem to be struggling with various elements of homework- the teacher is usually unaware, because they ask for help and I try to explain it, and go through it with them, so the teacher just sees correct homework) In subsequent years, dd1 has had teachers that she seemed to gel with better, who she was keener to please and possibly is just getting better with her concentration as she gets older. Personalities come into it too- some teachers seem to have more patience than others, and some seem to "get" my dds more than others. It's lovely when a teacher talks about your child with a smile and lots of nice insights that show they know and like your child, but that's not really what they are there for, I guess, so I take it graciously either way.
Just play down the negatives when your ds asks what was said! 