Is the annexe physically attached to the house, or does it have its own four walls?
As PP have suggested, adding extra space to a home can cause a council tax re-banding after a sale. If the annexe is attached to the house, I'd say the council should re-band your entire home, not charge an additional council tax for it.
Would your main home moving up a band or two cost more money than if you simply pay the extra council tax A for the annexe?
I use this tool to find out bandings:
https://www.mycounciltax.org.uk/content/index
With respect to annexe or not annexe, friends of mine had this argument with a council in North Yorkshire for several years. Theirs was a workshop, turned into an annexe (with planning approval), and not attached. The original purpose was to house an elderly relative but she died before it was complete.
Somewhat ironically, they were renting it out on Air B and B, and the council were all over this, in their adverts, looking at photos. It had a lounge with a kitchenette, a bedroom and a shower room.
The council argued that it was a separate dwelling because it had a kitchen. So, my friend removed the fitted oven, and put in a desktop oven with two rings on top of it.
Then the council argued that it was still a kitchen because it had an under-counter fridge in it. They said something along the lines of 'if it has white goods that you couldn't remove yourself in the event of an emergency, it's a separate dwelling.'
My friend said 'if there's an emergency, I'll put the fridge on my back and carry it out.' - at least - that's the story he told us, anyway.
They managed to make the council back off, but in all honesty that could just be because he's one of the most stubborn and pedantic people you have ever come across, it's possible they just didn't want to deal with him anymore, so they agreed just to get him off their backs.
Looking at his address now on mycounciltax, I see that two other properties in his postcode ARE paying an extra band A for a separate dwelling, but he is not.
So, my advice for you would be: make the kitchen(ette) as small as possible, remove any under counter fridges or freezers and/or ovens, leaving you with a few cabinets, plug points and a sink.
Here's what google says about council tax for older folks:
A granny annexe will be exempt from Council Tax charge so long as it is occupied by a dependent relative of the household as their main – or only, home. Family members are classed as dependent when any of the following applies: They are over the age of 65. They are substantially or permanently disabled, They are severely mentally impaired
As far as I'm aware, councils class dependents to be: sons, daughters, mothers, fathers and absolutely nobody else [edit: and husbands and wives]. So you couldn't move your uncle in and call him a dependent.
With respect to your kids using it, I would say it's their primary residence, and no charges should be applied.
I suggest you post over on Legal as to whether your Conveyancer should have told you it would attract a separate council tax.