YANBU. We did lots of "extras" with our son, particularly before he even started at school, he could read & write at a basic level, knew the alphabet, could do basic sums, etc - that came with bedtime reading and playing with traditional toys like flip over alphabet letters and a wooden abacus. He certainly didn't think it was "teaching", he thought it was fun!
Throughout primary school, we just made sure we were on top of things, i.e. making sure he did his homework, practicing times tables with him, ensuring he read at bedtime, helped him learn his spellings etc - i.e. nothing "more" than they were doing in school, but just keeping him at the top of his game.
We were much more hands off at secondary, again, just making sure he did his homework, made sure he revised for tests, helped him learn his MFL vocab for tests etc. The only time we intervened was after a particular bad year 8 in Maths (previously his strongest subject), but they had a crap teacher and he got less than 50% in the year of year test (after getting over 90% in the first year). That Summer between yrs 8 and 9, we got him a CGP book, printed off loads of Maths worksheets etc., for him to work through in the holidays (maybe an hour or so every 2/3 days), to find out his weaker areas, and then we really "hit" his weak spots by some 1-2-1 with us filling the gaps in his knowledge. He flew right from the start of year 9 and was back to 90%+ in the year 9 year end test and then grade 9 at GCSE and A* at A level, and he's just graduated with a First in his Maths degree. So that bit of effort in Summer after year 8 to get him back on track was more than worthwhile!!