I had always assumed there were many parents thinking a bit like me but I'm not so sure any more!
I tend not to worry about how DC are doing in infant school as long as they aren't behind in the expected targets. I think it matters more that they are happy at school and happy learning. The rest will come in time.
I don't read the school books as often as we are asked to. DC2 isn't keen and reading becomes a chore which seems a shame. But we play a lot of games and try to incorporate learning in a fun way. I also read every night for quite a while and my youngest will read bits of whatever book we have on the go. My eldest who can read independently reads at least an hour every day, often more so it hasn't hindered him long term at all.