No, you cannot give up on reading. To make expected progress, children need to be doing 10 minutes of reading at home to an adult almost every day. Teachers do not have the time to read with every child more than once or twice a week, so this is why it must happen at home. If it was a battle to get your child to clean their teeth, would you just give up? No, of course you wouldn't, and it should be the same with reading.
School teaches children the building blocks of reading, but the practice happens at home. The less you do, the more your child falls behind.
Turn the reading into a game. Throw a dice and that is the number of words (or sentences when older) you have to read. Take turns with your child and keep a tally. The winner is the first to get to n number of words or sentences (you choose how many). Reward as you see fit.
Also reading alternate words helps the flow of the sentence so you read the first and he the second etc..
After each sentence, re-read for meaning and prosody.
Keep working on those phase 3 digraphs and phase 3 and 4 tricky words; create a noughts and crosses board and put digraphs or tricky words in the nine squares. Your child has to say the sound/word to put their O or X in the box. Oh the frustration when they know where they need to go to block you but they are having to dig deep in their minds to recall the word or digraph!
Write the digraphs or tricky words on pieces of paper and hide them around the room. Then get your child to go and find the one you say (and so on), or have the sounds/words on the floor and them have to step on the ones you say. Can they make it around the room by stepping on all the right ones?
Keep going. Just keep thinking that the children who are well supported with their learning at home do better in life. It is as simple at that. And the more you do, the better he will get. His confidence will grow, it will become routine and it won't be a struggle any more.