OK - just starting out - I would recommend choose a book with no more than 100 pages, otherwise you'll both get fed up before they end. 70pp is the ideal.
The 'easy' Roald Dahl ones are great: The Giraffe the Pelly and Me springs to mind. Fantastic Mr Fox is great, but is rather 'gun-oriented' so may not suit everyone! Dick King-Smith is lovely, as others have mentioned. 'The Dragon Boy' is an all-time favourite here. also his Sophie stories are lovely "Sophie's snail" is the first one.
The boys like the Faraway Tree and the Wishing Chair, but I don't enjoy it, and we all usually get fed up before the end. But the chapters are very short, which is good.
I wouldn't worry about not reading to him books that he will enjoy himself - you can always read to him things that he will not be able to read himself for a year or two, and he will still enjoy reading them himself when he's older.
A good source at this stage is your local library - section of L - learner readers. Reading 'proper' books to your children at this age is IMO the best thing you can do for their literacy - expands their vocabulary immensely, and teaches them to listen well. Choose your books carefully and you will very soon be able to progress onto longer, more complex books, which are more fun for you all. DS1 is almost 6, and we have already read him things like The Hobbit, The Lion/witch/wardrobe, Harry Potter 1-3, The Railway Children, The Phoenix and the Carpet.
Hope that helps - sorry it was so long