Also have a boy with a September birthday and we didn't take up our place at 3 years old. This was mainly as our local nursery was full and I didn't want him to have to move later on. Haven't regretted it at all. I think at 4 he will be much better equipped to start e.g. at 3 although he was potty trained was not very reliable and would have got less help than at his private nursery/home. Also I work part time and we've been able to do lots of activities on my days off (swimming, parks, gymnastics) which I think he would have been too tired for after a half day at nursery (5 half days is only option here). Of course everyone has different circumstances but if you feel this is right for you I would go for it.
At our school, loads start at the age of 3 and come out of private nursery. Most come over for the total convenience that their siblings are there so one drop off, one pick up, all together and as siblings that's where they will go to school. They transition at 3 not 5, with their friends and it's a breeze. None of our school nursery DC had any issues going to reception whereas some others too, weeks to settle. They know the school and teachers from an early age. The school nurseries near us have wrap round care too or CM pick all DC up together for working parents. The school and private nurseries follow the same EYFS curriculum, so DC get same learning etc. School nurseries are generally a massive amount cheaper than private nursery if using more than just the free hours (1/2 the cost here). Round here most DC move across, so DC left at private nursery often end up as the oldest child with a number of younger ones and bored. Mine did 2 years school nursery and I wouldn't ever have changed it.