As kids vary so much there is no ideal age, but to me 4 is just too young and both my kids had miserable yr rs which could have been avoided with a later start date.
DS ignored the first 2 years of school (yr r and yr 1) as he wasn't ready socially and was miserable. He was fine academically. He just withdrew and pretended it wasn't happening. Now he is in yr 4 and doing OK socially, I do occasionally spy on him at break times on Mondays when I'm not working. This school year I've seen him playing with friends for the first time and laughing, which is a huge weight off my mind.
DD was summer born and a slow developer, physically especially. Yr r was hard for her, with lots of bigger, faster, less clumsy children to cope with, she got behind academically, was extremely aware of this and lost her self esteem and stopped trying. Fortunately she is now in another class with a lovely motherly supportive teacher and is now catching up rapidly, she has grown physically which has really helped and all of a sudden she can read and write. Last week she said she liked school sometimes, prior to this she has firmly stated that she hated it. Still not making friends much though.
Both were so tired in yr r it had a real negative impact on family life. No going out, quiet weekends, very little playing as both had to be in bed by 6pm or they couldn't manage the week without serial meltdowns.