i think camping is great fun
BUT I agree with a 4 year old it would be a bit more ideal to have more than 1 adult...
or else yes you will have to drag him with you to the loo / shower / dishwashing sinks etc.
also if he is asleep would you want to leave him while you used the loo?
yes you pitch your tent and treat it as home, dont need to pack it away etc.
BUT if you want to do it perhaps you should aim for a smaller site. (normally with children a site with more facilities etc would be better i would reckon)
IF you were on a small site ie here where i went last weekend then i think it would be do-able...
LOVELY 'small holding' like the good life! donkeys / free range chickens / ducks and geese etc... a VERY small site, 10 pitches, 1 shower room for each sex (loo shower sink in 1) and a washing up shed...
if you pitched in the corner near the loos you could see your tent while you washed up and there would be room for him to accompany you in the loo... and you would know if you trusted your 4 year old to stroke the donkey for 2 mins while you had a wee in peace
I have stayed on many campsites and most i wouldnt think would be ideal as a single adult with a young child but i do think it would be possible on that site.
also remember strong tent pegs (as that is what keeps your tent stable in wind) and a hyrdostatic head ideally of 3000 (i wont tolerate less) though some say 200 is good enough (in case it rains)
if you choose very small sites and try and pitch near loo / sink blocks then maybe its do-able
if not than in 2 / 3 years time it will definitely be much more do-able!