Harcourt Arboretum, Blenheim Palace, Millets Farm for whole day stuff. Or walk from Hinksey to the Abingdon Road playground (includes a railway bridge and going over a lake, very exciting).
For further afield, Beale Park, Cotswold Wildlife Park, Waddesdon, Bekonscot Model Village, Odds Farm, Stowe Gardens, The Living Rainforest, the Bus Museum, Buckingham Train Museum, Pendon Museum.
For wandering about - Marlow (river, park, swans), Henley ditto, Benson has a nice cafe and the river, Burnham Beeches is lovely at this time of year.
Try nature detectives for some tick sheets and activities that make a morning out in the woods into a cheap activity.