If your family likes Harry Potter there are quite a few venues that were used for location shooting. One place that is lovely to see is the Bodleian Library. You can look around Radcliffe Square and wander into the main quad (off Catte Street / Broad Street behind Sheldonian) and that is free. It is also possible to pay to get into the Divinity School and also take a tour of Duke Humfret's Library above it); the former is used in HP films as the Infirmary and the library is the hogwarts Library. Very beautiful in their own right though and the tour is good (and not too long). I am biased though as I got married in the Divinity School earlier this year!
Probably worth checking ahead to see if they are open on Christmas Eve / Boxing Day.
[http://www.bodleian.ox.ac.uk/whatson/visit/individual/divinityschool]
On the HP theme, a bit was also filmed around Christ Church (and the Howarts dining room is a replica of Christ Church dining hall).
I second getting into an evensong on Christmas Eve if you can. dailyInfomis your best bet to find which services are on, or you could call ahead to the likely colleges (New, Magdalen, Christ Church but also the smaller ones). And also the walk on Port Meadow on Christmas Day when not much will be open.
There are also some lovely fuggly pubs to go to which would probably be a good pit stop in the day with your boys - check out The Bear Inn on Alfred Street, just behind the High Street, for an amazing collection of cut-off ties and very cosy.
You may need to pay a few quid to get in and probably not open on Christmas Day at least but Magdalen College Deer Park (accessed through the college) is lovely.
I'll keep thinking - you'll need to check opening times though as most places will close for Christmas Day at least.