Maybe even use talking over the dinner table about things they are interested in as a way to build the list of curriculum topics for you - and also a way to get them debating and expressing themselves as it continues on beyond just your initial "idea gathering" phase - they can go research an issue and chat about it.
Baking has a lot of numeracy in it with weighing and measuring, and literacy in reading recipes. (There are a few recipes in DD's reading books this and last year - she's never actually made one but is talking about trying one from this week).
Gardening has a lot of science, maths, possibly literacy too involved - how do things grow, needing different ingredients (soil, water, light, heat), how many plants to grow, what do instructions say about how far apart, lets measure that, how does water move differently when we pour it from high up or low down or leave it standing still, will we grow our own peas to eat.....
The same with craft projects - things like make a piñata for Halloween which takes a few weeks including lots of drying time.
There are also lots of different homeschool resources online that I've found over the years (DD is MS still but with ADHD/ASD and I have contemplated it in the past, and have done various modules with her to stave off the "I'm Bored!!"-ness).
www.giftedkids.ie/kids.html
homeschooling.about.com/od/curriculum/u/teaching.htm
www.enchantedlearning.com/themes/
www.schoolexpress.com/fwsindex.php
www.dltk-holidays.com/easter/index.html
Activity Village is not bad either
They are just ones I had saved in my favourites, and used to start off bouncing around the web to find different things. There are lots there with themed sheets - like I know a few places would have writing, maths, and other factual learning sheets on a theme so the idea was obviously that you'd spend a week doing, say, penguins - maths sheets using penguin characters, word mining and story sheets using words related to penguins, colouring sheets and craft ideas on penguins, fact sheets and then quizzes for later on penguins, their lives and habitats etc.....then next week, move on to something else.
It certainly seemed like it would all take a fair amount of work to get organized, but there's lots that is "schooling" and learning without being formal about it. You just have to think about it yourself and decide how to structure it in your own head, and then you can justify it to the world and its' wife as you are comfortable you know what you're at!