I would echo not using special gluten free foods and expensive flours but choose naturally gluten free ingredients and meals instead. And don't over compensate on sugar or ready made GF products.
There are lots of Paleo and other websites and blogs (not all have recipes without odd ingredients but you will find some ideas.) Deliciously Ela, Helmsley and Helmsley, PaleoMom, Marks Daily Apple.
Lucy Burney's books are good if geared a bit towards toddlers.
I would also recommend looking into bulk buying the foods you need a lot of through one of the organic wholesalers, Suma in the North and Infinity Foods in the south. They can tell you if there are any food buying co operatives in your area. It's worth price checking because some of the supermarkets have cheap options but when I first started df and gf 8 years ago, this was not the case. Also look at some of the Indian / Chinese supermarkets for things like rice and depending on where you are, local farms for potatoes and vegetables (sacks of farm potatoes are much cheaper than supermarket potatoes and last well as they have not been cold stored or kept in a warm supermarket).
My favourite GF deserts are chocolate moose made with dark chocolate, fondant au chocolate made with dark chocolate with eggs, sugar and a tiny bit of rice flour or ground almonds and any combination of eggs, ground almonds / hazlenuts, dates, coconut, cocoa, coconut oil which makes macaroons, cakes, snack bars and biscuits in a variety of shapes, colours and flavours! Popcorn is a great and cheap GF snack! Oh and chopped up bananas put in the freezer for a bit then whizzed in the blender makes a good icecream.
It's not easy and I have often found myself crying in frustration and at the unfairness of it so I wish you luck!!
If anyone is willing to buy you a good blender that will grind nuts, frozen bananas it is worth the investment.