We did it at Easter on the way back from New York. I think 2-3 days probably realistic with young DCs. 5-6 days may be a bit too long.
We only stopped for 36 hours, which wasn't ideal, mainly because of the timings of the flights, which I hadn't really appreciated when I booked them. Our flight left NY in the evening and because of tail wind, only took 4 hours to get to Rekjavik, so we arrived early hours (NY time) 6am Icelandic time, which wasn't great. DCs (12 and 10) and DH slept but I didn't as severe turbulence.
In order to max our time there we booked a coach trip at airport for the Golden Circle tour (8 hours) and took bus into town to join that (grabbing some breakfast at nearby bakery!) at 8am. We did bus trip with the kids sleeping on and off. It was sometimes 45 mins between venues so we all had little kips. It was interesting but a bit too long for my liking. A friend had recommended the coach so the kids could sleep. The jeep tours go off road and are more bumpy etc so not so comfortable, apparently. A car may be better for you.
That afternoon we returned to Rekjavik and stayed in fab apartment (Rey Apartments - central, clean and toasty warm - check Tripadvisor). Slept like the dead that night and woke up next morning at 10am. I phoned to book a trip to Blue Lagoon en route to airport and by 11am we were in the pools! We had 2 hours there and it was lovely but eye-wateringly expensive. Not sure whether you'd get max value with little ones. Then went straight on to airport and flew back to London.
My impression of Iceland is that I couldn't imagine spending more than 2-3 days there in total. It was interesting and with some stunning scenery but at times a bit like a lunar landscape! If you're into walking etc then I can imagine longer but I'm from a country with similar geographical features so it wasn't that wow-ish for me. Rekjavik very nice but small. Everything very very expensive. Our humble bakery breakfast was over £35!! The people we dealt with were lovely - very friendly and helpful.
HTH.
BTW, If you're going on Icelandair, they were brilliant. We took a picnic lunch which we ate in Rekjavik airport when we were changing flights on the way to NY and on the way back had a meal before going to the airport.