Yeah you're not being controlling or unreasonable. He's being an absolute arse - both in DARVOing you on this issue, and also immediately not saying himself, I'll come back early.
He booked and paid for the glamping pods. It's not like he didn't know about the festival - he was an active participant in planning it. Yes, shit slips your mind. This is the kind of shit my husband does all the time, but he would immediately rectify it - maybe with some disappointment in his voice, but he would accept that it's his fuck up to mend and a family birthday comes first.
If it wasn't a weekend festival with 3 kids, AND my daughter's birthday, I'd be like, you idiot, don't worry, we'll make it work.
But it's his own daughter's birthday, he was part of the planning, and it's super unreasonable to expect her to understand why Daddy isn't there, and for you to wrangle 3 kids in the wild on your own when there's no goddamn need. If it's going to be special for your daughter rather than just frazzled, it needs a couple of pairs of hands.
I would be furious. I don't know how he isn't under the patio. You're doing well.