Yes you should know the family finances. Income, outgoings and savings. How else are you going to budget and plan and save? You should do that together. It's nice for each of you to have a personal amount each that you don't have to account for (apart from both of you agreeing how much it is), but for everything else of course you should both know.
You're a SAHM not a child.
I don’t think he appreciates my asking him about the family budget
I would be very worried too. Either he's hiding something or he doesn't trust you or he's controlling.
He’s just really evasive and snappy
He's hiding something. My first thought is gambling. Someone else said debt. But it could be a lot of things.
A few months ago I asked him to give me regular money ( after having three children like this ) as I told him my digging into my savings wasn’t sustainable
So what has he been doing with all his earnings, if he's been forcing you to dig into your savings for household expenses when he earns plenty? This is looking like financial abuse.
"he just give he money when I ask which is great"
You are not the controlling one. You should both know the family's financial situation, because you are jointly responsible for it. Controlling is him hiding from you the part of the family money
that he is responsible for, and doling just a little bit of it out when you ask.
So having read to the end of the thread, looks like he is controlling and has something to hide.