Not sure what to say.
I opened this thinking I might be able to help as I'm on a zero hours contract too but I can't think of anything useful to say regarding your situation. So this is mine.
In terms of financial planning, the contract has been a bit of a disaster. I only work term time, but as it's not in a school, even then there can be weeks when i get nothing. In a good month (October or June are usually good) I can earn around £900. In a poor month when I am still picking up work (December, say) I can earn as little as £200. Other months (August) I earn nothing at all. DH works full time but it's not a well paid job. Most months are in the middle (around the £400 mark).
The problem is that over the course of the year we just about earn enough between us to mean we don't qualify for much govt support. And I do mean just. We do fine on a month when I earn a reasonable amount, and then I put money aside, but a simple thing like having the car serviced or an oil delivery usually takes most of that savings and then the next month I look in my bank account, I've not worked as much and suddenly we only just have enough to cover grocery shopping. Child care is the other issue involved in this - I was bloody lucky because I found a nursery which would take DD on an ad-hoc basis, so days when I didn't work, she didn't go and I didn't have to pay. Most places would require you pay for the place even if you are not using it on a given week and that would have ended up with me paying more for child care than I earnt. I still paid a lot percentage wise out of my salary for child care. Now she's qualified for free govt hours it's made a big difference.
It makes form filling a logisitcal nightmare - stupid things like if someone wants proof of earnings to assess something it can be hard to get an accurate picture. Repaying my student loan, for example - they usually want the last three months worth which might not be an accurate picture of what happens across the year but often the person on the phone that you call to ask for help doesn't get that and trying to explain there is no such thing as an 'average' month is not easy! We just couldn't afford the repayments at the moment but proving it can be hard. Everybody wants an average monthly income and there just isn't one. I can average out last years income by the month, but that's not to say that this year will even come close to that, or it might massively outstrip it.
However, the reason I have kept up with it is two fold.
Firstly, it is a job I quite enjoy in the right sort of field. It's not exactly the job I did before i had DD but it's close enough, keeps my brain ticking over and looks very good on my CV so that when I want to get back into my 'proper' field later I can do so more easily. I put a lot into my career before having a child (I paid to do an MA and did it in my own time for example), loved the career and (I hope) was good at it and I didn't want to give it all up completely, or have to start again.
Secondly, although I love DD more than anything in the universe, I was lonely. We moved when she was 15 months and before then I had gone back to my old job pt, and I was surrounded by friends etc. so on days off there were people with other young children to hang out with. Then we moved and suddenly it was just her and me all day every day, not knowing anyone, or where to go on a rainy day. This job came up and I saw it as a chance to give both her and I a break from each other and for me to have some adult conversation once in a while!
Am now expecting no. 2. I will be considerably better off on mat leave than when I was available to work. C'est la vie. I have massively low moments. I keep telling myself (when we have a bad month and there isn't enough money to pay the bills) that this will be the worst point, while children are young, and that money is an artificial construct of society, not a real thing. Real things are things like health, and love, which my kids have in abundance, and that is what really really matters. Which doesn't necessarily help while standing in the supermarket working out which essential I need more but does stop me sinking completely!
Hopefully in there is something that might help.
I'd say maybe try it. It might be that getting out and doing something helps the anxiety, in which case great. But the financial side might make it worse. In which case, stop. Whatever decision you make doesn't have to be permanent. Hope you find a way through. Good luck.