"I think he was angry when he made that comment"
The way you say this, it's like it's an excuse for what he said. It's concerning actually and doesn't make things better it, makes them worse.
You came to Australia for a year primarily because he wanted to and you wanted him to have that opportunity, you got swept up in his excitement of ot all. The agreement was that if it wasn't working after a year, you'd return. Apart from the fact he lied to you, even at that early point, telling you what you wanted to hear to get you there (because if you're not planning on definitely staying you don't sign up for a 4yr training course). Now that you want to hold him to the agreement, he's angry.
Even if we look at it from the perspective of he'd have to give up a life he's loving to return or he'd have to lose you and the DC if you return without him and is angry about that, that's still no different to the situation you're in now and will be in forever if you stay. So it's alright for him to be angry at not being able to have everything how he wants it, but you're just supposed to suck it up not having things how you want it?
"Forever" because it won't be until DC are adults, by then they'll be Australian and likely won't want to uproot themselves to return to UK, especially the youngest who won't even remember UK. Even if you all did return at that point, all the people you left behind 20 years ago will have moved on, adjusted to not really having you in their lives and you won't be going back to what you left. You also couldn't turn back time and regain the relationship through the childhood years because those years would be past.
He's asked you to give up a lot, he's wanting you to remain in that situation for the rest of your life and his response to you not wanting to is anger? Not good. It shows he's not even considering your feelings, only his own.
Stay if you want to make a go of things in Australia, but I think you're foolish if you think he's going to change his mind about staying. He's already told you he won't.
That comment also wasn't as throw-away as you're making it out to be. When it came to it his main concern wasn't losing you or DC, it was losing Australia and his life there. Your and DC unhappiness wasn't on his mind at all, just anger at potentially losing the life he's built. It's really not a great response. Unhappiness (at needing to return to UK), yeh. Disappointment (at losing the life he's built in Oz), sure. But anger? At something happening that he knew was a risk when you all moved here, a risk that he agreed to? Uh, no. That's not on.
There's a reason your recent health scare has highlighted your lack of extended family support. Where was your husband during this scan, when you needed him? He should have taken time off work to take care of DC so you could attend this appointment without them. There's a reason your immediate family isn't enough for you. He's not a team player.
Anger also tells you that if he comes back to the UK he'll resent you, which I reckon will mean the end of your marriage ultimately. I wouldn't be surprised if he heads back to Australia at that point. If he's not a team player during marriage he's highly unlikely to become one upon divorce!
When he says "try for longer" to you, what's the flipside to that? He'll return with you if you can't make it work (not that I'd trust him anyway at this point, even if he said that)? Because if there's no flipside and he's staying regardless, then all "try a bit longer" really means is "don't leave me". Which is meaningless in terms of figuring out a way forward together or coming to a mutually beneficial solution. It's just a plea to maintain the status quo.
If you're going to change what you want to do (leave) then what's he changing to make the situation of you staying work? I suspect the answer is "nothing". And call me cynical but I suspect that even if the answer was "something", that behaviour wouldn't last and he'd slowly slide back to his old ways as soon as you stopped making noises about leaving.