Haha! @Noworrieshere, I've never thought about it like that!
What you do to stop it and how easy it is all depends on what stage you're at in the extraction process iyswim.
The first stage is as I described earlier, where the pressure of the oil basically forces it's way out once you've provided a route. If you imagine how the oil was created, with huge pressure from the rocks mushing up the organic matter, then physics basically means that when you first open up the route out the oil pressure forces out with the same pressure the earth was pushing down. But the more oil that comes out, the less pressure there is (there is more space in the well, so the oil can spread out and it's easier to do that than travel up the pipe). Letting the oil flow out by itself will only give you about 10% of the contents in the field.
So at that point, we start pumping water / gas down into the well to increase the pressure and push more oil out, which is called secondary extraction and in a good field you can get almost half the oil in this way. You can then go onto more difficult (and expensive) options once you've exhausted this method. Often it's not worth it. For example, this is the state that a lot of the north field oil supplies are in: waiting for the oil price to rise sufficiently for the cost of tertiary extraction to be worth it.
So the oil doesn't just keep pouring out regardless, quite quickly it needs assistance. I can't honestly tell you how extraction would be suspended in reality, or if you need to keep a minimum flow: I didn't do much work with extraction, I was more involved in pipelines. I would imagine that most countries would like to keep a minimum of extraction happening for practical reasons of being able to start up quickly (and beat competitors to market) when it's required. With oil, the political and economic reasons usually beat technical!
If you decide it isn't worth extracting the oil anymore, then the field can be plugged, but that's considered a permanent fix and involves a huge amount of work, so wouldn't be considered a short term solution.