The Richter Scale is logarithmic meaning a small 'earthquake' is really really really tiny.
Wikipedia explains it better than I can;
Because of the logarithmic basis of the scale, each whole number increase in magnitude represents a tenfold increase in measured amplitude; in terms of energy, each whole number increase corresponds to an increase of about 31.6 times the amount of energy released, and each increase of 0.2 corresponds to approximately a doubling of the energy released.
It's not a simple as the magnitude either as the damage a quake can cause depends on how deep it happens, the rock it happens in and the type of quake. Having said that magnitude 4 is probably when you start to notice something. So there's a long way to go before one of these quakes causes damage.
I think a lot of the fear on fracking stems from some of the frankly cowboy practices in the USA but there is a vast amount of regulation in the UK, which also explains why these quakes are even noted. Water quality is carefully monitored as are quakes.
I'm against fossil fuels, and would rather see investment into renewables but as it's the open market I can also see that some companies will develop one fuel whilst others chose a different one. I accept that we haven't got to the point where we can be entirely green just yet. If it was nuclear I'd say hell no, as the damage that can do plus the multi-decade waste issues are too high. In comparison fracking is safe as houses.