We too have a private supply @macshoto has provided a lot of useful information but I would add the following:
Do you know if the supply is natural fed, borehole or other type of supply as it does make a difference to pressure and if pumps are required.
Testing, it is only mandatory if you have any element of commercial business, such as B&B, self catering cottages etc. I think we test ours when ever we remember, it is not very often though! When you do test it the results can be quite scary depending on the area you are in and if the supply is natural fed. The testing will pick up minerals, some of which you raise an eyebrow at, but the levels are so low the water is safe!
Is the supply on land the property owns, many rural properties the source of the water is outside of the property boundaries, there should however be a document giving you access to the water and to draw as much as you need to use in perpetuity - it is important, as you do not want to be in the situation that someone decides you no longer have access to using the water.
Pipework from the source to the house, make sure there is a map with the details on (It will not stop someone hitting the pipework when erecting a fence!) so you know where to look, if the pipes are buried and there is a leak.
We have 4 holding tanks for water, 10000ltrs outside, then 2 pressurised cold water tanks in the cellar, plus an unvented hot water tank. This means if for some reason the water does not flow from the outside tank to the house, we always have enough in the house to keep us going for quite a while. For us the main reason for no water is the natural source freezes in winter. It is only happened 3 times in over 30 years though.
There are various ways you can get pressurised water for use in the house but it will depend on how you get the water in the first place. We can run showers,, baths, have washing machines on all at the same time without a drop in pressure, and our showers are not pumped but we achieve high pressure.
We change the sediment filters ourselves, normally about once every 6 weeks but shorter or longer dependant on how dirty they are. We buy them in bulk and they cost about £1 each. The UV filter we can also change ourselves but tend to get it done when we have a service done on the filter/pumps/tanks in the house. Ours are all in the cellar.
If it is a natural fed source, you get used to it being a bit brown after it rains, but the taste far outweighs its looks! Natural water is wonderful and you never get limescale in your kettle etc.
We also have non mains drainage - we empty our sceptic tank once every couple of years. I doubt we spend more than £100 per year on water and drainage.