I think alot is based on several things:
First, I can spend my money more effectively than the government. There are some things the government can do that I cannot, and I think that government rightly exists for those purposes (things like the provision of justice, international trade, effective monopolies that require hundreds of years to see the value of investment like railways, ports and docks, Water and gas infrastructure, roads, etc). But government provision of virtually any other economic good is generally trying to work out what I want and spend the money for me. The provision of local cultural services, for instance (like the recent local pagan festival) is not something I am interested in or interested in funding. Such things are therefore spending my money on things that I don't want in there mistaken belief that I do want them.
Second, the government spending my money involves necessary losses, which should, as far as possible be avoided. First, the collection of tax involves a loss- pissing tax men, paying auditors, etc, as well as list productivity while people attempt to comply with tax law (this is the more significant loss). Secondly, government spending is rarely efficient- because governments are political and their spending is political, a great deal of money has to be spent to ensure political compliance (data recording etc.). Additionally, there is a disproportionate effect from Union activity in public sector because the lack of services is felt politically rather than purely economically. And because politicians are keen to avoid political pain, they're happy to spend money that isn't their own and didn't cost them anything to make a problem go away. Related to that is the way governments spend money: if the money you spend isn't your own, you become less concerned with the way it is spent. If you are a weak manager, and find it hard to sack someone, or to deal with them effectively, you might think, oh it's fine I'll push them sideways, and we can just pay them until they retire in a meaningless role. Just easier than sacking them really. Which makes sense if you're not spending your money. Private companies tend not to do such things because it will push up the price of their products and make them less competitive. But if you're spending tax money, the belief that there's always more where that came from becomes easy to live with (think how much the NHS spends on paracetamol for instance).
I'm not opposed to a social safety net, but I am opposed to heavily taxing me to pay for things that I would pay for myself. Education and healthcare are things that I would be prepared to spend money on, and I would prefer that the NHS had a more moderate provision that could then be topped up by private spending (I like the Australian system for instance- and I strongly disagree that the only possible options are NHS or US style system, it's a complete non-sequiter and is a hugely convenient way of hiding the fact that we get quite a poor return on what we spend on the NHS).
In short, I don't think that Tory's are anti government or anti government spending or anti welfare, but it is coupled by a deep mistrust of the state to spend funds effectively, and therefore trends to be more circumspect at the potential of government to solve social problems through government spending. If you believe that government spending can solve all or most social issues then you will, I think, see Tory's as cold and callous. Some may be, but I think that most are just cautious about the potential of government to actually achieve what they set out to achieve, and this is further twisted when the government had it's hands on your money, and then gets lobbied to spend it in ways your disagree with (for instance I object that having taken taxes off me for the purpose of educating my children, there exist government proposals to introduce stonewall and mermaid policies into education). This is the problem with government spending.
The other problem with government spending is it undoes social capital. When the government spends money in a certain area (say, poverty relief) it means that local, social and community based efforts to provide that relief get replaced. Which means you are now dependent on government intervention forever more because people accept that poverty relief is government responsibility rather than their own social responsibility (because, of course, by paying taxes they are contributing to poverty relief). But going back to the previous points, a greater overall amount of money is required to achieve the same relief because of the inherent inefficiencies in government spending, as well as the fact that it ignores the natural efficiency of certain local social structures because they are often made up of people volunteering their time for free anyway.
So in place of what was probably a quite modest, probably overrun, but very efficient service which built up community cohesion, it's replaced by a professional, probably underfunded due to the inefficiencies service run by people who may not even live in the area who are just there cause it's their job, and the next time there's a funding cut the service gets cut because there's no more money, and all those volunteers who used to run a struggling but efficient local service are all doing something else now.
TLDR: governments are inefficient, probably don't know what I want anyway, overtly politicise what they get involved with, often replace and denude the services they do replace, and are not good vehicles of social cohesion.
Their good points: they can provide intergenerational investment in infrastructure as they exist long enough to see the returns, they can provide services that no-one else can justly provide themselves (like justice and military).
Interestingly and perhaps controversially, I would support government subsidies of farming. I think that food security is something that all countries should maintain, and so while we can all get our cheap veggies from Spanish poly tunnels, if that supply was ever cut off, it would be wise and prudent to have infrastructure in place to immediately kick in, rather than however many years it would take to get people farming with the necessary skills and experience, etc. Which I'm not sure if it's a Tory way of thinking of not.