I think your analysis of the cost-effectiveness of delivery of services via government is spot-on. I would like to point out that there are a couple of other axes of impact of government that are worth considering in forming a complete view of the role of government.

The first is non-spending activities of government. These are regulations and standards and other matters that enable self-interested people to interact profitably without shooting each other. It is easy to point to excessive regulation as a drag on growth and innovation, and that is true. You have felt this personally in your business. But it is also possible to point to areas of success, such as the Clean Water Act, or food safety rules that have increased quality of life. The main point is that frustration with inefficently-used taxes is likely to be associated with an aversion to regulation, when the elements of the cost-benefit analysis for each can be very different. The limit case of libertarianism is feudalism.

The second is government transfers. Your arguments are all essentially "efficiency" arguments. Government transfers such as pensions, unemployment insurance and the like are "equity" issues that are unlikely to have a market solution. Privitisation of delivery of these transfers and the related client interactions is probably a good idea, however.

The third is public goods. The scope of what is considered a pure public good has reduced considerably as technological changes allow new private markets to emerge. That said, there is still scope for government provision of goods and services that would not emerge from purly private markets, or where private markets would be unfair or less efficient. Fire Departments used to be private, but they would only put out the fire at your house if you had signed up for their services. This is not optimal for your neighbor, and leads to less fire-supression than people would like.

Again, the main point is that a government of size zero is a situation that is unlikely to be optimal. That leaves us with a more sophisticated problem--not "Government is bad" but "how much government is good"? This is a more complex problem as it involves evaluating tradeoffs.

There are good ways to improve the efficiency of government. Local provision of services is usually superior, so devolving government responsibility to the lowest possible level is prudent (city->county->state->federal). This is less true for regulations, where consistency of treatment is a fairness issue as well as impacting trade between regions.

Establishing performance metrics for government services, with an independant government auditor to report on these (as is done in Canada) has been proven to improve efficiency. Developing a customer-service perspective on the part of government agencies is key, and there are ways to encourage this.

A cautious view of the role of govenment is prudent and healthy. Having lived in France, I can say that when the government is expected to be the main actor, a mess of inefficiency follows, along with a withering of the individual's percieved role in the social contract. Having lived in the United States, I can say that excessive scepticism doesn't lead to less government, but rather less effective government as it prevents a fruitful discussion of how to improve matters.