Small-scale tax collections policies have large-scale distributive consequences. The central philosophical question addressed in this Article is whether the government can ever be justified in deliberately not collecting a tax owed, given the problematic distributive consequences that may result from non-collection. In brief, the government’s decision not to pursue full collection of a delinquent tax debt may give rise to three problematic distributive outcomes: (1) capture of the benefits of non-collection by the forgiven taxpayer’s other creditors; (2) imposition of the costs of non-collection upon compliant taxpayers through higher taxes; and (3) imposition of the costs of non-collection upon the public through decreased government provision. This Article argues that a social insurance framework, which conceptualizes tax non-collection as a transfer of the risk of financial distress from the taxpayer to the government in exchange for an insurance premium, may justify all three of these seemingly problematic outcomes and may hence justify tax non-collection. However, further research is required to determine the extent to which tax non-collection should be used to deliver social insurance, particularly given the existence of other social insurance delivery methods.