How the Tribune measured fairness and accuracy

To report this story, the Tribune gathered more than 100 million electronic property tax records, including data on assessments, sales and tax bills. The data covered tax years 2003 through 2015.

The newspaper used the data to conduct sales ratio studies, which assessing officers around the world use to evaluate the accuracy and fairness of their valuations.

The steps for the studies are straightforward. Sales ratio studies are done by dividing the assessor’s estimated market value of properties that have sold by the sales price. The sale must have occurred within one year after the mass appraisal.

Showing our work

The Tribune conducted 13 years of sales ratio studies for Cook County, following industry standards.

The newspaper first queried state property tax transfer data for all sales of residential properties between 2003 and 2015. Residential properties include single-family homes, condominiums and apartment buildings with fewer than seven units. Sales must be reported to the Illinois Department of Revenue because state and local governments levy a tax on property transfers.

The newspaper then eliminated transactions where the seller did not advertise that the property was for sale and sales between relatives or other related parties, limiting the list to what’s known as arm’s-length sales. Foreclosures, auction sales, judicial sales and short sales also were tossed out.

Data obtained from the assessor’s office included the market value and assessed value of each segment of a property. For example, each property has a land segment and a building, or improvement, segment. Properties can also have garages, additions, etc.

To determine the total value the assessor’s office used to assess each property, the newspaper summed those segments into one value, accounting along the way for any special exemptions, such as the home improvement exemption.

The assessment data were then joined with the arm’s-length sales data using Property Identification Numbers, or PINs. Each property in the county, including condo units, has a unique PIN, similar to a Social Security number.

Because assessments estimate prices as of Jan. 1 of a given year, the assessor’s data were matched to sales that took place within the same tax year.

If the ratio of the assessor’s estimated market value to the actual sales price is above 1, it indicates that the mass appraisal model overvalued a property, while a result below 1 means the property was undervalued.

For example, if the assessor estimated a value of $200,000 for a home that sold that same year for $300,000, the ratio would be about 0.7. A house valued at $300,000 that sold for $200,000 would have a ratio of 1.5.

The results of any mass appraisal model are never perfect. So sales ratio studies also include measures of fairness and accuracy.

One of the most important statistics produced in the studies is called the coefficient of dispersion, or COD. The COD measures the distribution of the ratios relative to the median and produces a score. A COD score over 15 means there is a large amount of variation, which indicates a high degree of random error in the assessments, or what assessors refer to as “horizontal” inequities.

Two other important measures used by assessing offices — price-related differential, or PRD, and price-related bias, or PRB — focus on “vertical” inequities in a mass appraisal. Both tests look at whether differences between ratios are correlated with differences in price. For example, the PRB measures the percentage by which ratios fall or rise as prices increase or decrease.

If higher-priced homes are undervalued while lower-priced homes are overvalued, the assessments are said to have a high degree of regressivity.

Before running the statistics, the newspaper trimmed outliers — ratios that were extremely high or low — to ensure they did not skew the results.

In Cook County, properties are reassessed every three years, depending on where in the county a home is located. To be as fair as possible, the Tribune cited only the results of sales ratio studies based on calendar years in which triennial reassessments took place.

The Tribune also examined how results differed if condominiums were excluded from the sales ratio studies.

The assessor uses computer-based regression models to value single-family homes and small apartment buildings. But a former county official has testified that condominiums were valued by tracking sales prices within an individual building and prorating the predicted value of other units.

The Tribune’s analysis was reviewed by three assessment experts who provided feedback and suggestions on how to improve it. Richard Almy served as executive director of the International Association of Assessing Officers, a professional organization that sets assessment standards, between 1982 and 1990, and later served as an assessment consultant and expert witness. Peter Davis, the ratio study supervisor for the Kansas Department of Revenue, helped write sales ratio study standards for the IAAO. Robert Denne is an assessment consultant and former director of research and technical services at the IAAO.

“I would say the Tribune conducted a quite rigorous study, following all of the principles and standards of the IAAO,” said Almy, now retired.

The Tribune also examined disparities in the system by analyzing effective tax rates, or the percentage of a home’s value that the owner pays in taxes. To do this, the newspaper compared property tax bills to sales data between 2009 and 2015.

Because the effective tax rate should be about the same for everyone in a single taxing district, calculating and comparing the rates for communities within Chicago was straightforward. The city is the largest single taxing district in the county.

To fairly compare effective tax rates across the county, which encompasses numerous taxing districts, the Tribune used the tiny fraction of property tax bills that funds Cook County government and the Forest Preserves of Cook County. Those are the only taxing districts that cover the entire county.