Who Counts?

Assessing Accuracy of L.A.'s Homeless Count

Overview

Within the past year, Los Angeles County and City voters approved $4.75 billion for services and housing to combat homelessness. The Greater Los Angeles Homeless Count is crucial for identifying how this money should be used to help people escape homelessness.

The Count is an increasingly comprehensive effort to count and describe Los Angeles’ homeless residents, but it is not yet sufficiently accurate to identify year-to-year changes in homelessness. Methodology that seemed reasonable when introduced in 2009 can now be seen to produce inconsistent estimates of the number of homeless residents and their attributes.

It is important to obtain more accurate and reliable information from the massive effort invested in the Count by volunteers, the Los Angeles Homeless Services Authority (LAHSA) and research collaborators. To be uncounted is to be unseen – to be left out of funding, planning and implementing programs to combat homelessness. To be helped, people experiencing homelessness must be seen and understood.

Objective

This report views Greater Los Angeles Homeless Counts from 2007 through 2017 as a body of work rather than discrete annual snapshots and assesses the extent to which the Counts present a consistent body of evidence and the extent to which there are inconsistencies among Counts, or with other data, indicating a need to strengthen the Count methodology. The objective is to strengthen the reliability of the Count as a tool for understanding and combating homelessness.

Assessment

The top level finding from this assessment is that the Homeless Count is valuable for providing a fresh picture of homelessness, but the Count data is not reliable enough to be used for comparing the number or population composition of homeless residents from different Counts. In addition, there are indications that the Homeless Counts have underestimated the number of people who are homeless.

Year-to-Year Comparability of Homeless Count Data

There does not appear to be reliable, year-to-year comparability in data produced through the Homeless Count. One source of this problem appears to be the demographic survey of unsheltered residents. A second source of discontinuity appears to be inconsistency in how the street count is carried out. The third source of discontinuity is lack of statistical tools for identifying and correcting measurement error: the difference between the Count and the actual number of homeless residents.

Since 2013, there has been a contradiction in Homeless Count reports between an increasing number of point-in-time homeless and a decreasing number of people homeless over the course of the year. This casts doubt on both the demographic survey and the formula for projecting annual homelessness. Aside from the accuracy of the demographic survey data, the formula used to project the annual homeless population has a shortcoming in that it does not account for people who exit homelessness after being homeless for more than one week but less than one year, and then are replaced by new entrants.

When information about the gender, ethnicity and age of the unsheltered homeless population from demographic surveys in different Homeless Counts is compared, there are large increases and decreases in the reported characteristics of the population that do not appear plausible. These shifts are even less plausible when compared to data for sheltered residents, which show very little year-to-year change in demographic characteristics.

There are year-to-year discontinuities in the share of the unsheltered homeless population that is reported to be chronically homeless as well as the share that is reported to be homeless for the first time. These do not appear to correlate with any trends in the underlying causes of homelessness such as unemployment or poverty.

Comparability of Homeless Count with the General Relief Caseload and School Records of Homeless Students

The General Relief caseload includes only part of the homeless population, which suggests that this caseload is smaller than the actual point-in-time homeless population. However, over the five Homeless Counts from 2009 through 2016, the homeless segment of the General Relief caseload plus homeless family members in other programs estimated to fit HUD’s definition of homelessness have been an average of 82 percent larger than the population estimate from the Homeless Count. In addition, the two populations have had opposite trends of growth and decline. This raises uncertainty about the comparability of one Homeless Count to the next and their reliability as an indicator of whether the homeless population has grown or declined.

School data from 2016 shows large concentrations of homeless students in the San Gabriel Valley and, Southeast/Gateway Cities. This geographic distribution is significantly different from the distribution of homeless children reported in the 2016 Count, which was based on demographic survey data for only 103 children. There is a strong possibility that the Homeless Count identifies geographic concentrations of homeless children inaccurately.

Recommendations

The core methodology for carrying out the Count has been unchanged since 2009. Progressively more effort and money has been invested in implementing the methodology, but the results still are not sufficiently accurate. These recommendations outline steps that should significantly improve the Count’s accuracy by reducing measurement error through more careful and consistent procedures, and by obtaining additional types of information for calibrating and correcting measurement error.

The research burden for improving the accuracy and consistency of the Homeless Count should be shared by researchers in the region and local governmental agencies that serve homeless residents, rather than falling solely on LAHSA, whose primary task is grant and contract management.

Street Count Improvements

Require Count volunteers to participate in more consistent, substantive training that includes standardized procedures for canvassing census tracts, assessing risks and making decisions about which areas to investigate.

Provide a suggested route on the maps that are given to both street count volunteers and teams that conduct the demographic survey to ensure that the entirety of their area is covered once and only once.

Maximize the number of enumeration teams that walk rather than drive their routes.

Use mobile apps on cell phones in the street count and the demographic survey to document the GPS coordinates of each homeless contact.

Where possible, integrate the demographic survey as a uniformly random component of the street count. This includes the youth survey, an improved version of the family survey, the follow-on surveys recommended later, and possibly some components of the street count.

Demographic Survey Improvements

Increase the number of families with children that are reached by the demographic survey or make greater use of HMIS data about children in order to provide more reliable information about homeless children.

Carry out the demographic survey in a random sample of locations rather than in locations influenced by opportunity or convenience.

Support detailed analysis and widespread dissemination of information from the demographic survey that is operationally important for combating homelessness, for example, barriers to employment, health conditions, justice system involvement, and needed services, and maintain year-to-year consistency in questions asked.

Assess whether Homeless Management Information System (HMIS) data, which represents shelter residents, is more reliable than the demographic survey and if it should have a larger role in describing the total homeless population.

Statistics and Data Analysis Improvements

Give the research organization working with the Count a fully independent and objective role in ensuring the data integrity of the Count rather than a secondary, supportive role.

Strengthen the integrity of the Count by identifying, quantifying and correcting measurement error. The statistical challenge is to describe the total homeless population based on a count that identifies only part of the population.

Make it a primary goal of the Count to calibrate year-to-year comparability in population estimates and to identify likely causes for major shifts in the number or composition of the homeless population.

Develop a more complete sampling frame for the demographic survey that includes benchmarks for social attributes. Local researchers and government agencies that serve homeless residents should participate in this effort.

Use a “decoy” quality assurance mechanism in which researchers deploy adults throughout each area of the county, posing as homeless individuals during the street count, to check whether they are found and counted as visible homeless persons in order to produce an estimate of the proportion uncounted among homeless people on the streets.

As an additional tool for quantifying the share of homeless persons who are not found by enumerators, conduct surveys at homeless provider locations in the days following the Count to determine whether individuals were counted.

Survey a stratified sample of vehicles that may have homeless occupants to determine the proportion of different types of vehicles that serve as homeless dwellings.

Develop a more accurate statistical model for estimating the annual homeless population using a more detailed and complete breakout of population turnover among individuals who experience homelessness.

Use other data sources to assess the accuracy and completeness of the Homeless Count. This includes the number, location and attributes of persons receiving public assistance from the county who are identified as homeless, health care provider reports of services to homeless individuals, and school data about homeless students.