Care Index Methodology, Limitations, and Acknowledgements

Summary

In order to quantify what can be an emotional and complicated experience for families, the Care Index measures three variables: cost, quality, and availability. Together, these pillars of care provide a picture of whether and how families are able to access the early care and learning system in each state.

For cost, we measured the average percentage of income spent on child care for each state (i.e. average cost of care relative to median income).

Because there is no single, consistent system for evaluating quality, we combined measures for three different kinds of providers in each state: the average Care.com rating of in-home care providers (nannies), the percentage of accredited family child care providers, and the percentage of accredited child care centers. We multiplied each by the percentage of children who access that kind of care, and the three were added together to create an overall picture of quality.

Availability for each state measured the proportion of early childhood workers, tracked by the U.S. Bureau of Labor Statistics and Care.com, to the number of children under 5 in a state.

In addition to measuring these pillars individually, we combined the three to provide a rank for each state’s overall landscape of care. However, because we are aware of the limitations of our data, we divided the states into quartiles rather than absolute rankings. For each quartile, we chose one state for further in-depth reporting to highlight any details our data might not fully capture: Massachusetts (1), Georgia (2), Illinois (3), and New Mexico (4).

The Care Index is a data and methodology collaboration between New America, Care.com and others.

Care Index Methodology, Limitations, and Acknowledgements

Background

Why, in 2016, haven’t we achieved full gender parity around the world? Why has the women’s movement stalled here and abroad? One big reason: We’ve been teaching both women and men that 2 a.m. work emails are more valuable than 2 a.m. diaper changes—and that getting to the C-Suite is a more prestigious goal than teaching toddlers their ABCs. In other words, we’ve devalued care work—both paid and unpaid—with devastating consequences for women, men, families, and economic growth.

New America, Care.com, and other partners are coming together to elevate and re-frame the conversation around care by producing a first-of-its-kind Care Index that examines the cost, quality, and availability of child care across the United States. Though other organizations have gathered data on some of these indicators, our index will be the first to combine the three, providing a snapshot of the care landscape across the country.

Care—like water, electricity and roads—is part of the infrastructure that enables all other work. And yet, we continue to pay care workers salaries equivalent to fast food employees—and to stigmatize individuals who leave the formal labor force to support their children or aging parents. Women still spend disproportionately more time on unpaid care work at home, often dropping out of the paid labor force to do so, while men are facing increasing levels of work-family balance anxieties. It doesn’t help that care work has historically been viewed as “women’s work,” and thus inherently unmanly.

Decades of devaluing care work has produced a grim landscape: a dearth of policy supports for, public investment in, and cultural acceptance of combining caregiving and breadwinning—for all genders. Simply put, there is not enough affordable, accessible, and quality care for those who need it. And that’s hurting us all.

The bottom line is this: We will all, at some point, need care or give care. It is not simply a women’s issue. Beyond amplifying the conversation around valuing, respecting, and elevating care work here and abroad, we have three goals for impact:

Give voice to the experiences of families and caregivers that are forced to make regular trade-offs between the cost, quality, and availability of early care and learning care depending on where they live, how many children they have, and their socioeconomic status.

Enable legislators to identify gaps in coverage so that they can make informed policy decisions and see where more investment, better training, regulations, laws, and work may be needed to build and maintain a high quality, accessible, and affordable care infrastructure (and to make it easier for their constituents to hold them accountable for the care environment in each state).

Supply businesses committed to improving their workforce policies and increasing their competitiveness with valuable data. We believe our comprehensive picture of care in America will illuminate for businesses where their efforts may be most needed to complement local, state or federal policies, and how they might attract and retain talent through such efforts.

The Index draws on unique proprietary data from Care.com(1) as well as publicly available data from other organizations including the U.S. Census Bureau, U.S. Bureau of Labor Statistics, the National Association for the Education of Young Children (NAEYC), and the National Association for Family Child Care (NAFCC) from 2015. (In order to provide a consistent snapshot of paid child care arrangements across the course of the year (school-year and non-school year), all data are taken from the full 2015 calendar year.) A new Care.com survey, conducted in October, 2015 by Hanover Research (a market research company), using an online panel maintained by ResearchNow (a digital data collection company), with design input from New America, provides additional data on paid child care arrangements in each state with a sample of households with children.(2) Survey questions include detailed information about the primary childcare arrangement for each child, time to find a caregiver, and reasons for changing child care providers. We use the index as exploratory research to identify key child care challenges and take a deeper look into what's happening on the ground.

(1) The Care.com dataset is an important new asset and lens into the overall care market; Care.com currently hosts 18.4 million members of which 56% are families seeking care and 44% are caregivers. Care.com users are present in all states and distributed roughly equal to the overall population distribution by state. In 2015, Care.com hosted nearly 7 million caregivers, families in approximately 93% of U.S. zip codes and caregivers in approximately 88% of U.S. zip codes. The typical care seeker for U.S. consumer business is female (82%), has an average household income of $75,000, and has at least one child under 18 in the house (73%). The typical caregiver is also female (94%) and well educated (61% indicating they have at least some college education). Care.com members seeking child care are similar to the U.S. population of families with children when it comes to the percentage of women who are the primary child care decision-makers and the number of families with dual incomes.

Nonetheless, Care.com members skew older than first-time moms, are somewhat more likely to live in an urban area, and have higher incomes on average. The higher income-skew is likely due to the fact that fewer low income families pay out-of-pocket for full-time child care. The Care.com survey was issued to more than 1,000 respondents in the "Less than $25K" income category in order to compare how low income households differ from high income households.

(2) Sampling frame, technique and execution: The Care.com survey uses household contacts available from online panel company ResearchNow, whose panel consists of 2 million people. The survey targeted a sample of online panel members in households with children under the age of 18. Quotas were set to ensure 385 valid responses per state (with additional quotas by income, race and marriage), and a margin of error of 5% at a 95% level of confidence. More specifically, the survey was sent out in three waves:

Wave one: households with children under 18 with quotas by race, income, and state.

Wave two: married couples with children under 18 with quotas by race, income, and state.

Wave three: persons age 18-64 with children under 18 (no quotas).

The response rate was 6.25%, meaning the percentage of opened survey invitations was 6.25%. It is possible that someone was surveyed more than once; ResearchNow was only able to ensure that each email address was only surveyed once. Total number of survey respondents was 15,038.

To prepare the data for state-level analysis, American Community Survey (ACS) Public Use Microdata Sample (PUMS) data were used to match the distribution of responding households on key characteristics to the state population distributions on those measures. If sample distributions on key household characteristics did not match state-level distributions of household characteristics, the data was weighted to eliminate known biases. The survey was weighted on income, marital status, and employment status.

To prepare the data for national level analysis, further weights were added based on 2010 U.S. Census data detailing the number of households with children under 18 per state. In other words, responses from states with higher proportions of total U.S. households with children under 18 were given greater weight when conducting national-level estimation, compared to responses from states with lower proportions of total U.S. households with children under 18.

This approach makes the survey responses representative of state and national distributions for these variables. Nevertheless, it does not account for any bias that could arise from non-random sampling. The known limitations of the survey are as follows:

Weighting data at the state and national level will approximate a representative sample. However, utilizing a voluntary panel rather than a probability-based panel, or random sampling through random digit dialing, introduces bias.

Selection bias on unobserved variables is not minimized.

Random sampling error cannot be estimated.

Care Index Methodology, Limitations, and Acknowledgements

Data and Methodology

Critically, the index and the New America Care Report translate what can be an emotional story for families into a narrative grounded in facts, data, and both quantitative analysis and qualitative storytelling—bridging the personal and the political. Data allows us to tell the stories of care in ways that we hope will resonate with and appeal to broader audiences, expanding the constituency who understand and are committed to this issue.

Each of the components of the Care Index brings new data to bear on the questions that most concern families when it comes to child care—cost, quality, and availability. The three components are combined into an overall Care Index and grouped in quartiles. By looking at quartiles within the index, and pulling out stories from four states—New Mexico, Illinois, Georgia, and Massachusetts—we give voice to shared challenges and opportunities in care.

Care Index Methodology, Limitations, and Acknowledgements

Cost

Cost is an important factor for families when considering child care options, yet there are few resources that allow for state-level comparisons of the complete cost of child care to families. (There is potentially a relationship between cost and availability we are unable to address in this iteration of the index.) The Care Index brings together data on the cost to families of in-home care (i.e. nanny-care), family child care homes, and center-based care, and takes into account tax credits (state and federal) for child care to develop a window into the out-of-pocket price for families in each state.

Child care costs can vary significantly by the age of the child. Cost metrics were calculated for infants (ages 0-1 year), toddlers (1-3 years), and preschoolers (3-5) for comparison in the Care Index. The average household cost for one child was used in the Care Index calculation. Cost data for in-home care came from Care.com hourly rates offered in jobs posted by families seeking full-time child care. Child care center rates were drawn from rate cards submitted by child care centers on Care.com. (Child care centers voluntarily submit (cost) rate information to be posted on their Care.com profile. In 2015, more than 14,000 child care centers, and 4,000 family child care homes had provided weekly rate information listed on Care.com.) Since we are primarily interested in comparing cost rather than hours used, the Care Index is based on a benchmark of 40 hours per week of care for 52 weeks for one child. The actual amount of time spent in paid child care varies across families depending on their needs, and may be more or less than 40 hours per week. The percentage of children in each type of care was calculated for each state using new (2015) survey data collected by Care.com. Since the impact of child care costs on a family depends on household income, the cost was normalized using U.S. Census, state-level data on median household income.(3) To test the accuracy of the cost data, we compared the state-level cost data to the state-level cost data compiled by Child Care Aware of America in their 2015 survey of Child Care Resource and Referral State Networks.

The equation below calculates the cost component of the child care index for state i with the following:

Cni = cost of paid individual caregiver (e.g. nanny) per hour (4)

Pni = % of children that have an individual caregiver out of the % that use paid child care

Cdi = cost of a child care center or family child care home per week for one child (5) (6)

Pdi = % of children that go to a child care center out of the % that use paid child care

Pfi = % of children that go to a family child care home out of the % that use paid child care

Xi = value of state and federal child care tax credits for one child for a family with the median household income

HHIi = median annual household income

This equation gives the expected cost of childcare as a percentage of median household income for each state. Ultimately, this shows us where families are spending more out-of-pocket to cover child care expenses.

(3) We discussed calculating the opportunity cost of a parent staying home in the index but didn’t feel confident we’d be able calculate accurately. A recent report and interactive calculator from the Center for American Progress provide a valuable approach to answering this question; however, the report was not available in time for us to use while designing our methodology. This consideration is also addressed in the 2016 McKinsey Power of Parity research on women’s labor force participation.

At this point in time we are unable to address the issue of causality, namely, we can’t explain why people’s child care preferences are what they are and why parents opt for a certain type of care. We will use our state deep dives to explore this question across four local contexts.

(4) We are not accounting for family care, such as a (paid or unpaid) grandparent or sibling, within our individual provider/nanny definition. However, we are including any non-relative, such as a friend or neighbor, who received payment and was listed on Care.com.

(5) Care.com data indicate that family child care homes are about 15 percent less expensive than childcare centers in the same area. However, because the state-level data were insufficient to distinguish between the two types of care in this iteration of the index, they are combined in the calculation for 2015. As a result, the Cdi data are likely deflated. Future studies should disaggregate the cost of child care centers and family child care homes.

(6) Child care businesses self-report this data, and know that it will be searchable online. While there are inherent limitations to self-reported data—chief among them the potential for inaccurate data designed to reflect positively on the business—child care centers and family homes have little to gain by misrepresenting their prices to potential customers.

Care Index Methodology, Limitations, and Acknowledgements

Quality

Quality child care provides the building blocks for healthy, happy kids and prepares them to succeed in school and beyond. However, it can be difficult to compare and measure the quality of different types of child care. There is no single child care quality metric with universal acceptance among child care researchers. The quality component of the Care Index combines quality data for in-home child care providers (nannies), family providers, and child care centers. (State licensing and monitoring requirements are not taken into consideration, as they are not consistent across states, primarily cover health and safety, not educational quality and are thus not reliable measures of quality that we can compare across states).

For in-home child care providers, the data comes from parent reviews of caregivers on Care.com. Reviews include both qualitative comments and a score on a scale from 1 to 5.

Q1i = average child care provider rating (out of 5 stars)

This refers to child care providers who come to the home of the child, such as nannies. In some studies (such as the National Institute of Child Health and Human Development (NICHD) Study of Early Child Care), researchers have observed interactions with caregivers in the home, which is probably the ideal way to measure quality. However, these studies are not large enough to compare across all 50 states. One of the advantages of reviews is that they are an evaluation of performance rather than just a characteristic of the individual. While ratings are subjective evaluations, it is probably the best currently available measure of quality for in-home care across all 50 states. The main source of bias is that people tend to submit reviews when they are passionate (positively or negatively) about a caregiver, but assuming that is true across all states, then the differences across states reveal different opinions about the quality of nanny care.

For family providers, quality is measured as the percentage of family homes in a state accredited by the National Association for Family Child Care (NAFCC). NAFCC is the leading accrediting organization for family child care homes. Standards cover the following categories: Relationships, The Environment, Developmental Learning Activities, Safety and Health, and Professional Business Practices.

Q2i = percentage of family providers in a state with NAFCC accreditation

For child care centers, quality is measured as the percentage of centers in a state that are accredited by the National Association for the Education of Young Children (NAEYC). NAEYC is the leading accreditation organization for child care centers. Standards cover the following categories: Relationships, Curriculum, Creating Caring Communities for Learning, Assessment of Child Progress, Health, Preparation and Skills of Teaching Staff, Families, Community Relationships, Physical Environment, Leadership and Management.

Q3i = percentage of child care centers in a state with NAEYC accreditation

The quality metrics for in-home child care providers and child care centers are not directly comparable, so these were scaled separately using a Z-score before being combined. As in the cost equation, the provider metrics were multiplied by the percentage of survey respondents who use each type of care (in-home child care or child care center).

The quality index compares the quality of paid child care options across states using summary variables that are comparable at the state level and represent the best proxies for quality available. It is important to note that we use accreditation as a proxy for quality. We chose NAFCC and NAEYC as standardized observational measures of child-teacher interactions across the states. Furthermore, accreditation is used here as the minimum expectation or valuation of quality. We went back and forth on how to measure quality, and particularly in-home (nanny) care, and, in settling upon accreditation, want to be transparent that it is an imperfect measure and that there is currently no standard, universally accepted measure of quality.

Conclusions may only be drawn for those who are accredited. Those that are not accredited may not be for one of three reasons:

Providers may be of high quality but uninterested in accreditation.

Providers of low quality may have tried and failed to become accredited.

Providers of low quality may not be aware or interested in accreditation.

Varying accreditation rates may be due to any of the following reasons:

Accreditation rules may be more stringent than states' licensing requirements, especially around ratios and group sizes. In such states, meeting licensing regulations may be a necessity, but going above and beyond them (especially in areas that entail additional costs to programs) may not appear to be in the best business interest of the program.

Similarly, as states rollout more early education requirements, how they set standards for those programs may move them closer to further from accreditation criteria. Some states may explicitly refer to accreditation criteria (or accreditation in total) in setting standards.

Some states, like Georgia, are moving away from NAEYC accreditation and pursuing quality rating and improvement systems (QRIS), and setting tiered reimbursement rates for federal and state child care subsidies based on QRIS ratings. Yet these systems vary by state and thus are not easily comparable, and include accreditation in differing ways, some of which more clearly incentivize pursuing accreditation than others.

In other states, such as Massachusetts and Connecticut, a comparably high percentage of programs are accredited, which may be due to efforts at the state or in larger districts (like Boston) to support accreditation as a model of quality improvement.

There are costs associated with meeting accreditation criteria, as well as pursuing accreditation generally. In some states or parts of states, the expense is too great compared with the revenue that a center can generate, so the ability to meet accreditation requirements may be limited.

Some large-market providers, like KinderCare and EduKids, have supported accreditation at an organizational level. To the extent that these chains are localized (i.e. based in one particular state), they affect the number of accredited programs.

Care Index Methodology, Limitations, and Acknowledgements

Availability

The third pillar of the Care Index is availability. Even if child care costs are low enough to be affordable for parents, and teachers trained and paid enough for the quality to be high, the lack of child care providers can be a major cause for concern. Accurately capturing the number of early care and learning workers in the current patchwork system is difficult. Care.com has unique data that can help answer questions about the availability of in-home care based on the number of active child care provider profiles on their site, including those both employed and those looking to be employed. (The U.S. Census does not track informal in-home workers. This project is one of the few that can address this important part of the child care workforce.) For child care centers, data on availability comes from the U.S. Bureau of Labor Statistics (BLS), which tracks the number of child care providers employed in day care centers. The U.S. Census employment data is based on the NAICS code 624410 for Child Day Care Services, which does not track pre-K teachers, including Head Start teachers, in public schools.

In order to understand availability of child care in relation to the demand for child care, each metric is calculated as a ratio of the number of child care providers to children under 5, as measured by the U.S. Census Bureau. This Index focuses on care for ages 0<5 for two reasons. First, this is an age group for whom access to learning is not currently provided by the state (i.e. prior to kindergarten-age). As a result, the time and monetary provision of early care and learning is left largely to families. This is also the age group sampled in the American Community Survey of the U.S. Census Bureau.

It’s entirely possible that someone who’s working for a child care center also has an individual profile on Care.com. Unfortunately, there is no easy way for us to estimate that overlap. The employee count data for daycares does not have individual level data that we could check against Care.com data. However, as long as the overlap % is not dramatically different by state, this should not impact our availability entry for the index. We are normalizing the entries, so assuming the double-counting factor is fairly similar across states, the overlap won’t have a major impact the outcome.

The implicit assumption of this project is that all children need care and an environment in which they can develop cognitive, linguistic, and socio-emotional skills. By counting all children, we are attempting to acknowledge the care work that is done by parents (in the broadest sense) but undervalued and unpaid. Care work isn't traditionally valued or paid but should be, irrespective of who is doing it.

The equation below calculates the availability for each state i using the following:

Ni = number of individual child care providers on Care.com

Ei = number of employees at day care centers

Hi = Number of children under age 5

By combining individual providers and child care centers, the availability index considers the number of paid child care options relative to the number of children under 5 by state. However, measuring state-level data is imprecise, as care may be more readily available in urban centers or in certain geographic areas, particularly those with more working parents or more resources. Thus, the metric more accurately captures broad coverage rates of care. Better geographic data, more accurately pinpointing child care demand with supply and identifying child care “deserts” is needed. The state of Georgia and the city of Philadelphia have begun this more precise geographic availability mapping.

Care Index Methodology, Limitations, and Acknowledgements

Combining Cost, Quality, and Availability

Together, the three components of the Care Index offers new ways to understand the child care landscape across the United States. Each component of the Care Index was given equal weight when ranking states.

We decided to use a Z-transform, which measures how far each state is from the mean. For all three normalized measures, a value of 100 indicates the state is at the mean. Each 15 points above or below 100 indicates one standard deviation above or below the mean.

Given that this research is exploratory, and the ranking a rough estimate, the ranking is then broken up into quartiles so that common experiences, challenges, and bright spots can be identified. Four states have been selected for in-depth reporting and analysis based on the following criteria:

One state for each geographic region (Northeast, South, Midwest, and West)

States representing a variety of urban, suburban, and rural experiences

States representing a variety of demographic make-ups, highlighting changing and diverse composition of American families and particularly the experiences of dual-language learners

States representing a variety of public infrastructure for ages 0-5, to highlight diverse delivery of early education resources

The states are as follows: Massachusetts (1), Georgia (2), Illinois (3), and New Mexico (4). View the index here.

Care Index Methodology, Limitations, and Acknowledgements

Limitations

This index is meant as an exploratory project, broadly mapping the landscape of child care, and highlighting where additional data is needed. We need to capture the invisible economy of unpaid and unregulated care for cost, develop standardized and reliable measures for quality, and conduct more research on the complex factors affecting care fit and accessibility considering geography, age of child, the match between parent work hours and hours care is available, and other metrics for a clearer picture of availability. Only then will we truly see the full picture of the challenges of the care infrastructure in America, which will give policymakers the best tools to begin to fix it.

Care Index Methodology, Limitations, and Acknowledgements

Acknowledgements

The Care Index is a data and methodology collaboration between New America, Care.com and others.

We would like to thank Care.com and A.T. Kearney for their generous support of this important research. Thank you to the numerous organizations and individuals who provided thoughtful contributions and reviews for this report. The conclusions reached in the New America Care Report are those of the authors alone. We’d especially like to thank

Louise Stoney, Alliance for Early Childhood Finance

Courtney McCaffrey, AT Kearney

Joyce Hodel, Care.com

Katie Hamm, Center for American Progress

Stephanie Schmitt, Center for Law and Social Policy

Lynette Fraga and Michelle McCready, Child Care Aware of America

Anne Mitchell, Early Childhood Policy Research

Barbara Gault and Ariane Hegewisch, Institute for Women’s Policy Research

Carmen Niethammer and members of the Gender Secretariat, International Finance Corporation

Kenneth Matos, Life Meets Work Institute

Helen Blank and Karen Schulman, National Women’s Law Center

Marica Cox Mitchell and Kyle Snow, National Association for the Education of Young Children