Years of funding shortfalls and stalled IT projects have placed the census in a precarious position.

A lot has changed since the first national census was conducted back in 1790, during the first term of the United States’ first president, George Washington, when the country’s population numbered around 3.9 million. But the need for the decennial census has not.

Conducted in every year ending in zero, the census serves several important purposes. It ensures that citizens are granted the correct number of representatives at the state, local, and federal level, and allows lawmakers and researchers to fairly lobby for them in Washington. It also reflects the overall, diverse makeup of the U.S. population and gives analysts the data they need to accurately prioritize people’s needs.

But the Census Bureau has found itself in increasingly dire straits in recent years, leaving the future of the decennial survey in question — which could spell disaster for communities that rely on the census for important representation.

As ThinkProgress’ Elham Khatami previously reported, a Sunlight Foundation survey published on October 23 found that 74 percent of city officials nationwide trust and rely on the results of each decennial survey to adequately serve their citizens. Specifically, they said, the information was “important” or “very important” for planning, development, innovation, and analytics purposes.

“Federal data is what lets us understand how we’re doing. It shows us, here’s how we compare to Jacksonville or Knoxville or Albany,” one survey respondent, a city hall officer in Arkansas, said. “No other entity can collect information on that scale across city or state lines. It’s not just us who needs this information. It’s everyone.”

Ethnic minorities and traditionally underrepresented groups especially rely on the census for the voice it gives them in government — and are at risk if the survey goes awry or if their communities are not accurately counted. Because the Census Bureau uses this data to draw congressional maps, an undercount of diverse populations that traditionally vote Democrat could end up benefiting Republican districts.

To its credit, the Census Bureau is developing methods for ensuring those communities are counted accurately and in a more efficient way, including overhauling existing technology to be able to handle the massive numbers they need to process.

But many of those IT systems have hit a brick wall in the form of under-funding. One of them, devised with the hopes of saving the agency billions of dollars down the line by centralizing data processing under a single platform, called CEDCAP, has been hit especially hard by the cash shortages.

“We get promises that are never delivered on”

Over the past decade, the Census Bureau has begun quietly building up its tech infrastructure to handle the burden of a cumbersome survey process faced with a growing population and a lukewarm response rate. New apps that would allow survey conductors (or “enumerators”) to log information on the go and software that would house that data in one place have garnered praise for their ingenuity and eventual ability to cut costs and save precious time.

But the Bureau has been missing its deadlines.

A promised list of “tech milestones” that the Bureau promised to present before the Oversight Committee in November 2016 simply never materialized, as Federal Computer Week reported.

Officials were also forced to scrap plans for testing in 2017 due to “budget uncertainties.” Instead, the agency has had to fall back on last minute, frenzied End-to-End testing in 2018. Currently, as The Los Angeles Times reported, “only four [of the Bureau’s technology systems] have completed development and 21 have some functionality.”

To address some of these issues, the Bureau is asking for more resources. On October 12, Commerce Secretary Wilbur Ross appeared before the House Committee on Oversight and Government Reform to request $3.3 billion in additional funding for the 2020 census. According to Ross, the increase would allow for increased cybersecurity preparation and testing and implementation of programs like COMPASS, a mobile app for enumerators to record responses in the field, which would also offer multilingual options.

This $3.3 billion would bring the overall cost of the 2020 census to $15.6 billion total, approximately 27 percent more than the $12.5 billion originally allotted.

The proposed increase has made Congress uneasy.

CREDIT: Government Accountability Office

“We get promises that are never delivered on,” said Rep. Jody Hice (R-GA) in the October 12 committee meeting, referring to similar funding requests for the 2010 census, during which the Bureau was forced to abandon technical upgrades at the last second and revert to paper and pencil reporting.

Oversight Committee members and critics alike have blamed census officials for the ongoing struggles to deliver.

“It’s leadership,” David Powner, director of IT at the GAO, told ThinkProgress over email, referring to the missed November deadline and the lagging IT schedule. “It’s primarily due to poor planning (starting late) and poor management (schedule and risk management).”

It’s true that leadership issues have been plaguing the Census Bureau for some time: a new CIO was only appointed in June 2016, following a nearly year-long gap; former Census Bureau Director John Thompson also resigned abruptly in May 2017, amid stormy budget talks. Most recently, the agency removed Lisa Blumerman, head of the decennial census program, from her role, replacing her with a former 2010 census regional official.

But leadership problems aren’t the agency’s only pitfall. Combined with funding shortfalls and an overly ambitious IT schedule, a mass of issues has left the agency spinning out of control.

Technological woes stretching back years

The agency’s technical endeavors have been accurately described as “ambitious”: the COMPASS app, new aerial imaging systems (to be used “instead of sending Census employees to walk and physically check 11 million census blocks”, the agency wrote in its 2020 Operational Plan), and a system meant to centralize all census data operations under one roof.

The last of those is the aforementioned CEDCAP program, or Census Enterprise Data Collection and Processing. According to its website, the CEDCAP system “will handle data collection and processing for the entire agency”, whereas the Bureau previously employed a variety of methods to handle survey samples, data collection, data capture, and data processing for hundreds of separate surveys. CEDCAP centralizes data capture and processing for the entire agency under one umbrella in order to cut back on waste and minimize burden on taxpayers.

But CEDCAP funding is strained.

“Only two months into my tenure, the Census Bureau suddenly announced a 40 percent cost overrun in one component, namely, the Census Enterprise Data Collection and Processing,” Ross lamented in the October 12 hearing. He claimed that he had not known about the shortfall until after he took over as secretary.

Commerce Secretary Wilbur Ross appears before the House Committee on Oversight and Government Reform to discuss preparing for the 2020 Census, on Capitol Hill in Washington, Thursday, Oct. 12, 2017. (CREDIT: AP Photo/J. Scott Applewhite)

At first glance, the timeline fits. On May 3 this year, more than two months after Ross was sworn in as secretary, former Census Bureau Director Thompson appeared before the House Appropriations Subcommittee on Commerce, Justice, and Science and announced that CEDCAP was at least $309 million over its original projected cost of $656 million. More troubling, Science Magazine noted, “a congressional watchdog group…pegged the overrun at $400 million and warned that costs might continue to rise.”

But CEDCAP’s troubles date much further back than that. At the very least, Ross should have had some inkling that the program — along with several other agency’s projects — was suffering, given that the GAO had already flagged CEDCAP for its list of “High Risk” programs.

When the 14-part CEDCAP program was first floated in 2011, it didn’t come with an official cost estimate, or “POE” (program office estimate). Even after an estimate was finally made available, the CEDCAP program office was forced to slash that total so that the budget would make it through Congress unscathed.

The Bureau finally kicked off work on the CEDCAP project in October 2014. The program came with a recalibrated $548 million price tag.

Initial work on its various components appears to have been undertaken by several contractors, including Portal Technologies, a small firm — it consists of fewer than 10 people, according to LinkedIn — run by Steven Nesbit. Nesbit’s team also built the aforementioned COMPASS mobile platform, among other things. (Portal Technologies declined to comment for this article or elaborate on its contracts with the Census Bureau.)

The Bureau also decided, as a cost-cutting measure, to buy several off-the-shelf programs meant to construct some of its apps, rather than hiring out and building them up from scratch.

Still, less than a year later in July 2015, an independent watchdog group noted that the initial cost estimates were ballooning: It flagged the price tag at closer to $1.14 billion.

Current official estimates are only slightly lower. According to Carol N. Rice, assistant inspector general for Economic and Statistical Program Assessment at the Commerce Department, who also testified before the Oversight Committee in October, the new CEDCAP cost estimate is closer to $1 billion — nearly identical to the original POE the Bureau had calculated initially, before being forced to slash their proposed budget to get it through Congress, according to some sources.

“In June 2017, the Bureau reported the program experienced increases, estimating that the CEDCaP program will cost $965 million,” Rice said on October 12. Offering a dire warning, she added, “If this enterprise-wide data collection solution falls short, the 2020 Census is at risk for accumulating even further escalating costs.”

Uncertain funding

A good chunk of CEDCAP’s budget is, understandably, housed under the 2020 census budget. By slipping the CEDCAP budget in with 2020 census budget, it’s less visible.

Congress is, in theory, less likely to dramatically slash funding for the 2020 census than for other discretionary programs. Therefore, CEDCAP funding should — also in theory — be more protected. One source, who chose to remain anonymous because they were not authorized to speak on the record, put it in simple terms: “It was safest to ‘hide’ it there because it was least likely to be cut.”

Unfortunately, that doesn’t seem to be the case.

With fiscal conservatives in charge and President Trump’s no-frills ideology in the White House, the 2020 census has apparently dropped on the list of government priorities — and with it, funding for CEDCAP and similar tech upgrades.

Compounding the problem is years’ worth of funding shortfalls. Congress has refused to adequately fund Census IT programs for the past several fiscal cycles, exacerbating the problem. At present, the Bureau is not only faced with the 2020 crisis and CEDCAP over-expenditures — it’s dealing with all of that on top of a mass of missing funds that was simply never resolved.

In some instances, Republicans in Congress used the 2020 census as a bank, withdrawing funds that the Bureau needed to conduct tests, beef up technology, and generally keep operations running smoothly.

In June 2015, House legislators voted largely along party lines to siphon $121 million in the 2016 Commerce spending bill that would have otherwise gone to 2020 census preparation. Republicans instead funneled the money to other agencies.

“The House appropriations committee cut the agency’s requested 2016 budget for census-related activities by $374 million,” Science Magazine reported that month. “After this week, that amount has grown to nearly half a billion dollars less than the $1.2 billion the administration is seeking for those activities. The swaps conform to the Republican leadership’s dictate that any proposed spending increases be revenue-neutral, that is, that legislators find the money from another agency.”

At least one person has argued that Secretary Ross himself is allowing Republicans to undermine the decennial census for their own gain.

“You might think…Ross is riding to the rescue with his recent plea to Congress for $3 billion more in census spending,” former California congressman and Al Gore campaign chairman Tony Coelho claimed in a column for RealClearPolitics. “In fact, the secretary’s new plan still does not fund the advertising or partnership programs credited with making the last two head counts the most accurate in history.”

He added that “Congress could make Ross a white knight on a ride to nowhere if [the] GOP conference fails to approve the funding in time to make a difference, if they approve the money at all,” and claimed that the secretary’s announcement may simply have been a way to “camouflage [the Republican] effort to starve the census.”

Commerce Department spokespersons did not immediately respond to Coelho’s accusations.

Claiming that Ross is intentionally undermining the decennial survey is a bit of a reach. But the secretary has previously attempted to shift blame onto the Obama administration for the current census crisis, calling its cost estimates and anticipated budgets “overly optimistic.”

“It’s unclear if Secretary Ross asked for enough in his revision of the full-cycle budget revision since we have not yet seen the details. Because only $187 million of the new request is included in the Administration’s FY 2018 budget request, the $3.3 billion life cycle cost increase he indicted is only a promissory note,” Phil Sparks, communications consultant and co-director of The Census Project, told ThinkProgress in an email.

Still, he added, “What we do know is the more should have been included in the FY 2018 revised request for investments in the Partnership and advertising programs targeted to Hard-To-Count populations in the 2020 Census.”

Ultimately, as one expert noted, it’s unlikely that Congress will deny additional funding requests altogether. If it does, the fate of the 2020 decennial census hangs in the balance.

“If the budget is not granted as requested or given additional funding,” they claimed, “the Bureau won’t be able to conduct the census itself. …Vital 2018 testing and programs would not be completed, and the 2020 survey would not take place as planned.” (Census spokespersons said the Bureau “does not speculate on hypothetical situations.”)

For now, the 2020 census is scheduled to take place as planned. But if its parent agency is, for some reason, denied by Congress or the Trump administration the money it needs, the country could suddenly find itself confronting a new challenge unlike any it’s faced before.