Formerly the non-partisan watchdog of the 2010 US Census, and currently an opinion blog that covers all things political, media, foreign policy, globalization, and culture…but sometimes returning to its census/demographics roots.

Popular Posts

Posts Tagged ‘IT’

To any investors out there, this is as much of a bull-shit alert as I can possibly give you. As MyTwoCensus has repeatedly noted, and the Census Bureau has repeatedly acknowledged, the PBOCS systems used during 2010 Census operations were complete failures that created problems resulting in severely delayed operations (thousands of workers sat around waiting for assignments) and mismanaged data (2010 Census forms had to be manually imported at a snail’s pace, and who knows how many of these never made it into the system at all…). But the PR teams below state otherwise:

WASHINGTON and BOULDER, Colo., Oct. 20 /PRNewswire/ — Rally®, the leader in Agile application lifecycle management (ALM), and ICS, a proven 8(a) information technology contractor, today announced that Rally’s Agile ALM platform played a central role in the success of ICS’s work in support of the U.S. Census Bureau’s 2010 decennial census.

Rally Unlimited Edition enabled the 2010 Census Agile ICS software development team to deliver the Census Bureau’s paper-based operations control system (PBOCS) software over 50% faster than delivery times of the 2000 or the 1990 census software, with just 1/3 of the staff. By tracking its software development process with Rally, ICS not only delivered software requirements and met immovable deadlines, but exceeded expectations by delivering an additional software module.

“The efficiencies we realized with Rally are a perfect example of the change being driven within the government to improve the performance of IT operations across the board,” said Khurram Shah, ICS founding partner and chief strategy officer. “The velocity and productivity gains Rally brought to the 2010 Census Agile ICS development team enabled us to deliver applications that processed more data at a much faster rate than during previous Census operations.”

About the United States Census

The United States Census is a decennial census mandated by the United States Constitution to gather statistics on the U.S. population. The data collected helps determine the number of seats states have in the U.S. House of Representatives. In the 2010 Census, this data also helps communities receive more than $400 billion in federal funds each year.

“Because Census deadlines are mandated by the Constitution, there’s no question that the execution, performance and timing of our software development operation was critical,” said Erika Peace, technical project manager at ICS. “Rally provided the right tools at the right time so we could cost-effectively deliver technology more accurately aligned with our client’s business objectives.”

Challenges

Software development requirements are defined by the mandate that decennial U.S. Census figures are based on actual counts of every person dwelling in U.S. residential structures. Delivery dates are immovable, as the Census Bureau is required by law to report the nation’s population and the allotment of seats in the U.S. House of Representatives for each state by the end of December. The 2010 Census Agile ICS software development team also had to adapt to changing requirements and unique circumstances, such as the challenges around accurately counting “group quarters,” like college students living in dormitories.

Solution and Results

The 2010 Census Agile ICS software development team brought in Agile development practices to deliver 12 key requirements for the Census Bureau’s paper-based operations control system (PBOCS) software. By implementing Rally Unlimited Edition to provide the real-time status, progress and quality of the Census Bureau’s software development processes, the Agile ICS team over-delivered ahead of schedule – completing all requirements in just 18 months with just 1/3 of the staff.

“By taking advantage of Rally’s Agile ALM platform, the 2010 Census Agile ICS software development team was able to help the Census Bureau improve the speed and accuracy of the 2010 census-taking process in response to the ever-increasing population of the United States,” said Ryan Martens, Rally’s founder and CTO. “Demonstrating that Agile practices meet federal schedule performance index requirements allows Rally’s Agile ALM platform to align with government projects.”

In order to achieve critical requirements within the allotted timeframe, every incremental build resulted in shippable, working software. The 2010 Census Agile ICS software development team used Rally to meet changing requirements, build incrementally and turn deliverables around quickly. Requirements, acceptance tests and source code changes were tracked in Rally’s Agile ALM platform, giving the team rapid feedback on the status and quality of each build.

Rally’s powerful reporting capabilities were critical for providing data analysis, progress reports and status updates to government officials on a daily basis. Providing real-time visibility to senior government officials was vital for making informed decisions, assessing scope change and tracking team progress to delivery.

While addressing the National Press Club, Census Bureau Director Dr. Robert Groves summed up the importance of the PBOCS software delivered by ICS and how well it was performing when he said, “This software system, called the Paper-Based Operation Control System (PBOCS), performs various functions that are really crucial for the non-response follow-up phase…we’re processing at rates that we never imagined we could process.” (1)

Government Agile Success Tour

Rally is hosting a special edition of its Agile Success Tour on October 21, 2010 in Bedford, MA for those working in Federal contracting environments. This free, interactive half-day seminar is intended for anyone who is adopting or considering adopting Agile development practices for government software projects. Northrop Grumman and Rally Software will discuss real-life Agile implementation stories from the Department of Defense, civilian agencies, and state and local governments.

About ICS

ICS is a certified 8(a), Small Disadvantaged Business (SDB), Minority Business Enterprise (MBE) founded in 2003 by seasoned technology professionals. ICS has proven an innovative designing experience that is client-focused, developing quality solutions in mission critical environments in the public and private sector.

ICS is comprised of experienced management and technically agile professionals with diverse competencies, creating a collaborative program and project management environment for clients. The size of the organization, coupled with the focused range of services performed, enables the company to rapidly source and retain thoroughly trained, certified professionals with tested, measurable performance and proven experience.

About Rally

Rally is the recognized leader in Agile application lifecycle management (ALM). We are dedicated to helping organizations embrace Agile and Lean development practices that increase the pace of innovation and improve product quality. According to a study by QSM Associates, software-driven companies that rely on Rally’s Agile ALM products and services are 50% faster to market and 25% more productive than industry averages. The company’s experienced services group, including training through Agile University, guides companies through the organizational change required to become innovative, Agile businesses. Rally’s products, including AgileZen, currently support more than 3,000 corporate customers, 76,000 projects and 138,000 users in 60 countries. For more information, visit www.rallydev.com.

(1) Dr. Groves briefing at the National Press Club on June 2nd, 2010; transcript available here.

Rally, the Rally logo, Rally Software Development, and AgileZen are trademarks of Rally Software Development Corp. Third-party trademarks are the property of their respective owners.

Vangent reported revenue from continuing operations of $214.8 million for the quarter ended July 3, 2010, an increase of $79.7 million or 59% compared to $135.1 million in the quarter ended June 27, 2009. The increase in total revenue from continuing operations reflects an $83.0 million or 72% increase in revenue from the Government Group. The increase is primarily attributable to $85.9 million in revenue from Vangent’s 2010 U.S. Census contract.

McKeon is the founder of Veritas Capital, a New York private-equity firm that invests in intelligence and defense contractors. Among the fund’s holdings: DynCorp International Inc. (DCP ) in Irving, Tex., whose name was tarnished in the late 1990s when some of its employees in Bosnia became embroiled in a sex trafficking scandal.

In the summer of 2005, another scandal-plagued military contractor caught McKeon’s eye. MZM Inc. was the focus of a federal investigation into whether its owner bribed California Representative Randy “Duke” Cunningham in exchange for hundreds of millions of dollars of government contracts providing technical and other support to intelligence agencies and the military. Sensing an opportunity, McKeon hired a former CIA general counsel to quiz Pentagon and intelligence officials about the viability of MZM’s business, which before the invasion of Iraq included helping with controversial analysis of Saddam Hussein’s nuclear capabilities. “We take on messy situations, and MZM was a mess,” says McKeon, whose top secret security clearance still doesn’t allow him to know all the details of what his companies do.

Not everyone is cheering. Critics argue that Athena’s apparent success rests on lucrative contracts landed under questionable circumstances. “Veritas is profiting from the spoils of congressional bribes,” says Keith Ashdown, vice-president of Taxpayers for Common Sense, in Washington.

He says his office does keep records of the amount of incidents associated with a particular employee. That also means tracking in-bound and out-bound traffic and worker use patterns. He will release this data to managers to help them make decisions, Ruland says, but he insists his office is “not the police.”

Ruland says his office has to reimage 25-30 computers per month because of inappropriate surfing and use of social networking websites like Facebook. He says this creates significant productivity loss.

“When you have to reimage a PC, you’re taking staff away from the IT directorate, but more importantly you’re taking the ability of that individual to do their job,” Ruland says.

In an attempt to respond to the growing cybersecurity problem, Ruland says his office has been working on a Web surfing handbook for new employees.

“We determined that there was a need to try and give something to somebody that they could have at their desk, that we could give them when they come in. Kind of a simple: ‘You’re at the Census Bureau; this is what you should think about.’”

He says the first draft of the guide is almost complete. It will eventually be distributed to new employees at orientations along with a general IT security handbook that is already distributed.

“It’s based on things that we’re seeing: what they should worry about when they’re going on Facebook, what they should worry about when they’re going on the Web and doing their business or surfing,” Ruland says.

Subject: DAPPS payroll software change
Message Body:
It appears that the following occurred on either 6/27 or 6/28. A change to D-308 timesheet editing rules in DAPPS was implemented which no longer permits timesheets to include entries for which one entry ends at the same time as the next begins. Such entries have been accepted up until now, and are *necessary* to properly record work across the midnight boundary. Rumor has it that the change was made to stop somebody from getting extra pay by exploiting a bug in the system.

It is not clear what the details are, for example how multiple task codes affect the results, but it is known for certain that work across the midnight boundary is affected, and that the new edit rules can cause errors between timesheets and batches. Washington I.T. support apparently is telling people they have to shift their time 1/4 hour away from midnight (trivially, falsifying their timesheets). My LCO discovered that payroll batches which had been entered without errors for the previous week, if re-opened and re-saved for any reason, generated NEW errors. It also appears that a key diagnostic report regarding overlapping time is now, sometimes, “broken” in a way that masks existing errors.

If my understanding of DAPPS is correct, it is *possible* that MANY LCOs experienced errors when payroll was transmitted 6/29.

The change was made the day before payroll had to be wrapped up and transmitted, and there was NOT SO MUCH AS AN E-MAIL NOTIFICATION. We can’t be the only ones who got burned; HQ I.T. support confirmed the situation and provided the workaround.

I would love to know how much of a problem this has caused, and/or whether the change had to be backed out of the code. As of Tuesday afternoon 6/29 the change was still in place.

OK, everyone, I am using this board to try to find some answers about the state of PBOCS (paper-based operations control system) across the US. As of this week (yesterday and today), is the PBOCS system working in your area? How have technical glitches harmed 2010 Census NRFU (non-response follow-up) operations? What have you heard from your higher-ups about 2010 Census operations for the coming week? I know it’s like beating a dead horse in that I’ve discussed these issues over and over again, but it seems like in many places the problems have still not been corrected, so let’s get some reports from the ground. Please state your approximate location in the comments section. Thanks!

Earlier this week, Dr. Robert M. Groves told NextGov that the Census Bureau’s infamous computer problems with the paper-based operations control system (PBOCS) software were fixed. He even went so far as to have a memo, obtained by MyTwoCensus, sent by his underlings to Census Bureau officials throughout the country, relaying this information:

Update: The technical team is ready to implement the fix for performance issues. In order to do so, the system will be brought down at 6:45 PM ET. All users need to log off prior to this time and remain out of the system until further notice. This process should take approximately two hour and once the system is available, there will be a staggered log-in. DOTS will send out another message regarding when PBOCS will be available and the staggered log-in schedule.

Update: The technical team is still testing the fix for the performance issues. They hope to have the system available sometime tonight however at this point we do not have a more specific time frame. DOTS will send out another update at 7:00 PM ET.

If you have any questions/concerns regarding this message, please respond to only DOTS 2010. Please do not reply to all.

Information: Due to log-in issues caused by the number of users trying to come onto the system right now, the staggered log-ins have been suspended. The six remaining RCCs will not be allowed to log-in until the current backlog has declined. DOTS will send out a message when the remaining RCCs can log-in.

The remaining RCCs are:

2499 = Detroit

2599 = Chicago

2699 = KC

3199 = Denver

2799 = Seattle

3299 = Los Angeles
If you have any questions/concerns regarding this message, please respond to only DOTS 2010. Please do not reply to all.

For weeks, we have reported on the serious PBOCS (paper-based operations control system) computer, tech, and software failures that have occurred in Washington and at local census offices throughout the country. According to NextGov, the problems are all solved:

The top Census Bureau executive said on Tuesday that the agency has fixed glitches that caused major outages in a computer system that manages information collected by census takers.

Census Director Robert Groves told Nextgov that the bureau enlisted developers to work with agency staff to solve the problem.

The problems occurred in the paper-based operations control system and could drive up costs beyond the $15 billion the bureau estimates it will cost to conduct the 2010 decennial count, according to a report from the Commerce Department’s inspector general that was released last week.

“In the past four days there have been dramatic improvements,” Groves said.

He did not disclose how much the repairs cost but added that investments in the system “cost a lot less money than it would have cost if that system didn’t work.”

He added, “The problem with the system created a backlog of completed work being checked in. The impact of these problems will be on the backend processing.”

The cost of sending out part-time workers to travel door to door to visit households that failed to complete and send back a census form remains at about $85 million for each percentage point of households that did not mail back a form, he said. That works out to about $2.38 billion because 28 percent of households did not mail back their forms by the April 27 due date.

On other topics, Groves said the next decennial census should offer an online option, which Congress has pushed the bureau to consider for years. “I can’t conceive 2020 without it,” he said. But he added that the bureau should proceed cautiously as it weighs procurement options because “nobody knows what the 2020 Internet will look like.”

“There are pressures in DC to lock into [software] designs very early and say how much you are going to spend on the 2020 [census] before you know what you’re going to do,” he said. “These pressures have to be managed carefully.”

Groves supported the Census Oversight Efficiency and Management Reform Act, sponsored by Sen. Tom Carper, D-Del., to make the job of the Census director a five-year, term-limited position to promote continuity across administrations and alleviate the fears of partisanship in the bureau.

Groves said it was problematic that so many census directors had been appointed in years ending with a nine – the year before the bureau began one of its largest undertaking, the decennial count. “To say that’s a good way to run this place, you must believe that the place is better off without a director,” he said.

Earlier today, the Associated Press released a short article (below) that discusses the Census Bureau’s repeated paper-based operations control system failures. In response to Census Bureau Director Robert M. Groves’ comment that, “the problems stem from a late change from a handheld device system to the paper-based system” I can simply point to a response I just received from Michael Cook, Chief of the Decennial Media Relations Branch at the Census Bureau’s Public Information Office. Cook wrote me, “The change to a paper-based NRFU operation ordered two years ago by then-Commerce Secretary Gutierrez, required us to develop PBOCS in a compressed time-frame.” Now, this makes no sense. TWO YEARS IS NOT A COMPRESSED TIME FRAME. It is a ridiculously long amount of time to use engineers to tweak and test a system to make it perfect. Once again, the Census Bureau’s IT failures are pathetic and unaccepptable in the year 2010.

LOS ANGELES — The U.S. Government Accountability Office says a computer system needed to finish the 2010 census may not be up to the job.

GAO Strategic Issues Director Robert Goldenkoff said Friday before a congressional hearing in Los Angeles that the Paper Based Operations Control System hasn’t demonstrated the ability to meet peak requirements of the census as it seeks to count residents who did not return forms by mail.

Census Director Robert Groves says in a statement that the problems stem from a late change from a handheld device system to the paper-based system.

He says the system has worked well so far but that the agency is not out of the woods yet.

The GAO says the Census Bureau is otherwise well-positioned to finish the door-to-door count, which begins Saturday.

Finally, a source other than MyTwoCensus.com has noticed and reported on the multitude of IT failures at the Census Bureau. H/t to Edwin Mora of CNSNews.com for the following piece…but hopefully the mainstream media — not just Conservative media outlets like CNSNews — will start to address these problems:

Census Still Struggling With IT Problems That May Affect Count’s Accuracy
Friday, April 30, 2010
By Edwin Mora

(CNSNews.com) – The U.S. Census Bureau is still having problems with its computer system that handles the data for households that did not return a census form. However, the Census Bureau director said the system has successfully printed out the assignments for the enumerators who will conduct in-person interviews with households that did not mail in their forms.

“We continue to struggle with the software system called the paper-based operation control system, but we passed, just amazingly, a wonderful threshold last week where we printed out assignments for all these enumerators,” said Census Bureau Director Robert Groves. “It worked.”

The Census director made the comments at a press briefing on the Census participation rate, which took place at the National Press Club in Washington on Thursday.

Groves said the Bureau is not fond of its paper-based operation control system (PBOCS), which is used to manage the non-response follow-up (NRFU). The NRFU, set to begin May 1, is the Census’ largest operation and involves census workers personally interviewing millions of people nationwide who did not respond to the mailed Census questionnaire.

“Slightly more than 72 percent of U.S. households believed to be occupied mailed back their 2010 Census forms, the same rate that was achieved in 2000,” the U.S. Census Bureau announced on Apr. 28.

“Not that it is the most loved piece of software in the Census Bureau, but it’s working well enough to get the census down so far,” said Groves.

“We have assignments ready for 600,000 people who are ready to hit the streets on Saturday,” he added. “So we’re proceeding.”

According to a Mar. 25 Government Accountability Office report entitled, “Data Collection is Under Way, But Reliability of Key Information Technology Systems Remains a Risk,” the Census Bureau was experiencing problems with two IT systems, one of which is the paper-based operation control system that Groves mentioned during the press conference.

The GAO reported last February that “key IT systems — most notably an automated system used to manage field data collection known as the Paper-Based Operations Control System (PBOCS) and a personnel and payroll processing system called the Decennial Applicant Personnel and Payroll System (DAPPS) — were experiencing significant performance issues.”

On Thursday, Robert Goldenkoff, the director of strategic issues for the GAO and author of the March 25 GAO report on the IT problems affecting the Census, told CNSNews.com: “The [paper based] operational control system used to manage the field follow-up operation was still having stability issues last week; the Census Bureau continues to work on it.”

On Mar. 25, Judith Gordon, the principal assistant inspector general for Audit and Evaluation at the Department of Commerce, which runs the Census Bureau, testified about the IT problems affecting the Census before Congress, saying that the Census’ decennial count’s accuracy was “at risk” because of IT issues.

“IT problems place the efficiency and accuracy of Non-Response Follow-Up at risk and final decennial costs remain uncertain,” Gordon told lawmakers, and as CNSNews.com reported. Gordon had testified before a subcommittee on the Census of the House Oversight and Government Reform Committee.

In the same Mar. 25 GAO report, Goldenkoff revealed that “an estimated 50 million housing units out of a mail-out universe of about 120 million” would be non-respondents and would require an in-person follow-up to count. The operating budget for the NRFU is $2.7 billion.

Article 1, Section 2 of the U.S. Constitution calls for a decennial enumeration (census) of the American people to be used for allocating U.S. House seats among the states.

Here’s the transcript from Census Director Robert M. Groves’ press briefing on Wednesday. Unfortunately, my microphone (on Skype) cut out at the moment that I hoped to ask Dr. Groves a series of questions, which concerned why there are still failures of the paper-based operations control system and who (if anyone) is being held accountable for these errors. I forwarded these questions to Stephen Buckner at the Census Bureau’s Public Information Office, but have still not received a reply. Dr. Groves did briefly acknowledge errors in the PBOCS during his speech, but it is unlikely that any members of the mainstream media who don’t cover 2010 Census operations regularly would know about such a system and what those errors mean. Dr. Groves said, “We continue to struggle with the software system called the paper base operation control system, but we passed, just amazingly, a wonderful threshold last week where we printed out assignments for all these enumerators. It worked. We have assignments ready for 600,000 people who are ready to hit the streets on Saturday. So we’re proceeding. Not that it is the most loved piece of software in the Census Bureau, but it’s working well enough to get the census down so far.”

STEPHEN BUCKNER: Good afternoon. Welcome to the Census Bureau’s news conference on the mail participation rates. I’d like to welcome everyone here in the room, and also those joining us online and via telephone. If you take a few moments, we have some information in your press kits, and also available online, including all the charts that Dr. Groves will be going over today as he walks you through America’s accomplishment in the mail participation rate for the 2010 census as we start to go door to door later this weekend.

Following Dr. Groves’ remarks, we will have a brief Q&A session for the media. Please state your name and organization prior to your question. We’ll try to get to as many questions as we can during the news conference. And with that, I’ll turn it over to Dr. Groves. Thank you, Dr. Groves. (Applause)

DR. ROBERT GROVES: This is the first time I’ve heard applause. Well, welcome. I’m happy to be here and happy to see friendly faces in front of me. Today is a big day for us because we announce the end of the first half of the 2010 census. And we have good news, because we can thank the American public for really the first major achievement of the 2010 census, I think. So first, I need to say why are we honoring the American public in this way, and why is this a notable achievement? You need to know something about levels of participation in surveys in this country, and in fact in the western world over the past few decades to understand how wonderful what happened really is.

For the last 20 years, response rates, the level of participation of the public in sample surveys, in all sectors, the commercial sector, the government sector and the academic sector, have been falling. In fact, in the Census Bureau’s own survey, take the American Community Survey, this very large thing that we do continuously, we’ve lost 5 percentage points from the 2000 experience to now. So when I took this job, I really expected that any achievement close to the 2000 rate was beyond each. In fact, I urge you, urge the journalists here, to call up your favorite survey researcher and ask them one question: could you achieve the response rate today that you received ten years ago on the same survey? And see what they say.

So we had low expectations on getting close to where we were in 2000. And where were we on 2000? The combined short form and long form participation rate in 2000 was 69 percent at the time we cut off for the non-response follow-up. So if we believe those lower rates, we would have expected something lower than that. The short form only rate, out of the 2000, the portion of the households that got the short form, their participation rate was about 72 percent. We chose that as a stretch goal. We were preparing for response rates between 65 and 72 percent in our simulations. Well, what happened was the American public hit that stretch goal, and it was a wonderful display, we think, of civic participation. And I can tell you, the folks at the Census Bureau are dancing down the hallways.

There are a lot of neat things about that; 28 states met or exceeded their 2000 rate, that’s cool. Some of these are pretty large states; Florida, Georgia, Illinois, Michigan, New York, Pennsylvania and Texas. In addition, the District of Columbia and Puerto Rico exceeded their 2000 rate. There are 11 other states within 1 percentage point of their 200 rate. North Carolina and South Carolina are kind of the poster children of 2010. They exceeded their rate in 2000 by 8 percentage points, just off the charts, so as they say these days, need a special shout out, I think. And there are thousands of jurisdictions around the country that have exceeded their 2000 rate.

Let me just give you a sense of this. In your press kit, I think you have notes that seven of the ten most populous counties equaled or surpassed their 2000 rates. Eight of the ten most populous cities did that, likewise. So once again, this is, I think, a congratulations due to the American public for their act of civic participation. And we are grateful and happy to report that.

It might be good to ask the question of why, why was that rate achieved? And the first thing that I want to point out are some hard data we have. The first unambiguous result in the why, I think, has to do with the replacement form. So if you look at this chart, let me help you read this chart. And for those of you on the phone, this is a chart that’s labeled 2010 average mail back participation rates for census tracks by assignment of 2010 replacement form. So it’s a busy graph, I’ll help you read it.

The X axis here is a day, sort of the life cycle of the mail out/mail back process. The Y axis is the participation rate. So the first thing you’ll see is that all of the lines rise as you go from left to right as the mail back rate built over days. Now, there are a lot of different lines here. The solid lines are from the 2010 data. The dashed, or dotted, lines are from 2000. There are three groups of lines color coded. The red line are those tracks that got replacement forms in a blanket fashion. If you lived in that track, whether you returned the form or not, you got a replacement form. The green line are tracks that were targeted. If you yourself did not return a form, you got a replacement form. If your neighbor did, they did not. And then finally, the blue line are the tracks where no replacement form was given.

Look at the dashed lines first. We targeted for blanket treatment the toughest tracks. They had low response rates in 2000. That’s the lowest line, the red dashed line, is low. And the green line were the targeted tracks. They had sort of medium participation rates. But you can see, both of those are pretty far below the blue. So we targeted the replacement form to those areas that tended in 2000 to have low participation rates.

What happened? Well, this replacement form worked. It’s just unambiguous in the data. Let your eyes focus on the solid red line. Look how after the blanket replacement mailing was complete around the 3rd of April, it starts pumping up. It just gets higher than the dashed line and stays higher. And then notice how the green line, solid green line, starts getting higher after the target replacement form. In fact, I can tell you, the green line is still pumping out cases as w speak. This is a good thing for the 2010 census because now let your eye go to the dashed lines and just look at the difference in height between the blue dashed line and the red dashed line. That variability in participation rate is not good for a census. We’d like everybody to have the same participation rate. We squeezed together those lines to this replacement form.

So the first question, why did we get a good participation rate, is the replacement form worked, and it worked in a wonderful way to reduce the variability in response rates. There’s another reason, and we have some data on this but it’s not as hard as these data; we think the advertising and the partnership worked. I think in prior discussions we’ve had, we showed you how the awareness of the census rose dramatically as the advertising and partnership campaigns evolved. That seems unambiguous, that that was a good reason why.

And then there are other things that have to do with judgments. I just told you that the participation rate in 2000 for the combined short and long form was 69 percent. And for the short form only was 72 percent. We’re pretty sure the short form was a great idea based on that comparison. We don’t have an empirical comparison of the long and short form in this census, but one clear, I’d be willing to make this speculation, one clear answer to the question why a high rate of participation rate, or why a high rate of participation, is that the short form reduced the burden on the American public and they cooperated at higher rates.

We are going to do tons of other analysis over the coming weeks to figure out other answers to the question why did this seem to work. That’s our job in order to prepare for the 2010 census. But those are the early findings on the reasons for success.

I want to turn to another issue, and that is the patterns of response rate. You can see on the map that we showed earlier that there’s variability in the response. If you just look at the colors, the colors are related to different response rates. Red is really good, blue is bad, and you can see how the colors vary. We’ve been publishing this map every day on the web, a lot of you have been watching it. You start looking at a map like that and you begin to make up hypotheses about geography as the cause of the participation rate. We’re pretty sure that’s a misinterpretation of this. And I want to give you a sense of the patterns of response.

You know that for several decades, the Census Bureau has tracked differential under counts. When the whole census is finished, differential under counts by race, ethnicity and age, and the patterns are very clear over decade after decade, that younger people, that minority groups, tend to be disproportionately missed in the census. I can tell you day by day, we were watching those differences on participation rates in this first half of the census to see if we were seeing those patterns.

The classic patterns emerge in 2010, they’re there. But the biggest drivers in the participation rate are not race and ethnicity in the 2010 census, but a variety of other indicators. And I wanted to give you a sense of what those look like. A lot of them are indicators of the socioeconomic status of the area. So let me describe what this is. This breaks the census tracks. There are about 65,000 census tracks in the country, into four groups of equal size by the rate of vacancy in the area. What proportion of the houses were vacant based on American community survey data over the past three years? So the X axis here separates all of the census tracks into four groups, from low vacancy rate to high vacancy rate.

The Y axis here is the participation rate. The pattern is beautifully monatomic, as they say, in the statistics business. It goes down with each added quartile from 76 percent way down to 64 percent. Tracks that have high vacancy rates tended to have low participation rates. And it’s a pretty strong effect.

Let me show you the next one. This does a similar sort of thing. It breaks the tracks into four groups by their level of percentage of multi-unit structures. And on the X axis, you see what those rates are. Similar sort of finding; pretty strong effects from 77 percent participation rate down to 64 percent. Tracks that have a lot of multi-unit structures or mobile homes versus those that have single family structures had lower participation rates.

Let’s look at the next one. Renter occupied housing units, same sort of divisions on the X axis, participation rate on the Y axis, moving from 77 to 64. Census tracks, neighborhoods with a lot of rental units had lower participation rates. Let’s look at the next one. Education. We can identify the proportion of people on the tracks with less than a high school degree, same sort to pattern. Tracks that have a lot of people with low education tend to have lower participation rates. And is that the final one? One more, poverty. The American community survey allows us to estimate the proportion of households under poverty threshold; 77 to 64, same thing.

Now in your press kit are other graphs that look at ethnicity and race. You’ll look at those and see smaller effects across those groups than you see across the socioeconomic indicators. Was this present in prior censuses? I suspect it was. Did we have the right data to do this sort of analysis? Now, this is a wonderful benefit of the American community survey that we can track in almost real time other indicators. These are the strongest drivers to participation rate, not the race, ethnicity indicators. But they’re pretty powerful drivers, as you see.

So we can say that tracks that have high rental units, low education, low income, are disproportionately where our non-response follow-up workload is. Those are the neighborhoods that we will be visiting disproportionately in the coming weeks.

I want to say just a bit about our current operations and then open it up for questions. We are proceeding on all sorts of operations now. I’m happy to report that every operation we’ve done since addressing canvassing in summer of 2009 are on time and on budget. Those are going well. We continue to struggle with the software system called the paper base operation control system, but we passed, just amazingly, a wonderful threshold last week where we printed out assignments for all these enumerators. It worked. We have assignments ready for 600,000 people who are ready to hit the streets on Saturday. So we’re proceeding. Not that it is the most loved piece of software in the Census Bureau, but it’s working well enough to get the census down so far.

I want to remind everyone that we have another press conference on Monday. And this is really a press conference to look forward to the non-response follow-up process. It’s a gigantic effort. It’s a complicated effort. We want to make sure you have all the facts that we do about how it’s going. We’ll talk about that. It’s important to note that although we report this mail back rate as if we’ve reached a final stage, this is really the end of the first half and that the census is not over. We will have a disposition on 100 percent of the units that we have on our list, those who didn’t get forms will be visited. Those who got forms and didn’t mail them out will be visited. We will collect information on everyone before we’re through, and that’s the second half that we’re now beginning. So I want to stop at this point and entertain questions. (more…)

The Harvard Business Review‘s Daily Stat for Tuesday, April 6, 2010, highlighted a disruptive innovation in, of all things, census-taking. According to the publication:

It’s a national census of hundreds of millions of people across 8 million square kilometers, using a workforce of 230,000 and budget of $1.4 billion. The 2010 U.S. Census? No, it’s Brazil’s 2010 census. The current U.S. headcount, by contrast, requires 3.8 million workers and $14 billion. Census takers in Brazil use PDAs and laptops; those in the U.S. still rely mainly on paper. – Source: Instituto Brasileiro de Geografia e Estatístic

While the United States tends to be seen as the technology innovation capital of the world, it seems we’re falling down in “government services innovation.” Doesn’t the US Census fall under the administration’s social innovation program? What other program is intended to have an impact on every single citizen of the US, if not this one?

On any corner of a typical US city, one can buy GPS-enabled, off-the-shelf, 3G-powered mobile devices, with local storage for data collection and Web-enabled connections to the back office. In short, the perfect mobile device for census collections is almost a commodity.

But it seems the US Census Bureau made the classic Innovator’s Dilemma mistake of choosing the slow, safe (and expensive) player, while the consumer mobile world blew by.

The supplier in question is Harris Corp. (NYSE: HRS), which started work on the mobile census project in 2006. It turns out that four years is an eternity in the modern mobile world, and Harris simply couldn’t match the speed of the market with its own proprietary, custom-built devices.

In contrast to the US approach to having custom devices built, Brazil partnered with LG Electronics Inc. (London: LGLD; Korea: 6657.KS) , as noted in TechTicker: “Brazil will start taking its population census in the second half of this year and to ensure a smooth and efficient counting, the Instituto Brasileiro de Geografia e Estatistica (IBGE) has roped in LG to supply 150,000 GM750 phones.”

The GM750 is ruggedized and comes with its own application; but otherwise, the core phone itself is a device anyone with $100 or less could buy at the corner mobile store.

You may be thinking, who really cares? Perhaps we all should. According to CNN, Hermann Habermann, a former deputy director of the Census Bureau, thinks that without handheld data collection via mobile phone, the government is missing out on a chance to get information more quickly and cheaply than through the mail.

The technology would also help to better identify which Census tract a home is in, which determines an area’s representation in Congress and the distribution of more than $435 billion in federal funds every year. With GPS, according to CNN, the Census Bureau’s Daniel Weinberg, assistant director for the decennial census, anticipated placing residences within a 0.5 percent error rate into the correct tract. Without GPS, the Census Bureau places approximately 5 percent of residences in the wrong tract.

Unfortunately, with the fallback to paper, the error rate is likely to remain, as is the (inadvertent) mis-distribution of funds. While a 4.5 percent error differential may not seem like much, when you’re distributing billions of dollars each percentage point is quite significant in its effect on the local and state economies.

Lessons learned? Pay attention to the trends happening in parallel to your “normal” business and technology world — or you may find that your technological innovation has been disrupted by the fast movers you couldn’t be bothered to notice.

Whether in private, public, government, or other spaces, look around, and you’ll see disruption creeping up on you. Be aware of the technology environment and evolve! Or suffer the costs.

— Dan Keldsen is a Principal and Strategic Advisor at Information Architected.

Under the Freedom of Information Act, 5 U.S.C. § 552, I am requesting all e-mails sent to and from Brian Monaghan, Barbara Lopresti, and Marilla Matos from February 4, 2010 through February 12, 2010.

As you probably already know, I run MyTwoCensus.com, the non-partisan watchdog of the 2010 Census. My work has also appeared on MotherJones.com,governingpeople.com, and other publications. Since this is a non-commercial request and the release of these documents will serve the public interest (because analyzing these documents is likely to contribute significantly to public understanding of the operations or activities of the government), I am requesting that all fees be waived.

I am also requesting expedited processing of these documents under the clause on your web page that states I can do so if this information is “urgently needed to inform the public concerning some actual or alleged government activity.” With the 2010 Census just around the corner, and recent reports by the Associated Press and other organizations that language translations have been inadequate and sub-par, this request deserves your prompt attention.
If you deny all or any part of this request, please cite each specific exemption you think justifies your withholding of information. Notify me of appeal procedures available under the law. If you have any questions about handling this request, please feel free to contact me.

Sincerely,

Stephen Robert Morse

PS – I’m not sure why, but you never responded to my FOIA request for hotel information from February 25, 2010. Any updates on that situation?

Under the Freedom of Information Act, 5 U.S.C. § 552, I am requesting the records of all technical and information technology glitches, failures, and errors that involved the Census Bureau and its technology systems from January 1, 2006 through the present. This includes everything from e-mail systems going down to fingerprint scanners not working properly to the recent paper-based operational control system failure. Most important to me are items pertaining to the 2010 Census. I would appreciate if you started with the most recent failures and worked your way back. These should include every piece of technology that the Census Bureau uses at field offices as well as at headquarters in Suitland.

As you probably already know, I run MyTwoCensus.com, the non-partisan watchdog of the 2010 Census. My work has also appeared on MotherJones.com, governingpeople.com, and other publications. Since this is a non-commercial request and the release of these documents will serve the public interest (because analyzing these documents is likely to contribute significantly to public understanding of the operations or activities of the government), I am requesting that all fees be waived.

I am also requesting expedited processing of these documents under the clause on your web page that states I can do so if this information is “urgently needed to inform the public concerning some actual or alleged government activity.” With the 2010 Census just around the corner, and recent reports by the Associated Press and other organizations that language translations have been inadequate and sub-par, this request deserves your prompt attention.

If you deny all or any part of this request, please cite each specific exemption you think justifies your withholding of information. Notify me of appeal procedures available under the law.

The following piece comes to us from a Census Bureau official from the Mid-West region who has requested anonymity but has had her identity verified by MyTwoCensus.com. Her discussion of a major week-long IT systems failure has effected every Census Bureau office across the country and demonstrates how disgraceful operations are being handled (and subsequently covered up, as MyTwoCensus has requested information about this failure numerous times yet has not received any responses from the Census Bureau’s Public Information Office):

I have a problem in my local census office. The problem is 493 of those offices across the nation are having the same exact problem. You see, in November and December 2009 when I first started at the Census Bureau, one of my first assignments was assisting in a system load test known as the Decennial Application Load Test (or DALT for short). DALT simulated the conditions of every census office employee logging on at the same time across the nation and what would happen. This was my first taste of our federal tax dollars hard at work. As is typical of any large scale government program the whole system went kaput; users couldn’t log-on and applications crashed.

The Bureau’s contingency plan was to limit users and ask offices to spread out their staffing. We started to run an evening shift which meant paying night differential (an additional 10% pay) to our night employees. One application of concern was PBOCS the Paper Based Operational Control System that was unveiled in January 2010 to complete the rest of their field operations. The system was developed after the Census decided to nix the Harris Corporation as their contractor. The PBOCS system was used to check out assignments into the field, make assignments to enumerators, check work into the office after completion and check out for shipping. In February 2009 after the two big snowstorms that hit Washington DC the same thing happened. The Bureau’s contingency plan for PBOCS was to limit users to four per local census office and remove functionality they didn’t deem was mandatory.

In the last few weeks we are experiencing many more problems as we near non-response followup the most massive operation of the decennial census. Some of PBOCS problems include but are not limited to:
* individual census reports (ICRS) from group quarters enumeration that PBOCS claims are missing for shipment.
* daily progress reports that are outdated or showing the wrong numbers
* numerous system crashes where work is lost or has to be rescanned
* wrong or missing work being selected for reinterview

At the beginning when these problems started, the RCC pointed fingers at the local census offices accusing them of not using the system or processing the work properly. However when several offices reported the same issues the blame then shifted to the software.

To further complicate things is the divide that also occurs between the local census office employees and the the regional census center and headquarters staff. The headquarters and regional census center staff are mostly career employees. The local census office employees are temporary Schedule A hourly employees. Although both are referencing the same procedures in the same manuals the local census office employees are the ones who are doing the grunt work (the enlisted men per say). The RCC and headquarters staff (the commissioned officers) manage and oversee but do not realize the difficulties and nuances because they are not out there getting their hands dirty.

The LCO employees are finding that the PBOCS system actually will not update the report numbers or sometimes show the wrong numbers. The RCC and headquarters are lead to believe if the work doesn’t show in the system as completed then it is not completed. However when the reports don’t show the quotas are being met the career census employees usually get on the phone to threaten the temporary employees and even sometimes terminating their employment.

This weekend is the third weekend PBOCS has gone offline. On Wednesday April 14 it was announced PBOCS was to go down at 8pm (actually went down about 5pm) and will not be back up until Monday morning April 19th. We are somewhat relieved to hear through the grapevine that the RCC and headquarters will be more lenient and readjust their production goals for the entire nation. However I think that much of the intimidation and harassment will occur again so that the career census employees can cover their asses and recover any of the money they were probably paying in overtime for programmers to fix this crap piece of software. This is all the while they tell the temporary employees at the LCO that overtime will be strictly prohibited.

The career employees at RCC and headquarters hold the temporary LCO employees accountable for mistakes they [RCC and headquarters] made in the ten years they had to prepare since the last decennial. But who is held accountable at the highest levels when a multi-billion dollar piece of software doesn’t work and they constantly have to fix it? I hope the Inspector General and my congressmen are reading this because PBOCS is government waste at its finest. How ironic it is happening in the nearing days of the largest operation of the decennial census.

The Public Information office is still bouncing e-mails back to us….and this has gone on since Monday! Is there an IT department at the Census Bureau? Is anyone being held accountable for this? Some answers please!