Report 2012-122 Recommendation 6 Responses

Report 2012-122: Mental Health Services Act: The State's Oversight Has Provided Little Assurance of the Act's Effectiveness, and Some Counties Can Improve Measurement of Their Program Performance (Release Date: August 2013)

Recommendation #6 To: Health Care Services, Department of

To ensure that Health Care Services and other state entities can evaluate MHSA programs and assist the Accountability Commission in its efforts, Health Care Services should collect complete and relevant MHSA data from the counties.

Annual Follow-Up Agency Response From August 2017

DHCS contracted with a vendor to perform data clean up on the data included in the DHCS Client Services Information System and MHSA Data Collection and Reporting System. The purpose of the project is to provide DHCS, the MHSOAC, and the counties with accurate and complete data that may be used to track performance outcomes for Community Services and Supports, Full Service Partnership clients. This project is scheduled for completion over the next 12-18 months. With DHCS maintaining data quality thereafter. Counties have already received their comprehensive data error reports, and technical assistance calls to support the interpretation and use of these reports are currently underway. The data clean up project is to be completed by Fall 2018. Upon completion, this recommendation will be fully implemented.

Annual Follow-Up Agency Response From October 2015

All of the data that DHCS collects is relevant data. DHCS is working to improve the timeliness of the data that is reported and the completeness of the data that is reported. DHCS has implemented processes that have improved the timeliness of data reporting in both the Client and Services Information (CSI) data system and the Data Collection and Reporting (DCR) data system. Counties are now reporting data timely.

- Client and Services Information (CSI) Data: According to the DHCS September 2015 county submission status report, 50 counties are current in submitting. County submissions average 86% current with data submission.

- Data Collection Reporting (DCR): According to the DHCS September 2015 county submission status report, 57 counties are current in submitting DCR full service partnership data. County submissions average 97%.

DHCS is starting to focus on improving the completeness of the data submitted. Once DHCS is completed the work necessary to improve the completeness of data, this recommendation will be fully implemented.

DHCS is currently working on relevancy in its data modernization project. Once the data modernization project is done, this recommendation will be fully implemented.

1-Year Agency Response

According to the DHCS June 2014 county submission status report, 46 counties are current in submitting CSI data. The number of counties submitting late went from 9 counties in January 2014 to 12 counties in June 2014. Late submissions went from an average of 7 months delayed in January 2014, to an average of 3 months delayed by June 2014.

Data Collection Reporting (DCR):

According to the DHCS June 2014 county submission status report, 56 counties are current in submitting DCR full service partnership data. The number of counties submitting late decreased from 8 counties in January 2014 to 3 counties in June 2014. The number of counties that never submitted data decreased from 5 counties in January 2014 to 2 counties in June 2014. Late submissions went from 3 counties being an average of 8 months late in January 2014 to 1 county being 21 months late in June 2014.

6-Month Agency Response

For all counties late in reporting data, DHCS maintains monthly contact with county staff to obtain an update on their reporting status and timeline for becoming current with data reporting, and to offer any technical assistance. Submitted reports are attached to this document.

Client and Services Information (CSI): According to the DHCS January 2014 county submission status report, 49 counties are current in submitting CSI data. The number of counties submitting late decreased from 14 counties in September 2013 to 9 counties in January 2014. Late submissions went from an average of 10 months delayed in September 2013 to an average of 7 months delayed by January 2014.

Data Collection Reporting (DCR): According to the DHCS January 2014 county submission status report, 51 counties are current in submitting DCR full service partnership data. The number of counties submitting late decreased from 9 counties in September 2013 to 8 counties in January 2014. Late submissions went from an average of 15 months late in September 2013 to an average to 8 months late by January 2014.

60-Day Agency Response

DHCS continues to review county data submission for completeness and accuracy. According to the DHCS September 2013, county submission status reports, 44 counties are current in submitting Client and Services Information (CSI) data and 14 are not current in reporting. Regarding the Data Collection and Reporting (DCR) full service partnership data, 49 counties are current in submitting DCR data and 9 are not current.

To assist all counties in becoming fully compliant with their required reporting, DHCS established and operates a helpdesk for county data reporting questions; provides technical assistance to county and county vendor information technology staff; posts monthly county submission status reports on the DHCS website; and coordinates with CMHDA and the Accountability Commission for any needed system improvements and updates to data reporting requirements. In September 2013, DHCS began contacting the County Mental Health Director in each county without current data to request the county's reporting status and timeline for becoming current with data reporting.

DHCS will begin an in-depth assessment of CSI and DCR data completeness and relevancy for outcomes reporting purposes in the winter of 2013.