There is ongoing interest in developing more eMeasures for use in federal programs and obtaining NQF endorsement. However, developers have indicated that it can be challenging to test eMeasures to the extent necessary to meet NQF endorsement criteria – at least until they have been more widely implemented. NQF endorsement may provide the impetus to implement measures; but a submitted measure with very limited testing that does not meet NQF endorsement criteria could be prematurely abandoned.

The Trial Use Program is specifically for eMeasures that are ready for implementation, but cannot yet be adequately tested to meet NQF endorsement criteria. The program seeks to identify and support eMeasures that address important areas of performance measurement and quality improvement. As program participants, all eMeasures must be assessed as technically acceptable for implementation and have a plan to conduct more robust reliability and validity testing that takes advantage of clinical data in EHRs.

Candidacy for the Trial Use program is similar to eMeasure endorsement candidacy and is reviewed by the standing committee based on the standard NQF evaluation criteria. Approved for Trial Use carries no endorsement label, but may be considered a pathway for measures to prepare for endorsement. Approved for Trial Use eMeasures are indexed in QPS and are indicated as part of the program. See Table 12 (PDF) for comparison of activities related to endorsement and approved for Trial Use.

Trial Use Program Candidate Screening

Candidates for the eMeasure Trial Use Program are initially screened by the NQF eMeasure and Maintenance Measure Review team prior to standing committee review and consideration. Initial screening includes:

Must meet all criteria under Importance to Measure and Report (clinical evidence and opportunity for improvement/performance gap)

Completion of the Measure Testing Form for Trial Use (DOCX) and use results from either a simulated data set from the BONNIE Tool, or a test data set from another source to demonstrate that the QDM and HQMF are used appropriately and that the measure logic performs as expected.

Plan for future use and discussion of how the measures will be useful for accountability and improvement.

Identification of related and competing measures with a plan for harmonization or justification for developing a competing measure.

Once eMeasure candidates have completed a successful screening, they may be submitted via online Measure Submission Form for committee consideration in the Trial Use Program.

Maintenance of eMeasure Trial Use Program

Approved for Trial Use designation expires three years after the initial committee approval date (if the eMeasure has not been submitted for endorsement prior to that date).

There is no expectation that every trial use measure will be submitted for endorsement consideration – some may fail during testing.

If submitted for endorsement three or more years after the Approval for Trial Use date, the measure must be submitted and evaluated on all criteria similar to any measure being submitted for initial endorsement consideration.

eMeasures approved for Trial Use may be submitted for endorsement at any time prior to the three-year expiration. There are two options for endorsement evaluation:

Option 1: Submit for the evaluation of only the Scientific Acceptability of Measure Properties criterion, including the final eMeasure specification and all testing. If endorsed, endorsement date will assume the Approved for Trial Use date. Endorsement maintenance will be scheduled on the regular three-year cycle and the measure will be subject to evaluation on all criteria.

Option 2: Submit for evaluation on all criteria. If endorsed, an endorsement date will be identified and endorsement maintenance will be scheduled from the endorsement date, at which time it will be submitted for endorsement maintenance and subject to evaluation on all criteria.

Related Information

Contact

Contact

There is ongoing interest in developing more eMeasures for use in federal programs and obtaining NQF endorsement. However, developers have indicated that it can be challenging to test eMeasures to the extent necessary to meet NQF endorsement criteria – at least until they have been more widely implemented. NQF endorsement may provide the impetus to implement measures; but a submitted measure with very limited testing that does not meet NQF endorsement criteria could be prematurely abandoned.

The Trial Use Program is specifically for eMeasures that are ready for implementation, but cannot yet be adequately tested to meet NQF endorsement criteria. The program seeks to identify and support eMeasures that address important areas of performance measurement and quality improvement. As program participants, all eMeasures must be assessed as technically acceptable for implementation and have a plan to conduct more robust reliability and validity testing that takes advantage of clinical data in EHRs.

Candidacy for the Trial Use program is similar to eMeasure endorsement candidacy and is reviewed by the standing committee based on the standard NQF evaluation criteria. Approved for Trial Use carries no endorsement label, but may be considered a pathway for measures to prepare for endorsement. Approved for Trial Use eMeasures are indexed in QPS and are indicated as part of the program. See Table 12 (PDF) for comparison of activities related to endorsement and approved for Trial Use.

Trial Use Program Candidate Screening

Candidates for the eMeasure Trial Use Program are initially screened by the NQF eMeasure and Maintenance Measure Review team prior to standing committee review and consideration. Initial screening includes:

Must meet all criteria under Importance to Measure and Report (clinical evidence and opportunity for improvement/performance gap)

Completion of the Measure Testing Form for Trial Use (DOCX) and use results from either a simulated data set from the BONNIE Tool, or a test data set from another source to demonstrate that the QDM and HQMF are used appropriately and that the measure logic performs as expected.

Plan for future use and discussion of how the measures will be useful for accountability and improvement.

Identification of related and competing measures with a plan for harmonization or justification for developing a competing measure.

Once eMeasure candidates have completed a successful screening, they may be submitted via online Measure Submission Form for committee consideration in the Trial Use Program.

Maintenance of eMeasure Trial Use Program

Approved for Trial Use designation expires three years after the initial committee approval date (if the eMeasure has not been submitted for endorsement prior to that date).

There is no expectation that every trial use measure will be submitted for endorsement consideration – some may fail during testing.

If submitted for endorsement three or more years after the Approval for Trial Use date, the measure must be submitted and evaluated on all criteria similar to any measure being submitted for initial endorsement consideration.

eMeasures approved for Trial Use may be submitted for endorsement at any time prior to the three-year expiration. There are two options for endorsement evaluation:

Option 1: Submit for the evaluation of only the Scientific Acceptability of Measure Properties criterion, including the final eMeasure specification and all testing. If endorsed, endorsement date will assume the Approved for Trial Use date. Endorsement maintenance will be scheduled on the regular three-year cycle and the measure will be subject to evaluation on all criteria.

Option 2: Submit for evaluation on all criteria. If endorsed, an endorsement date will be identified and endorsement maintenance will be scheduled from the endorsement date, at which time it will be submitted for endorsement maintenance and subject to evaluation on all criteria.