This Funding Opportunity Announcement (FOA) seeks research
applications designed to develop and test methods for improving the fidelity,
and ultimately the effectiveness, of empirically supported behavioral
treatments (ESBTs) implemented by front-line therapists in community practice
settings.

Because achieving this aim presumes valid and reliable
assessment of fidelity, the FOA uses the sequential R21/R33 funding mechanisms
to encourage research designed to develop and test (a) methods for assessing
theory-derived ESBT fidelity components (R21 phase), and (b) interventions
that enhance and maintain the fidelity with which clinicians implement an
ESBT in community practice settings (R33 phase).

Key Dates

Posted Date

March 31, 2011

Open Date (Earliest Submission Date)

May 9, 2011

Letter of Intent Due Date

May 20, 2011

Application Due Date(s)

June 20, 2011, by 5:00 PM local time of applicant
organization.

AIDS Application Due Date(s)

Not Applicable

Scientific Merit Review

October 2011

Advisory Council Review

January 2012

Earliest Start Date(s)

April, 2012

Expiration Date

June 21, 2011

Due Dates for E.O. 12372

Not Applicable

Required Application Instructions

It is critical that applicants follow the instructions in
the SF
424 (R&R) Application Guide except where instructed to do otherwise (in
this FOA or in a Notice from the NIH Guide for Grants and Contracts).
Conformance to all requirements (both in the Application Guide and the FOA)
is required and strictly enforced. Applicants must read and follow all
application instructions in the Application Guide as well as any
program-specific instructions noted in Section
IV. When the program-specific instructions deviate from those in the
Application Guide, follow the program-specific instructions. Applications that
do not comply with these instructions may be delayed or not accepted for review.

This Funding Opportunity Announcement (FOA) seeks research
applications that aim to study methods for improving the fidelity, and
ultimately the effectiveness, of empirically supported behavioral treatments
(ESBTs) as implemented by front-line therapists in community settings.

Because achieving this aim presumes valid and reliable
assessment of fidelity, the FOA uses sequential R21/R33 funding mechanisms to
encourage research designed to develop and test (a) methods for assessing
theory-derived ESBT fidelity components (up to two years of R21 funding), and
(b) interventions that enhance and maintain the fidelity with which clinicians
implement an ESBT in community settings (up to three years of R33 funding).
There are many factors that are relevant to all ESBTs (e.g., therapist level of
training, quality of supervision, therapist motivation); however, the intent of
this FOA is that investigators should select a particular ESBT for study, and
focus on its particular therapeutic components.

Because valid assessment will ideally link theory-derived
fidelity components to clinical outcomes, a by-product of the FOA may be
increased knowledge about how ESBTs work (i.e., their mechanisms of action).
This would guide further research and subsequently facilitate paring these
treatments down to their most essential elements.

Background and Rationale

Although efficacious behavioral treatments for many mental
disorders exist, research suggests that people with mental disorders who seek treatment
in community settings rarely receive them. A major factor in this widely
acknowledged science-to-service gap is treatment fidelity (also known as
treatment integrity), which refers to the implementation of an intervention in
a manner consistent with principles outlined in an established manual.

Only a small fraction of clinicians who routinely provide an
ESBT such as cognitive-behavior therapy are able to do so with adequate
fidelity (e.g., one study found 5% based on direct observation, Santa Ana et
al., 2008; Journal of Substance Abuse Treatment, 35, 369-379). This may reflect
the insufficiency of commonly used ESBT training and dissemination methods such
as workshops and lectures, which by themselves effect little substantive change
in clinician behavior. Moreover, even documented acquisition of fidelity skills
under close supervision does not guarantee continued fidelity maintenance.

The fidelity of an ESBT is important for several reasons. First, the integrity of a psychosocial intervention is highly dependent on clinician behavior: what the clinician does and does not do (based on an ESBT manual) defines a multi-component independent variable – the treatment itself – encompassing domains such as adherence, competence, and differentiation from other treatments.

Second, if ESBT fidelity predicts outcome, fidelity failures may explain some of the fairly dramatic reduction in effect sizes associated with transporting ESBTs from university-based efficacy trials to effectiveness studies in community settings (e.g., Henggeler, 2004; Journal of Family Psychology, 18, 420-423). Unfortunately, while successful fidelity acquisition and maintenance are feasible in randomized efficacy trials, surprisingly little is known about how to extend effective methods of ESBT training and fidelity maintenance used in controlled studies to community practice (e.g., how much and what kind of direct observation and feedback are sufficient?).

Third, a sharpened focus on fidelity has the potential to
increase knowledge about specific targets for behavior change, which could make
ESBTs more efficient and attractive for broader adoption. For example, the
complex, multi-component nature of some ESBT treatment packages may itself pose
a barrier to successful community implementation. In the absence of specific
knowledge about how these treatments work (i.e., which components are active
and which are inert), a common practice is to train therapists in all elements
of the package, emphasizing active and inert components equally. Findings from
the assessment-development phase (R21 phase) could begin to illuminate which
fidelity components of an ESBT are most and least crucial to immediate (e.g.,
session) or longer-term (e.g., end-point, follow-up) patient outcomes. Although
discarding components on the basis of how well they relate to outcome would be
premature, the intervention-development (R33 phase) presents opportunities to
experiment in a more focused manner with methods for enhancing and maintaining
fidelity components that do correlate with outcomes, which could lead to
subsequent refinement of an ESBT.

Specific Areas of Research Interest

By “fidelity component,” this FOA refers to definable
elements or clusters of therapist behavior presumed to activate theory-relevant
mechanisms of change, which in turn lead to positive patient outcomes such as
symptom alleviation or improved quality of life. The optimal number and
molarity of components should depend not only on conceptual considerations
specific to a given ESBT, but also on psychometric (e.g., factor analytic) and
validity results, such as how the measure of a component relates to concurrent
expert judgments and/or to independent markers of session or intermediate
outcomes.

Responsive applications must establish, first, that a given
ESBT is in fact “empirically supported” for a given mental disorder according
to accepted standards of scientific evidence. Investigators should explain the
criteria on which they base this determination and cite supporting research. Eligible
ESBTs encompass a wide range of behavioral and psychosocial treatments for
mental disorders of children and adults.

A second prerequisite is that the ESBT (and the manual(s)
defining it) reflects a coherent theory of change, specifying the psychosocial
processes or change mechanisms through which the intervention presumably has
its beneficial effects. This theory of change guides what the therapist does to
implement the ESBT and ultimately grounds the definition of fidelity
components.

Third, when operationalizing the components of ESBT
fidelity, responsive applications will account for at least the following core
dimensions or domains: (a) how well the intervention components are delivered
(i.e., quality), in addition to their frequency or quantity as captured by
adherence check-lists; (b) proscribed as well as prescribed therapist behavior
(what the therapist should and should not do); (c) components that are unique
and essential to the ESBT, as well as those that are essential but not unique;
and (d) expected pacing and sequencing of intervention components, given that
some might precede or build upon others.

Applications should aim to demonstrate reliable and valid
measurement of treatment fidelity and its active components. Essential aspects
of reliability include consistency of the items or rating elements within each
fidelity component construct, as well as consistency between the judgments of
different raters (the latter indicating that different people who use the
measure would come to similar conclusions regarding therapist fidelity).
Demonstrating the validity of fidelity measurement can take several forms,
including concurrent validity (e.g., relating a measure to judgments of the
same behavior sample by unimpeachable experts, such as ESBT developers);
predictive validity (e.g., relating a measure to markers of a relevant change
mechanism in the same session or later sessions); and construct validity (e.g.,
demonstrating that a measure of one fidelity component is distinct from
measures of other components). Closely related to validity is the question of
how fidelity components, alone or in combination, correlate with intermediate
and ultimate outcome, with relevant criteria ranging from session outcome to
markers of theory-relevant change mechanisms and longer-term symptom
alleviation.

The following examples, while far from exhaustive,
illustrate possible fidelity components and their relationships to
theory-derived change mechanisms:

In Prolonged Exposure (PE) treatment for PTSD, the fidelity
component “skillful elicitation and normalization of patient reactions to the
trauma” should activate effective emotional processing by the patient (change
mechanism), leading ultimately to symptom reduction;

In Cognitive-Behavior Therapy (CBT) for depression, the fidelity
component “skillful Socratic questioning about cognitive reactivity to negative
moods” should activate effective reconsideration by the patient of his or her
negative cognitions, leading to symptom reduction;

In Motivational Enhancement Therapy (MET), the fidelity component
“skillful exploration of pros, cons, and ambivalence to change” should elicit
patient “change talk”, leading to behavior change.

Needless to say, measurement in this area presents
formidable challenges, not the least of which is the inherently interactional
nature of many fidelity components, where intervention quality reflects not
only what the therapist does but also how the client responds, and how the
therapist takes this response into account while staying faithful to the ESBT’s
theory of change. On the therapist side, fidelity measures will need to include
clearly defined behavioral anchors for each point on a given scale; and on the
patient/response side, investigators will need to consider whether to emphasize
intermediate outcomes, longer-term outcomes, or both, as this will influence
the possibility of detecting putative mediators or mechanisms of change.
Finally, given the ultimate goal of assessing fidelity in a variety of
community practice settings, selection and/or refinement of fidelity tools
should consider efficiency and respondent burden.

To summarize, responsive applications in the
assessment-development (R21) phase should assess the quality of ESBT delivery
in addition to the mere adherence to a treatment manual; go beyond an overall
fidelity scale to examine the fidelity of specific, theory-derived components
of an ESBT; validate measures of specific fidelity components, e.g., by
correlating these component measures with intermediate and/or longer-term
markers of mechanisms or outcomes. Ideally, scales that capture variability in
therapist fidelity also predict clinical outcomes, either alone or in
combination with other variables.

Although the R33 phase should entail new data collected from
interventions in community settings, the R21 phase might utilize archival data
from either efficacy or effectiveness trials, especially if these data sets
entail sufficient variability in fidelity to identify meaningful
fidelity-outcome correlations. An advantage of archival data for the R21 phase
of this FOA is having large enough sample sizes to examine measurement models.

Research questions relevant in the R21 phase include, but
are not limited to, the following:

What are the crucial treatment-specific elements (components)
that need to feature highly in fidelity assessment for a given ESBT, based on
associations with outcome?

What are the most reliable and valid methods of assessing these
components?

For observational measurement of fidelity, what samples of
therapist behavior are sufficient?

To what extent do “common factors” (e.g., therapist-patient
alliance) contribute to outcome variance for a given ESBT? How does
common-factor fidelity compare to specific-factor fidelity in this respect? How
do common- and specific-factor components interact?

Applications in the R33 phase should establish access to
community therapists with caseloads that would support the application’s aims.
Priority will be given to applications that include community-based consultants
advising on the feasibility and practicality of the research plans. Relevant
aspects of fidelity enhancement or maintenance include: (a) therapist training,
(b) ongoing supervision, and (c) ongoing monitoring and feedback. While an application
may address one of these aspects more than others, it is important to consider
how approaches to enhancing all three will contribute to knowledge. In
addition, investigators should use fidelity measures developed in the R21 phase
to evaluate the success of interventions designed to enhance and maintain
fidelity in the R33 phase.

The intervention-development (R33) phase is intended to
yield valuable knowledge about how to help community therapists maintain
high-fidelity intervention over time in ways that are relatively cost
effective. The goal of this work is to inform strategies for enhancing ESBT
fidelity in community settings (e.g., via peer- and supervisor-based assistance
models, ongoing collaborative learning networks). At the conclusion of the R33
phase, investigators should be able to show preliminary evidence regarding the
effectiveness of procedures for enhancing and maintaining ESBT fidelity in
community settings. Tangible byproducts of this effort might include
empirically-supported training and supervision manuals, and, if applicable,
other implementation products such as demonstration videos and technology aids.
In addition, investigators should be able to provide information on the
feasibility of developing an R01 application that would implement fidelity
enhancement strategies on a larger scale and assess the impact of such
interventions on provider behavior (i.e., increased fidelity) and patient
outcomes. Partnering with organizations that already train clinicians and
monitor intervention quality may be useful in these endeavors.

Research questions relevant in the R33 phase include, but
are not limited to, the following:

How can fidelity feedback systems work to optimize the delivery
of a particular ESBT? Who should provide the feedback? Is there value added by
live supervision? What methods and schedules of reinforcement (feedback) most
facilitate fidelity maintenance in community settings? Do fidelity ratings by
the therapists themselves add performance value to other training or monitoring
methods?

What technology could be applied to make fidelity monitoring more
immediate, efficient, and economical? For example, under what conditions can
observations (reports) by supervisors, therapists, and even clients serve as
reliable and valid proxies for fidelity ratings by independent expert
observers?

How much supervision, for whom and by whom, is sufficient to
maintain high levels of treatment fidelity;

Can theories from basic behavioral science (e.g., theories of
adult learning, social-interaction theories, theories of behavior change) be
used to inform interventions designed to enhance and maintain fidelity?

What is the relative importance of clinician conceptual (case
formulation) and behavioral (implementation) skills, and how are the two
related?

To what extent does the parallel between supervision and
intervention models enhance the fidelity with which the supervisee carries out
an ESBT (e.g., when a cognitive-therapy supervisor uses Socratic questioning,
or when a structural family-therapy supervisor models firm but flexible
hierarchical boundaries in the supervisory system)?

Finally, this FOA does not address other factors that may
play an important role in improving the effectiveness of ESBTs (e.g.,
interventions aiming to change organizational climate and attitudes towards
ESBT adoption). Investigators interested in these areas should refer to PAR-10-038 - Dissemination and Implementation Research in Health (R01).

Section II. Award Information

Funding Instrument

Grant

Application Types Allowed

New

The OER
Glossary and the SF 424 (R&R) Application Guide provide details on
these application types.

Funds Available and Anticipated Number of Awards

The NIMH intends to commit approximately $1,500,000 in FY
2012 to fund up to 7 grants in response to this FOA.

Award Budget

Direct Costs, across all years are limited to $900,000,
excluding consortium F&A costs, if applicable.

The R21 phase may not exceed two years or $300,000 in
direct costs over the two year period, with no more than $225,000 in direct
costs in any single year.

The R33 phase may not exceed three years with no more than
$225,000 in direct costs per year.

Award Project Period

The total project period for an application submitted in
response to this FOA may not exceed five years. The R21 phase may not exceed
two years and the R33 phase may not exceed three years.

NIH grants policies as
described in the NIH Grants
Policy Statement will apply to the
applications submitted and awards made in response to this FOA.

Section III. Eligibility
Information

1. Eligible Applicants

Eligible Organizations

Higher Education Institutions:

Public/State Controlled Institutions of Higher Education

Private Institutions of Higher Education

The following types of Higher Education Institutions
are always encouraged to apply for NIH support as Public or Private
Institutions of Higher Education:

Applicant organizations must complete the following registrations
as described in the SF 424 (R&R) Application Guide to be eligible to apply
for or receive an award. Applicants must have a valid Dun and Bradstreet
Universal Numbering System (DUNS) number in order to begin each of the following
registrations.

All Program Directors/Principal Investigators (PD/PIs) must
also work with their institutional officials to register with the eRA Commons
or ensure their existing eRA Commons account is affiliated with the eRA Commons
account of the applicant organization.

All registrations must be completed by the application due date. Applicant
organizations are strongly encouraged to start the registration process at
least four (4) weeks prior to the application due date.

Eligible Individuals (Program Director/Principal
Investigator)

Any individual(s) with the skills, knowledge, and resources
necessary to carry out the proposed research as the Program Director/Principal
Investigator (PD/PI) is invited to work with his/her organization to develop an
application for support. Individuals from underrepresented racial and ethnic
groups as well as individuals with disabilities are always encouraged to apply
for NIH support.

Applicant organizations may submit more than one application,
provided that each application is scientifically distinct.

NIH will not accept any application in response to this FOA
that is essentially the same as one currently pending initial peer review
unless the applicant withdraws the pending application. NIH will not accept any
application that is essentially the same as one already reviewed.

Section IV. Application and Submission Information

1. Requesting an
Application Package

Applicants must download the SF424 (R&R) application
package associated with this funding opportunity using the “Apply for Grant
Electronically” button in this FOA or following the directions provided at Grants.gov.

2. Content and Form of Application Submission

It is critical that applicants follow the instructions in
the SF424
(R&R) Application Guide, except where instructed in this funding
opportunity announcement to do otherwise. Conformance to the requirements in
the Application Guide is required and strictly enforced. Applications that are
out of compliance with these instructions may be delayed or not accepted for
review.

Letter of Intent

Although a letter of intent is not required, is not binding,
and does not enter into the review of a subsequent application, the information
that it contains allows IC staff to estimate the potential review workload and
plan the review.

By the date listed in Part 1. Overview
Information, prospective applicants are asked to submit a letter of intent
that includes the following information:

Descriptive title of proposed research
Name, address, and telephone number of the PD(s)/PI(s)
Names of other key personnel
Participating institutions
Number and title of this funding opportunity

The forms package associated with this FOA includes all
applicable components, mandatory and optional. Please note that some
components marked optional in the application package are required for
application submission. Follow all instructions in the SF424 (R&R)
Application Guide to ensure you complete all appropriate “optional” components.

Page Limitations

All page limitations described in the SF424 Application
Guide and the Table of
Page Limits must be followed.

PHS 398 Research Plan Component

All instructions in the SF424 (R&R) Application Guide
must be followed, with the following additional instructions:

Resource Sharing Plan

Individuals are required to comply with the
instructions for the Resource Sharing Plans (Data Sharing Plan, Sharing Model
Organisms, and Genome Wide Association Studies (GWAS)) as provided in the SF424
(R&R) Application Guide.

Appendix

Do not use the appendix to circumvent page limits. Follow
all instructions for the Appendix as described in the SF424 (R&R)
Application Guide.

Foreign Organizations

Foreign (non-US) organizations must follow policies
described in the NIH Grants
Policy Statement, and procedures for foreign organizations described
throughout the SF424 (R&R) Application Guide.

3. Submission Dates and
Times

Part I. Overview Information contains information about Key Dates. Applicants are encouraged to submit in
advance of the deadline to ensure they have time to make any application
corrections that might be necessary for successful submission.

Organizations must submit applications via Grants.gov, the online portal to find and apply for grants
across all Federal agencies. Applicants must then complete the submission
process by tracking the status of the application in the eRA Commons, NIH’s electronic system for grants
administration.

Applicants
are responsible for viewing their application in the eRA Commons to ensure accurate
and successful submission.

Information on the submission process and a definition of
on-time submission are provided in the SF424 (R&R) Application Guide.

For assistance with your electronic application or for more information on the electronic submission
process, visit Applying
Electronically.

Important
reminders:All PD/PIs must include their eRA Commons ID in the Credential
fieldof the Senior/Key Person Profile Component of the SF 424(R&R) Application
Package. Failure to register in the Commons and to include a valid PD/PI
Commons ID in the credential field will prevent the successful submission of an
electronic application to NIH.

The applicant organization must ensure that the DUNS number it provides on the
application is the same number used in the organization’s profile in the eRA
Commons and for the Central Contractor Registration (CCR). Additional
information may be found in the SF424 (R&R) Application Guide.

Upon receipt, applications will be evaluated for
completeness by the Center for Scientific Review and responsiveness by NIMH, NIH. Applications that are
incomplete and/or nonresponsive will not be reviewed.

In order to expedite review, applicants are requested to
notify the NIMH Referral Office by email at nimhreferral@mail.nih.gov when the
application has been submitted. Please include the FOA number and title, PD/PI
name, and title of the application.

The R21/R33 Phased Innovation Award application must be submitted
as a single application with one PHS398 Cover Page Supplement component and one
Research & Related Budget component.

Within the Research Strategy, applicants should describe both the
R21 phase and the R33 phase, including the milestones they aim to achieve in
order to proceed to the R33 phase.

Applicants should indicate in the budget justification whether
the budget year is for the R21 phase or the R33 phase.

The total length of the Research Strategy section cannot exceed
12 pages, including milestones, tables, graphs, figures, diagrams, and charts.

For the R21/R33 Phased Innovation Award application, the
initial review group will evaluate the specific goals for each phase and the
investigator-generated expected milestones that would justify expansion to the
R33 phase. A single impact/priority score will be assigned to each
discussed application. For the R21/R33 application the initial review
group may recommend that only the R21 phase be supported, based on concerns
related to the application’s specific goals and the feasibility of achieving the
milestones that justify the expansion to the R33 phase. The
recommendation to delete the R33 phase by the review panel or presentation of
inadequate milestones in the application may affect the merit rating of the
application.

The milestones proposed in the application should be well
described, quantifiable, and scientifically justified to allow program staff to
assess progress in the R21 phase and readiness for the R33 phase. The
milestones should provide data on the reliable and valid assessment of the
ESBT’s fidelity and its components, as described in Section 1 of Part 2, of
this FOA. The milestones will be considered in evaluating the approach proposed
by the investigator. A discussion of the milestones relative to the
progress of the R21 phase and the implications of successful completion of the
milestones for the R33 phase should be included. Applications lacking
this information, as determined by NIH staff, will not be reviewed. The
clarity and completeness of the R21/R33 application with regard to specific
goals and feasibility milestones are critical.

The R33 phase, investigating interventions to enhance and
maintain ESBT fidelity, is open to a wide range of experimental and
naturalistic research designs. However, applicants must make clear how they
will use assessment methods developed in the R21 phase to evaluate the success
of these interventions. At the conclusion of the R33 phase investigators
should be able to show preliminary evidence regarding the effectiveness of
procedures for enhancing and maintaining ESBT fidelity in community settings.
Tangible byproducts of this effort include empirically-supported training and
supervision manuals, and if applicable, other materials such as demonstration
videos, and technology aids. In addition, investigators should be able to
provide evidence of an active partnership with community clinics developed
during the grant period and information on the feasibility of developing an R01
application that would (a) implement fidelity enhancement strategies on a
larger scale (i.e., across a variety of clinical populations and community
settings), and (b) assess the impact of such interventions on provider behavior
(i.e., increased fidelity) and patient outcomes.

Prior to award, the Program Officer will contact the
applicant to discuss the proposed milestones and any changes suggested by the
review panel as indicated in the Summary Statement. The Program Officer
and the applicant will negotiate and agree on a final set of R21 milestones.
These will be the basis for judging the success of the R21 work. For
funded applications, the Project Director/Principal Investigator (PD/PI) will
submit a progress report to the Program Officer upon completion of the R21
milestones. Receipt of this progress report will trigger an
administrative program review that will determine whether or not the R33 should
be awarded. The release of R33 funds will be based on successful
completion of negotiated scientific milestones, on program priorities, and on
the availability of funds.

The R21 and R33 cannot be funded in the same fiscal year.

Post Submission Materials

Applicants are required to follow the instructions for
post-submission materials, as described in NOT-OD-10-115.

Section V. Application Review Information

1.
Criteria

Only the review criteria described below will be considered
in the review process. As part of the NIH mission,
all applications submitted to the NIH in support of biomedical and behavioral
research are evaluated for scientific and technical merit through the NIH peer
review system.

The R21/R33 phased innovation grant support
exploratory/developmental investigation of novel ideas and strategies that have
the potential for significant improvements in prevention or treatment of mental
disorders. The R21/R33 grant application need not have extensive background
material or preliminary information (although the availability of archival
source data would allow investigators to conduct a more efficient R21 phase,
especially if these data sets entail sufficient variability in fidelity to
afford meaningful fidelity-outcome correlations). Accordingly, reviewers will be
asked to focus their evaluation on the conceptual framework, the level of
innovation, and the potential to significantly impact the outcomes and
functioning of individuals with mental disorders, or those at high risk for
mental disorders. Appropriate justification for the proposed work can be
provided through literature citations, data from other sources, or, when
available, from investigator-generated data. Preliminary data are not required
for R21/R33 applications; however, they may be included if available.

Overall Impact

Reviewers will provide an overall impact/priority score to
reflect their assessment of the likelihood for the project to exert a
sustained, powerful influence on the research field(s) involved, in
consideration of the following review criteria and additional review criteria
(as applicable for the project proposed).

Scored Review Criteria

Reviewers will consider each of the review criteria below in
the determination of scientific merit, and give a separate score for each. An
application does not need to be strong in all categories to be judged likely to
have major scientific impact. For example, a project that by its nature is not
innovative may be essential to advance a field.

Significance

Does the project address an important problem or a
critical barrier to progress in the field? If the aims of the project are
achieved, how will scientific knowledge, technical capability, and/or clinical
practice be improved? How will successful completion of the aims change the
concepts, methods, technologies, treatments, services, or preventative
interventions that drive this field? Is the project likely to advance knowledge
of how to best assess theory-derived fidelity components of a specific EBST,
and are the methods for enhancing and maintaining fidelity likely to be
effective and transportable to community practice settings?

Investigator(s)

Are the PD/PIs, collaborators, and other researchers
well suited to the project? If Early Stage Investigators or New Investigators,
or in the early stages of independent careers, do they have appropriate
experience and training? If established, have they demonstrated an ongoing
record of accomplishments that have advanced their field(s)? If the project is
collaborative or multi-PD/PI, do the investigators have complementary and
integrated expertise; are their leadership approach, governance and
organizational structure appropriate for the project? Do the investigators have documented expertise relevant to treatment fidelity and the specific ESBT
under investigation, and more generally to developing behavioral measures and
evaluating behavioral interventions? Does the investigative team include
community-based consultant(s) to advise on the feasibility and practicality of the
project?

Innovation

Does the application challenge and seek to shift
current research or clinical practice paradigms by utilizing novel theoretical
concepts, approaches or methodologies, instrumentation, or interventions? Are
the concepts, approaches or methodologies, instrumentation, or interventions
novel to one field of research or novel in a broad sense? Is a refinement,
improvement, or new application of theoretical concepts, approaches or
methodologies, instrumentation, or interventions proposed? Does the application entail innovative approaches to assessing treatment fidelity (R21 phase) and to
enhancing and maintaining fidelity in community settings (R33 phase)?

Approach

Are the overall strategy, methodology, and analyses
well-reasoned and appropriate to accomplish the specific aims of the project?
Are potential problems, alternative strategies, and benchmarks for success
presented? If the project is in the early stages of development, will the
strategy establish feasibility and will particularly risky aspects be managed?

If the project involves clinical research, are the plans for 1) protection of
human subjects from research risks, and 2) inclusion of minorities and members
of both sexes/genders, as well as the inclusion of children, justified in terms
of the scientific goals and research strategy proposed? Specifically, in the assessment-development (R21) phase, does the application (a) go beyond an
overall fidelity scale to examine specific, theory-derived fidelity components?
(b) propose to assess the quality ESBT delivery in addition to mere adherence?
(c) attempt to validate measures of specific fidelity components? and (d) plan
to correlate these components with intermediate and/or longer-term markers of
mechanisms or outcomes? Are the investigators likely to establish a reliable,
valid, and practical approach to ESBT fidelity measurement applicable in
community settings?

In the intervention (R33) phase, is the application
likely to (a) yield valuable preliminary knowledge about ESBT training and
methods for helping therapists maintain high-fidelity intervention over time in
community settings? (b) produce treatment-specific implementation products such
as ESBT training and supervision manuals, and, if applicable, demonstration
videos, and technology aids? and (c) produce compelling feasibility data for
developing an R01 application aimed at enhancing ESBT fidelity and thus
improving provider behavior (i.e., improved fidelity) and patient outcomes
across community settings and patient populations?

Environment

Will the scientific environment in which the work
will be done contribute to the probability of success? Are the institutional
support, equipment and other physical resources available to the investigators
adequate for the project proposed? Will the project benefit from unique
features of the scientific environment, subject populations, or collaborative
arrangements? Specifically, can the investigators obtain data appropriate to
the aims listed in the R21 phase, and to community therapists and caseloads as required
for the R33 phase?

Additional Review Criteria

As applicable for the project proposed, reviewers will
evaluate the following additional items while determining scientific and
technical merit, and in providing an overall impact/priority score, but will
not give separate scores for these items.

Milestones

Given the critical nature of the milestones for the
transition from the R21 to the R33 phase, are the proposed milestones
well-defined with quantifiable measures that are appropriate for assessing the
success of the R21 phase of the application? Do the investigators make clear
how they will use the assessment methods developed in the R21 phase to evaluate
the success of the fidelity-enhancing interventions they plan to develop in the
R33 phase? Is it clear how the R33 phase will develop once the R21 milestones
are achieved?

Protections for Human Subjects

For research that involves human subjects but does
not involve one of the six categories of research that are exempt under 45 CFR
Part 46, the committee will evaluate the justification for involvement of human
subjects and the proposed protections from research risk relating to their
participation according to the following five review criteria: 1) risk to
subjects, 2) adequacy of protection against risks, 3) potential benefits to the
subjects and others, 4) importance of the knowledge to be gained, and 5) data
and safety monitoring for clinical trials.

For research that involves human subjects and meets the criteria for one or
more of the six categories of research that are exempt under 45 CFR Part 46,
the committee will evaluate: 1) the justification for the exemption, 2) human
subjects involvement and characteristics, and 3) sources of materials. For
additional information on review of the Human Subjects section, please refer to
the Human
Subjects Protection and Inclusion Guidelines.

Inclusion of Women, Minorities, and
Children

When the proposed project involves clinical research,
the committee will evaluate the proposed plans for inclusion of minorities and
members of both genders, as well as the inclusion of children. For additional
information on review of the Inclusion section, please refer to the Human
Subjects Protection and Inclusion Guidelines.

Vertebrate Animals

The committee will evaluate the involvement of live
vertebrate animals as part of the scientific assessment according to the
following five points: 1) proposed use of the animals, and species, strains,
ages, sex, and numbers to be used; 2) justifications for the use of animals and
for the appropriateness of the species and numbers proposed; 3) adequacy of
veterinary care; 4) procedures for limiting discomfort, distress, pain and
injury to that which is unavoidable in the conduct of scientifically sound
research including the use of analgesic, anesthetic, and tranquilizing drugs
and/or comfortable restraining devices; and 5) methods of euthanasia and reason
for selection if not consistent with the AVMA Guidelines on Euthanasia. For
additional information on review of the Vertebrate Animals section, please
refer to the Worksheet
for Review of the Vertebrate Animal Section.

Biohazards

Reviewers will assess whether materials or procedures
proposed are potentially hazardous to research personnel and/or the
environment, and if needed, determine whether adequate protection is proposed.

Resubmissions

Not Applicable.

Renewals

Not Applicable.

Revisions

Not Applicable.

Additional Review Considerations

As applicable for the project proposed, reviewers will
consider each of the following items, but will not give scores for these items,
and should not consider them in providing an overall impact/priority score.

Applications from Foreign
Organizations

Reviewers will assess whether the project presents
special opportunities for furthering research programs through the use of
unusual talent, resources, populations, or environmental conditions that exist
in other countries and either are not readily available in the United States or
augment existing U.S. resources.

Select Agent Research

Reviewers will assess the information provided in
this section of the application, including 1) the Select Agent(s) to be used in
the proposed research, 2) the registration status of all entities where Select
Agent(s) will be used, 3) the procedures that will be used to monitor
possession use and transfer of Select Agent(s), and 4) plans for appropriate
biosafety, biocontainment, and security of the Select Agent(s).

Reviewers will consider whether the budget and the
requested period of support are fully justified and reasonable in relation to
the proposed research.

2. Review and Selection
Process

Applications will be evaluated for scientific and technical
merit by (an) appropriate Scientific Review Group(s) convened by the NIMH , in accordance with NIH peer
review policy and procedures, using the stated review
criteria. Review assignments will be shown in the eRA Commons.

As part of the scientific peer review, all applications:

May undergo a selection process in which only those applications
deemed to have the highest scientific and technical merit (generally the top
half of applications under review), will be discussed and assigned an overall impact/priority
score.

Will receive a written critique.

Applications will be assigned to the appropriate NIH Institute or Center. Applications will compete for available funds
with all other recommended applications submitted in response to this FOA. Following initial peer review, recommended applications will receive a second level of
review by the National Advisory Mental Health Council . The following will be
considered in making funding decisions:

Scientific and technical merit of the proposed project as
determined by scientific peer review.

Availability of funds.

Relevance of the proposed project to program priorities.

Applications in the R33 phase should establish access to
community therapists with caseloads that would support the application’s aims. Priority
will be given to applications that include community-based consultants advising
on the feasibility and practicality of the research plans.

3. Anticipated Announcement
and Award Dates

After the peer review of the application is completed, the
PD/PI will be able to access his or her Summary Statement (written critique)
via the eRA
Commons.

If the application is under consideration for funding, NIH
will request "just-in-time" information from the applicant as
described in the NIH Grants
Policy Statement.

A formal notification in the form of a Notice of Award (NoA) will be provided
to the applicant organization for successful applications. The NoA signed by
the grants management officer is the authorizing document and will be sent via
email to the grantee business official.

Awardees must comply with any funding restrictions described in Section IV.5. Funding Restrictions. Selection
of an application for award is not an authorization to begin performance. Any
costs incurred before receipt of the NoA are at the recipient's risk. These
costs may be reimbursed only to the extent considered allowable pre-award costs.

The Federal Funding Accountability and Transparency Act of
2006 (Transparency Act), includes a requirement for awardees of Federal grants
to report information about first-tier subawards and executive compensation
under Federal assistance awards issued in FY2011 or later. All awardees of
applicable NIH grants and cooperative agreements are required to report to
the Federal Subaward Reporting System (FSRS) available at www.fsrs.gov on all subawards over $25,000. See the NIH Grants
Policy Statement for additional information on this reporting requirement.

Section VII. Agency Contacts

We encourage inquiries concerning this funding opportunity
and welcome the opportunity to answer questions from potential applicants.

Awards are made under the authorization of Sections 301 and 405 of the Public Health Service Act as amended (42 USC 241 and 284) and
under Federal Regu78lations 42 CFR Part 52 and 45 CFR Parts 74 and 92.