The assessment of fidelity in a motor

The PROMPT Institute, Santa Fe, NM, USA, 2Department of Speech-Language Pathology, University of Toronto,
Canada, 3Department of Paediatric Rehabilitation, Princess Margaret Hospital for Children, WA, Australia,
4
Faculty of Medicine, School of Paediatrics and Child Health, Dentistry and Health Sciences, The University of
Western Australia, Perth, WA, Australia
Objective: To demonstrate the application of the constructs of treatment fidelity for research and clinical
practice for motor speech disorders, using the Prompts for Restructuring Oral Muscular Phonetic Targets
(PROMPT) Fidelity Measure (PFM). Treatment fidelity refers to a set of procedures used to monitor and
improve the validity and reliability of behavioral intervention. While the concept of treatment fidelity has
been emphasized in medical and allied health sciences, documentation of procedures for the systematic
evaluation of treatment fidelity in Speech-Language Pathology is sparse.
Methods: The development and iterative process to improve the PFM, is discussed. Further, the PFM is
evaluated against recommended measurement strategies documented in the literature. This includes
evaluating the appropriateness of goals and objectives; and the training of speech–language
pathologists, using direct and indirect procedures. Three expert raters scored the PFM to examine interrater reliability.
Results: Three raters, blinded to each other’s scores, completed fidelity ratings on three separate occasions.
Inter-rater reliability, using Krippendorff’s Alpha, was >80% for the PFM on the final scoring occasion. This
indicates strong inter-rater reliability.
Conclusion: The development of fidelity measures for the training of service providers and treatment delivery
is important in specialized treatment approaches where certain ‘active ingredients’ (e.g. specific treatment
targets and therapeutic techniques) must be present in order for treatment to be effective. The PFM
reflects evidence-based practice by integrating treatment delivery and clinical skill as a single quantifiable
metric. PFM enables researchers and clinicians to objectively measure treatment outcomes within the
PROMPT approach.
Keywords: PROMPT, Fidelity, Treatment, Motor speech disorders, Integrity, Evidenced-based practice, Intervention, Inter-rater reliability

Introduction
The implementation of evidence-based practice (EBP)
principles in the identification and selection of the
most appropriate treatment for a given client is well
established in the field of speech–language pathology
(Dollaghan, 2008) and requires a speech–language
pathologist (SLP) to critically evaluate the best available scientific evidence. Traditionally, this evaluation
has been formulated on the basis of methodological
rigor and strength of findings associated with the
dependent variable (outcome measures). These
strengths may include reliability of the outcome
measures, control of subjective bias, and hierarchy of
evidence (Dollaghan, 2008). While these elements
are essential to understanding threats to internal
Correspondence to: Aravind Kumar Namasivayam, Department of
Speech-Language Pathology, 160-500 University Avenue, Rehab
Sciences Bldg., Toronto, ON M5G1V7, Canada.
Email: a.namasivayam@utoronto.ca

30

validity, they do not address the quality of the independent variable, that is, the fidelity of the intervention
reportedly administered (Schlosser, 2002; Kaderavek
and Justice, 2010).

What is treatment fidelity?
Treatment fidelity refers to the methodological strategies utilized to monitor the reliability and validity
of therapy interventions. It is inextricably linked to
the framework of EBP and defined as ‘… the degree
to which administration of a treatment corresponds
to the prototype treatment, also referred to as the
“gold standard” implementation’ (Kaderavek and
Justice, 2010, p. 369). The underlying assumption is
that the best possible outcomes for a client can only
be achieved when an empirically supported treatment
is delivered in a systematic manner with high fidelity
(Kaderavek and Justice, 2010).

g. and social-emotional
(Hayden et al. Furthermore. cognitive-linguistic. verifiable and
theoretically grounded treatment protocols (e. 2010.
Schlosser.
One treatment approach that has demonstrated the
implementation of these key strategies for establishing
and measuring treatment fidelity is Prompts for
Restructuring Oral Muscular Phonetic Targets (i. Treatment quality is the skillfulness with
which the clinician delivers a given treatment.
1
31
. A summary of
the fidelity strategies reported in each of these
studies. 2010). systematic intervention
delivered with high fidelity has been shown to result
in better and more consistent outcomes (e.
18
NO. 2013.
reporting treatment fidelity is essential to the interpretation of treatment outcomes. the
gold standard in assessing the delivery of an intervention protocol is the administration of an evaluation
protocol/checklist by a trained and reliable coder.
2006. 2002).g. 2006). 2011). the PROMPT Fidelity Measure). The approach was developed
by Chumpelik-Hayden (1984) and first reported as a
single case study. Günther
and Hautvast.
the PROMPT approach)..
examining the effectiveness of PROMPT intervention.g.
Borrelli.’ Given the above. procedural
integrity.g. 2010).. consistency in training and mentoring. Otterloo
et al. et al.
Treatment fidelity within the PROMPT approach
The PROMPT approach has implemented three strategies recommended in the literature for the establishment and measurement of intervention fidelity. Of 342 articles evaluated. 2002) and blinded to the intervention.
Procedural fidelity..
2010). 2005. they all report
Speech. as one is unable to determine whether the lack of treatment effect is due to
deficits inherent in the treatment program or excessive
alteration from the ‘gold standard. 2002. Whyte and Hart. Both these aspects of treatment fidelity require assessment as even an excellent
intervention with strong empirical support for its efficacy and effectiveness may not yield expected outcomes if the intervention is not delivered with high
fidelity (Kaderavek and Justice.
according to a priori criteria (Bellg et al. This table illustrates that all three studies met at least three of the five
fidelity strategies recommended by Borrelli et al.
To date.
The importance of treatment fidelity
The goal of treatment research is to assess the causal
relationship between the intervention administered
and the outcome measures. Schlosser. treatment quality.g. 2005)
that strives to achieve normalized speech movement
patterns via hierarchical goal selection and the use of
coordinated multi-sensory inputs during task-related
production of contextual and age-appropriate
lexicon.. These include explicit.. 2002. PROMPT is a motorspeech treatment approach framed within the principles of Dynamic Systems Theory (Thelen. refers to the
clinician’s adherence to the prescribed intervention
procedures and techniques.
have reported treatment fidelity consistent with guidelines recommended in the literature (Schlosser. Bellg
et al.
Borrelli et al.
Borrelli et al... The quantity and quality
of treatment administered has been shown to strongly
correlate with effect sizes and influences the validity
and power of intervention studies (e. Ward et al. three peer-reviewed studies (Rogers.
The establishment of treatment fidelity
The literature indicates key mutually exclusive components essential to the establishment of treatment
fidelity (Schlosser.
the clinician’s ability to adjust or customize certain
active ingredients of an intervention according to the
needs of a client (Mihalic.e. procedural fidelity). 2010). 2005. Borrelli. The failure to
report treatment fidelity negatively impacts upon our
ability to evaluate the efficacy of an intervention
under investigation. physicalsensory. 2004. 2011). For example. Dale and Hayden. PROMPT not only addresses speech production. that is. is provided in Table 1. 2004. based on the five-part treatment fidelity framework. on the other hand. 2004). treatment integrity.
These include manualization of the intervention. this
paper will differentiate between treatment quality
and procedural fidelity (Kaderavek and Justice. 852). a clinician may increase the amount of speech motor practice along with auditory/visual cues to support the
acquisition of sound sequences in a child with
apraxia of speech relative to a child who has a
speech sound disorder without verbal apraxia. ‘the cost of inadequate fidelity can be rejection
of powerful treatment programs or acceptance of ineffective programs’ ( p. 2005. but the development and organization of
speech motor behavior as a coordinated action
across several interrelated domains viz.
Assessment of fidelity in a motor speech treatment approach
supervision and certification) and systematic demonstration of adherence to the treatment protocol. Kaderavek and
Justice. and the development of a fidelity assessment tool with a priori
criteria for the evaluation of fidelity in treatment delivery (i. between
1990 and 2000. Borrelli. Borrelli et al. 2011).. developed by the National Institutes of Health’s
Behavioral Change Consortium (Borrelli et al. That is.Hayden et al.. 22% provided supervision and 27% checked adherence to the
delivery of the intervention protocol.. 2013).
While treatment fidelity has been referred to using a
variety of terms (e. (2005) assessed the reporting of fidelity
across 10 years of health behavior research.
(2005. treatment manual). Currently. As stated by Borrelli et al. as few as
35% reported use of a treatment manual.. adequate training and supervision in
the implementation of the treatment protocol (e.
(2005).e. 2003. Language and Hearing
2015
VOL.

(2013)
Not specified. visited the site
quarterly.
completed a case study
requiring assessment by a
PROMPT Instructor. Two
fidelity measures per
participant. and
attended a mentoring day
held by the developer of the
technique. Fidelity above
95% was maintained
Administration of the PFM
Administration of the PFM
Delivery of treatment
Method to ensure that the
content of the
intervention is delivered
as specified
Ward et al. Dynamic Systems Theory
Not provided
All therapists had completed:
Introduction to Technique
training. one was
trained to PROMPT
Introduction to Technique
level. per intervention
phase were taken to generate
a total of four ratings per
participant. Fidelity ranged
between 77.
1
.
Fidelity of at least 85% was
maintained
Single blinded assessor
completed the PFM two
occasions during the
intervention. Dynamic Systems Theory
Yes. 10 weeks each
8 weeks
6
PROMPT – without tactile input
50 minutes
N/A
16
PROMPT – without tactile input
8 weeks
N/A
N/A
N/A
Yes
Yes
Yes. All therapists met 80%
fidelity prior to commencing
the intervention
Single blinded assessor
completed the PFM.Hayden et al. Dynamic Systems Theory
model
Training providers
Description of how
Treatment developers viewed
providers were trained
and coded tapes of the
therapist (frequency not
stated).7% and 97%
Administration of the PFM
Continued
32
Speech.
Assessment of fidelity in a motor speech-treatment approach
Table 1
Components of treatment fidelity in three treatment studies evaluating the PROMPT approach
Fidelity components
Study design
Study population
Rogers et al.
18
NO. Three therapists
also completed PROMPT
Bridging to Intervention
Yes
Standardized provider
training
Not specified
Measured provider skill
post training
Three consecutive fidelity
measures at 85% of greater
required before
commencing the
intervention
Described how provider
skills were maintained
over time
Single blinded assessor
completed the PFM on 25%
of therapist sessions. (2006)
Single subject research
design
Nonverbal toddlers and
preschoolers with autism
10
Number of participants
Treatment design
Provide information about
treatment duration and dosage:
Length of contact
60 minutes
(minutes)
Number of contacts
12
Content of treatment
PROMPT Intervention
Duration of contact over
12 weeks
time
Provide information about
treatment dose in comparison condition:
Length of contact
60 minutes
(minutes)
Number of contacts
12
Content of treatment
DENVER model
Duration of contact over
12 weeks
time
Mention of provider
Yes
credentials
Mention of a theoretical
Yes. used the technique
for a minimum of 9 months. and provided
telephone supervision
monthly
Dale and Hayden (2013)
Single subject research design
Single subject research design
Preschool children with
childhood apraxia of speech
4
Children with cerebral palsy
50 minutes
45
16
PROMPT – full
20
PROMPT Intervention as
described within the
intervention manual
x2 blocks. Language and Hearing
2015
VOL. PROMPT
certification requires
completion of the
Introduction to Technique
and Bridging Technique to
Intervention workshops. as
well as completion of a 4month certification project
Two certified SLPs with
extensive experience using
PROMPT
Four SLPs administered the
intervention – three were
trained to PROMPT Bridging
to Intervention level.

Hayden et al. as rated by a single blinded
assessor across the phases of the treatment study.
However.g. the question
addressed in this paper is: what is the inter-rater
reliability of the PFM? Reliability of the PFM was
assessed through measures of inter-rater agreement. how the scores
were weighted. a clinician
is judged based on his/her ability to intentionally use
a therapeutic strategy that results in an observable
change in the client’s behavior. (2013)
Not specified
Not specified
Not specified
PFM scored based on videorecording of randomly
selected intervention
sessions
Not specified
PFM scored based on videorecording of randomly
selected intervention
sessions
Not specified
PFM scored based on videorecording of randomly
selected intervention sessions
Yes
Yes
Yes
PFM
PFM
PFM
Not specified
Not specified
Not specified
Outcome measures stated and
data analyses of these
measures provided
Outcome measures stated and
data analyses of these
measures provided
Outcome measures stated and
data analyses of these
measures provided
Outcome measures stated and
data analyses of these
measures provided
Outcome measures stated and
data analyses of these
measures provided
Outcome measures stated and
data analyses of these
measures provided
Reporting of outcome
measures
Not specified
Reporting of outcome
measures
Not specified
Reporting of outcome measures
Not specified
Not specified
Note. not applicable.
18
NO.
Cognitive-linguistic (e. (2006)
Dale and Hayden (2013)
Ward et al.
Total of 60 points possible.. PFM.
Assessment of fidelity in a motor speech treatment approach
Table 1 Continued
Fidelity components
Method to ensure that the
dose of the intervention
is delivered as specified
Mechanism to assess
provider adhered to the
intervention plan
Assess nonspecific
treatment effects
Use of treatment manual
Receipt of treatment
Assessed subject
comprehension of the
intervention during the
intervention period
Included a strategy to
improve subject
comprehension of the
intervention
The participants’ ability to
perform the intervention
skills will be assessed
during the intervention
Assessed participant’s
ability to perform the
intervention skills
Enactment of treatment skills
Assessed subject
performance
Assessed strategy to
improve subject
performance
Rogers et al.
while language used matches/slightly exceeds the
Speech. the purpose of this paper is to document progress toward establishing the psychometric
properties of the PFM. other than mentioning that a Likert style
rating was used.
In each study fidelity to the intervention was calculated at 85% or greater.
1
33
. Examples of some of the key elements
within each domain and the total possible points for
each domain on the PFM include:
Physical-sensory (e. whether appropriate prompting
is given at the right time and for the right purpose). how the final scores were calculated or the
inter-reliability of the fidelity measure itself. none of these three studies provide
information on what the items were. N/A. Language and Hearing
2015
VOL. PROMPT Fidelity Measure. what domains and dimensions were
assessed.
Method
Participants
Participants consisted of three raters: the first and
third author and another independent rater. 2010).
Given that the evaluation of treatment fidelity is
dependent on the psychometric soundness of the fidelity instrument.
The PROMPT Fidelity Measure
The PMF (see Appendix A) consists of 36 items and
utilizes a 4-point Likert style rating system based on
behavioural frequencies.
information regarding treatment fidelity strategies
including training provided and the evaluation of
treatment delivery using the PROMPT Fidelity
Measure (PFM). All participants were certified SLPs specializing in
developmental motor speech disorders with more
than ten years’ experience in using the PROMPT
approach. where a score of 1 indicates
that a behaviour is rarely observed while 4 indicates
that a behaviour is always observed. The PMF represents
each of the domains ( physical-sensory. and social-emotional) described in the
PROMPT conceptual framework and is a composite
view of a clinician’s adherence to nine core elements
of the PROMPT protocol and philosophy (Hayden
et al. Specifically. activities that are used are at
the appropriate cognitive level to engage the child.g. cognitive-linguistic. Thus.

criterion and cumulative
was implemented. The three raters blinded to each other’s
scores viewed and assessed fidelity on a randomly
chosen treatment sample video that was uploaded on
to the PROMPT Institute’s secure server. This
means the rater has passed the PROMPT certification
process. one videotaping session every 2
weeks in an 8-week treatment block) and uploads the
videos on to a secure server at the PROMPT
Institute. clinician consistently
reinforces positive behavior). it became apparent that two
additional descriptors were required.
c. Each rater
then emailed the fidelity scores directly to the researcher
. inter-rater reliability measures were piloted by a
four person team. Pilot testing to test psychometric properties. The instructors view tapes. and iterate the
process if fidelity scores are below 70%. Once
the definitions for each item had been completed to consensus. Thus.
The PFM is a direct measure completed either live
or via video recording. To score
each item. based on frequency. the rater assigns a rating (1–4) based on
their judgment as to whether the clinician delivering
the PROMPT intervention is consistently applying a
certain technique or strategy. it became apparent that consensus was
improved with some items being assigned a cumulative
or criterion-based score. a
34
Speech.g.
Initially (consensus meeting 1). until all items of
the PFM reached 100% consensus. Positioning between the clinician and client allows
for appropriate eye contact. Total of 32 points
possible. Total
of 16 points possible. Therefore.g.
Procedure
Reliability of the three raters was assessed through
measures of inter-rater agreement.
Step 3.
b. A score of ≤79 (<55%) requires
a complete re-do of the treatment project (a new
peer reviewer is assigned). Once all the items within one domain
(e.g. The intent
of each item of the PFM was discussed and the definitions were operationalized through a consensus
approach. attended the Instructor training program and
attends yearly Instructor updates run by the Prompt
Institute.
Assessment of fidelity in a motor speech-treatment approach
receptive language of the child). during the process of operationalizing the
definitions. Typically. The clinician is appropriately positioned to allow
clinician a neutral or ‘at rest’ shoulder position. The child/clinician pair are physically positioned in
close proximity to allow the clinician a neutral or ‘at
rest’ shoulder position. The child/clinician is positioned for appropriate
head/neck alignment. as follows:
a. work space is used
appropriately given the nature of the activity). two key descriptors
were identified as essential:
a. Language and Hearing
2015
VOL.
1
de-identified certification project was randomly
selected and scored independently by each rater.
18
NO. Total of 36 points
possible.
To compute the total fidelity score. Positioning between the clinician and client enable
joint interaction with the materials. clinician interaction optimizes
child arousal & joint attention.
d.
However.Hayden et al. Operationalization of the definitions. using
item 2 in the physical-sensory domain of the PFM:
‘The child is positioned closely to the clinician
for adequate prompting and support’. The following example illustrates
the process of fine-tuning the definitions and the
implementation of a cumulative score format. Clinicians may re-do
selected sections if they achieved a score between 80
and 99 (∼55–69%). The Instructor meetings and the certification
project/process are both recognized by the Continuing
Education Board of the American Speech-LanguageHearing Association and clinicians are typically
allowed to accrue continuing education credits. score fidelity
and provide feedback to the clinician. Approximately 20% of these videos are randomly chosen and rated for fidelity by any PROMPT
Instructor from any geographic location logging into
the system.
Currently.
Clinicians must earn a minimum of 100 points out
of 144 total points (∼70%) to pass PROMPT fidelity
(certification) requirements. physical sensory domain) had been defined. the
descriptors for this item were further refined (consensus meeting two). The process was
as follows:
Step 1.
Therapy set up and strategies (e. a rater must be
trained to the level of a PROMPT Instructor. Live assessments are not generally carried out due to the time and personnel costs
involved. Positioning between the clinician and client is comfortable with good head/neck alignment for the
client.
A single point has been allocated to each item to a
maximum of 4 points.g.
Social-emotional (e. and
b. the 4-point Likert
system. This process was repeated
on three occasions (see Table 2). the SLP being assessed for fidelity
is required to videotape four sessions across the entire
treatment block (e. The PFM used a frequency of behavior rating system for all items. This process involved the same three
experienced PROMPT instructors and an SLP
researcher familiar with psychometrics and test construction. to administer the PFM.
However. The definitions for these
items were further refined.
Step 2. One
rater collated the fidelity measures and identified
items with poor agreement. subsequent to scoring a de-identified certification project. Item testing and revision. all items are
tallied and converted to a percentage score.

Minimize the reactivity of observations: therapist may behave
differently whilst being watched or video-taped.
Report treatment fidelity (item 7). 2002. and external validity.
Calculate treatment fidelity (item 6). enabling the clinician to self-monitor or seek
mentoring to promote adherence to the approach. Freelon. the tenets of
the approach are well defined. time between cues. treatment session fidelity) as percentage scores (i. The PROMPT approach is manualized. The
PROMPT approach utilizes both direct and indirect
assessment procedures. and scoring information.
7.
2. The 36-item checklist-based rating system is supplemented by additional
questions that target communication focus. the underlying theory is detailed. the technique and
prompts are described and taught in workshops.g. etc. The following discussion addresses how the
PFM meets these components. words.
18
NO.).e. descriptions. procedural. As clinicians progress in the training. Of principle concern is the change in
clinician behaviour as a result of being watched or
video-taped in a session (i. tone issues). 44)
Recommended measurement strategy for treatment fidelity
Intervention design
1.
Assessment of fidelity in a motor speech-treatment approach
the PFM. Furthermore.).
4.
Total = 144) which allows the examination of overall
fidelity and component fidelity (quality vs. motor
speech hierarchy and priorities selected. etc. score x of total 144 or score x
of domain score. items are rated on a 4-point
Likert system.
9. Ascertain the number of observations: observe between
20–40% of all sessions to have adequate representation of
treatment process.
In the current PFM.g.
1
this manner the PROMPT fidelity measure meets
requirements for items 1 and 2 (Table 3). 3. Use of
random schedule for recording sessions is recommended.
Execution: how fidelity is measured and assessed
3.
construct. Prepare data recording (fidelity scoring) sheets consistent
with assessment methods (direct or indirect).
Execution
How fidelity is measured and assessed (items 3–5). These nine items are
shown in Table 3 as categorized according to intervention design. as these negatively impact treatment fidelity. The fidelity
assessment is carried out systematically with Likert
rating scales that have been operationally defined
using a priori coding categories. spatial and
temporal parameters (e.
The PROMPT clinical training program provides the
operational definition for the independent variable
and the procedural steps necessary to carry out the
treatment in line with the core PROMPT principles
(Hayden et al. 2010). and
clinicians are taught how to determine intervention
objectives. p. Decide procedural steps: steps carried out during treatment
monitored using a checklist (by an independent observer or
the clinician). Determine an assessment method: (a) Direct assessment
through behavioural observations (video-taped or live) or (b)
Indirect assessment though self-monitoring/reporting. Minimize experimenter bias: self-reporting or indirect
measures are inadequate by themselves
36
Speech.
Minimizing threats to validity
Minimizing reactivity of observations (item 8). and
phrases) chosen to embed speech motor movements. Each section or
domain in the fidelity measure has a sub score
(Physical-sensory = 60 Cognitive-linguistic = 32 Socialemotional = 36 and Therapy set up and strategies = 16.
Minimizing threats to validity
8. treatment may be less
effective in untapped sessions). Schlosser (2002) recommends nine key components essential to the assessment of treatment fidelity and associated consequences to internal.Hayden et al.
6.
Intervention design
Define the independent variable operationally (item 1). criterion (1 or 4) or cumulative (1.
Schlosser (2002) further recommends that a good fidelity measure should minimize threats to both internal
and external validity. component fidelity. Report fidelity data: as overall fidelity. co-morbid
conditions that may have implications for intervention
(e. they are also encouraged to selfmonitor and report clinical issues to the PROMPT
instructor group where feedback and mentoring is
offered. based on either frequency of occurrence. 2.
Define procedural steps (item 2). The PROMPT fidelity
process somewhat ameliorates this by the requirement
. execution and managing threats to validity.. Calculate treatment fidelity using % accuracy scores along
with % inter-observer agreement or reliability scores.
session fidelity etc. set up of
room.
5. write goals and activities. target lexicon (syllables. physical. Language and Hearing
2015
VOL. Define the independent variable operationally: operational
definitions must include verbal. 4)
points scored. the itemized fidelity
measure in Appendix A easily permits the calculation of
both domain-by-domain or overall inter-rater reliability
using point-by-point percentage agreement index
(# agreements/(# agreements + #disagreements) × 100)
or inter-rater reliability coefficients like Krippendorff’s
alpha which account for chance agreements between
two or more raters using freely available software (e.e.
ReCal OIR. Over-all the results indicate good inter-rater
reliability between three raters following operationalization of the definitions for each item contained within
the PFM. The PFM contains
detailed definitions. choose correct
communication focus and target lexicon that are true
to the approach. In
Table 3 PROMPT fidelity measure as compared with
recommended measurement strategies for treatment fidelity
(adapted from Schlosser. 2013). All intervention sessions are
video recorded in entirety with approximately 20%
randomly selected for direct (live or video based)
evaluation of the treatment process.g. verbal instructions/feedback.
pre-treatment clinician fidelity vs.

including validity
and clinical utility is required.
Dyslexia. 2012.
Ethics approval The paper discusses test construction
and literature searches – ethical approval was not
required for this study. 73(5):
852–860. further evaluation of
additional psychometric properties. The PROMPT fidelity measure
utilizes a scoring system that integrates treatment delivery (procedural fidelity) and clinical skill (treatment
quality) as a single quantifiable metric.
References
Bellg A. and ratio intercoder
reliability as a web service’. Treatment integrity in a home-based pre-reading intervention programme.
McCauley R.g..A. A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior
research. 8(1): 10–16.
Borrelli B. Hayden D.
Günther T. 2013. and enhancement of
treatment fidelity in public health clinical trials. Veldkamp E. Sepinwall D. uniform intervention delivery. Olsen L.
Otterloo S..J. as
mentioned before. 2004. Ory M.. Journal of Consulting and Clinical Psychology... 2010. 8: 23–34. et al. 2010). which facilitate
interpretation of internal/external validity study replication and dissemination (Kaderavek and Justice. 19(4):
369–379. 4(4): 83–105..
Dale P. therapeutic techniques and dosage) must be
present in order for treatment to be effective..
Minimize experimenter bias (item 9). Statistics and write-up by
Write up of discussion and paper inteediting. The PROMPT system of therapy:
theoretical framework and applications for developmental
apraxia of speech. The importance of implementation fidelity. Tutorials in Quantitative
Methods for Psychology.G. apart from providing
details of the fidelity measure. It is acknowledged that a larger sample size is required and efforts
are in progress to collect inter-rater reliability across 40
independent raters.J. 2010. preliminary data on
inter-rater reliability and the iterative process that followed to improve on the reliability scores are provided
to allow future researchers to plan and develop fidelity
measures for other intervention approaches. van der Leij A.N.A.
Limitations
The evaluation of treatment fidelity is dependent on
the psychometric soundness of the fidelity instrument.. Fidelity: an essential component
of evidence-based practice in speech-language pathology. 5:
139–156. 2004. PROMPT: A
Tactually Grounded model.
The general procedure currently used for the assessment of fidelity of the PROMPT intervention is. 2011.
for video-taping of all sessions.
Dollaghan C. Minicucci D. 2010.
Disclaimer statements
Contributors
Measure by
2nd author. Journal of
Public Health Dentistry. Baltimore: Paul H. The construction of the PFM enables researchers and clinicians to objectively measure treatment outcomes and
reflects EBP. Brookes
Publishing.
Borrelli B. The handbook for evidence-based practice in
communication disorders. Brookes Publishing. the PROMPT approach utilizes a systematic. which entails the measurement of content
Assessment of fidelity in a motor speech treatment approach
validity and scaling up the study generalizability by
using a larger sample size (N = 40) for the calculation
of inter-rater reliability. fidelity measure refinement by
Funding None... and
consistent treatment outcomes.M.
Conclusion
The development of fidelity measures for the training
of service providers and treatment delivery is especially
important in specialized treatment approaches where
certain ‘active ingredients’ (e. 71(Suppl..Hayden et al. McLeod S. Ernst D.. 2002)..
gration and
3rd author.
Freelon D. 2006. Czajkowski S.. 1): 52–63. Walker A.) Interventions for speech sound disorders
in children. manualized.. the PROMPT treatment fidelity measure
meets recommended measurement strategies reported
in the literature (Schlosser. 1984. Biases which
may arise from indirect measures such as self-reporting
may be ameliorated by recording and assessing video
samples and by using two or more observers and calculating inter-rater reliability for the fidelity measure..
Conflicts of interest None.
Hallgren K.
1
37
.A. 2013. independently peer-reviewed clinical certification process.
Acknowledgements
The authors would like to thank Cheryl Small Jackson
for her assistance with the fidelity definitions and
Jennifer Hard for her editorial assistance. Health Psychology. Borrelli B. Language and Hearing
2015
VOL. it is exploratory in nature with a limited sample size. 12(3): 155–176.
Mihalic S. Resnick B. Hecht J.
American Journal of Speech-Language Pathology. In addition. the assessment of video sample(s)
by blinded/naive prompt instructors.. Baltimore: Paul H.23. Computing interrater reliability for observational data: an overview and tutorial. from which one or two
sessions are randomly selected for assessment.
Kaderavek J. specific treatment
targets.
Development of PROMPT Fidelity
first author. interval. Addition of contingency management
to increase home practice in young children with a speech sound
disorder. International Journal of Internet
Science.. Eigen J.L. which allows
for the reporting of both domain-by-domain or overall
fidelity scores and/or reliability.
Emotional and Behavioral Disorders in Youth. p. This allows for higher
clinician reliability. International Journal of Language and Communication
Disorders. 2008. Seminars in Speech and Language.J. 2005.
Speech. Enhancing treatment fidelity in health behavior
change studies: best practices and recommendations from the
NIH Behavior Change Consortium. This process
minimizes threats to validity.. The next
steps are already underway in the development of
this PFM.S.
In summary. The assessment.. Treating speech subsystems in childhood
apraxia of speech with tactual input: the PROMPT approach. Hautvast S.5.. Bellg A. 22(4):
644–661. In: Williams A. monitoring. (eds.
18
NO.
Finally. 45(3): 345–353. Justice L...443. 23(5):
443–451.
et al.
Hayden D.
American Journal of Speech Language Pathology. 453–474.
Chumpelik-Hayden D. ReCal OIR: ordinal. doi: 10. Breger
R.
Despite the contributions of this study. In this study.1037/0278-6133.

J. asks models. and provides feedback
expected response if child does not automatically
produce it
10 Prompting achieves the desired effect. Kinematic changes in jaw
and lip control of children with cerebral palsy following participation in a motor-speech (PROMPT) intervention. Hall T. 2006.g. Hepburn S. On the importance of being earnest about
treatment
integrity. and models expected response if
child does not automatically produce it
If needed. 15(2):
136–155. complex. Teaching young nonverbal children with
autism useful speech: a pilot study of the Denver Model and
PROMPT interventions.
spitting.. tantrums. 2003. or words are embedded within a
meaningful and appropriate context in activities
Child arousal and joint attention optimized by choice of
materials/activities
Clinician uses language that matches or just slightly
exceeds the receptive language level of the child
Clinician provides labels for associations between objects. Journal of Autism and Developmental
Disorders. Leitão S. 2002. stage 1. asks. it is addressed in the intervention session
2 Child is positioned closely to the clinician for adequate
prompting and physical support
3 The appropriate motor level. 2005. and people
Clinician states. for the child’s sensory motor
capacity. 15: 255–283. Strauss G. American Journal of
Physical Medicine and Rehabilitation. self-injurious)
Clinician affect is appropriate and natural to the situation
Total possible points (36)
Therapy set-up and strategies
Materials are out of child’s reach..
refined.
18
NO. Language and Hearing
2015
VOL. 18: 36–44.
Ward R. throwing.. if appropriate
Work areas are clearly and visually delineated
Space is used appropriately given the nature of the activity
Clinician provides changes in location a few times during
the session (if appropriate)
Total possible points (16)
. is chosen
4 Appropriate prompting (parameter.Hayden et al. 36: 1007–1024.
Continued
Clinician:
1
2
4
(A) Prompt fidelity score form
5
Clinician:
1
2
3
4
Client:
Reviewer:
Date:
6
Motor speech hierarchy stages and priorities correctly
identified
Communication foci correctly identified
Treatment stage has been correctly identified (i. screaming... it’s a Russian
doll: defining rehabilitation treatments.
Thelen E.
1
Reviewer:
Date:
13 If ‘motor-phoneme’ practice is seen..W. clinician changes task demands of activity to reengage child
Total possible points (32)
Social–emotional domain
Overall positive affect displayed by child
Reciprocal turn-taking is observed in most activities
between the child and clinician
Child’s behavior indicates their ability to predict session
routines
Clinician interaction optimizes joint arousal and joint
attention
Clinician provides opportunities for child to communicate
and interact at almost every turn (every 30–60 seconds)
If needed. or 3)
Purpose of prompt correctly identified
Physical – sensory domain
If tone is identified as an issue on the motor speech
hierarchy.. syllables.
Hayes A. hitting. child imitates or
is able to approximate the target in a more relaxed.R.
Whyte J.
actions.e. Charlifue-Smith R. the practice is
appropriate to child’s motor levels
1
Continued
38
Speech. Psychoanalytic Dialogues. e.
Augmentative
and
Alternative
Communication. 2013. 82: 639–652.g.
Assessment of fidelity in a motor speech-treatment approach
Rogers S. syllable. or intelligible manner
11 Some ‘motor-phoneme’ practice is seen during the session
12 If ‘motor-phoneme’ practice is seen.
International Journal of Speech-Language Pathology. Hart T.
crying. It’s more than a black box. or
surface prompting) is given at the right time and for the
right purpose
5 Observed prompting technique is accurate
6 Clinician provides prompting for both (1) accuracy of motor
phonemes and (2) whole word/phrase approximations
7 Frequency of prompting is appropriate
8 Child appears to understand what the goal and expected
response of prompting the clinician is expecting?
9 Clinician states. kicking. Dynamic systems theory and the complexity of
change. the practice is
appropriate to the context of the activity in which the
sounds will be embedded
14 The selected PROMPT lexicon created for use in the routine
or activity is consistent with identified motor-phonemes
15 Chosen PROMPT lexicons are used functionally within the
environment whenever opportunities arise
Total possible points (60)
3
Appendix
Client:
7
8
1
2
3
4
5
6
7
8
9
1
2
3
4
Cognitive–linguistic domain
Chosen activities are at the appropriate cognitive level to
engage the child
Chosen activities facilitate interaction and reciprocal turntaking
Sounds..
Schlosser R.
2. clinician reframes task to elicit a response in the
child
Clinician consistently reinforces positive behavior
Clinician appropriately addresses difficult behaviors (e. Hayden D.