You may be trying to access this site from a secured browser on the server. Please enable scripts and reload this page.

Enter your Email address:

Wolters Kluwer Health may email you for journal alerts and information, but is committed
to maintaining your privacy and will not share your personal information without
your express consent. For more information, please refer to our Privacy Policy.

The opinions presented in this article reflect the views of the authors and not necessarily those of the National Institutes of Health or the US Public Health Service. As we are US federal employees, this article fits the description of a “US Government Work” and cannot be copyrighted (Copyright Revision Act, 1976). It is available for publication and there are no restrictions on its use, now or subsequently.

Abstract

Purpose: The Brief Assessment of Motor Function Fine Motor Scale (FMS) allows rapid assessment, independent of age. This study was done to establish content validity of the FMS and to demonstrate FMS reliability.

Methods: A standard questionnaire (“Disagree” to “Agree,” 1–4) was emailed to 28 expert panel members. Ten children with diagnoses including Proteus, Sheldon-Freeman, Smith-Lemli-Opitz, and Smith-Magenis syndromes were videotaped for reliability trials.

Results: Expert panel members agreed that all 28 items should be included (means, 3.43–3.89); were functionally relevant (means, 2.93–3.82), were clearly worded (means, 2.71–3.61), and were easily discriminated (means, 3.32–4.0). Kappa values for interrater and intrarater reliability were 0.978 and 0.993, respectively.

Conclusions: Feedback from an expert Panel supported content validity of the Brief Assessment of Motor Function FMS. Kappa values for interrater and intrarater reliability suggest this is a reliable instrument for rapid, objective fine motor assessment.

INTRODUCTION

The development of normal fine motor function follows a specific sequence,1 which has been documented extensively in the literature. From birth to two years, the development of fundamental hand skills occurs, including grasp, release, and bimanual skills.2 A child typically learns to button between ages three and four years3 and to snap between ages three and a half and four years. During the preschool years (ages four to six years), precision handling and manual dexterity develop1 such that in-hand manipulation,4,5 lacing and tying,3 tool use, visuomotor, and self-care skills1 are refined.

Development of any of these skills can be directly affected by delays1 resulting from a variety of causes, including developmental disabilities, such as cerebral palsy.6 Even low birth weight can affect fine motor outcomes later in life.7,8 In both children and adults, musculoskeletal and neuromuscular disorders such as inflammatory joint disease9,10 and Wilson’s disease11 may affect fine motor skills, as can stroke12 and head injury,13,14 as well as HIV/AIDS.15

Substance-induced declines in fine motor function have also been documented in populations, including acutely psychotic patients.16 Finally, studies have shown that even the normal aging process is associated with a decline in fine motor function: evidence of a critical decline in hand movements in geriatric populations suggests that there may be an identifiable point in midlife when fine motor decline either begins or significantly worsens.17–19 Others have emphasized the heterogeneous nature of fine motor decline in geriatric populations, suggesting that the observed decline may not be simply age-dependent.20

Instruments commonly used to assess fine motor skills in children include the Fine Motor Scales (FMSs) of the Peabody Developmental Motor Scales (PDMS-II),21 the original PDMS,22 and the Bruininks-Oseretsky Test of Motor Proficiency.23 For children and adults, hand function is often evaluated using the Jebsen Hand Function Test.24 Additional measures used to assess fine motor function among adults include the Nine-Hole Peg Test,25 the Motor Assessment Scale,26 the Purdue Pegboard,27 and the Smith Hand Function Evaluation.28

Through wide application, many of these tests have become “gold standards” in their respective domains. The validity and reliability of these instruments has generally been supported6,29–35; however, results of standardized tests designed to identify developmental stage or rate skills with limited direct applicability to daily life may not be appreciated as reflective of true functional ability, and as a result, the tests may not be used routinely in clinical practice to demonstrate treatment response.36

Many assessments are considered valuable for their ability to generate normative comparison scores, although the scores may not clearly describe what an individual actually can do with respect to motor performance.37 Comparisons to normal scores may not be useful for an individual with a disability; for example, they may not be relevant to tracking performance changes of children who have functional limitations associated with neuromuscular or musculoskeletal impairments.9 In a study generally supporting the interrater reliability of the PDMS FMSs, Stokes et al34 found greater agreement between raters for a group of children who were not delayed than for a group of children with motor delay. An additional consideration is the fact that normal values may not be relevant across cultures.38

Brief Assessment of Motor Function

The Brief Assessment of Motor Function (BAMF) consists of five hierarchically ordered motor scales for gross, fine, and oral motor domains.37 Designed to be used independent of age, each BAMF scale is organized to allow very rapid identification of motor skill level. The authors designed the BAMF Scales to assess observable, unequivocal, adaptive functional behaviors.37 While the BAMF FMS (see Appendix) grew out of a need for rapid assessment of fine motor skill in infants and children with disabilities, behaviors selected for inclusion in all five of the scales represent observed performance capabilities, rather than disability-imposed limitations. The BAMF was developed from the observation that a brief assessment represents a valid and clinically useful first step in some circumstances. This is underscored by the fact that a BAMF score corresponds to a specific motor performance level, allowing the examiner to rapidly determine, and subsequently recall, an individual’s level of function.

Used together, the five scales of the BAMF provide a complete profile of an individual’s motor functioning in five different domains in as little as 10 minutes. Ease of BAMF administration makes it a useful tool for tracking an individual’s progress over time. Although a single assessment on which to base initial referrals may be useful, a need clearly exists for screening over multiple time points and across domains.39 The BAMF can be used by professionals and students of various disciplines in numerous contexts, including clinical research and educational settings. Among the BAMF scales, the Lower Extremity Gross Motor Scale37 has been shown to demonstrate good interrater and intrarater reliability for children with various diagnoses, as well as good concurrent validity with standard measures of gross motor performance used in children with osteogenesis imperfecta.

Brief Assessment of Motor Function Fine Motor Scale

The BAMF FMS can be used to rapidly assess fine motor skill. Based on observable functional behaviors, and requiring a dichotomous choice of “present” or “absent,” the BAMF FMS is a 0 to 10 scale, and the score given is equivalent to the highest, most challenging observed item completed on the scale, regardless of whether items lower on the scale were performed (Figure 1). From the universe of possibilities, the BAMF FMS was made up of skills with functional salience, organized developmentally, and has some value in documenting change (gain or loss) over time. In testing a child, emphasis is placed on seeing the highest skill level (0–10) possible, so instructions are made as clearly as possible, repeated as necessary, and time is given to allow the child to demonstrate the presence or absence of the skill being tested. Although every task below the observed best performance could be assessed, this is not consistent with the intent of the BAMF FMS scale, and other assessments already exist for this purpose. Measures like the BAMF FMS that incorporate a range of levels, in this case from essentially no isolated fine motor skill to a highly coordinated skill like writing, also minimize the “floor effect” on scores, which sometimes makes it difficult to detect change in performance over time.40 Physical performance measures like the BAMF FMS are useful not only because they document observable and reproducible behaviors, but also because they can capture meaningful changes in functional ability and reflect distinctions between an individual’s usual function and the maximum level possible.41 The BAMF FMS is both a screening instrument and follow-up tool, offering a functionally meaningful and concise way to assess the outcome of specific interventions and track subsequent changes in function (see Appendix).

METHODOLOGY

Participants and Procedures

Content Validity.

Content validity is frequently the initial step in the process of instrument development,42 and establishing content validity is encouraged by the tests and measurements standards of the American Psychological Association43 and American Physical Therapy Association.44 Content validity has been defined as “the degree to which test items represent the performance domain the test is intended to measure.”45(p. 154) One common way to assess the validity of an instrument is to have it evaluated by a panel of expert judges.42,45 The present study employed one of the methods described by Dunn,45 in which an expert panel is provided with both test items and a list of test objectives. Quantitative and or qualitative feedback provided by the panel is then used to modify individual items and overall test content. Other studies have used similar approaches to establish content validity.46–48

The expert panel for the BAMF FMS consisted of 28 experts in fine motor assessment, all occupational therapy clinicians and researchers with doctoral degrees (PhD, DSc, ScD). Selected for their terminal degrees and recommended to us by colleagues, they came from widely diverse locations around the United States and Canada. More than 90% of the panel had 20 years or more experience in their field, which is indicative of their collective expertise (Table 1).

Questionnaire.

Following an email invitation to participate and individuals’ consent to do so, a standard questionnaire was emailed to each prospective panel member. None of the panel had used the BAMF FMS before participating in the study, and they were asked to assess the BAMF FMS items on the basis of six standard questions for each skill level (0–10). Respondents were included as expert panel members only if they agreed to participate before receiving the questionnaire, and completed and returned the questionnaire. The standard six questions for each BAMF item are as follows: (1) This item should be included; (2) This item is clearly worded; (3) Item should be reordered higher on scale; (4) Item should be reordered lower on scale; (5) This is a functionally relevant motor behavior; and (6) This behavior is easily discriminated from others on the scale. Respondents were asked to respond to each question using a four-point scale, ranging from 1 = Disagree to 4 = Agree. This scale allowed clear expression (no neutral rating) of judgment, with some room to express intensity or strength of agreement or disagreement (4 vs 3, 1 vs 2).

Reliability.

A convenience sample of 10 children (ages seven months to 15 years; three boys, seven girls) with a range of diagnoses including osteogenesis imperfecta, Proteus, Sheldon-Freeman, Smith-Lemli-Opitz, and Smith-Magenis syndromes, was selected to represent a broad range of skill levels; no children who were typically developing were included in the sample (Table 2). Following parental informed consent (and child assent where possible), the children were videotaped performing specific upper extremity fine motor skills, items 0 to 10 on the BAMF FMS. High quality videotaped segments of the children’s performances were then randomly ordered on a single videotape. Ten children were selected as the maximum permitted under the Department’s IRB-approved Screening Protocol, which allows data collection on small numbers (10 or fewer) in pilot studies, for cases such as instrument development. In the reliability study, five raters were used; all were employed in the Rehabilitation Medicine Department of the National Institutes of Health and had varying skill levels and training; among the group were a board certified pediatric specialist, a newly graduated therapist, and a therapist having more than 20 years of experience mainly with adults with mental health disorders. Five raters were chosen, so as to generate robust results for data analysis. Before viewing the composite videotape, the group was prepared with training consisting of an overview and brief explanation of each item 0 to 10 and the descriptive criteria. The same three occupational therapists and two physical therapists viewed the videotape together and silently rated the children’s performances on the BAMF FMS (with revisions incorporated from expert panel suggestions) on three occasions: at baseline, after 24 hours, and three weeks after the original viewing. The same tape was used on all three rating occasions; if raters needed to see specific taped segments over, they could ask for a rewind or repeat play.

Data Analyses

Content Validity Questionnaire.

Descriptive statistics were used to determine central tendency and range of responses. Range, median, and mean were calculated for responses to each of the six questions for each of the 10 BAMF FMS items (Table 3). The percentage of responses assigned ratings of 1 through 4 was also calculated for items 0 through 10 (Table 4).

Although not required to do so, many expert panel members provided item-specific commentary (we received 84 separate comments), which were used in addition to the quantitative feedback to further refine individual items on the BAMF FMS. After systematic analysis of qualitative feedback in the form of comments on the items, several items were modified and several descriptive criteria for scoring items were rewritten. To evaluate the applicability of the qualitative feedback and ultimately the decision to modify specific items, written comments provided by the panel were examined based on these criteria: (1) Does the comment appear more than once? (2) Is the comment useful for clarifying the description of an item? and (3) Is the comment useful with respect to reordering an item? Items revisions were incorporated into the scale used by raters during the reliability trials.

RESULTS

Content Validity

Table 3 and Figure 2 provide and illustrate the mean response values for each of the BAMF task items. Table 4 displays the percentage of responses allocated to ratings of 1, 2, 3, or 4 for all items; Table 4 also shows the median and mode of responses for all items. For items 0 to 5, 9, and 10, some expert panel members did not provide a rating for every criterion. When this occurred, it was typically accompanied by a written response.

Rated on a scale from 1 to 4, the expert panel members generally agreed that all items should be included: means ranged from 3.43 to 3.89 (Table 3). Depending on the BAMF task item, 57.1% to 92.9% of panel members gave “This item should be included” a rating of 4 (Table 4). There was also general agreement that the items could easily be discriminated from others on the scale (means 3.32–4.00). “This behavior is easily discriminated from others on the scale” was given a rating of 4 by 60.7% to 100% of panel members, depending on the BAMF task item.

The panel also found the rank order of the items appropriate. In response to the statement “This item should be reordered higher on scale,” means ranged from 1.14 to 1.61 (Table 3), indicating strong agreement that the order of BAMF task items should not be changed. Depending on the task item, 67.9% to 92.9% of panel members assigned a rating of 1, indicating they did not recommend reordering the items (Table 4). Similarly, for “This item should be reordered lower on scale,” means ranged from 1.08 to 1.88 (Table 3), and, depending on the item, 61.5% to 92.3% of panel members assigned a rating of 1, again not recommending a change in order (Table 4).

Although less, considerable agreement existed overall on the clear wording of the items: means ranged from 2.71 to 3.61, and 50.0% to 89.3% of panel members assigned the criterion “The item is clearly worded” a rating of 3 or 4. It merits mention that almost all of the written comments on the questionnaire related to changes in wording to describe the criterion behavior more clearly. For the criterion “This is a functionally relevant motor behavior,” means ranged from 2.93 to 3.82 (Table 3), and 60.7% to 96.5% of panel members assigned a rating of 3 or 4 (Table 4).

In addition to completing the questionnaire, 18 respondents also provided 84 written comments on the questionnaire. Items 1, 2, and 10 elicited the greatest number of comments: 10, 12, and 13 comments, respectively. Table 5 provides specific examples of feedback from panel members.

Item 3, “Reaches while supine or sitting,” had its criterion description modified to clarify that the seated position was at a 90-degree trunk angle with support, if an erect trunk position required it.

Item 10, “Writes 10 words or characters legibly,” was modified as was its criterion description, because many comments pointed out that requiring a “dynamic tripod grasp” in the original item hierarchy ignored the fact that not all typical children and adults use this type of grasp.50 The rewritten criterion, “Using a mature grasp,” acknowledges the widely recognized finding that there are acceptable variants of the dynamic tripod grasp, which are used by adults who are successful, including medical students.51 Item 10 itself, “Writes 10 words or characters legibly,” was rewritten from the original, “Uses pencil with dynamic tripod grasp,” to decrease cultural bias (eg, many Asian languages use characters, rather than words); clarify the emphasis on motor output (written or copied product); and set a specific number (“10 words or figures”) of units to be written or copied accurately.

Some expert panel members questioned the functional relevance of “isolated dynamic digital extension.” The authors thought that the ability to isolate dynamic digital extension represents a specific and higher level of function that can only be developed through movement which is not dominated by reflexive or mass finger flexion patterns.

Reliability

Robust coefficients of reliability demonstrate success in achieving the objective of refining the clarity of the items, so that scoring remained stable when the instrument was put to use by the raters on a series of occasions (Table 6). Kappa values were 0.978 for interrater reliability, and 0.993 for intrarater reliability.

DISCUSSION

The language of an instrument is of primary importance; an instrument is only as good as its ability to reflect recognizable, meaningful behavior, and its ability to provide stable scores from one administration to the next. From the standpoint of language, the first of the study’s two objectives was to develop an instrument in which the language would reflect increasing developmental skill, such that scores would stand for representative, readily observable behaviors easily staged in a clinical or research situation. The second objective was to make the language so clear that raters would score consistently, agreeing with each other when assigning a specific score, and making interrater and intrarater reliability acceptably high. Some may criticize the items for their potential lack of stability or their lack of ability to be sensitive to small changes. The BAMF FMS was developed out of a need in our clinical research setting for a clearly differentiated, straightforward, clear-cut, and reliable assessment to quickly verify what children can do, based on a standard pertinent to the broadest possible variety of ages and disabilities. According to statistical analyses of their responses, the expert panel thought the instrument was functionally relevant with items easily discriminated from each other. They also thought, in general, that items were properly ordered, not requiring movement either up or down the hierarchy; given the tool’s objective of establishing baseline and monitoring change over time, appropriateness of item order is imperative.

Our experience using the BAMF FMS for clinical and research purposes has shown us that children’s functional abilities can be identified and improved through actual interventions based on BAMF FMS scores. We developed the BAMF FMS to be used solo or in conjunction with more comprehensive assessments (eg, PDMS-II or Bruininks-Oseretsky Test of Motor Proficiency, Second Edition); the two types of assessments play complementary roles for children who are followed over a long period of time. Functionally relevant information can be obtained longitudinally with the BAMF FMS, while more detailed information across a range of skills is available with conventional motor assessments: the BAMF FMS identifies a child’s best skill in its domain at each monitoring session, while the other assessments provide norm-based, comprehensive information across a range of skills.

Several potential limitations of the study should be mentioned. The expert panel was comprised of occupational therapists, a majority of whom come from academia, and whose perspective may be different from that of a panel comprised of full-time clinicians. Using a 1 to 4, Disagree to Agree scale (with no descriptors for ratings of 2 or 3) for items examined by the expert panel may have had an effect on our results: positively, it forces a clear decision on one side or the other; negatively, it may not have offered the panel members sufficient range of ratings from which to choose. During reliability trials, a single videotape with randomly ordered samples of children’s performance was used; this could have caused an ordering effect, as well as recall bias, which may result in inflated Kappa values. An additional potential effect on Kappa values was the use of videotape, rather than live performances: use of videotape allows greater convenience in carrying out reliability trials, but it also limits the degree to which the trials approximate real life. Conceivably, in real life children may perform differently at 24 hours and three weeks later than baseline.

Although the instrument has been used most often with children up to now, there is a proposal to incorporate its use into the care of adults with closed head injury. To further refine its psychometric properties, there is a plan to examine the quality of the rating scale and its structure using a Rasch-measurement approach. The instrument is currently included in five research protocols, which use other “gold standards” like the PDMS-II; it is projected that concurrent validity studies will be undertaken in the future.

CONCLUSIONS

Through their responses to a standard content validity questionnaire, an expert panel agreed that the BAMF FMS is a valid hierarchical scale of fine motor performance. Changes were made to the BAMF FMS based on expert panel members’ quantitative feedback and 84 qualitative responses. Kappa values for interrater and intrarater reliability suggest this is a highly reliable instrument for baseline and screening purposes when rapid motor performance assessment is desired. The BAMF FMS expands the array of fine motor assessment options available to clinicians and researchers. Particularly for those who seek to quickly and easily document baseline fine motor skills and track subsequent progress, the BAMF FMS provides a useful solution.

ACKNOWLEDGMENTS

The authors acknowledge the computer graphics expertise and invaluable advice provided by Gloria Furst, OTR/L, MPH, Rehabilitation Medicine Department, National Institutes of Health. They also thank the parents and their children who participated in the reliability study, and the Occupational Therapists who shared their time and expertise to complete the content validity study.

Enter and submit the email address you registered with. An email with instructions to reset your password will be sent to that address.

Email:

Password Sent

Link to reset your password has been sent to specified email address.

Remember me

What does "Remember me" mean?
By checking this box, you'll stay logged in until you logout. You'll get easier access to your articles, collections,
media, and all your other content, even if you close your browser or shut down your
computer.

To protect your most sensitive data and activities (like changing your password),
we'll ask you to re-enter your password when you access these services.

What if I'm on a computer that I share with others?
If you're using a public computer or you share this computer with others, we recommend
that you uncheck the "Remember me" box.