This feature is available to Subscribers Only

OBJECTIVE. We studied the psychometric properties of the dynamic version of the Lowenstein Occupational Therapy Cognitive Assessment (DLOTCA) and examined the most frequent level of mediation used for planning for intervention.

METHOD. Participants included 83 clients hospitalized after first stroke (mean age = 57.7, standard deviation = 8.33) and 45 healthy control participants. All were assessed with the DLOTCA after providing informed consent.

RESULTS. Interrater reliability showed high correlations between all pairs of raters. Internal consistency reliability showed moderate to high αs (.602–.813) for all domains except Visual Perception. We found significant differences between the groups of participants before mediation; both benefited from mediation, showing moderate to high effect sizes. Stroke clients needed higher levels of mediation.

CONCLUSION. The DLOTCA is effective in providing insight into whether participants need mediation and the level and type of assistance they require. The DLOTCA provides guidance for planning intervention for people with cognitive disabilities.

Occupational therapists are in a unique position to assess and treat cognitive deficits in neurological populations to facilitate their participation in all areas of occupational performance, such as self-care, education, play, leisure, and social participation (Hartman-Maeir, Katz, & Baum, 2009; Katz, 2005; Law, 2002; Youngstrom, 2002). The DLOTCA aims, first, to identify cognitive abilities and limitations of people with neurological disabilities in the primary cognitive areas related to function. Second, it aims to allow identification of learning potential and change as well as thinking strategies by means of analysis of the test’s mediation process. Third, it aims to provide a starting point from which to begin intervention appropriate to the specific person tested.

Dynamic Assessment

Conventional standardized cognitive tests are static in nature; they examine individual performance in the here and now for the purpose of identifying and quantifying cognitive deficits (Toglia, 2005). Dynamic assessment is an interactive procedure that systematically and objectively measures the degree of change that occurs in response to cues, strategies, feedback, or task conditions that are introduced during testing. In contrast to static assessment, dynamic assessment focuses on individual variations and changes rather than on comparison with normative or typical performance. The goal is to measure how and to what extent performance can improve with guidance (Haywood & Lidz, 2007).

In recent years, we have seen an increased awareness within the health professions of the potential for dynamic assessment techniques to provide professionals with the opportunity to estimate the person’s potential for learning or receptiveness to instruction (Feuerstein & Feuerstein, 1991). Dynamic assessment is based on Vygotsky’s (1978) concept of the zone of proximal development, which refers to the discrepancy between what a child can do independently and what he or she can do with the help and guidance of others (Grigorenko & Sternberg, 1998; Sternberg & Grigorenko, 2002). This concept is somewhat similar to that independently developed by Feuerstein, Rand, Hoffman, and Miller (1980) of the mediated learning experience, in which adults serve as catalysts for learning by modifying the person’s internal arousal and the specific task demands to allow for improved cognitive performance (Feuerstein, 1979; Feuerstein et al., 1980; Hadas-Lidor & Weiss, 2005). In dynamic assessment, mediation is defined as the process whereby the tester or evaluator provides cues that may facilitate the person’s ability to perform the task.

Toglia (1994, 2005) introduced the use of a structured, graded system of cues to the assessment of cognitive and perceptual deficits among adults with cognitive impairments. Following in the footsteps of earlier dynamic cognitive theorists, she believed that the examiner could learn much about underlying information-processing strategies by observing a client’s responses to such cues. In this way, dynamic assessment becomes naturally linked to intervention and can be used as a baseline for choosing and designing an intervention program (Lidz, 1992; Lidz & Elliot, 2000; Toglia, 1994). Toglia and Cermak (2009) demonstrated that in an intervention based on dynamic assessment, clients with unilateral neglect showed significant differences—more left-sided search and reduced neglect—compared with a control group performing an object search task.

In dynamic assessments such as the DLOTCA, a person’s initial incorrect response is not taken as the final product of the evaluation task. Instead, the examiner uses a systematic approach to modify the task, through prompts or other forms of mediation, to understand the type of information that is essential so that the person can best complete the task. This feature, which is unique to dynamic assessments, also enables the examiner to collect information that can be helpful in developing effective remediation strategies (Lidz, 1992). Indeed, the use of such feedback during and after testing demonstrates how dynamic assessment ascribes to the principle basic to the learning-test concept—namely, the assumption that learning can take place even within the actual testing process (Guthke & Beckmann, 2000). Other researchers have applied a process similar to ours in changing the LOTCA to a dynamic assessment to well-known cognitive assessments. For example, Uprichard, Kupshik, Pine, and Fletcher (2009) tested 77 people with acquired brain injury using the Wisconsin Card Sorting Test (Heaton, 1981) using the regular static administration and using a dynamic procedure. The dynamic process scores showed better prediction of community integration than did the static scores. This adaptation of the Wisconsin Card Sorting Test suggests the same underlying assumptions that dynamic measures will be ecologically more valid than static testing.

The goals of this study were (1) to study the DLOTCA’s psychometric properties, interrater reliability, internal consistency, and construct validity; (2) to determine the level of change from before to after mediation on performance of the DLOTCA subtests, indicating the potential for learning and change; and (3) to examine the most frequent level of mediation used in the different domains for further planning of appropriate methods of intervention.

Method

Research Design

The study was a comparative study of clients with stroke and healthy control participants to determine the DLOTCA’s psychometric properties. The study was approved by the Human Rights Committees of the Lowenstein Rehabilitation Hospital and the Sheba Medical Center, and every participant signed an informed consent before testing.

Instrument

The DLOTCA battery consists of 28 subtests in seven cognitive domains: Orientation, Awareness, Visual Perception, Spatial Perception, Praxis, Visuomotor Construction, and Thinking Operations. The battery is designed to assess cognitive performance of adults ages 18–69 yr. The DLOTCA is a dynamic assessment; in five of the domains (i.e., all but Orientation and Awareness), each subtest has a structured, four- to five-step mediation option.

The objectives of the assessment are as follows:

Identify the cognitive abilities and disabilities of the individual in the different domains

Measure learning potential and change

Recognize thinking strategies through the use of dynamic assessment

Identify the person’s level of awareness of his or her condition and cognitive disabilities.

Level 4: Structured category—The examiner gives cues through the use of key points.

Level 5: Copying or subtracting amount—The examiner performs the task and then asks the client to perform it, or the examiner reduces the number of stimuli (e.g., the number of cards in the categorization task) and then asks the client to complete the task. (Katz, Livni, et al., 2011, p. 16)

Although the mediation process is structured, each subtest has detailed options for each level of mediation. The evaluator chooses the best options according to the person’s performance.

Administration of the entire battery takes from 1 hr to 2 hr, depending on the amount of mediation needed. If the client is unable to complete the assessment in one session, it is possible to administer it in more than one session within a reasonable period of time. In the domains of Visuomotor Construction and Thinking Operations, time is measured for each subtest with a maximum of 2 min before mediation starts. Scoring consists of three components for each subtest: (1) a basic before-mediation score (higher scores = better performance), (2) a mediation score (for which 1 = less mediation and 5 = extensive mediation), and (3) an after-mediation score (higher scores = better performance).

Participants

Participants included 83 clients hospitalized following first stroke, who were entered into the study consecutively (mean age = 57.7 yr, standard deviation [SD] = 8.33) and 45 healthy control participants who volunteered for the study and fulfilled the inclusion criteria (mean age = 62.67 yr, SD = 9.22). Mean yr of education for stroke clients was 12.88 (SD = 2.65) and for control participants was 9.91 (SD = 4.88).

Inclusion criteria for both groups were (1) a score above the cutoff point of 24 for dementia on the Mini-Mental State Evaluation (Folstein, Folstein, & McHugh, 1975); (2) age <70 yr; (3) for stroke clients, first stroke with no previous neurological or psychiatric illness; and (4) for healthy control participants, no previous neurological or psychiatric illness. All were assessed with the DLOTCA after providing informed consent.

Procedure

All participants were assessed with the DLOTCA in either one or two sessions. Assessment sessions lasted no more than 1 hr according to participant fatigue or attention span. If needed, a follow-up session was conducted within a few days.

Data Collection and Data Analysis

The data were collected by six experienced occupational therapists from two rehabilitation hospitals whom we trained to administer the instrument. Interrater reliability was performed by pairs of the therapists who collected the data.

Data analysis included, first, Spearman correlation coefficients for interrater reliability. After data were collected from all participants, Cronbach’s α was calculated for internal consistency of the DLOTCA domains. We calculated means and standard deviations for each group’s performance on individual subtests before mediation, and comparisons were performed using the Mann–Whitney U test. Moreover, we calculated descriptive statistics (means, standard deviations, and quartiles) for all domain scores in each group. Because the sample size was large enough, we performed two-sample t-test comparisons between the healthy control participant groups and the stroke client groups on the domain scores before and after mediation and for time.

We calculated frequencies of level of mediation in each domain and presented them in a histogram. To test the change within the stroke client group from before mediation to after, we performed paired t tests for those who needed mediation and calculated Cohen’s d. In addition, because participant groups’ years of education ranged widely, we calculated Pearson correlation coefficients for the association of years of education with domain scores.

Results

The mean age of the 83 hospitalized clients following first stroke was 57.7 (SD = 8.33), and that of the 45 healthy control participants was 62.67 (SD = 9.22). Mean years of education was lower for the healthy control participants, but the difference was not statistically significant.

Psychometric Properties of the DLOTCA

Interrater Reliability.

We calculated interrater reliability for three pairs of raters for 10 participants in the different testing locations. Results showed high Spearman correlation coefficients between all pairs of raters for the domains (rs = .90–.98).

Internal Consistency Reliability.

The α coefficient level for each domain was found to be moderately high (Spatial Perception, α = .665; Praxis, α = .602; Visuomotor Construction, α = .813; Thinking Operations, α = .737; and Verbal Mathematical Questions, α = .647), except for the Visual Perception domain (α = .313). The shape identification subtest was found to involve naming difficulties for the second set of the four more difficult items that are highly correlated with years of education. We therefore decided to remove this subtest from the final version (Katz, Livni, et al., 2011).

Construct Validity: Comparison Between Groups.

Using Mann–Whitney U tests, we calculated means and standard deviations for each group’s performance on all subtests before mediation. We found significant differences on most of the subtests before mediation, p < .05, on which healthy control participants performed better than stroke clients.

Means, standard deviations, and quartiles are presented in Table 1 for performance before mediation and were compared with two-sample t tests between healthy control participants and stroke clients on the DLOTCA domains. Significant differences can be seen on the domains of Orientation, Visual and Spatial Perception, and Praxis at p < .05 before mediation and in the Visuomotor Construction and Thinking Operations domains; the difference was not significant because the healthy control participants also had difficulties, probably as a result of the correlation with years of education (presented later). However, time was significant at p < .01, showing that the stroke clients needed much more time to accomplish the tasks (see Table 1). The subgroups who needed mediation improved in both groups, and therefore we found no significant differences after mediation except for Thinking Operations; whereas before mediation, there was no difference, after mediation a significant difference was found (t = 5.441, df = 126, p < .01; see Table 1).

Table 1.

Mean Performance of Healthy Control Participants and Clients After Stroke on DLOTCA Domains Before Mediation and Time by Group With Two-Sample t-Test Analyses

Mean Performance of Healthy Control Participants and Clients After Stroke on DLOTCA Domains Before Mediation and Time by Group With Two-Sample t-Test Analyses×

The quartile data provide the scoring for the 25%, 50%, and 75% quartiles. The 25% score means that a lower score should be considered as low performance that indicates possible deficit and should be further tested. Regarding Thinking Operations, it can be seen in the quartile columns that both groups improved, but the healthy control group received scores of 4.33, 4.60, and 5.00 for the 25%, 50%, and 75% quartiles, respectively, whereas the stroke client group received lower scores (3.70, 4.00, and 4.50; see Table 1).

In addition, we performed Pearson correlation analysis on the DLOTCA domain scores before mediation to study the relationship of years of education with performance because the healthy control group had a wide range of years of education. For the healthy control group before mediation, we found moderate to high significant correlations ranging from .295 (p < .05) to .699 (p < .001); however, in the stroke client group, the significant correlations were mild to moderate, ranging from .203 (p < .05) to .449 (p < .01), probably as a result of the acquired brain injury that overshadowed the difference in education.

Performance in the Stroke Client Group Before and After Mediation.

Comparing the scores of the stroke client group before and after mediation for those who needed mediation revealed significantly high differences on all domains at ps < .01 and .001 (Table 2). Moreover, effect sizes were moderate to high in all domains, ranging from 0.35 for Thinking operations to 2.11 for visual Perception (Figure 1).

Table 2.

DLOTCA Domain Means Before and After Mediation With Paired t Tests Within the Stroke Client Group

DLOTCA Domain Means Before and After Mediation With Paired t Tests Within the Stroke Client Group×

The percentages of participants who needed mediation at each level of each domain are presented in Figure 2. As can be seen, stroke clients needed more mediation in the Visuomotor Construction and Thinking Operations domains. They needed higher levels of mediation in specific feedback (Level 3, specific feedback), structured category (Level 4, partial intervention), and reduced amount (Level 5, copying or subtracting amount; see Figure 2).

Figure 2.

Frequency of levels of mediation in the DLOTCA domains in the stroke client group.

Our findings support the use of the DLOTCA. We found high interrater reliability and good internal consistency reliability for almost all domains. The results partially support construct validity because the test significantly differentiated between clients after stroke and healthy control participants in the basic domains (Orientation, Visual and Spatial Perception, Praxis), but the higher cognitive domains (Visuomotor Construction and Thinking Operations) showed only a trend in favor of the healthy control participants.

Moreover, we found significant differences in the stroke client group before and after mediation, suggesting potential change and providing an indication of the level of mediation from which the person would benefit in intervention. These findings strengthen the original LOTCA’s psychometric properties (Averbuch & Katz, 2005) and further substantiate the dynamic procedure, which is a fundamental change and addition to the evaluation. Thus, the assessment provides the level of cognitive performance along the different domains as well as the potential for change for each individual and the starting point for intervention.

The levels of mediation used by the stroke clients suggest that for the domains of Perception and Praxis, Levels 1 and 2 (general intervention and general feedback) were enough for most participants to reach best performance. In the Visuomotor Construction and Thinking Operations domains, Levels 3 and 4 (specific feedback and partial intervention) were frequently needed, and in some cases, Level 5 (reduced amount) was needed, especially in the verbal mathematical questions (see Figure 2). These results support the hierarchy of cognitive skills that were the basis for the LOTCA development (Katz et al., 1989) and further support the notion that the higher the cognitive demands are, the more mediation is required to change and learn.

Pretest/static scores provide information on domains of abilities and difficulties as well as the level of difficulties

Mediation scores describe what level of mediation was most frequently used in which domain. This level should be used in the initial intervention process.

Posttest or after-mediation scores describe learning potential and the amount of improvement in each domain. They provide therapists with information on the learning potential of the person assessed. Learning potential is based on the discrepancy between pretest and posttest scores.

In addition, response time, or length of performance time, describes whether the person is able to perform the task over a longer time. The intervention approach will differ on the basis of response time.

Limitations and Future Research

We assessed interrater reliability on only 10 participants, and the number could be larger. The sample size is small for test standards, so the results have to be regarded as first indications. Healthy control participants had, on average, fewer years of education, a variable that has been found to be significantly correlated with most of the domains. Therefore, this group is not an ideal control group. The client population included only people after stroke; thus, the data cannot be generalized to other neurological groups.

Future research should use a larger sample size; collect data from additional groups, such as people with traumatic brain injury, Parkinson’s disease, multiple sclerosis, and schizophrenia; translate and adapt the DLOTCA for use in other countries and languages; and collect data on and compare the DLOTCA across cultures. Beginning work is already underway in Brazil and Chile; namely, the DLOTCA has been translated into Portuguese and Spanish, and data collection has begun. The DLOTCA has also been translated into Arabic, and data have been collected on clients with stroke and healthy control participants among the Arab population in Israel (Abdelqader, 2010). Results have shown findings similar to the ones reported here. The next major step in validation should include determining the DLOTCA’s predictive or ecological validity for daily functioning and participation in the community.

Acknowledgments

We thank the occupational therapists from Lowenstein Rehabilitation Center and Sheba Medical Center who helped in collecting the data for the development of the DLOTCA.

References

Abdelqader, W. (2010). DLOTCA assessment among patients following stroke: Validity and reliability in an Arab population. Thesis submitted to the School of Occupational Therapy, Hebrew University, Jerusalem.

Abdelqader, W. (2010). DLOTCA assessment among patients following stroke: Validity and reliability in an Arab population. Thesis submitted to the School of Occupational Therapy, Hebrew University, Jerusalem.×