Abstract

Background:
Executive functions are among abilities which school children require for learning in the future and deficit in executive functions in preschool children can continue into the older age and leads to serious problems in children in relation to doing their homework and other personal affairs.

Objectives:
The objective of the present study is to determine the validity, reliability and factor structure of the per-school version of behavioral rating inventory of executive functions (parent’s form) in Iranian children.

Patients and Methods:
The present study comprised 592 children aged from 2 - 5 years selected from pre-school centers of the city of Isfahan in 2013 - 2014 using cluster random sampling method, with their parents answering the questions asked in this inventory. The correlation coefficient among items with a total score of factors, Cronbach’s alpha coefficient, confirmatory factor analysis, and the correlation coefficient among the subscales were used to measure the reliability and internal consistency of the inventory.

Results:
Confirmatory factor analysis confirmed embedding items and the five-factor structure of the inventory including inhibition, shift, and emotional control, working memory and planning. In addition, Cronbach’s alpha coefficient was at satisfactory level for each of the factors and the total score of the scale (≥ 0.60).

Conclusions:
In general, it can be concluded that the behavior rating inventory of executive function (BRIEF) for preschool-aged children is a reliable and valid instrument for measuring executive functions of Iranian children, and can be used as a suitable means for psychological research and clinical situations.

1. Background

Executive functions is an inclusive term for various hypothetical cognitive processes including planning, working memory, inhibition, attention, self-control, and self-regulation which of executed by prefrontal areas of the frontal lobe (1-3). Some studies indicate that the deficit in executive functions in preschool children can persist across higher ages and confront children with serious problems in doing their homework and personal affairs (4). Therefore, clinical experts are interested in helping parents, teachers, and clients in different environments to contribute to treating executive dysfunctions, particularly neurodevelopment disorders. These include speech and language disorders, brain injuries via identifying different elements and components of executive actions (5), children with cancer (6), attention deficit hyperactivity disorder (ADHD) (7), Autism spectrum disorders (ASD) (8), and other disorders of childhood and adolescence. Accordingly, it deemed necessary to measure these variables in clinical non-laboratory, using reliable and valid instruments (9).

Since at least 30 structures can be placed under the term “executive functions”, their practical definition is difficult, and various terms have been used to define executive functions. Some authors have tried to define the concept of “executive functions” using models which cover a range of different components. Therefore, researchers and therapists apply different methods for measuring this concept and so far, many instruments and scales have been designed and developed for measuring executive functions (1, 10).

The cambridge neuro-psychological test automated battery (CANTAB) (1990) represents a group of tests which are appropriately combined with standardized cognitive tests and have a new and innovative format. Another instrument is cognitive assessment system-second edition (CAS2) (1, 11) which includes neuropsychological components such as goal-oriented attention, impulse control, cognitive flexibility, visual planning and organization, and the division of attention. Moreover, the continuous performance test is subset of the WISC-IV, visual attention tests of the NEPSY-II, and Delis and Kaplan’s psychological subtest of drawing and designing lines are used for evaluating executive functions (1, 6, 12). Furthermore, one of the behavioral rating scales is the Barkley deficits in executive function scale-children and adolescents (BDEFS-CA) (2012) designed for measuring behaviors with executive functions by parents of children aged 6 - 17 years-old and includes five secondary scales including self-management to time, self-organization, self-restraint, self-motivation, and self-regulation of emotion. Another scale, the Delis rating of executive functions (D-REF) is a set of rating for measuring executive functions in 5 - 18 year-old individuals in three forms of parents, teachers, and individuals themselves, each containing 36 items. In addition, the comprehensive executive function inventory (CEFI) (3, 8), is an instrument for evaluating observed behaviors related to executive functions. This inventory is to be complemented by parents or teachers for children aged from 5 - 18 years (12, 13).

One of the first and most valid instruments for measuring executive functions is behavior rating inventory of executive function (BRIEF), (14). The motivation for developing this inventory for authors in 1994 was to create an agreement between contradictory reports of parents and teachers from the daily functions of children at home and at school that presented a performance indicator (such as tests) from executive functions. At the time, several performance indicators were available for children and adolescents, which did not provide a suitable scale or structural method for evaluating executive functions, and there were few published studies conducted on executive functions in children. Although performance tests attempted to measure executive functions implicitly and specifically, different confounders caused the limitation of ecological validity and its generalizability. It has been argued that neuropsychological tests are insufficient for the assessment of executive functions because they artificially and ambiguously degrade an integrated system (1). Non- integrated, multidimensional and relative indicators are evaluated during a short-term temporal framework. These are based on personal performance in executive functions system and prioritized according to decision making required in the real world (14) the developers of the BRIEF found that collecting structured observations from parents and teachers enjoy potential efficiency. This behavioral assessment approach is not replaced by the traditional performance indicator, and can be used as a criterion of ecological validity in clinical and laboratory situations. The framework of designing the BRIEF was based on literature review on executive functions during life with particular attention to developmental models in this framework (15). The authors of the BRIEF considered diverse components in the framework of executive functions. Most of models on these variables such as inhibition of dominant responses, rivals’ actions and distracting stimuli, flexible shifting of cognitive set or strategies of solving problems as necessary, beginning the goal-oriented behavior, planning and organizing information and problems solving behavior, and the capacity of working memory which has a fundamental role in active and timely retention of information and are used for problem solving (16). Another important point is that executive functions include behavioral and emotional control, i.e. hot executive functions (11, 15), and are not limited to cognition or cold executive functions. (11, 15).

For developing the initial test, the BRIEF items were extracted by interviews with parents and teachers, and agreement was made regarding the least covering of domains of executive functions. Furthermore, common rating scales such as behavior assessment system for children (BASC) and child behavior checklist (CBCL) were sued to prepare the BRIEF, and related experts commented on the domains. Finally, indicators were developed, adjusted and studied and their validity was calculated over a period of 6 years. The first version of the BRIEF was then published in 2000. After the first version of the BRIEF, other version comparable to the BRIEF-P for pre-school children were prepared and published. So far, different versions of the BRIEF have been used which are translated into various languages.

2. Objectives

With regard to the foregoing issues, it deemed necessary to have valid and suitable instruments for assessing executive functions in pre-school children of Iran and other countries. Therefore, the aim of the present study was to investigate the behavior rating inventory of executive functions, using its translated version, in Iranian pre-school children and to determine the psychometric indicators of this inventory in the Iranian sample.

3. Patients and Methods

3.1. Population, Sample, and the Method of Conducting the Research

The present study was conducted using a descriptive-correlational method. The population of the present study included all children aged between 3 and 5 years registered in pre-school centers of Isfahan city in the academic year 2014 - 2015. In most studies the minimum sample size considered for factor analysis is 10 - 20 samples per each variable. Therefore, the present study included 650 participants selected by cluster random sampling method. In this context, initially of 14 regions of the education organization 4 regions, each having 30 pre-school centers were randomly selected. From each of these 4 regions, 20 pre-school centers were selected randomly. Finally, by simple random sampling 600 children entered this study where, after eliminating invalid questionnaires, analyses were conducted on 592 of children aged from 5 - 13 years consisting 342 boys and 250 girls.

The Persian version of the original BRIEF for pre-school children (BRIEF-P) was translated into Persian by the authors of the present study and was subsequently revised, amended and back-translated by some child psychologist and English translators. Then, by an initial study, its psychometric properties were obtained using a small sample of 60 participants. During this preliminary study, the translation of some of the items were revised, and the final translated version was investigated and confirmed by three experienced child psychologists in terms of formal and content validity. In the next stage, after coordinating with pre-school centers, the parents of pre-school children in each center were invited, received a copy of the BRIEF and were informed about the research project. The parents and pre-school centers not willing to cooperate were excluded from the study and replaced with randomly selected pre-school children from other centers.

3.2. Instrument

The preschool version (BRIEF-P) of behavior rating inventory of executive function inventory was developed by Gioia et al. (17) and used for evaluating and screening executive functions in pre-school children, and included 63 items for children aged 2 - 5 years and 11 months. It assesses five domains of executive functions consisting of inhibition (11 items), shift (10 items), emotional control (10 items), working memory (17 items), and planning and organization (10 items). The time required to complete this questionnaire is 10 - 15 minutes, and the minimum literacy requirement for respondents is elementary school degree. Parents and teachers answered the questions related to children as mostly (3), sometimes (2), and never (1). These indicators were calculated and presented as t-scores, percentage, and confidence interval of 90% for indicators. Gioia et al. (17) reported Cronbach’s alpha coefficients for scales in the parents’ study as follows: inhibition (0.90), shift (0.85), emotional control (0.86), working memory (0.88), planning and organization (0.80), and total indicator (0.95). These coefficients for teachers are as follows: inhibition (0.94), shift (0.90), emotional control (0.91), working memory (0.94), planning and organization (0.97), and total indicator (0.97). The validity and reliability of the inventory for parents were calculated in the present study (17).

3.3. Data Analysis

To measure the reliability of the BRIEF-P, the internal consistency of items was calculated by Cronbach’s alpha coefficient and the Pearson correlation coefficient of each item was calculated by related sub-scales. In addition, content and construct validity of this inventory is confirmed using confirmatory factor analysis and by answering the question of whether the five-factor structure attested the original questionnaire.

4. Results

The demographic characteristics of the studied population are demonstrated in reported in the Table 1.

Factorial validity is a form of construct validity obtained via factor analysis which is confirmatory or explanatory, and used to find the underlying constructs of a phenomenon or summarizing a set of data. In confirmatory factor analysis, the researchers’ aim is to determine a particular factor analysis. Since the number of BRIEF factors has been identified by its developers, this inventory is considered as a standard instrument. Therefore, in the present study, the factorial structure of the inventory was investigated using confirmatory factor analysis. Test for goodness of fit of absolute Chi square (χ2), (GIF), and comparative fit index (CFI) and Tucker Lewis index (TLI) and parsimony goodness-of-fit index (PGFI) including The root mean square error of approximation (RMSEA) and (PCFI) and (PNFI) were used to investigate the factor fit. If the value of RMSEA is less than 0.08 and the values of GFI, CFI, and TLI are higher than 0.90, and those of PCFI and PNFI are higher than 0.5, these values would indicate the acceptable model fit. In the present study, the tests of the primary model fit related to factors, did not favor appropriate fit. Therefore, regarding the proposed amendments of the software, some of the amendments were conducted and items 13 (inhibition) and 26 (shift) which did not have significant factor loadings, were excluded and then, fit statistics were investigated (Table 2).

The chi square statistic in the factor structure has significance levels less than 0.05 in all factors. Since this statistic is sensitive to the sample size, and is significant in high sample size, as well as regarding the fact that other fit indices suggest acceptable model fit, it can be concluded that the fit statistics of the modified model of items on factors indicate the desired theoretical structure of the scale.

To assess the concept that all components of the inventory are placed in the framework of a concept called executive functions, factor analysis of factors was conducted by the general structure of the inventory. As observed in the Table 3, most fit indices, except PCFI and PNFI, indicate relatively favorable fit of factors with the general structure of the scale.

As observed in the Table 4, the lowest correlation coefficient is between the items of the sub-scale intention with the total score of the sub-scale related to 0.04 of item 13, and Cronbach’s alpha coefficient of this sub-scale increases from 0.80 - 0.82 by deleting this item. In addition, item 26 has a very small correlation coefficient with 0.05 of the total score of the sub-scale of shift, and deletion of this item increases Cronbach’s alpha of this sub-scale from 0.69 - 0.73. Therefore, regarding the foregoing results, and also the results of confirmatory factor analysis, items 13 and 26 are excluded as inappropriate items from the questionnaire.

Table 4. Correlation Coefficients of Items With Total Score of Each Sub-Scale and Cronbach’s Alpha In Case Of Deleting Each Item

Item

Correlation Coefficient

Cronbach’s Alpha

Inhibition

1

0.34

0.80

2

0.29

0.80

3

0.37

0.79

4

0.45

0.79

5

0.50

0.78

6

0.47

0.79

7

0.49

0.79

8

0.55

0.78

9

0.50

0.79

10

0.48

0.79

11

0.43

0.79

12

0.48

0.79

13

0.04

0.82

14

0.39

0.79

15

0.34

0.80

16

0.43

0.79

Shift

17

0.48

0.64

18

0.43

0.66

19

0.41

0.66

20

0.50

0.64

21

0.31

0.68

22

0.24

0.69

23

0.44

0.66

24

0.45

0.66

25

0.32

0.68

26

0.05

0.73

Emotional Control

27

0.53

0.77

28

0.48

0.78

29

0.63

0.76

30

0.57

0.76

31

0.55

0.77

32

0.56

0.77

33

0.34

0.79

34

0.45

0.78

35

0.26

0.80

36

0.35

0.79

Working Memory

37

0.49

0.89

38

0.50

0.89

39

0.59

0.88

40

0.62

0.88

41

0.57

0.88

42

0.62

0.88

43

0.58

0.88

44

0.60

0.88

45

0.64

0.88

46

0.50

0.89

47

0.56

0.88

48

0.51

0.89

49

0.50

0.89

50

0.38

0.89

51

0.48

0.89

52

0.53

0.88

53

0.57

0.88

Planning/Organization

54

0.46

0.77

55

0.40

0.77

56

0.42

.77

57

0.51

0.76

58

0.41

0.77

59

0.49

0.76

60

0.40

0.77

61

0.38

0.77

62

0.57

0.75

63

0.51

0.76

As reported in Table 5, there is direct and significant correlation between all the sub-scales of the questionnaire and the total score of correlation coefficients. Further, Cronbach’s alpha of all scales are higher than 0.7, thus, having favorable internal consistency.

Table 5. Correlation Coefficients Among Sub-Scales and Cronbach’s Alpha and Their Descriptive Indices

5. Discussion

Executive functions are a set of processes responsible for guiding, leading, and directing cognitive, emotional and behavioral actions particularly in case of solving new problems (18). To date, numerous instruments and inventories have been designed and developed for assessing executive functions. One of these instruments is the behavior rating inventory of executive function-preschool version (BRIEF-P) (17) which has been translated into different languages in the world. In the present study, the psychometric properties of parents’ form of this inventory were investigated for children aged from 2 - 5 years in Isfahan. To investigate the construct validity of the inventory, confirmatory factor analysis was employed. Chi-square test is frequently used to investigate the goodness of fit in confirmatory factor analysis. The suitable fit of the inventory is indicated by insignificant chi-square coefficient (P < 0.05). In the present study, chi-square statistic was significant which is due to the large research sample size.

Tanaka (19) posits that the larger sample size increases the strength of the study and make chi-square statistic strong, with increasing statistical significance, suggesting the concurrent use of other fit statistics (20). Other absolute, parsimony and comparative fit indices obtained from confirmatory factor analysis indicate appropriate fit of the items of the inventory with related factors, except items 13 and 26 which had no significant factor loadings with the related factors. In addition, correlation coefficients between items with the total score of factors and Cronbach’s alpha of factors in case of deleted items were investigated. The results indicated that items have satisfactory correlation coefficient more than 0.40 with related factors, and only items 13 and 26 had low correlation coefficient with related factors. Therefore, Cronbach’s alpha increases by deleting these items (9), which is consistent with the findings of the present study, indicating that some of the items of the BRIEF-P have not favorable factor loading with the final factor, therefore, were deleted from the questionnaire.

In analyzing the questionnaire of present study, the items 13 and 26 were vague for parents and were not significant in the statistical analysis. For instance, item 13, has low correlation coefficient with the total score of the final factor, and deleting it cause increase in the Cronbach’s alpha coefficient. Therefore, item 13 like item 26 were deleted from the questionnaires as inappropriate items. Ultimately, deleting items 13 and 26 in the questionnaire increased Cronbach’s alpha coefficient, and also compensated the lack of statistically significant in confirmatory factor analysis.

The values of Cronbach’s alpha as 0.7 and higher indicate the internal consistency of the inventory. In the present study, the Cronbach’s alpha coefficients of the sub-scales ranged from 0.73 - 0.82 and the total Cronbach’s alpha coefficient of the inventory was 0.96; therefore, the questionnaire enjoys satisfactory internal consistency for Iranian children. Moreover, the correlation coefficients of sub-scales and the total number were investigated. Moderate correlation coefficients (0.20 - 0.70) indicate the independence of sub-scales from each other. This result confirms the multifactorial nature of the inventory. Further, confirmatory factor analysis was conducted for investigating the fit of the five factors with the general structure of the inventory of the BRIEF-P/parents’ form. The results indicate the acceptable model fit. As a result, the above-mentioned factors accurately measure the underlying structure of the inventory. In general, considering the results obtained, it can be concluded that the BRIEF-P/parents’ form consists of five factors of inhibition, shift, emotional control, working memory, and planning/organization. In assessing the general structure, executive functions enjoy acceptable validity. These results are consistent with the findings of Gioia et al. (17), the developers of the inventory and other researchers such as Mashhadi et al. (9), Bonillo et al. (21), Ezpeleta et al. (22), and Duku and Vaillancourt (23). The consistency of the results of the present study with those of foregoing investigations indicates the acceptable validity of the instrument in different countries and communities, which can be used as a global instrument for evaluating children’s executive functions in real-life situations and everyday life (24, 25).

Since the approach adopted in this instrument refers to behavioral assessment and conducted using structured observation of parents, it does not suffer from traditional performance instruments such as the continuous performance test (17, 26), the Wechsler cancellation sub-test and the NEPSY-II visual attention sub-test. The administration of this instrument is less time consuming compared with performance tests. In addition, it is amenable to easy scoring and interpretation. Therefore, it can be used as an index of biological validity and as a complementary inurement along with performance instruments in clinical and laboratory situations in order to obtain a compressive evaluation of children’s executive functions (10, 15, 27).

In general, the results of the present study indicate the acceptable validity of the BRIEF-P/parents’ from. In other words, it can be used for evaluating the executive functions of children aged from 2 - 5 years and 11 months in clinical situations as suggested by related researches. In this research, the reliability of the inventory was calculated only by using the Bisection method. Therefore, it is recommended to calculate the reliability by the inter-rater method in future studies. Furthermore, further studies are warranted for calculating the diagnostic validity of the normal and abnormal samples, and to determine the convergent and divergent validity of the inventory using other related instruments and concepts. The sample of the present study only covered children aged from 2 - 5 years and 11 months enrolled in pre-school centers of Isfahan city. Therefore, generalizing the results of the present study to children who have not entered pre-school centers and the children of other urban and rural areas should be done with caution. Conducting this inventory at national level and its comparison between different cities and urban and rural areas, and also different ethnic groups are other suggestions for further studies.

Acknowledgements

The authors would like to thank the parents and staff members of the pre-school centers in Iran for their assistance with data collection. This study was conducted as part of the first author’s PhD thesis.

Footnote

Authors’ Contributions:
Ahmad Abedi, Ali Akbar Ebrahimi, Salar Faramarzi, Aziz Kassani, and Rostam Menati contributed to conceiving and designing the study. The data was collected by Ahmad Abedi, Ali Akbar Ebrahimi, Ahmad Yarmohammadian and Salar Faramarzi. The gathered data analyzed and interpreted jointly by Aziz Kassani, Ahmad Abedi and Rostam Menati. All authors contributed equally in writing the manuscript and had an equal share in revising and approving the manuscript.