Abstract

How can the consequences of self-enhancement (SE) be tested empirically? Traditional two-step approaches for investigating SE effects have been criticized for providing systematically biased results. Recently, we suggested condition-based regression analysis (CRA) as an approach that enables users to test SE effects while overcoming the shortcomings of previous methods. Krueger et al. (2017) reiterated the problems of previous two-step approaches and criticized the extent to which CRA could solve these problems. However, their critique was based on a misrepresentation of our approach: Whereas a key element of CRA is the requirement that the coefficients of a multiple regression model must meet two conditions, Krueger et al.’s argumentation referred to the test of only a single condition. As a consequence, their reasoning does not allow any conclusions to be drawn about the validity of our approach. In this paper, we clarify these misunderstandings and explain why CRA is a valid approach for investigating the consequences of SE.

Previous empirical studies addressing such questions have typically applied an analytical approach involving two steps. A discrepancy score (e.g., an algebraic difference or residual) between individuals’ self-view and their value on some criterion measure was computed in a first step, which was then correlated with an outcome variable in a second step. Researchers have repeatedly emphasized that this two-step approach is not appropriate for testing SE effects because it is systematically biased toward mistaking main effects of the self-view for effects of SE (e.g., Asendorpf & Ostendorf, 1998; Edwards & Parry, 1993; Griffin, Murray, & Gonzalez, 1999; Humberg et al., 2018; Krueger & Wright, 2011; Ullrich, 2009; Zuckerman & Knee, 1996). Recently, Humberg et al. (2018) introduced condition-based regression analysis (CRA) as an approach that can be applied to overcome the problems of prior two-step approaches. Unlike previous approaches, CRA enables users to test SE effects without mistaking the main effects of the self-view for SE effects.1

After their recap on the statistical problems of two-step approaches, Krueger et al. (2017) noted that CRA has been suggested2 as a way to solve these problems, but then they misrepresented the CRA approach and drew a conclusion that would be very unfortunate for the field: They concluded that CRA should be avoided because it suffers from the very same problems as discrepancy score correlations. Specifically, the authors first correctly introduced CRA by stating that two conditions must be tested to detect an SE effect but they then referred to only one of the conditions when evaluating the utility of CRA. Consequently, the argumentation that the authors provided cannot justify their conclusion about CRA. In this rebuttal, we clarify CRA, explain why the critique in Krueger et al. (2017) is unjustified, and explain why CRA does, in fact, solve the problems of prior two-step approaches that have been applied for investigating SE effects.

Note that Krueger et al.’s statistical considerations about discrepancy scores and about CRA constituted only one message presented in their paper. Entangled with their statistical critique, the authors also voiced concern about whether the typically applied conceptualization of SE as a comparison of the self-view and some criterion measure (e.g., a person’s self-view vs. this person’s rating of the average other; a person’s self-view vs. an objective assessment of the ability under consideration) could indeed capture the concept of SE that researchers have aimed to capture. As an alternative, they suggested an approach that categorizes individuals as making a “self-enhancement error” if they believe they have a higher-than-average standing on the ability (i.e., self-view > rating of the average other) when their true standing is in fact below average (i.e., objective ability < average ability in the sample). Our evaluation of these conceptual ideas is that they are worthy of further elaboration. However, these ideas are unrelated to the statistical question of how researchers whose conceptual definition of SE is best reflected in a discrepancy measure (e.g., algebraic difference or residuals) can validly test the association between SE and an outcome variable. We wrote the present article to show that, although Krueger et al. (2017) had indicated otherwise, CRA is an appropriate tool for achieving this aim. We therefore focus on the relevant methodological contents from Krueger et al.’s paper and refer readers who are interested in the authors’ conceptual ideas to the original article.

The CRA Approach for Testing SE Effects

Imagine that we want to test whether people are happier the more they overestimate their cognitive ability, that is, we want to test for a positive SE effect.3 We would assess participants’ self-ratings S about their ability, some reality criterion T (e.g., intelligence test scores), and happiness H as the outcome variable. We would need to specify what we mean by the term SE, and we could, for example, decide that our understanding of SE is best reflected by the algebraic difference (S-T) between a person’s self-rating and his or her criterion value.4 In this case, the question of a positive SE effect translates into the question of whether higher values on the discrepancy (S-T) tend to go along with higher values on H. As was repeatedly emphasized in the literature and also described in Krueger et al. (2017), the correlation of (S-T) with H might indicate an SE effect when it is in fact not present (e.g., see Griffin et al., 1999; Ullrich, 2009; Zuckerman & Knee, 1996) because, inter alia, Cor(S-T,H) is biased toward mistaking self-view main effects (i.e., an effect of S) for effects of the difference (S-T).

CRA was designed to solve this problem. It shows whether individual differences in H are associated with individual differences in (S-T), or, rather, “only” with individual differences in S. CRA is based on the linear regression model5 that predicts H from S and T:

The key element of CRA is that it states which conditions on the coefficients β1 and β2 must be satisfied before we can conclude that there is a positive SE effect. That is, the CRA approach states that the discrepancy (S-T) is positively related to the outcome H if and only if the following two conditions are satisfied (see p. 9 in Humberg et al., 2018):

For the intuitive derivation of these conditions, we made use of the fact that the coefficient β1 reflects the effect of S when T is controlled for, and β2 is the effect of T when S is controlled for.6 The first condition ensures that when comparing two people with equal values of ability T, the person who has the higher self-view and thus the higher value on the discrepancy (S-T) will be predicted to be happier. The second condition ensures that for two people with equal self-views S, the person who has the lower ability level, who is again the person with a higher SE value (S-T), will be predicted to be happier.

Both conditions in Equation 2 must hold before we know that there is a positive SE effect: When the first condition β1 > 0 holds but the second condition is violated because β2 > 0, the regression model would predict that for two people with the same self-view value, the person with the higher ability level would be happier (because β2 > 0), but this is in fact the person with lower SE. This situation would contradict a positive SE effect. Similarly, when β2 = 0, two people with the same self-view values are predicted to be equally happy, independent of their ability levels and thus independent of their levels of SE. When the first condition is violated (i.e., β1 = 0 or β1 < 0), the model predicts that two people with the same ability level are equally happy independent of their self-views and thus independent of their levels of SE (if β1 = 0), or it even predicts that the person with the lower self-view and thus the person with lower SE would be happier (if β1 < 0). All of these scenarios contradict a positive SE effect, indicating that testing only one of the two conditions is not sufficient to test for this effect (see also OSF Material B at osf.io/e8p5r).

The fact that CRA requires that two conditions must be tested before the user can conclude that there is a positive SE effect might seem counterintuitive at first glance. However, this requirement can be seen as the key property of CRA because it is the reason why CRA can be applied to overcome the problems of two-step approaches. Intuitively speaking, the two conditions ensure that both sources of information about individual differences in SE, namely, self-views and ability levels, are taken into account in the analysis. Thereby, contrasting CRA with previous approaches, the approach enables researchers to disentangle SE effects from main effects of the self-view.

Instead of testing the two intuitive conditions in Equation 2, an equivalent set of conditions can be tested. This set is less intuitive but provides statistical benefits (i.e., it avoids alpha error accumulation because only the first condition has to be significant, and the second condition has to hold only numerically; see Humberg et al., 2018; see Figure 1). Equivalent to the conditions in Equation 2, the CRA approach states that the discrepancy (S-T) is positively related to the outcome H if and only if the following two conditions are satisfied (see the theorem on p. 10 in Humberg et al., 2018; see OSF Material F at osf.io/e8p5r for the proof that the set of conditions in Equation 3 is equivalent to the set of conditions in Equation 2):

A visual representation of the test of SE effects with CRA. See osf.io/e8p5r for details on the significance test of abs (OSF Material G) and, for example, the R code that demonstrates CRA. The depicted strategy refers to the operationalization of SE as the algebraic difference S-T. For different operationalizations, the computation and test of abs must be adapted (see Table 4 in Humberg et al., 2018, see OSF Materials C, D, and E at osf.io/e8p5r).

Again, both conditions in Equation 3 must be considered before drawing a conclusion, because none of the conditions considered alone is sufficient for identifying a positive SE effect. For example, when only the second condition β1 – β2 > 0 is tested and satisfied, we only know that β1 is larger than β2. It could be the case (Case A) that β1 is positive and β2 is negative (indicating a positive SE effect), (Case B) that both β1 and β2 are positive (contradicting a positive SE effect), or (Case C) that both coefficients are negative (also contradicting a positive SE effect). If we considered the condition β1 – β2 > 0 to be sufficient to test for a positive SE effect, we would falsely claim support for such an effect in many cases when it is in fact not present (i.e., in Cases B and C).7 CRA prevents us from drawing such a false conclusion by requiring that we additionally test the condition abs > 0 because this information is needed to conclude whether Case A (if abs > 0; positive SE effect) or rather one of the latter cases (Case B or C if abs ≤ 0; no SE effect) is present.

To sum up, CRA enables researchers to test whether higher values on the discrepancy (S-T), which are often considered a proxy for SE, are associated with higher values on an outcome variable H. Testing for a positive SE effect with CRA involves the estimation of a regression model predicting H from S and T and simultaneously testing two conditions on its coefficients or on auxiliary parameters computed from the coefficients (either both conditions stated in Equation 2 or both conditions in Equation 3).

Besides algebraic difference scores, some other operationalizations of SE have been suggested in the literature. For example, researchers have suggested that SE might instead be reflected by residual scores (John & Robins, 1994) or by the Kwan index (which includes a way to control for rating tendencies; see Kwan et al., 2004). The decision about which SE measure is appropriate for the specific research question at hand should be based on conceptual considerations. In addition, one should take into account the potential pitfalls of discrepancy measures, which were discussed in the long-standing debate on the use of difference scores for measuring change (e.g., the influences of regression toward the mean; for an overview, see Campbell & Kenny, 1999). Whereas individual researchers should identify an SE measure that accurately reflects the constructs they are interested in, the basic principle of CRA allows them to test SE effects for a variety of possible operationalizations (see Table 4 in Humberg et al., 2018, see OSF Materials C, D, and E at osf.io/e8p5r). This principle is always the same as outlined above: The basic element is a multivariate regression model, and (at least) two conditions must be satisfied for its coefficients before it is possible to conclude that the chosen SE score is positively related to the outcome variable.

An Evaluation of Krueger et al.’s Critique of CRA

Krueger et al. (2017) called for an approach that unequivocally identifies SE effects. They reviewed the CRA approach and came to the conclusion that it does not achieve this aim. Their key argument was, basically, that if one considers the condition β1 – β2 > 0 as sufficient to test for a positive SE effect, then one would falsely claim support for such an effect in many cases when it is in fact not present. In the words of the authors, “the difference between the weights β1 and β2 is dependent on β1 much like S – T is dependent on S. … Self-enhancement effects [are] not disentangled from simple self-perception effects” (Krueger et al., 2017, p. 7).

These observations are correct, but they refer exclusively to the test of β1 – β2 and thus do not allow any conclusion to be drawn about the validity of CRA. CRA explicitly states that two conditions must be tested before any conclusion can be drawn about a positive SE effect. Moreover, the authors’ argument against CRA is fully consistent with one of the central insights provided by the CRA approach, namely, that testing only one condition (e.g., the condition on β1 – β2) is not sufficient for concluding that there is a positive SE effect. In addition to the condition β1 – β2 > 0, CRA requires abs > 0. However, the latter condition for abs, a key element of CRA, is not mentioned at all in Krueger et al.’s review and critique of CRA.

Krueger et al. (2017) mentioned the two conditions required to apply CRA at two points in their article. First, when the authors introduced CRA (p. 6), they initially provided the correct description, which is that, according to the CRA approach, a positive SE effect requires the two conditions β1 > 0 and β2 < 0 to hold. Afterwards, however, the authors presented the results of simulation studies to demonstrate that testing the difference β1 – β2 inherits the same problems as testing the correlation of (S-T) with H. Neither their simulations nor their logical reasoning referred to the set of two CRA conditions.89 This misrepresentation of CRA is also reflected in their statement that “Humberg et al. proposed to let … the difference between β1 and a negative β2 reflect the self-enhancement effect”10 (Krueger et al., 2017, p. 7). This statement is wrong because it is not the difference between the coefficients that is tested in the CRA approach. Second, Krueger et al. (2017) stated that “requiring both regression weights to be significant and of opposite signs, the multiple-regression approach may provide a conservative test for the presence of a self-enhancement effect” (p. 7). The validity of this statement depends on how one interprets the term “conservative”: The two conditions ensure that main effects of the self-view are not mistaken for SE effects. Thereby, the CRA approach is “conservative” in the sense that it prevents researchers from drawing systematic false-positive conclusions about their data as they would have done if they had applied any previous two-step approach. However, Krueger et al. (2017) did not integrate this observation into their final evaluation of CRA, but they falsely concluded that CRA was not sufficient for differentiating SE effects from main effects of the self-view. To sum up, although the authors mentioned the two key conditions of CRA at two points in their article, their reasoning exclusively referred to a statistical test that was different from and not equivalent to the CRA approach.

A second concern that Krueger et al. (2017) voiced about CRA refers to terminology: They stated that “the regression weight β1 is what most traditional researchers consider the self-enhancement effect” (p. 7). Independent of whether this claim is true or not, it is not related to the validity of CRA which tests whether individual differences in an outcome variable H are associated with individual differences in the discrepancy (S-T) (or in other discrepancy scores such as residuals or the Kwan index for the adapted versions of CRA, see Table 4 in Humberg et al., 2018, see OSF Materials C, D, and E at osf.io/e8p5r). Its ability to do so does not depend on the question of how many researchers have thus far been interested in this kind of effect. This being said, we doubt that most researchers have used the term “self-enhancement effect” to refer to self-view main effects while controlling for the criterion, but this is the effect reflected by the coefficient β1. As can be seen in Krueger et al.’s overview of previous research in this domain, for example, researchers typically operationalized SE as some kind of discrepancy score and stated that higher values on this score should be associated with higher outcome values. In all of these cases, the researchers’ methods have implied that they were interested in the effect of a discrepancy score and not in the main effect of self-view.11

To sum up, Krueger et al. (2017) questioned whether CRA can provide an appropriate test of SE effects, but their conclusion was based on a misrepresentation of this method as they referred to only one of two necessary conditions stated in the CRA approach. In fact, when applying CRA as originally described in Humberg et al. (2018), this approach actually solves the problems that Krueger et al. (2017) and many others before them have pointed out.

A More Detailed Explanation of Why CRA Solves the Problems of Previous Two-Step Approaches

We will now provide a more detailed explanation to illustrate that CRA is, in fact, an appropriate tool for investigating SE effects. The key problem of two-step approaches for investigating SE effects is that they lose information about subjects’ self-views and criterion levels when discrepancy scores are computed in the first step of the analysis (Griffin et al., 1999; Ullrich, 2009; Zuckerman & Knee, 1996). For example, two people with very different levels of self-view and ability (e.g., Ray with S = 10, T = 5, and Sam with S = 6, T = 1) can be assigned the same algebraic difference score (S-T = 5; analogous observations can be made for any other SE operationalization, see Humberg et al., 2018). As a consequence, in the second step of the analysis, one cannot tell whether a positive correlation of (S-T) and H reflects an effect of the discrepancy (S-T) or rather an effect of only one of its components S and T. This approach can, for example, falsely indicate that H is related to SE when there is in fact only a positive main effect of the self-view because the information that would be needed to identify the self-view effect is not available in the second step of the analysis.

Consider, for example, the simulated data in Table 1, which was constructed such that H = 2S + 0T. That is, H is positively related to S when T is controlled for, but H is unrelated to T when S is controlled for. To ease interpretation, we added no measurement error so that the data perfectly reflect the underlying effect. In this example, the two-step approach leads to a correlation of Cor(S-T,H) = .38, and one would conclude that there is a positive SE effect. However, the data in fact contradict such an effect: Although Kate (S-T = 4) has a higher SE value than Tom (S-T = 1), Tom has a higher value on the outcome variable (HTom = 12, HKate = 10). That is, the correlation of (S-T) and H falsely indicates a positive SE effect.

Table 1

Example Data Set Reflecting a Self-View Effect but no SE Effect.

Self-view S

Criterion T

Discrepancy score S-T

Happiness H

Ray

10

5

5

20

Sam

6

1

5

12

Tom

6

5

1

12

Kate

5

1

4

10

Note. Happiness H was constructed as H = 2S + 0T.

CRA solves this problem of the two-step approach. The reason is that CRA (a) preserves all information necessary to identify effects of the self-view and the criterion by including S and T as separate predictors in the regression model (H = β0 + β1S + β2T) and (b) also makes use of all of this information by stating that the common constellation of two parameters must be inspected (i.e., two regression coefficients or two auxiliary parameters) before deciding whether the data indicate a positive SE effect. These properties, which contrast CRA against all previous approaches, enable researchers to unequivocally identify SE effects without misinterpreting simple self-view or criterion main effects as effects of SE (see OSF Material B at osf.io/e8p5r for the mathematical proof).

When applying CRA in practice, the differentiation between self-view main effects and SE effects can be explicitly observed (see also p. 10 in Humberg et al., 2018): A positive self-view effect (controlled for potential effects of T) is present when the coefficient β1 of the self-view is positive (β1 > 0). This is the case for the example data in Table 1, where β1 = 2 and β2 = 0. For a positive SE effect, the two conditions “abs > 0 and β1 – β2 > 0” (Equation 3) must hold, where abs > 0 must be significant and β1–β2 > 0 must hold numerically. For the example data, abs = 0 is not significantly positive, revealing that the data are not in line with a positive SE effect. That is, CRA, in contrast to the two-step approach, correctly indicates that only the self-view but not SE is positively related to happiness (see Humberg et al., 2018, for a more detailed explanation of CRA; see Tables 5, 6, and 7 in Humberg et al., 2018, for more examples).

Conclusion

In contrast to Krueger et al.’s (2017) conclusion, CRA provides a valid test for identifying effects of SE. Krueger et al.’s (2017) flawed assertion can be traced back to a misrepresentation of CRA as their argument focused on testing a single condition on the parameters of the regression model. CRA, by contrast, states that two conditions must be satisfied to indicate such an effect. These two conditions ensure that SE effects are clearly differentiated from self-view main effects. Thus, CRA overcomes critical limitations of prior intuitive but systematically biased two-step approaches, which have been typically applied to analyze the consequences of self-enhancement.

Data Accessibility Statement

No data was analyzed for this article.

Notes

1For some discrepancy measures (e.g., algebraic difference scores), the two-step approach has the additional problem of mistaking simple main effects of the reality criterion for SE effects. For simplicity and to be consistent with Krueger et al. (2017), we will not delve into this problem. CRA solves both problems because it differentiates SE effects from the main effects of the self-view and also from the main effects of the criterion.

2The source “Humberg et al. (2017),” which Krueger et al. (2017) cite for CRA, is the advance online version of the source “Humberg et al. (2018),” which we refer to in the present article.

3Here, we focus on the hypothesis of a positive SE effect because Krueger et al. (2017) also considered this hypothesis in their article. Naturally, CRA can also be used to test for a negative effect of SE, and the respective procedure is outlined in Figure 1 and in Humberg et al. (2018).

4CRA can be adapted to fit other operationalizations of SE (see below).

5Estimation of the regression model that forms the basis of CRA requires the classical assumptions of multiple regression. Moreover, as is typical for multiple linear regression models, estimation problems can arise when the predictor variables are highly collinear. In most domains in which SE effects are investigated, self-views and objective criteria are moderately correlated (e.g., around .3 for intelligence, see Mabe & West, 1982; Zell & Krizan, 2014; between .20 and .35 for personality, see Back & Vazire, 2012; up to .13 for physical attractiveness, see Gabriel, Critelli, & Ee, 1994), such that multicollinearity should not occur in typical applications. We nonetheless recommend a pre-analysis to test for potential multicollinearity problems (e.g., by applying the variance inflation factor; VIF; Fox, 2016).

6CRA focuses on the associations of S and T with H, respectively, when the respective other predictor is controlled for (i.e., semi-partial correlation). By contrast, the raw correlations Cor(S,H) and Cor(T,H) are not relevant to the existence of an SE effect and are thus not considered in the CRA approach. Given that semi-partial correlations can differ from raw correlations (e.g., see Horst, 1941; Smith, Ager, & Williams, 1992), this implies that there can be an SE effect even if, for example, the raw correlation between T and H is positive (see also Humberg et al., 2018).

8To see that the set of CRA conditions “β1 > 0 and β2 < 0” is not equivalent to the single condition “β1 – β2 > 0,” consider the example values β1 = 5 and β2 = 3, which satisfy β1 – β2 > 0 but not the CRA conditions (because β2 > 0).

9Krueger et al. (2017) referred to another observation during their critique of CRA, namely, that the regression weights β1 and β2 can correlate across studies. It is unclear why they assumed that this observation would have any implication on the validity of CRA. A nonzero correlation between coefficient estimates is a statistical consequence when the predictors are correlated. Whereas one should rule out the possibility that collinearity could be too high to reliably estimate the regression coefficients in the multiple regression model (see Footnote 5), correlated regression weights do not per se affect the validity of CRA.

10The phrasing “negative β2” in this quote might indicate that the authors are suggesting that β2 < 0 should be tested before the difference β1 – β2 is considered (although this seems unlikely, given that the condition on β2 does not show up in combination with β1 – β2 in the rest of their critique). However, even if this were the case, the statement provides a misrepresentation of CRA because the conditions “β2 < 0 and β1 – β2 > 0” are not equivalent to the conditions stated for the CRA approach: “β1 > 0 and β2 < 0.” For example, the coefficients β1 = –2, β2 = –3 satisfy the first set of conditions but not the CRA conditions.

11The question of whether it was theoretically justified to expect such effects is yet another area of discussion (see Humberg et al., in press).

Funding Information

This article was supported by the Open Access Publication Fund of the University of Münster.