Intro

Yet while EUCS has been validated and found generalizable across several applications (Doll & Torkzadeh, 1988), it has not been validated with users of ERP systems.

This paper reports the results of a study that further examines the theoretical meaning, structure and dimensionality, and reliability and validity of EUCS when used to measure end-user satisfaction with ERP software applications.

ERP: Enterprise resource planning systems, also referred to as enterprise-wide systems or enterprise systems, offer the “seamless integration of all the information flowing through a company—financial and accounting information, human resource information, supply chain information, and customer information” (Davenport, 1998, p. 121) and present a holistic view of the business from a single information and information technology (IT) architecture (Gable, 1998). #

ERP penetration is 67%, and 15% of companies that do not have ERP today plan to implement it within the next 12 months (Scott

ERP PROBLEMS:

many ERP installations achieve only partial implementation.

only 10% of newi nformation systems failures can be attributed to technological problems (Bikson&Gutek, 1984)

12-item survey instrument that was a synthesis of the Ives et al. (1983) measure of UIS, and which is a widely used, validated, and generalizable instrument

EUCS is a multifaceted construct that requires subjective self-reports of five subscales that measure end-user satisfaction with the content, accuracy, format, timeliness, and ease of use of a computer application and a single overall second-order construct called EUCS.

Past research has demonstrated instrument validity (content validity, construct validity, and reliability [Straub, 1989]) as well as internal validity and statistical validity

NEED TO CROSS VALIDATE

Klenke (1992) highlighted the importance of cross-validation of measurement models and stressed the need to retest EUCS with different samples.

Since the instrument was established, a number of researchers have applied it to various advanced information technologies.

Responding to the call for “reinstating replication as a critical component of research” (Berthon et al. 2002), we believe EUCS as developed by Doll and Torkzadeh (1988) should be reinvestigated in light of emerging technologies with new data to demonstrate robustness of the measurement model.

DATA

Data used in this study were collected via a nationwide mail survey of users of ERP systems.

End users were asked to indicate the ERP module(s) they were using and to answer questions about their specific application(s)

Data came from 214 organizations with the number of respondents per organization ranging from a minimum of one to a maximum of three

TABLE 1: End User Characteristics

Seventy-nine per- cent had revenues of $500 million or more,

almost half (48%) had revenues exceeding $1 billion.

Approximately 32% were manufacturing organizations,

18% were financial services organizations, and

13% were utility organizations.

37% came from a variety of sectors, including education, insurance, retail, high tech, health care, or government.

Approximately 86% of the organizations implemented ERP systems regionally or nationally

14% reported global implementations

TABLE 2: Profile of end users.

Table 2 shows that end users were college educated with approximately 92% possessing bachelor’s or master’s degrees.

Typically, end users had used ERP systems for approximately 3 years.

RESULTS

TABLE 3

Doll and Torkzadeh’s (1988) 12-item instrument to measure EUCS along with descriptive statistics is shown in Table 3.

ERP systems are profoundly complex pieces of software that require large investments of money, time, and expertise (Davenport, 1998).

CFA PROCEDURE

The EQS 5.6 (Bentler, 1995) program with maximum likelihood estimation was used to estimate the confirmatory and structural equation models in this study.

We tested for multivariate normality of the observed variables to ensure observations were independently and identically distributed.

No significant outliers were detected.

Test for identification: The solutions converged at the same point each time, indicative of model identification

ALTERNATIVE MODELS

4 proposed by Doll et al. (1994)

Tested the fit of each hypothesized model to determine its consistency and applicability with the sample data of end users of ERP applications.

Comparison studies examined in this paper have assumed that a second-order factor structure applies, consisting of a single factor called EUCS.

Figure 1: Model 1: One first-order factor.Hypothesized one first-order factor (EUCS) accounting for all the common variance among the 12 items.

Figure 2: Model 2: Five first-order factors (uncorrelated).Hypothesized that the 12 items form into 5 uncorrelated or orthogonal first-order factors defined as content, accuracy, format, ease of use, and timeliness.

Figure 3: Model 3: Five first-order factors (correlated).Hypothesized that the five first-order factors are correlated with each other.

Figure 4: Model 4: Five first-order factors and one second-order factor.Hypothesized five first-order factors and one second-order factor (EUCS). The second-order factor EUCS exists but cannot be directly measured by indicator variables. It can only be inferred from the first-order factors, which in turn are measured by their respective indicator variables.

Model 4 is the only model that has been tested in previous studies over the past decade

TABLE 4: Characteristics of the previous validation studies with EUCS

The CFA estimation proceeded in a two-step approach in which the confirmatory factor models were tested prior to testing the structural model.

TABLE 5: The results of testing the four models

Six common model-fit measures, assess each model’s overall goodness of fit.

Hair et al. (1998) χ2 is appropriate for sample sizes between 100 and 200, with the significance test becoming less reliable with sample sizes outside this range.

The null model serves as a good baseline model against which to compare alternative models for purposes of evaluating the gain in improved fit, and to establish a zero point for the normed fit index (NFI)

RESULTS

Doll and Xia (1997) found that neither models 1 nor 2 were even close to being considered a good fit with the sample data. Our results concur with their observations. Model 3 either, model 4 is the best fit.

TABLE 6

Compares model 4 goodness-of-fit indices from the present study with those of previous studies.

In sum, the absolute indexes (GFI, AGFI, and RMSR) in this study compare favorably with the values reported by the other studies.

Convergent validity was evaluated using three criteria recommended by Fornell and Larcker (1981):

all indicator factor loadings (λ) should be significant,

construct reliabilities should exceed .80, and

average variance extracted (AVE) by each construct should exceed .50.

RESULTS

The λ-values for all scale items in the CFA models were significant at p ≤ .001

Convergent validity is established if the loadings of the measures to their respective constructs are at least .60 (Bagozzi & Yi, 1988).

All of the original 12 items demonstrated loadings of .62 or greater and are statistically significant (t > |2.00|)

R-square, the proportion of the variance in the observed variables that is accounted for by its latent variable, ranges from 0.39 to 0.88 providing evidence of acceptable reliability for all individual items

The structural coefficients exceed .70 and are significant, indicating good construct validity of the latent factors comprising the EUCS construct.

R-square values range from .62 to .95, providing evidence of acceptable reliability for all factors.

GENERAL RESULTS

The results are in consonance with the other studies and support the structural equation model and underlying theory.

A higher-order EUCS factor is confirmed as accounting for or explaining all variance and covariance related to the first-order factors in capturing end users’ satisfaction with ERP systems.

The replication of these results enhances our confidence in the generalizability of EUCS, and its robustness as a measure of UIS.

IMPLICATIONS AND CONCLUSIONS

Our study shows that the EUCS instrument maybe used to evaluate ERP systems in organizations.

In addition, the instrument provides not only an overall assessment of end user satisfaction, but also the capability to an-alyze which aspects of ERP implementation efforts are most problematic.

The magnitude of path coefficients provides useful insights into the relative impor-tance of each subscale of EUCS and thus the major areas of satisfaction or dis-satisfaction with the use of a givenERPapplication. Managers could focus on those factors as significant contributors to overall satisfaction to improve ERP system effectiveness.

Similar to other studies, content and format had the highest loadings, indicating that these two play a critical role in EUCS. The third highest loading in this study was for ease of use.

the results also underline the need for designing highly effective user documentation or providing additional training to end users.

top five important IS success factors indicated by the IS managers to be accuracy of output, reliability of output, relationship between users and the IS staff, user’s confidence in the system, and the timeliness of output.

For researchers, the major contribution of this study lies in the area of measurement by rigorously validating EUCS (Doll & Torkzadeh, 1988) and thus enabling researchers to use the EUCS instrument with increased confidence

In sum, as a result of our study, EUCS can be better understood and applied as a standardized measure of advanced information tech- nologies, and can provide a summary evaluation for researchers and a means of formally evaluating ERP implementations for practitioners.

Abstract

Orgs make sig investments in ERP (Enterprise Resource Planning)

Investments in ERP depends on:

supporting effective use of information technology (IT)

satisfying IT users (Most important determinant of the success of ERP)

Sample of 407 end users of ERP systems

working within the framework of confirmatory factor analysis (CFA),

Aim: this study examines the structure and dimensionality, and reliability and validity of the end-user computing satisfaction (EUCS)instrument posited by Doll and Torkzadeh (1988).

In response to Klenke (1992) motion to cross-validate management information system (MIS) instruments and to retest the end user computing satisfaction instrument using new data

this study’s results, consistent with previous findings:

confirm that the EUCS instrument maintains its psychometric stability when applied to users of enterprise resource planning application software.

Implications of these results for practice and research are provided. #