Bibliometrics

Grantee Research Project Results

This is a bibliometric analysis of the papers prepared by intramural and extramural
researchers of the U.S. Environmental Protection Agency (EPA) on topics related
to drinking water (DW). For this analysis, 691 papers were reviewed. These
691 papers, published from 1994 to 2005, were cited 8,334 times in the journals
covered by Thomson’s Web of Science1. Of these 691 papers, 567 (82%) have
been cited at least once in a journal.

The analysis was completed using Thomson’s Essential Science Indicators
(ESI) and Journal Citation Reports (JCR) as benchmarks. ESI provides access
to a unique and comprehensive compilation of essential science performance
statistics and science trends data derived from Thomson’s databases.
The chief indicators of output, or productivity, are journal article publication
counts. For influence and impact measures, ESI employs both total citation
counts and cites per paper scores. The former reveals gross influence while
the latter shows weighted influence, also called impact. JCR presents quantifiable
statistical data that provide a systematic, objective way to evaluate the world’s
leading journals and their impact and influence in the global research community.

Summary of Analysis

More than one-quarter of the drinking water publications
are highly cited papers. A review of the citations indicates that 178 (25.8%) of the drinking
water papers qualify as highly cited when using the ESI criteria for the top
10% of highly cited publications. Nineteen (2.8%) of the drinking water papers
actually qualify as highly cited when using the criteria for the top 1%, and
2 (0.3%) of these papers qualify as very highly cited when using the criteria
for the top 0.1%.

The drinking water papers are more highly cited than
the average paper. Using
the ESI average citation rates for papers published by field as the benchmark,
in 9 of the 13 fields in which the EPA drinking water papers were published,
the ratio of actual to expected cites is greater than 1, indicating that the
drinking water papers are more highly cited than the average papers in those
fields.

Nearly one-third of the drinking water papers are published in very high impact
journals.
Two-hundred two (202) of 691 papers were published in the top 10% of journals
ranked by JCR Impact Factor, representing 29% of EPA’s drinking water
papers. Sixteen percent (110 out of the 691 papers) of the drinking water papers
are published in the top 10% of journals ranked by JCR Immediacy Factor.

Seven of the drinking water papers qualify as hot papers. Using the hot paper
thresholds established by ESI as a benchmark, 7 hot papers, representing 1%
of the drinking water papers, were identified in the analysis.

The 691 drinking water papers reviewed for this analysis covered 13 of the
22 ESI fields. The distribution of the papers among these 14 fields and the
number of citations by field are presented in Table 1.

Table 1. Drinking Water Papers by ESI Fields

No. of Citations

ESI Field

No. of EPA DW Papers

Average Cites/Paper

2,162

Environment/Ecology

193

11.20

1,743

Pharmacology & Toxicology

129

13.51

1,298

Chemistry

77

16.86

1,095

Biology & Biochemistry

109

10.05

818

Engineering

115

7.11

545

Clinical Medicine

22

24.77

422

Immunology

23

18.35

142

Neuroscience & Behavior

10

14.20

79

Mathematics

6

13.17

16

Plant & Animal Science

3

5.33

8

Agricultural Sciences

2

4.00

3

Computer Science

1

3.00

3

Physics

1

3.00

Total = 8,334

Total = 691

There were 178 (25.8% of the papers analyzed) highly cited EPA drinking water
papers in eight fields—Biology & Biochemistry, Chemistry, Clinical
Medicine, Engineering, Environment/Ecology, Immunology, Mathematics, and Pharmacology & Toxicology—when
using the ESI criteria for the top 10% of papers. Table 2 shows the number
of EPA drinking water papers that met the top 10% threshold
in ESI. Nineteen
(2.8% of the papers analyzed) of these papers qualified as highly cited when
using the ESI criteria for the top 1% of papers. These 19 papers covered seven
fields—Engineering, Pharmacology & Toxicology, Environment/Ecology,
Mathematics, Clinical Medicine, Chemistry, and Biology & Biochemistry.
Table 3 shows the number of EPA papers in those seven fields that met the top
1% threshold in ESI. There were 2 (0.3% of the papers analyzed) very highly
cited EPA drinking water papers in two fields—Engineering and Pharmacology & Toxicology.
These two papers met the ESI criteria for the top 0.1%
of papers.

Table 2. Number of Highly Cited Drinking Water Papers by Field (top 10%)

Citations

ESI Field

No. of Papers

Average Cites/Paper

% of EPA Papers in Field

1,584

Environment/Ecology

62

25.55

32.12%

1,054

Chemistry

33

31.94

42.86%

907

Pharmacology &Toxicology

20

45.35

15.50%

669

Engineering

40

16.72

34.78%

427

Biology & Biochemistry

13

32.85

11.93%

387

Clinical Medicine

3

129.00

13.64%

174

Immunology

3

58.00

13.04%

78

Mathematics

4

19.50

66.67%

Table 3. Number of Highly Cited Drinking Water Papers by Field (top 1%)

Citations

ESI Field

No. of Papers

Average Cites/Paper

% of EPA Papers in Field

149

Engineering

6

24.83

5.22%

348

Pharmacology & Toxicology

4

87.00

3.10%

190

Environment/Ecology

3

63.33

1.55%

53

Mathematics

3

17.67

50.00%

322

Clinical Medicine

1

322.00

4.55%

98

Chemistry

1

98.00

1.30%

7

Biology & Biochemistry

1

7.00

0.92%

The citations for the 19 highly cited papers are presented in Table 4, and the citations for the 2 very highly cited papers are listed in Table 5.

The expected citation rate is the average number of cites that a paper published
in the same journal in the same year and of the same document type (article,
review, editorial, etc.) has received from the year of publication to the present.
Using the ESI average citation rates for papers published by field as the benchmark,
in 9 of the 13 fields in which the EPA drinking water papers were published,
the ratio of actual to expected cites is greater than 1, indicating that the
EPA papers are more highly cited than the average papers in those fields (see
Table 6).

Table 6. Ratio of Average Cites to Expected Cites for Drinking Water Papers by Field

ESI Field

Total Cites

Expected Cite Rate

Ratio

Biology & Biochemistry

1,095

1,063.04

1.03

Chemistry

1,298

481.71

2.69

Clinical Medicine

548

230.32

2.38

Engineering

818

269.25

3.04

Environment/Ecology

2,162

1045.76

2.07

Immunology

422

308.69

1.37

Mathematics

79

9.91

7.97

Neuroscience & Behavior

142

147.59

0.96

Pharmacology & Toxicology

1,743

1013.34

1.72

Plant & Animal Science

16

16.19

0.99

Agricultural Sciences

8

8.27

0.97

Computer Science

3

0.53

5.66

Physics

3

8.03

0.37

JCR Benchmarks

The Impact Factor is a well known metric in citation analysis. It is a measure
of the frequency with which the average article in a journal has been cited
in a particular year. The Impact Factor helps evaluate a journal’s relative
importance, especially when compared to others in the same field. The Impact
Factor is calculated by dividing the number of citations in the current year
to articles published in the 2 previous years by the total number of articles
published in the 2 previous years.

Table 7 indicates the number of drinking water papers published in the top
10% of journals, based on the JCR Impact Factor. Two-hundred two (202) of 691
papers were published in the top 10% of journals, representing 29% of EPA’s
drinking water papers.

Table 7. Drinking Water Papers in Top 10% of Journals by JCR Impact Factor

EPA DW Papers in that Journal

Journal

Impact Factor

(IF)

JCR IF Rank

37

Environmental Science & Technology

3.592

487

30

Applied and Environmental Microbiology

3.820

418

24

Environmental Health Perspectives

3.408

538

18

Analytical Chemistry

5.250

248

10

Journal of Analytical Atomic Spectrometry

3.200

605

9

Chemical Research in Toxicology

3.332

555

9

Journal of Infectious Diseases

4.481

311

8

Mutation Research-Fundamental and Molecular
Mechanisms of Mutagenesis

3.433

530

7

Infection and Immunity

3.875

403

6

Carcinogenesis

4.663

292

5

Epidemiology

4.220

350

4

Journal of Virology

5.225

251

4

Journal of Clinical Microbiology

3.489

519

3

Mutation Research-Reviews in Mutation Research

5.783

210

3

International Journal of Epidemiology

3.289

575

2

Emerging Infectious Diseases

5.340

240

2

American Journal of Epidemiology

4.486

310

2

TrAC–Trends in Analytical Chemistry

3.539

502

1

New England Journal of Medicine

34.833

5

1

Nature Medicine

30.550

9

1

Chemical Reviews

21.036

23

1

Lancet

18.316

28

1

Cancer Research

8.649

105

1

Mass Spectrometry Reviews

7.364

143

1

FASEB Journal

7.172

149

1

Bioinformatics

6.701

168

1

Nucleic Acids Research

6.575

171

1

Journal of Biological Chemistry

6.482

179

1

Free Radical Biology and Medicine

5.063

260

1

Drug Discovery Today

4.943

271

1

Mutation Research-DNA Repair

3.987

386

1

Drug Metabolism and Disposition

3.652

462

1

Methods

3.622

469

1

Mental Retardation and Developmental Disabilities
Research Reviews

3.479

522

1

American Journal of Public Health

3.363

551

1

Journal of the American Society for Mass
Spectrometry

3.321

563

1

Journal of Nutrition

3.321

563

Total = 202

Immediacy Index

The journal Immediacy Index is a measure of how quickly the average article
in a journal is cited. It indicates how often articles published in a journal
are cited within the year they are published. The Immediacy Index is calculated
by dividing the number of citations to articles published in a given year by
the number of articles published in that year.

Table 8 indicates the number of EPA drinking water papers published in the
top 10% of journals, based on the JCR Immediacy Index. One-hundred ten (110)
of the 691 papers analyzed appear in the top 10% of journals, representing
16% of EPA’s drinking water papers.

Table 8. Drinking Water Papers in Top 10% of Journals by JCR Immediacy Index

EPA Papers in that Journal

Journal

Immediacy Index

(II)

JCR II Rank

24

Environmental Health Perspectives

0.869

304

18

Analytical Chemistry

0.657

493

9

Journal of Infectious Diseases

0.889

287

8

Mutation Research-Fundamental and Molecular
Mechanisms of Mutagenesis

0.721

420

7

Infection and Immunity

0.624

544

6

Carcinogenesis

0.775

379

5

Epidemiology

0.938

264

4

Journal of Virology

1.124

188

3

International Journal of Epidemiology

1.376

131

2

American Journal of Tropical Medicine and
Hygiene

1.024

216

2

Emerging Infectious Diseases

1.007

225

2

American Journal of Epidemiology

0.908

281

2

Journal of Applied Toxicology

0.759

391

1

Free Radical Biology and Medicine

0.712

432

1

New England Journal of Medicine

11.719

2

1

Nature Medicine

6.749

5

1

Lancet

5.826

10

1

Chemical Reviews

2.955

40

1

Drug Discovery Today

1.882

86

1

Nucleic Acids Research

1.370

133

1

FASEB Journal

1.247

154

1

Journal of Biological Chemistry

1.231

160

1

ATLA-Alternatives to Laboratory Animals

0.964

247

1

Cancer Research

0.935

268

1

Drug Metabolism and Disposition

0.791

368

1

Mental Retardation and Developmental Disabilities
Research Reviews

0.788

371

1

Bioinformatics

0.736

408

1

American Journal of Public Health

0.682

465

1

Journal of Nutrition

0.647

507

1

Methods

0.596

577

1

Infection Control and Hospital Epidemiology

0.590

586

Total = 110

Hot Papers

ESI establishes citation thresholds for hot papers, which are selected from
the highly cited papers in different fields, but the time frame for citing
and cited papers is much shorter—papers must be cited within 2 years
of publication and the citations must occur in a 2-month time period. Papers
are assigned to 2-month periods and thresholds are set for each period and
field to select 0.1% of papers. There were no hot papers identified for the
current 2-month period (i.e., January-February 2005), but there were a number
of hot papers identified from previous periods.

Using the hot paper thresholds established by ESI as a benchmark, 7 hot papers,
representing 1% of the drinking water papers, were identified in five fields—Chemistry,
Clinical Medicine, Environment/ Ecology, Engineering, and Pharmacology & Toxicology.
The hot papers are listed in Table 9.

Self-citations are journal article references to articles from that same author
(i.e., the first author). Because higher author self-citation rates can inflate
the number of citations, the author self-citation rate was calculated for the
drinking water papers. Of the 8,334 total cites, 468 are author self-cites—a
5.6% author self-citation rate. Garfield and Sher2 found that authors working
in research-based disciplines tend to cite themselves on the average of 20%
of the time. MacRoberts and MacRoberts3 that approximately 10% to 30%
of all the citations listed fall into the category of author self-citation.
Therefore, the 5.6% self-cite rate for the drinking water papers is below the
range for author self-citation.

1 Thomsons Web of Science provides access to current and retrospective multidisciplinary information from approximately 8,700 of the most prestigious, high impact research journals in the world. Web of Science also provides cited reference searching.

2 Garfield E, Sher IH. New factors in the evaluation of scientific literature through citation indexing. American Documentation 1963;18(July):195-201.