We have identified 575 citations to nanoHUB in the scientific literature with 56% of these citations attributed to investigators who are not in any way affiliated with NCN. The citation map provides a visual representation of affiliated and non-affiliated investigators. Non-affiliates appear outside of the dashed line. Researchers affiliated with but not funded by NCN are responsible for many of the remaining citations. NCN clearly is strongly networked through research papers, with networks being developed outside of NCN. Some of the outside networks are completely decoupled from NCN. 509 (89%) of the citations appear in peer-reviewed journals, conference proceedings, book chapters. Ph.D. or Master’s thesis. 469 (82%) pertain to research in the nanotechnology field. Thus, the ''vast majority of citations refer to actual nanoHUB.org use''.

6

7

[[Div(start, class=clear)]][[Div(end)]]

8

9

-

=== nanoHUB h-index ===

+

== nanoHUB h-index ==

10

Two years ago we began charting citation network maps to address the question of whether nanoHUB can indeed be used for research. The documented citations and their extent into the non-NCN-affiliated nano community exceed those of any other science gateway we are aware of. Next the question arose: “Is it good research?” In the past year we have begun to address that question by asking: “Are the papers that cite nanoHUB subsequently cited by other authors?” We have interacted with Luo Si, professor of computer science at Purdue, who helped us to mine the Google Scholar service to obtain the secondary citations to the nanoHUB citations. Let’s imagine nanoHUB as the author of the 575 papers citing nanoHUB. We have found 3,251 citations to these 575 primary papers such that the h-index is 27. That means 27 of the primary papers have at least 27 citations. Considering that the first primary papers appeared in 2000, the “beginning of nanaoHUB’s scientific career,” nanoHUB exceeds the typical value of 10 for the h-index of a professional with 10 years of experience.

11

12

-

[[Div(start, class=clear)]][[Div(end)]]

+

The most recent citation lists can be found here, ordered by the number of [/citations/browse secondary citations]. The interactive table can be explored easily - you can look at "nano" publications only, look at NCN external papers, etc.

13

-

+

14

-

=== nanoHUB use by Experimentalists ===

+

15

-

+

16

-

[[Image(research_and_exp_datateal_44in.png, 360, class=align-left)]]

+

17

-

Documenting the impact achieved by researchers who are using a remote cyberinfrastructure is a challenging task and part of our assessment effort. This year we have re- examined all 575

+

18

-

nanoHUB citations in the literature to find out if the citation is either given by experimental group that has clearly designed or improved an experiment and utilized nanoHUB resources along the way, or, which is a bit easier to identify, if the paper is plotting real experimental data. In the cosmos of 469 citations that reference nanoHUB usage in nano research, we have identified 55 (12%) papers that are clearly driven by experimentalists, and 142 (30.3%) papers that plot experimental data. We consider these numbers to be a strong evidence of extensive use of nanoHUB by experimentalists doing experimental nanotechnology.

+

19

-

+

20

-

We have also determined that 41 of the papers are written by authors with industrial

+

21

-

affiliations. This is just about 9% of the 469 nano research papers. As a concrete example we reference an experimental Phys. Rev. B paper where the authors use the conceptual model of molecular conduction in the tool MOLCtoy by Supriyo Datta to explain fundamental conduction results.

+

22

-

+

23

-

[[Image(vosszanderfoninrudigerburgertgroth.gif)]]

+

24

-

+

25

-

The authors state in the abstract: “A fewparameter scalar model for ballistic current flow

+

26

-

through a single energy level is sufficient to describe the main features observed in scanning tunneling spectra of individual Mn12 molecules and offers a deeper insight into the electronic transport properties of this class of single-molecule magnets.” The authors plot their data and MolCtoy results in their paper on the same chart as shown in the figure here. The authors logged into nanoHUB some 65 times and ran around 280 simulations with MolCtoy consuming less than 20

+

27

-

minutes of CPU time. This paper was published in Oct. 2008 and already has been cited

+

28

-

three times.

+

29

-

+

30

-

As another example we list the use by Judy Hoyt’s MIT research group of nanoHUB FETtoy tool to examine their experimental Si/Ge slabs. The work is published in IEEE transactions electron devices. Prof. Hoyt, who is a very well respected experimentalist in the nanoelectronic community and a Fellow of the IEEE, is planning to attend the NCN site visit virtually through a teleconference link to describe her nanoHUB interactions.

+

31

-

+

32

-

A newly developed nanoHUB tool that modelselectron transport in high mobility InAs/InGaAs based transistors. The particularly important modeling capability is the mapping of the non-parabolic bandstructure to a simplified model and the ability to compute the gate tunneling. The visualization of the current flow drove a nanoHUB development that now enables an intuitive representation of gate tunneling. The new OMENfet code has now been released and is now available

+

33

-

openly on the nanoHUB. The scientific results generated in this experimental and theoretical collaboration has been published in a co-authored IEDM proceedings article involving the experimental MIT group of Jesus del Alamo and the theory group of Klimeck at Purdue. The funding for the science was obtained through leveraged grants of the FCRP / MSD center where del Alamo and Klimeck participate.

+

34

-

+

35

-

[[Image(cntbundle.gif)]]

+

36

-

+

37

-

[[Image(kureshihasan.gif)]]

+

38

-

+

39

-

The analysis of our external literature citations revealed some examples where external

+

40

-

theory-driven researchers utilized a nanoHUB tool that has been contributed to nanoHUB

+

41

-

from outside of the NCN. For example the authors Kureshi and Hasan working at a

+

42

-

university in India published their study CNT bundles as possible copper replacement

+

43

-

interconnects in the “Journal of Nanomaterials” in April 2009. In their work they utilized

+

44

-

the tool “Carbon Nanotubes Interconnect Analyzer (CNIA)” for over 1,600 simulations in

+

45

-

the year 2009 alone. CNIA was contributed by Tanachutiwat and Wang of the University

+

46

-

of Albany. We cannot see any relationship between the two research groups and NCN

+

47

-

other than the contributed, efficiently served, and utilized tool CNIA. The same CNIA

+

48

-

tool has previously also been cited in a Master’s thesis at the University of Cincinnati in

+

49

-

27 the year 2005. The concept of community contributions coming from outside the NCN

+

50

-

and being useful for an unrelated research group is becoming reality.

+

51

52

[[Div(start, class=clear)]][[Div(end)]]

53

54

-

=== Lowering Barriers to Simulation ===

+

== Use by Experimentalists ==

55

-

+

The NCN does have the specific goal to place simulation tools into the hands of experimentalists. The article [/topics/nanoHUB_UseByExperimentalists "Use by Experimentalists"] provides a few examples of such impact on experimental work.

56

-

[[Image(rapptureintroncn_tools.png, 360, class=align-left)]]

+

57

-

+

58

-

Accelerating Deployment of Scientific Software

+

59

-

+

60

-

In June 2005, NCN released Rappture 1.0, a new, open toolkit to facilitate the rapid development and robust deployment of scientific simulation packages. Designed to work as a library for a variety of programming languages, including Matlab, C/C++, Fortran, Python, Perl, Tcl, and Ruby, Rappture automatically generates a graphical user interface from a description of the inputs and

+

61

-

outputs of the simulation program. Rappture has become the main vehicle for software deployment on nanoHUB. Students across the NCN network are using Rappture to enable simulation applications for use on nanoHUB.org. Simulation usage on nanoHUB has grown seven-fold to over 7,900 users

+

62

-

in less than four years and we attribute this increase to the friendly interfaces afforded by

+

63

-

Rappture.

+

64

-

+

65

-

Prior to Rappture, a significant portion of nanoHUB users downloaded software and installed it on their own computers. However, since the introduction of Rappturized applications, the number of software downloads has all but vanished – Rappturized software is user-friendly. Rappture is also central to NCN’s long-term strategy to link simulations in complex workflows. Because of the scale of the NCN initiative, we expect that Rappture will have broad impact on scientific computing outside NCN. There are currently over 282 Rappture software development projects under way with 318 developers. In all, 140 tools have been deployed on nanoHUB, with all but seven using

+

66

-

Rappture. Although we do not force developers to use Rappture, approximately 95% of the deployed projects use it, which indicates developer buy-in and the ease of development with Rappture technology.

+

67

-

+

68

-

[[Image(schred_ref_type_44in.png, 360, class=align-left)]]

+

69

-

+

70

-

[[Div(start, class=clear)]][[Div(end)]]

+

71

-

+

72

-

=== Leadership in Assessment and Virtual Organizations ===

+

73

-

+

74

-

Open Usage Statistics

+

75

-

+

76

-

The idea of science portals that enable the rapid dissemination of scientific and engineering results (and that enable other researchers and educators to use these results) has been pursued by many organizations since the early-to-mid-1990s. The Purdue University Network Computing HUB (PUNCH) was one of the first. We believe there are five critical elements of a successful science gateway:

+

77

-

+

78

-

1) Connection to outstanding science/engineering.

+

79

-

2) Willingness to make the results useful to others, outside the core community.

+

80

-

3) Efficient, dependable infrastructure operations.

+

81

-

4) Technology that enables rapid development and deployment.

+

82

-

5) Open assessment and usage statistics.

+

83

-

+

84

-

We believe that while most science gateways are based on criterion 1), most struggle to

+

85

-

meet points 2) and 3) and are lacking 4) and 5). HUBzero can help to address 3), 4) and

+

86

-

5) as discussed above. Even extremely well–funded NSFbased infrastructures, such as TeraGrid or NEES, do not make their usage data and usage patterns as available as does nanoHUB. Usage data guides nanoHUB technology development, and its availability to our contributors and

+

87

-

users bolsters our engagement efforts. We believe that the detailed study and openness of the

+

88

-

46 nanoHUB usage statistics has advanced nanoHUB capabilities and has given nanoHUB a standing as the premier science gateway. nanoHUB can now provide contributors with usage and impact statistics that can be used in proposals as hard evidence of their impact on a community.

+

89

-

+

90

-

[[Image(vasileska_impact_9736.gif, 360, class=align-left)]]

+

91

-

+

92

-

The figure above for example shows the monthly and cumulative numbers of user served by Prof. Dragica Vasileska at Arizona State University.

+

93

-

+

94

-

Content Characterization by Usage, User Feedback, and Community Involvement

+

95

-

+

96

-

As nanoHUB content increases, we find that users are struggling to find the high quality content. As a result, we have continued to improve the search mechanisms on nanoHUB to enable rapid information retrieval. One key element in this effort is to characterize each content item by a

+

97

-

variety of criteria that ultimately influence the ranking of the resource. Each simulation tool is

+

98

-

characterized by:

+

99

-

+

100

-

1) A Google-like ranking based on user reviews and use.

+

101

-

2) A target audience rating, or, the expertise level expected from the user.

+

102

-

3) An indication if this is an NCN Supported tool, or a community supported tool

+

103

-

4) Data including number of users and simulation jobs, average run time, and average

+

104

-

number of stars awarded in reviews.

+

105

-

5) Number of citations in the scientific literature–this indicates the vetting of the tool

+

106

-

and its use in research.

+

107

-

6) Number of questions, indicative of the liveliness of the community. A large number

+

108

-

of open questions suggests a poorly supported tool. Conversely, large numbers of

+

109

-

closed questions indicates a live code with tool owners interested and dedicated to its

+

110

-

support. The introduction of a virtual economy has proved to have a positive

+

111

-

influence on the question and answer forum.

+

112

-

7) A wishlist enables users to express tool improvement wishes and the tool

+

113

-

development team to handle tool improvement processes.

+

114

-

8) User reviews: anyone can give a 0- to 5-star review and submit written comments.

+

115

-

9) Users can also declare nanoHUB content items as their favorites, which they can later

+

116

-

easily find again on their favorite list. Furthermore, they can share their favorite

+

117

-

nanoHUB items on six different social network sites, including Facebook, Twitter,

+

118

-

and Google.

+

119

-

10) A list of associated and recommended documents that support this tool.

+

120

-

+

121

-

Processes for User Surveys

+

122

-

+

123

-

Dr. Diane Beaudoin, Director of Assessment for the College of Engineering at Purdue, has served as

+

124

-

NCN Director of Assessment for two years. She leads the effort to formalize nanoHUB assessment and

+

125

-

user survey processes. Last year we had begun a systematic user survey process. We have

+

126

-

categorized registered users by their usage patterns:

+

127

-

+

128

-

1) one-time,

+

129

-

2) nonsimulation, and

+

130

-

3) heavy users.

+

131

-

+

132

-

The so-called “One-time” users utilize nanoHUB content for a single visit only and never return. We have also devised specialized user surveys for these groups. One interesting result of the specialized survey that went to the “one-time” users (which was

+

133

-

accommodated by a large 10% return of survey requests) is that these one-time users are

+

134

-

overall quite satisfied with what they received from nanoHUB. Another surprising result

+

135

-

was that users want to interact with other users more. This has driven nanoHUB

+

136

-

component developments that enable connections to social network sites.

+

137

-

This reporting year we contacted 1,431 users in December 2009 who had registered their

+

138

-

account within the prior 3 months. Also on this survey we received a very high 9%

+

139

-

response rate with 130 people. 52% were not using nanoHUB as part of a course. Of

+

140

-

these, 49% were graduate students, 21% were professional scientists/engineers, and 16%

+

141

-

were faculty members. Users who utilize the nanoHUB primarily in the context of

+

142

-

coursework – we call course users. Of these course users, 90% were undergraduate

+

143

-

students. The majority of our new non-course users (users of nanoHUB for purposes

+

144

-

other than coursework) discovered nanoHUB by surfing the web. This result

+

145

-

reconfirmed our effort that we must improve our presence and linkage to other web sites

+

146

-

like Wikipedia and iTunes U, which helps our Google ranking.

+

147

-

Components of these surveys address research questions that the Education Research

+

148

-

team has posed. Finally, we are also working with external groups that have approached

+

149

-

the nanoHUB team because of a desire to study nanoHUB as a virtual organization and

+

150

-

coordinate the user populations that are being surveyed.

+

151

-

How users found out about nanoHUB

+

152

-

48

+

153

-

• Independent Studies by VOSS Projects

+

154

-

In the spring of 2008, NSF solicited proposals for studies on “Virtual Organizations as

+

155

-

Sociotechnical Systems (VOSS)”. A portion of the program announcement reads as

+

156

-

follows: “A virtual organization is a group of individuals whose members and resources

+

157

-

may be dispersed geographically, but who function as a coherent unit through the use of

+

158

-

cyberinfrastructure. Virtual organizations are increasingly central to the science and

+

159

-

engineering projects funded by the National Science Foundation. Focused investments in

+

160

-

sociotechnical analyses of virtual organizations are necessary to harness their full

+

161

-

potential and the promise they offer for discovery and learning. The Virtual

+

162

-

Organizations as Sociotechnical Systems (VOSS) program supports scientific research

+

163

-

directed at advancing the understanding of what constitutes effective virtual organizations

+

164

-

and under what conditions virtual organizations can enable and enhance scientific,

+

165

-

engineering, and education production and innovation. …..”

+

166

-

Two proposal teams, one each from Northwestern (PI, Noshir Contractor) and Purdue