1998 User Survey Results

Respondent Summary

NERSC has completed its first user survey since its move to Lawrence Berkeley National Laboratory. The survey is intended to provide user feedback about every aspect of NERSC's operation, help us judge the quality of our services, give DOE information on how NERSC is doing, point us to areas we can improve, and show how we compare to similar facilities. We intend to repeat the survey yearly to monitor our progress.

Users rated NERSC in the areas of computational and file storage resources, account and allocations support, consulting services, documentation, and training. An area that wasn't included explicitly in the survey, but that users commented on, was software support.

There were 138 respondents. About 56 percent said they use the C90; another 54 percent use the T3E and 46 percent the J90s. There was less representation for the archival systems: 32 percent said they use CFS and 26 percent HPSS.

The three areas rated most important are network access, the center overall, and consulting services. The three areas that received the highest ratings are consulting services, network access, and account support services.

Overall our users say we provide a high level of service. On a scale of 1 (very unsatisfied) to 7 (very satisfied) our scores ranged from 5.8 for consulting services, C90 uptime, and CFS reliability to 4.4 for C90 and T3E turnaround time. Many users pointed out that NERSC is a good high-end production center, with a good user focus. Below are a sample of the comments that reflect this:

"Gives us access to high-end, high-capacity and capability machines without too many limitations on the way codes can be run. There is a good mix of machines, serial and parallel, and a good mix of running modes, interactive and batch."

"Supplies reliable cycles to a broad user base. Excellent user support, probably the best in the world for high-end computing."

"As a service center NERSC does a great job keeping the T3E's up and providing easily accessible information relating to hardware and software. Moreover, the intellectual quality of NERSC support and research staff is excellent and they are easy to communicate with."

"Over the years NERSC has been the most reliable source of production computing for scientific research and development, by concentrating on scientific computing ahead of computer science."

When asked what they wanted more of, most users said "more cycles". Specific software requests were also popular, as were longer T3E queues and additional batch queues in general. Areas that need improvement include:

turnaround time: we are addressing the need for more cycles with the NERSC-3 procurement process underway.

queue structures: a new T3E queue structure with more 4-hour queues and a new 12-hour queue will be implemented in December. The J90s were unified under NQE this October; further enhancements will be considered in early 1999.

software, visualization, and What's New web pages: these are being worked on and a new organization of NERSC information will be ready in December.

more direct communications with users: as a result of this survey we have been using email more often to communicate system changes to users.

the training process: we are now conducting teleconference classes in order to reach all users more effectively.

Below we present the results of the survey. (For the survey itself, click here.) The responses to some of the questions have been combined. To read the replies, click on the questions.

We would like to thank all the users who participated. If you wish to make any comments on these survey results, please send them to webmaster@nersc.gov.

Q3. What NERSC resources do you use?

Resource

N

NERSC consulting services

86

NERSC website

86

C90

77

T3E

75

J90

64

NERSC account support services

49

CFS

44

HPSS

36

NERSC Computer Operations and Network Support

10

Visualization server

5

Math server

4

Q4. Your satisfaction with NERSC: ( Scale: 1 (low) to 7 (high) )

Topic

N

Avg.

Consulting services

126

5.87

Network access to NERSC

118

5.70

Account support services

124

5.67

Overall satisfaction with NERSC

135

5.43

C90 Overall

82

5.29

Technical information on website

128

5.27

T3E Overall

91

5.20

CFS Overall

61

5.18

HPSS Overall

64

5.09

J90s Overall

79

4.92

Training

103

4.91

Allocations process

112

4.60

How important is this to you? ( Scale: 1 (low) to 3 (high) )

Topic

N

Avg.

Network access to NERSC

116

2.84

Overall satisfaction with NERSC

129

2.77

Consulting services

128

2.70

Technical information on website

127

2.51

T3E Overall

99

2.45

Account support services

123

2.31

Allocations process

118

2.31

C90 Overall

101

2.27

J90s Overall

94

2.19

CFS Overall

80

2.09

HPSS Overall

82

2.04

Training

117

1.99

Q5. Comments on the ratings in Q3 and Q4?

Comments on Section 1 Ratings (all responses here)

T3E Comments: 11 responses

4 - Happy and satisfied with the T3E; positive experience

2 - unsatisfied with queue policy

queuing system extremely awkward; doesn't support very large simulations,

"Our research depends very much on HPCC technology. More computing resources available to us will allow us to complete century long simulations and comparisons."

7 - more cycles in general

1 - SMP

"More emphasis on SMP architectures. I fear the MPP approach may be dead in a few years and is not relevant to many computing projects in DOE."

NERSC is addressing this concern through the NERSC-3 procurement process.

Better software support: 20 responses

wanted for the T3E:

multithreaded Perl

full ports of CERN libraries

GNU tcsh

full ssh to/from with authentication

Concurrent Versions System (CVS)

more end-user oriented tools like PSPASES, pPCx, optimization tools

latest version of emacs

improve ease of use

wanted for the PVPs:

3 requests for keeping cf77

f77 to f90 translator like VAST

faster debugger than totalview

MOLPRO for chemistry

CERNLIB

tcsh

ezget libraries from LLNL

wanted in general:

increase variety of supported software

better support for fixing Cray Fortran bugs

better file manager and editor

ghostview

pop mail server on SAS, email on SAS

a set of Unix scripts on line for making batch file submission easier and more user friendly

in-house software mode as needed to improve use of systems

Some of the software mentioned above is already on the machines, for example: tcsh, CVS, emacs. We can't provide some of the software listed because it is no longer supported, for example, f77. We evaluate all software requests, and consider the anticipated usage and the cost.

"I am wondering whether the T3E scheduler could predict when a given job in the queue would be expected to start execution (this prediction could perhaps be included in the qstat listing); this prediction of course would be subject to change, but might be helpful for planning."

"you have to figure out some way that it is possible to do medium sized development work in a acceptable time frame. some bugs show up only if the job is a little larger - and in the present set-up with about 2 jobs / day ... it takes for ever to find anything."

7 - better batch software

4 - different PVP queue management

"give more priority to large job which can not be done locally"

NERSC has developed a new proposal for the T3E queue structure which should be implemented by December 1998:

The J90 Cluster queue structure will be evaluated next. We will try to make the NQS software easier to use and understand by writing better web documentation.

Better/different allocations process: 11 responses

make it easy to get increases during the year

better notification to users (not just PIs)

make allocations more easily available to local users

simplify / rationalize process

need larger T3E allocation

change and make more flexible

ERCAP should be replaced - need to be able to use powerful editor like LateX

earlier deadline this year was a major inconvenience

make allocations based on science, not whether DOE funded

Improve documentation/training: 10 responses

make info easier to find

improve writing style

more info

better info on using math and graphics libraries

better batch documentation

need Unix tutorial

want manual and tutorials in hand

more high-end training / computational science support

more remote site training

better communication of machine status

better communication of available software

We do continuous work on web pages; the survey tells us we need to focus especially on the software pages and batch documentation (also on the visualization pages, but these affect fewer users). We will provide a new search engine by the end of January, 1999. We are now offering teleconference classes: see NERSC Training Classes.

1 - resource allocation should reflect user needs: talk to users in account before setting up the account

1 - better account and queuing policies that allow a group to work as cohesive unit

NERSC is working on a plan to redesign CUB, the database which manages accounting information. Deployment of the new system is expected for FY'2000. More account creation procedures will be automated this year, especially for HPSS accounts.

Q17. How do you find out about changes at NERSC?

Q18. Do you feel you are adequately informed about NERSC changes? If not, why not?

Yes 94, No 21

If not, why not?

I don't usually know if there has been a system change that is not listed on the message of the day.

I don't log in very often

Don't login much, didn't know about the website.

If I don't log into the system some time or check the web, I usually miss some announcements

Because I rely only on log-in messages. I don't have a lot of extra time to check websites frequently for messages.

I miss the Buffer - the Web is not a complete replacement.

I ought to check your NERSC news regularly

I don't check the what's new or change pages regularly.

I don't check the website very frequently and notices via email don't appear very often.

Sometimes I don't know about changes until after they occur.

I did not know that the NQS script format has changed until I accidentally found that.

I've had problems with my email and didn't realize free time on Pierre was over, thus overrunning my allocation.

I am mostly logged in on one of the workstations around the lab, since I can't make use of the CRAYs I am seldom logged in there. However, news are posted only there. In addition it would be nice to send a reminder each time even though it's a scheduled maintenance at least to PDSF, which is a part of NERSC, right? This is mainly related to HPSS.

Q19. How would you like to be kept informed?

32

email (of which 6 email + web; 1 email to PDSF users re maintenance)

8

motd (1 said there used to be a lot more)

2

automated What's New

2

from colleagues

1

online bulletin board accessible during telnet session

1

are there email discussion groups?

1

more advanced notification of when T3E will/won't have scheduled down

1

broadcast messages

Q20. How satisfied are you with NERSC's consulting services?

Topic

N

Avg.

Overall quality of consulting responses

113

5.88

Problem resolution by consultants

109

5.73

Software bug resolution

84

5.61

Consultants follow-up to questions

99

5.57

Q21. Comments about consulting support

24

satisfied

4

don't use / don't need

2

mixed evaluation

2

consultants can't fix things: "I'll report it to Cray; "we don't have that software"; "there is no way to do that" rather than "You can't do that now, but we'll get our system software programmers to write a utility which will allow it to be done in the future"

1

not useful

1

get more consultants

1

sometimes emails get lost

1

want faster email responses

1

didn't get response on query for CERNLIB on T3E

Q22. What is needed to adequately cover your training needs?

Method

N

Online tutorials

85

Web-broadcasted training

28

Classes given in Berkeley

15

Classes given at remote sites

12

Q23. Have you taken any of our training classes?

Yes 19, No 102

Q24. Comments about our online tutorials, suggested improvements and other ideas for training