Systems for the management of information in a university context: an investigation of user need

Rita Marcella
Aberdeen Business School
The Robert Gordon University
Aberdeen, AB10 7QE, UK

and
Karl Knox
School of Informatics
Northumbria University
Newcastle upon Tyne, NE1 8ST, U.K

Abstract

The accessibility, reliability, consistency, and relevance of data underpinning information systems are crucial to its use and effectiveness in a university setting. This article reports on the findings of a research project carried out at a new university, which highlighted the role information plays in the success of the operation and in allowing the institution to evolve and meet the challenges posed by the government, students and other stakeholders. Data were gathered from the academic and administrative staff of the university through interviews with senior managers, and a Web-based questionnaire completed by 863 respondents (a 47.9% response rate). The project aimed to explore data and information activities supporting management and strategic decision making in a new university. Project results indicate that there are real deficiencies in the realization of the case institution's information strategy and that these deficiencies must be addressed in developments focusing on improving strategic effectiveness in the future. Particular issues identified included the lack of clarity in responsibility regarding information and concerns about the validity of much of the internally created and maintained data.

Introduction and background to the project

This paper describes a study initiated as part of the remit of a task group which had been set up to explore the effectiveness of existing data and information systems supporting the staff – management, academic, research and administrative – of a higher education institution, a new university. As a priority it was thought desirable to investigate the needs of the internal users of the systems under investigation, at varying levels of seniority and across the broad spectrum of administrative and academic departments. The task group was aware that there were likely to be some deficiencies in data and information currently held and a study to investigate staff satisfaction with the systems and data presently in place would be, therefore, a valuable exercise in determining the limitations in provision, where issues had arisen in operation and use of systems and in investigating those improvements that were most urgently required.

Lucey argues the significance of management information in the university context:

Management information is data converted to information which allows managers at all levels in all functions to make timely and effective decisions for planning, directing, and controlling the activities for which they are responsible. (Lucey, 1995)

As Norton and Peel (1989) predicted, the information management environment has changed radically in the last few years, with the development of new and improved systems potentially outstripping our capacity to manage and exploit them fully, resulting in a situation where the role of information management has undergone dramatic transformation.

Given the changing nature of higher education and the pressures placed on institutions to satisfy numerous and various stakeholders (Salter & Tapper, 1994); the role of information has been perceived to have increased dramatically in significance in addressing stakeholders' needs. Its importance is emphasised in government reports (Dearing, 1997); as well as through initiatives undertaken by the Joint Information Systems Committee (JISC) and the Higher Education Funding Councils for England (HEFCE) and Scotland (SHEFC) to improve institutional information strategies. The JISC, in particular, has for several years sought to support universities and colleges in developing effective information strategies. In 1995 the Committee published Guidelines for Developing an Information Strategy (JISC, 1995) and established six pilot sites to test the implementation of the guidelines. Atkinson, et al. report on the success of three of these at, respectively, the University of Glasgow, the University of North London and the University of Glamorgan. The University of Glasgow had adopted a 'projects driven approach' in seeking to 'benefit the entire University community', exploring information needs through focus groups and briefing sessions in relation to these chosen projects (Atkinson, et al., 1997: 3). The University of North London developed an information toolkit which includes an information audit questionnaire, the use of a software package for mapping information flows and a grid for attaching notional value to information assets. They have adopted a project approach rather than seek to implement an holistic strategy. Glamorgan explored the user perspective through workshops with 'senior academic staff and teaching staff' of two selected departments, in order to, 'obtain a 'grass roots' picture of the ways in which information is utilised, created and shared' (Atkinson, et al., 1997: 5). Again, specific, rather than overarching, initiatives relating primarily to the administration of teaching and learning were identified and put in place as a result of the pilot scheme. Presumably such results were fed back to the JISC and lessons learned accordingly. One recent result has been the establishment in 2002 of a JISC information strategy advisory service, based at University of Northumbria at Newcastle.

Interestingly, in an ethnographic study of the cultures of two universities and their implementation of information strategies, Neyland and Surridge reported that,

...it was not clear that there was universal agreement that an Information Strategy was a good or useful idea and there was no clear agreement on what should be included in an Information Strategy (or how an agreement should be reached. (Neyland and Surridge, 2002: 2)

The authors characterise this situation as 'messy' and lacking in clarity or understanding and note that the institutions studied were observing the rhetoric of requiring and producing a strategy without taking on board in spirit what was necessary for the strategy to be meaningful.

More broadly, Davy argues the importance of developing an effective information strategy by placing information at the centre of all corporate success, whether in the public or private sectors, where the manner in which information is used by an organisation is the 'determining factor as to how competitive, efficient and, ultimately how profitable they are' (Davy, 1998: 1). Information in terms of its collection, relevance, dissemination, and retrieval supports a university's integral functions. In its many forms it lies at the heart of a university's primary activities, whether these relate to teaching, learning or research. Davenport (1997) emphasises the importance of understanding an organisation's information environment and the way in which people use information. He also draws attention to the need to place less emphasis on information technology and more awareness on how people interact with information.

Burrows and Harvey dentify the variety of categories of stakeholder in a university context all of whom potentially require different types of information, of varying levels of detail and complexity and for differing reasons. These internal and external stakeholders include, students and their families, employers, teaching and administrative staff, government and funding bodies, accreditors, validators, quality auditors and assessors, and professional and employer bodies (Burrows & Harvey, 1992). To develop an information strategy and systems to support it that would meet the needs of all of these stakeholders is no small challenge for universities, in particular when we remember that the same sets of data may be used, simultaneously, as the basis for informative and promotional messages.

Although the JISC tells us that, 'information is the lifeblood of higher education. It is a resource and needs managing as such; this puts it on par with finance and human resources' (JISC, 1995 as cited in Dhillon, 2001), the present authors would argue that, typically, information has not been regarded holistically as a management issue in the clear manner that finance, for example, has. It has tended, rather, to be seen as subsidiary to the perceived greater priorities it serves. It has often failed, so far, to achieve its appropriate strategic status within institutions' management hierarchies, as recommended by Drucker (2002), who suggests that information and finance managers should each attain a comparable position within the institutions' executive level. As a result, systems have frequently developed and been applied in a discrete, unsystematic and unrelated manner, driven by function rather than value, which has led to the chaotic and visionless situation in which we presently find ourselves. Neyland and Surridge (2002) also examine the most effective ways in which information strategies may be realised in terms of their institutional management, identifying a very complex set of relationships between information strategy committees, centres of key players and local hubs of information users, recognising the pervasive nature of information within institutions, used by all, but with, in the two cases they examine, no single central responsibility. They recognise the importance of 'representation', that is the development of mechanisms by which the needs of the disparate user community may be fed into strategy formulation. Strategy must be ongoing, flexible and inclusive.

Greene, Loughridge and Wilson (see, Greene & Loughridge, 1996; Loughridge, 1996; Greene, et al., 1996) describe variously the results of a project designed to investigate the management information needs of academic Heads of Department which identified a number or failings in existing provision, including barriers in accessing external data, lack of integration of systems and poor response in areas such as financial management. They were also doubts about information quality and in particular the accuracy, specificity and currency of information available. The project also revealed a mismatch between the views of academic Heads of Department and senior managers and administrators of information gathering and distribution units. Greene and Loughridge (1996) conclude that 'there is a need for universities to clarify and, if possible, simplify the structure of their information provision', emphasising the importance of the technology supporting information services rather than allowing the technology to 'determine the information services provided'.

Cullen, et al. (2003), comment on another problem, that is the extent to which strategies in 'old' universities have been IT-oriented and procurement driven, with the corollary that they have failed to meet the needs of users. We would argue that this problem is equally common in the newer universities and that, given the high proportion of investment in HEIs in information systems, it is one that requires exploration as a matter of urgency.

Allen and Wilson (1995, 1997) also recognise that, in discussions of information strategy, there has been too great a focus on the technologies, on internal institutional factors and on processes, in arguing that in,

...the late 1980s and early 1990s most higher education institutions (HEIs) developed strategic plans which were information technology focused... over 73% of these were perceived to have either failed or been only partially successful. (Allen & Wilson, 1997: 179).

They describe problems in formulating information strategies as a result of terminological mismatch and bring together nine characteristics that influence an institution's capacity to implement information strategies successfully, drawn from a number of authors such as Galliers and Wilson (as cited in Allen & Wilson, 1997: 185). These issues are considered influential in terms of both information systems and information strategy development.

There is a need to align strategies across all organisational areas.

Key stakeholders should be involved within the process of formulating the strategy.

The process should allow opportunity for debate and consultation.

The process should be reviewed on a regular basis.

The approach taken must fit the organizational structure and culture.

Motivational issues are important, developed to be implemented for internal reasons not simply to satisfy an outside body.

Those who implement the strategy should feel that they have ownership of the policies.

The process should have a powerful champion.

The strategy should be adequately resourced.

Equally, much of the discussion about information strategy has focused on the management of internal information. Drucker argues that information strategy must look beyond its traditional interpretation by managers and IT specialists as dealing with the creation and maintenance of sound systems to manage internal information as 'the most important changes affecting an institution today are likely to be outside ones, which present information systems usually know nothing about.' (Drucker, 2002: 294). A judicious mix of systems designed to facilitate access to internal and external sources is, therefore, required.

Guan, et al. (2002) identify three of the ways in which information is used strategically by both internal and external stakeholders in universities:

By administrators and policy makers and faculties to:

assist in resource management and strategic planning,

assist in the recruitment and retention of students.

By external oversight agencies to:

assess the performance of the institutions.

assess programmes offered on a variety of indicators.

By students requiring subject specific information on:

lecture notes, tutorial and case study briefs;

examination timetables, and

results of assessments.

The importance of placing the needs of the user at the centre of any information strategy development is paramount if the institution is to be successful in meeting stakeholders' needs. If we accept the centrality of information to the health and success of an HEI, in line with the JISC, then the systematic evaluation of its use, by staff and others will ensure that institutions will be better equipped to generate relevant, accurate and consistent data and better placed to meet both broad institutional aims and objectives and the variety of needs demonstrated by internal and external stakeholders. Given the variety of need, the present project began by examining the primary user group, that is the academic and administrative staff of an HEI, incidentally investigating their use of information in serving other stakeholders, students, external bodies, government, etc. This paper describes one attempt to introduce greater clarity into our understanding of the ways in which information strategy may serve institutional needs. It should be borne in mind, however, that the paper focuses exclusively on research carried out to gather data about staff of the university's information needs and does not seek to examine issues associated with information use by students and external stakeholders, areas which would repay further exploration.

Research Approach

Prior to the design of the present project, research had been carried out which illuminated two main areas of concern in relation to the case university's data and information management and suggested that the project objectives should focus from the beginning on the investigation of the information needs of academic and administrative staff in the drafting of an information strategy. Research design involved two elements: the exploration of senior managers' perceptions of data and information quality issues; and the gathering of data from a representative sample of university staff as to their attitudes to accessing and using data and information systems as these were currently configured. As a result, the methodology consisted of two elements: (i) interviews with senior staff of the university and (ii) a survey by questionnaire of all staff of the university.

In stage one, in-depth, semi-structured interviews were carried out with senior managers, responsible for managing central departments and/or academic subject divisions. These individuals were chosen to represent the range of different interest groups acting as key expert informants about the processes and systems that drive information strategy development and evolution and about the needs which the strategy must be designed to meet. They consisted of:

two Deputy Vice Chancellors (Resources and Academic Affairs);

Director of Human Resource Management;

Academic Registrar;

Director of Finance;

Head of the Strategic Planning Unit;

Head of Management Information Systems;

Director of Learning Resources;

a former Head of School; and

the Manager of the Information Systems Department

The semi-structured interview schedule was piloted and sent to interviewees before the face-to-face meetings. It had been designed to enable free-form exploration of issues and concerns from the perspective of the interviewee, rather than to follow a rigid set of pre-determined questions. Given the varying contexts in which the informants operated and the difficulties associated with identifying an agenda in advance of the data collection, it was felt that this approach was essential. Interviewees were open in response and identified a number of significant areas for exploration that were subsequently to form part of the survey by questionnaire of all staff.

Given that the survey took the form of exploratory research designed to draw a broad picture of perceptions of the data and information systems used, it was felt that a positivist approach would be appropriate, where data would be collected in sufficient quantity to allow statistical analysis to be carried out, in order to determine if characteristics such as respondents' level of technological expertise impacted upon the results. Although some argue that surveys may be unrevealing and poorly responded to in the organisational setting (see, for example, Dhillon, 2001), it was believed that a well-designed survey instrument would gather meaningful and informative data. The survey was based on an electronic questionnaire to be completed by the general staff population, which, while addressing issues raised by senior managers, gave respondents opportunities to identify unpredicted areas of concern through judicious use of open questions. It sought to explore, through a variety of questions, the following research questions:

Which sources of data were used by which groups of staff, their frequency of use and whether or not the data required were available from the existing data warehouses.

Whether data were duplicated internally, that is, within departments, schools or faculties and, if so, the reasons for such duplication.

Whether it was possible to characterise categories of data which were held to be more important or the use of which had a higher priority for certain groups of staff.

Whether there were other factors contributing to respondents' satisfaction with the university's information systems, such as skill level, training provision or the ease of use of system interfaces.

The questionnaire was disseminated to all staff of the university, identified through the university's e-mail listings and posted on its intranet, as a Web-based form. Staff were sent an e-mail, apprising them of the purpose of the survey and directing them to a Unique Resource Locator (URL), which allowed them to access the questionnaire quickly and easily. This approach enabled the researchers to disseminate the survey instrument very rapidly to the widest possible population. The software package used, SNAP, also supported automatic collation of the responses, helping to ensure greater accuracy in data input than with conventional approaches, in that no manual transference of data was required. The questionnaire took the form of a series of Web pages, with features such as drop-down boxes and automatic navigation for the users through the questions based upon the responses they made. This presentational style enabled respondents to complete the questionnaire quickly and with minimal input error. [The questionnaire is shown in the Appendix. (Close the window, after use)]

The ease of access from the desktop, coupled with the assumed novelty of completion and submission of the questionnaire online, contributed to the achievement of a higher than anticipated response rate, that is 47.9%, for what is traditionally held to be a method of data collection to which response will be poor. The fact that the questionnaire was sent to all staff with an e-mail account, notionally equivalent to the entire complement of the university, including administrative staff, academics, researchers and operational and facilities staff, ensured that the response was truly representative of all groups in the study population. Equally the large number of responses achieved, 863 in total, enabled valid statistical manipulation of the data to be carried out. Staff had access to the on-line questionnaire for a four-week period in order to ensure that those staff who were away from their desks for any particular reason would have the opportunity to complete the questionnaire upon their return. A reminder of the survey, again directing users to the URL, was sent after a period had elapsed and this clearly helped in attaining the response achieved.

Results

Key informant interviews with senior staff of the university

Eighteen interviews were conducted with senior staff, responsible for the management, collation, creation, dissemination and analysis of university data both internally and externally. A number of significant outcomes emerged from this process:

No single department or unit held ultimate responsibility for the creation, maintenance, accuracy and reliability of all of the original data sets, in a manner which was responsive to all categories of user.

There were unclear lines of responsibility which meant that staff might be prevented by system restrictions from gaining access to the data necessary for the performance of their duties.

There was confusion with regard to terminology, deadlines, and meeting the needs of outside organisations or governing bodies in providing information.

There was frustration over who holds data (faculty, administration or central services) and, therefore, who is allowed access to it.

There was uncertainty about the responsibility (the Strategic Planning Unit, Registry etc) for creating data and information reports for both internal and external clients, raising concerns that data might not meet the needs of all people who might use it.

There was concern about data being unavailable or about users' inability to manipulate the data to meet precise requirements.

Interviewees talked very openly about their information concerns and there was a good deal of consensus about the major issues associated with its management. As may be inferred from these findings, availability of data or its lack was a recurring theme. Non-availability might arise from a variety of factors: non-generation of data; system restrictions; bureaucratic access restrictions; failure in quality checking; or failure to process data sufficiently. However, the significant inference must be that, whatever the cause, staff of the institution frequently encountered difficulties in locating the data they needed. Equally, interviewees argued that there was no sense of a strategic vision in the management of information as it was realized at present within the case institution.

Questionnaire survey of all staff of the university

Respondents' demographics

In total, 863 people of a target of 1,805 (47.9%) replied to the questionnaire. This was a broadly representative sample and responses from all faculties and schools were achieved in the responses, both in terms of staff (academics, administrative and research) and sex. All twelve faculties were represented as well as a significant proportion of support services - the University Student Association, sports facilities' staff, and central administrative services, such as finance and human resources. A high percentage of respondents was female (63.5%) and while of the University staff complement as a whole 57.4% are female and 42.6% are male, the relatively high proportion of female respondents reflects a particularly good response from administrative staff (49.6%), a category where women are represented in a higher proportion than for the University as a whole. We also believe that the high response from administrative staff reflects the frequency with which administrative staff use information systems and, arguably, the extent to which they are constrained and challenged by deficiencies in the systems available.

Table 1: Respondents by age

Age

Percentage

25 and under

6.6

26-35

28.1

36-45

31.3

46-55

24.7

55 and over

9.3

A spread of responses was achieved across the four categories of staff. Overall responses were highest from administrative staff at 49.6%. This is understandable given that these are the individuals whom one would expect to be using data sources in the fulfillment of their duties. It should be noted that although the researchers had assigned respondents to four main categories, the respondents were of course varied within these categories both in terms of level or grade and job classification. No attempt was made to distinguish staff in terms of seniority or management responsibility, although with hindsight it is acknowledged that this data might have been revealing.

Table 2: Respondents by job category

Category

Percentage

Academic

29.9

Administration

49.6

Research

10.8

Management

9.7

The largest single group of respondents (52.8%) identified their work as falling into the operational section of data manipulation.that is, those activities that support the day-to-day running of the operation. This is not surprising given the high percentage of responses from administrative staff. However, there were significant if smaller proportions respectively of respondents who recorded that they used data tactically (31.5%) and strategically (15.7%). It should be noted that this response reflects the respondents' own assessment of their use of data in operational, tactical and strategic ways rather than being categories that have been objectively applied by the University. Again the significance of this result would have been aided by an understanding of levels of seniority amongst the respondents, as we argue that the tendency to use data strategically and tactically is dictated by the extent to which individuals have decision making powers rather than the university function to which they belong. Future research should consider this aspect of respondents' activities.

Table 3: How respondents view their use of data

Respondents view their use of data as...

N

%

Operational - supporting the day to day operational activities

347

52.8

Tactical - addressing department/faculty/school/ areas needs

207

31.5

Strategic - addressing the raison d etre of the university

103

15.7

Total N & (% of overall total)

657

100

The next question sought to explore respondents' assessment of their level of self-assessed computing skills identified, as it was felt that there might be a relationship between their feeling of comfort with information systems and their confidence in their own capacity to handle information technology. Table 4 below shows that there is a 99% level of confidence that the difference between the 'length of employment' and the 'level of computing skills' is not due to chance factors, shown by a Chi-squared value of 59.47 and an indication that there is a significant relationship at the 1% level.

Table 4: Competency by years employed

Current level of competency

Belowaverage

Average

Good

Competent

Verygood

Excellent

Time employed

N

N

%

N

%

N

%

N

%

N

%

N

%

<1 year

168

4

2.4

44

26.2

20

11.9

43

25.6

40

23.8

17

10.1

2 to 5 years

258

4

1.6

51

19.8

41

15.9

79

30.6

58

22.5

25

9.7

6 to 10 years

79

3

3.8

27

34.2

9

11.4

27

34.2

11

13.9

2

2.5

>10 years

52

10

19.2

13

25.0

6

11.5

7

13.5

14

26.9

2

3/8

Total (% of overall total)

557

21

(3.8)

135

(24.2)

76

(13.6)

156

(28.0)

123

(22.1)

46

(8.3)

The table identifies that those staff members who rate their computing skills as below average tend to cluster around the 6 to 10 years or more length of service (23%). Similarly staff who rate their 'computing skills' as very good or excellent tend to be grouped amongst the newer members of staff with less than one year through to five years of service (19.8%).

Use of data sets by categories of staff

The most frequent users of 'student data' are administrative staff. They access these data constantly, indeed 50.3% them identified that this is done on a frequent or daily basis. This would suggest that the reliability of this data set is vital for all administrative staff concerned. As one might expect, the majority of academic staff also require student data on at least an occasional basis. Student data, therefore, represent a core area of record keeping for universities, with significance for a wide variety of stakeholders, not least the funding bodies who will base financial allocations on the numbers of students enrolled. The fact that it is so widely used makes it essential that the raw data set be the same for all staff, whether based in central support or academic departments.

Table 5: Use of student data by category of staff

Student data used...

Not at all

Sometimes

Frequently

Daily

What is your current position?

N

N

%

N

%

N

%

N

%

Academic

157

27

17.2

96

61.1

26

116.6

8

5.1

Administration

258

70

27.1

58

22.5

45

17.4

85

32.9

Research

58

36

62.1

16

27.6

3

5.2

3

5.2

Management

50

24

48.0

13

26.0

11

22.0

2

4.0

Total N & (% of overall total)

523

157

30.0

183

35.0

85

16.3

98

18.7

Academic and administrative staff similarly outweigh any other group who need access to enrolment data. Over 40% of staff access this data set 'sometimes or on a daily basis'. This is the initial data set, which feeds into student records. It is vital that it accurately relates precise student figures, which are highly significant in terms of HEFCE and SHEFC funding. The majority of respondents indicated that wastage data were not frequently used. Where they were used it was by academic and administrative staff. However, as wastage again reflects on success measures for academic programmes and departments, its accuracy is highly significant.

It can be seen from Table 6 below that, on the whole, both academic and administrative staff use course information. The need for reliability and accuracy is important as these data will also be accessed by external users, most significantly by potential applicants who may require the data to make choices about courses and institutions.

Table 6: Use of course information by category of staff

Course information used...

Not at all

Sometimes

Frequently

Daily

What is your current position?

N

N

%

N

%

N

%

N

%

Academic

163

14

8.6

71

43.6

64

39.3

14

8.6

Administration

246

65

26.4

81

32.9

58

23.6

42

17.1

Research

57

29

50.9

22

38.6

6

10.5

-

-

Management

49

18

36.7

22

44.9

9

18.4

-

-

Total N & (% of overall total)

515

126

24.5

196

38.1

137

26.6

56

10.9

In contrast (see Figure 1), human resource management (HRM) data sources are not so widely accessed and tend, in the main, to be used by management and administrative staff. This would suggest that HRM data sets could be restricted to certain groups who really need them. This would allow more tailoring to specific group needs. Indeed, data protection concerns make it necessary that certain HRM data are restricted to a very small group of essential users. It is questionable, therefore, that, for example, research staff would be in a position where they would require or should be allowed to access HRM data. Similarly, only administrative and managerial staff use financial information, 23% of whom use these data frequently or on a daily basis. Where it is used, it is again vital to the successful management of units and departments in that budgetary data will, for example, very likely have a major impact on future plans and developments, even, in extreme situations, on the survival of units.

Figure 1: Use of HRM data by category of staff

A large proportion of all respondents from academic to managerial staff used external data on a regular basis, that is, over 44% in total. What is surprising in relation to this finding is that external data are not accessed more frequently. The Library Resources Web site is also used very frequently by staff, providing links to internal and external sources through electronic journals, learning support materials and the catalogue. This resource is used by over 40% of respondents daily, primarily academics and researchers.

The primary significance of data about students is highlighted in these results, which, in itself, is not surprising, given their centrality as the customer of higher education and its major product. When asked to indicate which university information systems were presently used, results show that the Student Administration System (SAS) is the source accessed most frequently, on a number of occasions each day. However, the finding suggests that a universally-accepted format for data about students should be established as a priority in providing the basis for generating the required information from and for all staff who input and access these data.

Data quality issues

A significant proportion of respondents (20.1%) highlighted concerns about the 'quality or reliability of data'. In conjunction with the comments from interviewees about the reliability of data at present, a prime objective of the university's information strategy, therefore, should be to put in place better systems for checking the accuracy of data input and update.

Other problem areas identified by respondents included:

Sourcing data—data were frequently unavailable or non-existent and a significant number of staff were required to create their own, frequently duplicate, data sets. This was felt to be a factor which impinged on 13.5% of respondents' capacity to carry out their roles within the university.

Inappropriate data format—15.2% of respondents reported that data were at times provided in an 'incorrect format' and that further manipulation or creation of data sets were necessary in order to meet specific requirements.

Table 7: Respondents' views of the availability of data

When accessing data, it is...

N

%

Usually available on time, every time

76

15.1

Usually available most times

366

72.8

Usually available after many requests

44

8.7

Not available at all - therefore another source has to be used

17

3.4

Total N & (% of overall total)

503

100

When asked to rate the ease with which data could be accessed, respondents were generally positive about their ability to collect the required data, in that 72.8% felt that most of the time data were available. However, there remains a significant minority, 12.1%, of respondents who state that data are not available when needed or only after a number of requests. Given that many of the university's strategically significant operations are based on the availability and manipulation of data, even a small percentage of data unavailability is cause for concern. Of the 12.1% of the respondents who encountered problems with the availability of data, the following were noted as areas for potential improvement:

reliability of financial data;

availability of course information on the Web;

academics' ability to access student data directly, rather than being required to channel such requests through administrative staff.

Very significantly, in total over one third of staff (34.2%) indicate that they store, create or find it necessary to manipulate data in some way, shape or form locally, so that it can meet their specific needs or requirements, in addition to the systems provided by the university.

certain data sets are only available to administrative and not academic staff, necessitating the creation of local, modified data stores;

staff are uncertain about relying on the central financial reporting system;

data are not stored in easily usable formats within the university systems, for a significant proportion of respondents.

Use of information support systems

Although a majority of staff (over 60%) report positively that the current computer interfaces and/or systems are user-friendly, problems that were identified included:

lack of expertise in the applications and in data manipulation;

lack of knowledge of how or where the data are 'housed' in terms of the inter-relationship of data sets.

When asked which types of support assist individuals in their duties, the largest proportion of respondents identified technical support offered centrally as being the most frequently used (35.0%) in the form of the computer help desk, although all other forms of support are used by between a fifth and a quarter of staff.

Table 8: Use of Support

Types of support made use of:

N

%

Technical support - centrally provided

385

35.0

Guidance, instructions, training

257

23.4

On-line instructions, prompts

227

20.6

Training sessions - specific

231

21.0

Total N & (% of overall total)

1100

100

Meeting the needs of the respondents' customer base

The following series of questions sought to explore the extent to which respondents could meet the information needs of their customers with the data available at present. The majority of respondents indicated that their main customers were either students (32.2%) or academics (25.6%) in line with the overall profile of a high response rate from administrative staff. A relatively high percentage of respondents also identified 'external agencies' as customers (13.1%) indicating that a major role for these individuals involved making data available to third parties, on behalf of the university.

Table 9: Customer Base

Who are your main customers?

N

%

Central Services

131

10.4

Students

405

32.2

Academic Staff

322

25.6

Researchers

128

10.2

Executive Staff

107

8.5

External Agencies

165

13.1

Total N & (% of overall total)

1258

100

Overall, a positive response was received when asked whether or not individuals could access the data they needed to meet their customers' needs, with 71.3% of the respondents indicating that they could access data to meet the needs of their customers 'most of the time'. However, when we examine the results in Table 10 more closely it is worth noting that only 8.2% of respondents indicated that they could meet their customers' needs all of the time. Although 'most of the time' might seem to be an acceptable level of achievement, if there are any occasions on which staff cannot provide data to meet their customers' needs, then we might argue that the systems in place at present are potentially failing 91.8% of staff on at least an occasional basis. When coupled with the fact that approximately a third of respondents report that data require modification in order to be useful, there are clear deficiencies in the present strategy.

Table 10: Ability to meet customers' needs in supplying information

Can you access data which meets your customers' needs?

N

%

Totally, 100% of the time

43

8.2

Most of the time

372

71.3

Some of the time, with modifications

107

20.5

Total

522

100.0

Similarly, when asked if current systems allowed respondents to deal with personalised requests for data, the majority felt that the systems in use were fairly robust. However, a significant 32.8% encountered difficulties dependent on the nature of the request.

Table 11: Ability to meet personalised requests for data

How well the system allows personalization

N

%

Extremely well

27

5.7

Good - depending on what is requested

289

61.5

Some difficulty due to current data setup

107

22.8

Not well due to lack of flexibility in data

47

10.0

Total N & (% of overall total)

470

100

Respondents' aspirations and suggested improvements

A number of desired improvements were suggested by staff with regard to information systems:

A desire for faster and easier access to electronic data.

Data to be made available on the intranet in a downloadable format.

A single, central and linked system that works for everybody.

A system that links all current systems instead of separately configured and accessed systems as at present.

Systems able to handle multiple course entries and progressions across standard academic years.

Some of these suggested improvements relate to features that are already in place, indicating that staff may be unaware of the full functionality of existing systems and that training might be enhanced to cater for such lack of awareness. However, the desire for a single holistic and flexible system is clearly demonstrated. Respondents also indicated areas where they felt their own personal effectiveness could be enhanced:

The heavily used student data system was rated as very slow and lacking in versatility, not being able, for example to cope with overseas students' name order.

General training was required on use of the student data system.

Improvements were thought necessary in the Marks Record System (MRS).

A new and more user-friendly database, supported by internal staff, was sought.

The reliability of data sets was thought to be in some doubt.

Better communication between centrally held information systems and those that are more locally derived was thought desirable in order to increase data reliability.

A desire was expressed for improved student retention data, indicating reasons for wastage as well as global figures, and subject to rigorous validity checks.

When these suggestions for future improvements are considered they provide further support for the view that there are real deficiencies in existing systems, indicating a greater level of dissatisfaction than might otherwise be inferred from some of the responses to closed questions.

Conclusions

The project gathered data from a significant sample of staff from across the full range of university functions and levels suggesting a high level of interest in the topic. The online questionnaire was felt to be a particularly useful and appropriate tool for this purpose, although inevitably, as tends to be the case when using survey tools, further exploration of issues would be desirable using qualitative methods.

Senior managers identified a number of issues associated with existing systems, including:

a lack of cohesion, consistency, and reliability across all data sets;

no single institutional responsibility for the data and information management function, with a resultant uncertainty about lines of responsibility;

confusion and frustration arising from the multiple sources presently existing;

concerns about the location of data and its appropriateness in meeting user needs; and

a lack of flexibility in existing systems.

It is worth noting that although some of the issues identified remain remarkably similar to those discussed by Greene, et al. (1996), there is at least in the results of the present project a clear indication that the views of information users and information managers are converging to a greater extent than ever before, with consensus around the failings of existing systems and the need for a more coordinated approach with data quality as the core priority. Perhaps the most compelling issues to emerge revolve around lack of direction and understanding of information strategy, lack of clearly articulated responsibility for information management (in line with Drucker, 2002, and Neyland and Surridge, 2002) and concerns about data validity which suggest that auditing of data quality is urgently required. The last point was supported by the staff survey, the findings of which indicated that quality, accuracy, and reliability of data are clearly of concern to users and where there is evidence of failings in achieving confidence in existing data sets. Improved systems for accuracy checking should be a high priority in future strategy.

Staff of the university tended to use existing data and information systems frequently, in particular administrative staff being very reliant on data for the execution of their duties. Respondents were on the whole comfortable with their computing skills, although on the whole those who had been longer in service tended to be less confident, suggesting a need for continuing training and development of staff. There is also evidence from the survey that staff training on specific existing systems may be desirable where staff are clearly unaware of the full functionality of existing systems.

Individual data sets are used predominantly by particular categories of user and, with the exception of student and external data, access to individual data sets such as HR information could usefully be tailored and restricted. Problems were also identified by users in relation to: data not being available in an appropriate format; the quality of course data made available on the Internet; and the systems' inability to deal with increasingly flexible forms of study.

In light of such problems and very significantly, over one third of staff feel impelled by deficiencies in current systems to create duplicate or parallel systems of their own. This is a very costly phenomenon, in terms of resources, and creates the potential for increasing confusion and mismatch of data, somewhat ironically allowing additional errors to creep into or embed in systems. Where, for example, the local School data set is accurate, there may be no sense of urgency in ensuring that the central data set is similarly corrected.

Suggested areas for improvement to existing systems indicate that respondents, although positive about their capacity to work with existing systems, are clear that things could be better. A single integrated system with the flexibility to cater for individual requirements would appear to be a high priority. Users are concerned again about achieving greater accuracy and reliability from systems that are quick and easy to use and designed to meet real operational functions, reflecting the current pattern of increasingly flexible and non-standard means of educational delivery rather than focused upon a single paradigm of full-time undergraduate provision.

These results would suggest that there is little room for complacency about current information strategy in higher education institutions, if the case institution may be regarded as not unique. Although information systems have proliferated and resources, both technological and human, have been dedicated to their development, there are still deficiencies in the ability of systems to meet the broad range of institutional need. Equally as systems have embedded, staff have failed to update skills and/or have found pragmatic ways around the limitations and failure of the central systems, with the result that deficiencies are not made manifest. Information strategy must step back to consider in depth the implications of current problems rather than simply put new systems in place which appear to cover the cracks - a strategy that has proven singularly fruitless, although sadly not uncommon, in the past. It will be interesting to monitor the progress of the JISC's new information strategy advisory service, as there is by no means any certainty that this agency will be able to impose sense and a critical understanding of the importance of a holistic strategy formulation in any more constructive a manner than has hitherto been achieved. Certainly to be successful the service would require far better data than are presently available about the measures of success in information strategy formulation and delivery.

Attempts have and continue to be made to establish integrated systems within universities or, in the 1980s, for consortia of universities (see, for example, the efforts of the HE/FE Records Management Group (2003), which aims to encourage consistency of practice and share best practice, and JISC (2002) projects aiming to deliver benefits such as joined up systems for institutions and an integrated student record ). Further research will explore the extent to which these initiatives have been successful and where they have failed, as has frequently been the case, which factors have influenced the outcome. Solutions which might apply across universities are clearly highly desirable: however, there are often difficulties associated with the identification and exploration of failure which have bedevilled the investigation of information systems. Truly critical, open, honest and reflective accounts of development projects are required to add to our understanding of system quality.

Universities, when planning and implementing information provision strategies, need to take into account to a much greater extent than they presently do the perceived or anticipated management information needs of academic Heads of Department.

The present authors would go further to suggest that it is only based upon a much fuller and more precise understanding of the complex and multifaceted needs of all users, internal and external, in all functional areas of the institution, that systems will be developed which are truly responsive and which function to meet overall university objectives.

How to cite this paper:

Marcella, R. and Knox, K. (2004) Systems for the management of information in a university context: an investigation of user need. Information Research, 9(2) paper 172 [Available at http://InformationR.net/ir/9-2/paper172.html]