Neil Jones
Surveys and Evaluative Studies
New South Wales Department of Technical and Further Education

Curriculum research in TAFE

Curriculum research in TAFE has typically been characterised by a reliance
on quantitative research methods - most often survey research. Such an
emphasis has served us well in providing useful answers to curriculum problems
and in collecting data to be used in the design, implementation and evaluation
of curriculum products.

During the years of diversification and high growth in the TAFE sector
following the Australian Council of Technical and Further Education Report
of 1974 (Kangan), survey methods served as the basic tool for many of the
new curriculum initiatives undertaken as a result of the directions heralded
in that report. A wide range of new courses were designed to meet hitherto
unmet community/industry needs and demands; statewide mechanisms were established
to undertake the systematic evaluation of courses, curricula and materials;
and representative or multi-disciplinary task forces or working parties
were commonly established to undertake curriculum planning and developmental
activities. In most States and Territories, centralised pools of curriculum
research and development professionals were established (or increased in
size) to meet the demands of these new curriculum initiatives.

That these initiatives relied heavily on survey methods for their research
base was probably due to a number of factors. The established market responsiveness
of TAFE to industry/community demands and governmental policies was served
by the adoption of recognised and unsophisticated market research (survey
or polling) strategies. As well, during the 1970s, TAFE curriculum researchers
in Australia often chose to adopt occupational analysis techniques which
had been demonstrated as useful by vocational and technical training developers
within the armed forces of both Canada and the United States of America.

The research skills which many of the new TAFE curriculum professionals
brought with them were those gained in the physical sciences and in psychological
measurement. Moreover, many of the senior decision makers in TAFE were
products of their training in engineering and/or the physical sciences.
It was therefore both pragmatic and purposeful to employ research methods
which TAFE had the capacity to use effectively. Further, such methods were
understood amongst TAFE's advisory clientele, including industry training
spokespersons. These methods most often resulted in quantifiable and comparative
data which could be interpreted readily to make evaluation, planning and
resource allocation decisions.

The first half of the 1980s has witnessed a number of changes in the
curriculum research milieu in Australian TAFE. Amongst these has been the
marked trend towards improving TAFE's responsiveness to community/industry
demands - a trend which has led researchers to seek faster, more economical
means for curriculum problem solving (Anderson & Jones, 1986). This
trend has been further pronounced in the past two years by the increasing
and more prescriptive accountability requirements of State and Commonwealth
governments.

A more subtle change has occurred in the backgrounds and experiences
which some recent curriculum professionals have brought to TAFE. The wider
recognition of TAFE as a legitimate and dynamic educational enterprise
has attracted educational professionals with more specialised curriculum
research qualifications and more diverse research experiences. Of significance
amongst these have been skills and experiences based in the social sciences,
leading to a greater capacity and interest in employing enquiry methods
from, for example, the sociological and anthropological domains.

Coupled with these changes has been an increasing awareness by TAFE
researchers that use of quantitative enquiry methods alone sometimes fails
to solve the curriculum problem in question or, at best, overlooks important
elements of the evidence that might be available, This awareness has been
accentuated by the increasing complexity of curriculum problems and the
political environment in TAFE, and by the recent sharper focus on equitable
allocation of scarce financial resources amongst a diversifying range of
competing TAFE priorities.

Case study approaches

TAFE researchers therefore have sought and experimented with methods
that are able to deal with the complexities and changes that have been
outlined and are realistic in a TAFE environment. Qualitative methods have
been employed, mostly in conjunction with the more established survey methods.
In the area of needs, or occupational, analysis, for example, Anderson
and Jones (1986) have identified a range of "group process methods"
and techniques that have been successfully adapted from other research
fields to TAFE curriculum. Included amongst these are the Search Conference,
DACUM (Developing a Curriculum), and the Delphi group method.

These methods generally have substituted the representative or random
sampling platform of survey research with the judgemental selection of
a small number of stakeholder representatives. In using these methods,
a small group of selected experts or stakeholders is "facilitated"
by a skilled group leader who abides by a set of "rules" which
vary depending on the theoretical basis of the method and "value orientations"
of the facilitator. In most instances, however, the rules emphasise the
importance of participants' behaviour being democratic, sensitive and communicative,
and the data collection outcomes are acknowledged as group outcomes generated
by consensus.

In other curriculum endeavours, particularly in evaluations of curriculum
products and processes, researchers have employed a range of ethnographic
or interpretative research methods (see Richards, Sharp, Ashurst and Funnell
in this volume). Researchers have employed a range of structured and unstructured
interview techniques and to a lesser extent have adopted action research
and participant observation methods in certain circumstances.

Case study research methods1 have also been used, sometimes as a sole
method of enquiry, but more often as a partial method. Interestingly, many
of the earlier examples of case study work in TAFE are found amongst the
research and evaluation literature relating to the Participation and Equity
Program (PEP), or the Transition Education Program prior to 1984. Examples
of these are found in the published reports of Hailstone and Pulsford in
South Australia (1981), Bartlett in Queensland (1980), Victoria's Choice
and Diversity Project (1984), Rustomji in NSW (1983a; 1983b), and researchers
who participated in the series of reports which formed part of the Transition
Education Case Study Project in Victoria, including Smith (1982). More
recently this case study research tradition has been continued by Thorn
and Chapman (1987) and Chapman, Lappan and Thorn (1987) in their innovative
evaluative work on the New South Wales TAFE PEP, which has included the
preparation of both student and community centred case studies. In some
instances the research was undertaken by researchers from outside TAFE.
In some others at least, the research methods were selected by researchers
who at that time were newly recruited to TAFE, often as temporary or contract
researchers employed to undertake a PEP or other specific research study.

Frequently the instances of curriculum case study research undertaken
in TAFE have been retrospective - looking back on a curriculum process
or activity, or building a picture of the perceived impact of a curriculum
product or process after the event. In addition to some examples of retrospective
case studies amongst the PEP researchers noted earlier, two of the studies
contained in this volume are of this nature (McBeath, Furber). Others are
found in the macro-evaluations of national TAFE programmes, and are evidenced
in the works of Broderick (1981), Poole and Kuhl (1983) and Jones (1983).

Researchers who have employed case study methods most often justify
their selection of method in terms of their wish to gather qualitative
data - data which are subjective and judgemental and which afford a richness
of understanding of the complexities of the curriculum problem which does
not often emerge from survey research.2 This reason would appear to reflect
the developments or adaptations to research approaches being made by researchers
in response to the changed TAFE environment outlined earlier. Curriculum
decision making in TAFE requires a better understanding of the complex
educational, economic, political and social issues that might be involved,
as well as of the perceptions of different stakeholder groups, than is
generally possible from quantitative methods alone.

The writer has employed case study research method on several occasions
for evaluating curriculum processes and products in a national TAFE context
(Jones 1983; Anderson & Jones 1986; Jones & Krzemionka 1987). In
these instances, case studies were supplemented by other research methods
in varying degrees. The reflections on case study research methods and
practice that follow draw largely from the experience gained in undertaking
those national projects.

National Core Curricula case studies

The prime method of data collection for the project An evaluation of
the development and implementation of National Core Curricula in Australian
TAFE (Jones 1983) was case study. Case studies of seven national core curricula3
initiatives were undertaken during 1982. Three of the case studies were
retrospective in that their subjects were curriculum initiatives that already
had commenced and, in two instances, completed prior to 1982. Four of the
case studies were concurrent in that they commenced at the same time that
the curriculum initiative commenced in 1982. These four case studies were
also participative, in that they were undertaken by the researcher who
was simultaneously a participant in the curriculum initiative.

The retrospective case studies were constructed following interviews
with key participants in the curriculum development process (including
curriculum professionals, TAFE teachers and senior teachers/administrators)
and with a small sample of industry representatives who were familiar with
TAFE curricula in the subject area under study. The concurrent case studies
were constructed using a diary method by the researcher, who was also participating
in the curriculum development process as a curriculum adviser.

The seven case studies were undertaken to enable evaluative comparisons
to be made in relation to the effectiveness and efficiency of the process
of curriculum design, the quality and relevance of curriculum outcomes,
the level of awareness and adoption of the curriculum outcomes by teachers
and the effectiveness of implementation. The purpose of the evaluation
was to design a model or models for the development and implementation
of future national core curricula activities.

The three retrospective case studies required a more intensive research
effort to secure data which would be as comprehensive as the data obtained
for the concurrent studies. The concurrent studies were served effectively
by the use of a diary of events, decisions and observations made by the
researcher, both during and immediately after participation in the curriculum
development process. For the three retrospective studies, however, effective
data collection was dependent firstly on a complex process of tracking
key people and documents across all of the State and Territory TAFE Authorities.
Often this tracking process led the researcher down the wrong track! Once
the right track was identified, there followed the exhaustive process of
gathering the required data by interview, often up to seven hours long,
and document examination. Over 100 people were interviewed for the three
retrospective case studies. Collection of the required data for these was
undertaken over a 12 month period and consumed approximately the equivalent
of half a working year. By comparison, data collection strategies for the
concurrent participative case studies were time efficient. They could be
predetermined to a large extent; they were directed towards single data
sources and locations; and they were task- and process-oriented rather
than people-oriented. Data collection for each of these four case studies
required approximately the equivalent of one week.

Reflection on the different qualities of the two case study approaches
also leads to comparisons of research effectiveness and validity. The involvement
of the researcher as a participant in the concurrent case studies raises
questions concerning the validity of the observations made by the researcher,
especially since the ultimate purpose of the observations was to be evaluative.
To what extent, for example, was the researcher able to distance himself
from the curriculum development initiative in order to make unbiased observations
about the curriculum development process? And to what extent was the researcher
able to perform his dual roles as researcher and curriculum development
facilitator effectively? Would not effective participation in the curriculum
development process preclude, or at least diminish, an effective research
effort? Moreover, what possible confounding effect did the double role
of the researcher have on the other participants in the curriculum development
initiative? Indeed did (or should) they know of the double role?

The writer is not able to provide definitive answers to these questions.
Suffice to say, the researcher was aware of the complexities and potential
conflict of roles and, from the outset, resolved to place greater priority
on participation in the cooperative curriculum development act rather than
the research act, lest his recognition as a participating member of the
group might not be secured. It was felt that failure to gain recognition
in this way would have undermined both his purposes as a curriculum developer
and as a researcher. This preference for engagement in the curriculum development
act, therefore, probably diminished his effectiveness in the research act.
Others will have to reflect upon the questions concerning research validity
which arise from the objectivity/subjectivity of the researcher's position
in the research environment. The other participants were aware of the researcher's
dual role and did not manifest any concern about it.

Despite concerns for the researcher's participant position in the research
environment in the concurrent case studies, it is likely that the qualitative
nature of the research data collected was enhanced because of this position.
It is felt that the participant position enabled the researcher to form
a deeper understanding of the curriculum development process under study,
leading to the identification of important process factors (such as the
potency of States' rights in core curriculum decision making) that might
otherwise have been overlooked. Also, as a participating member the researcher
was able to gain additional insights arising from the dynamics of the group.

Whilst the concurrent case studies afforded greater research power in
the examination of the curriculum development process, the retrospective
studies appeared to afford greater power to the examination of curriculum
outcomes. There was less opportunity to reflect on outcomes in the concurrent
studies quite simply because there were fewer outcomes at that stage. This
research advantage, however, also appeared to derive from the opportunity
for introspective reflection by participants in the one-to-one interview
situation. Often, observations about unintended outcomes, or unanticipated
process factors, were made by interviewees who had been able to form an
impression about the whole curriculum development process in the light
of the outcomes and their experiences in implementing outcomes.

The concurrent case studies were necessarily formative in nature. By
comparison the retrospective studies enhanced the researcher's capacity
to make summative judgements about the curriculum process and outcomes.
While no measures of case study impact were recorded, either during or
after the studies, it appears to the writer some years after the event
that the retrospective studies had a greater impact on determining future
policies and processes for national core curriculum development than did
the concurrent studies. It is possible this was due to the primary audience's
(TAFE administrators) greater readiness to place weight on summative evidence
rather than formative evidence. If so, this would have followed, at least
in part, from the methodological difference between the two types of case
study. It is also thought likely that the audience had a greater familiarity
with a research method which, although not a survey method, did have a
closer resemblance to the traditionally used quantitative research methods
based on representative sampling strategies.

Case studies of curriculum research methods

In 1983/84 a national study of TAFE curriculum research methods was
undertaken. The study resulted in the report TAFE curriculum research:
A review of group process methods (Anderson & Jones, 1986). The purpose
of the study was again evaluative - an evaluation of the relative merits
of different curriculum research methods to the TAFE context in Australia.
The study was particularly concerned with examining research methods which
held promise for enabling TAFE systems to make speedy and cost effective
curriculum development responses to changes in industry/community requirements.
Case studies of curriculum research methods were an integral part of the
design for the project.

As in the discussion arising from the national core curricula project
above, the case studies that were undertaken displayed qualitative differences.
These differences likewise followed from the different methodologies used
to study each, which followed from the different circumstances of the research
method being studied. Some case studies were generative in that the case
study data were generated from live (and video-recorded) examples of the
research method in practice in a TAFE context. Others were projective in
that they relied substantially on methodologies (found in the literature)
in order to project what a particular method would be like if used in a
TAFE context. The generative case studies were also in part participative.

It would again be possible to make some methodological comparisons between
these two kinds of case study (the generative and projective). The comparisons
however, would appear to have less efficacy than in the discussion in the
previous section because of the quite different circumstances and purposes
of the case studies, and because of the different levels of data accessibility
for each research method that was studied. Some useful reflections however,
are possible.

In the writer's experience, there is no one case study method or approach.
The use of case studies should not be constrained by past or model case
study practices, although intending case study practitioners of course
should learn from and take advantage of the experiences and reflections
of other practitioners. Most frequently, however, research problems and
circumstances have unique characteristics. As is the case with social enquiry
methods in general, the researcher should not be encouraged to duplicate
a case study method that has been used successfully in other circumstances.
Rather the researcher should be encouraged to adapt and modify successful
approaches depending on the circumstances for the proposed study. For example,
the references to retrospective, concurrent, participative, generative
and projective kinds of case study made earlier are not text book classifications
of case study methods. They are quite simply stylistic descriptors for
case studies that have been conducted, and they arose from the different
circumstances and issues of the research problem at hand in each case.

Nor are case studies only suitable for serving a single kind of research
purpose or agenda. Case studies can and should be used in varying research
circumstances and for the resolution of different kinds of curriculum problems
and issues. My own experience has shown me the value of case study enquiry
methods in examining (evaluating and improving curriculum development and
implementation processes and products (in relation to National Core Curricula
for example). They also have proven valuable in the provision of the building
blocks for the development of training models in both curriculum development
and research methods (as in the writer's study of curriculum research methods).
The work of the PEP researchers already mentioned, and reflections on the
third study outlined below, provide evidence that case studies have been
integral to strategies aimed at promoting and disseminating descriptive
and instructional material and information on new curriculum initiatives.

Case studies of schools/TAFE cooperative programmes

In 1985/86 a national study of schools/TAFE cooperative programmes
was undertaken. This study resulted in the report Schools TAFE cooperative
programs: A review of Australian practices (Jones & Krzemionka, 1987).
The purposes of the study were descriptive, evaluative and promotional.
It sought to identify the range and nature of Schools/TAFE cooperative
programmes in Australia, to identify curriculum and accreditation issues
in the design and implementation of such programmes and to promote the
development of programmes that were considered to have potential for enhancing
the post-Year 10 study options and pathways of young people.

Case studies of 14 cooperative programmes were conducted. These case
studies formed a major part of the research effort for this project. Because
most of the 14 case studies were to be undertaken under commission by on-site
practitioners, the research team designed a set of case study guidelines.
The guidelines were intended to achieve comparability of ease study reporting
across the 14 studies. They guided writers to report their case study in
terms of the history of the cooperative programme, its design process,
how the programme had been accommodated within the full range of school
and TAFE college offerings, a description of 33 pre-determined programme
characteristics, impact of the programme, and any evaluative impressions
of the programme.

The remote control management of these case studies, compared to the
direct control experienced in the studies reported in the sections above,
enables some useful practical observations to be made.

The case study guidelines that were designed were detailed and structured.
They did not require the case study writer to pursue a specific data collection
methodology. Rather they required the writer to report the study around
key headings noted above. Reporting control, rather than methodological
control, was sought because it was known that some of the on-site practitioners
necessarily would write their case study concurrently; others retrospectively.
The guidelines were effective in achieving reporting control and, in so
doing, made the subsequent analysis of case study findings, to be undertaken
by the project team, fairly straight forward.

Some case study writers collected data by individual interview using
a checklist of questions, whereas others used group interviews with a checklist.
In some cases, writers relied on document searching and analysis and then
validated their findings with key participants in the programme initiative.
Methodologically these approaches were quite different but, as noted in
relation to the case studies of curriculum research methods referred to
earlier, the different methodologies arose from the different circumstances
for each. In any case they served well the research purposes of the project.

Selection of the 14 cooperative programmes to be studied was made from
over 200 such programmes that had been identified by the project team in
the States and Territories. This selection was made according to 17 selection
criteria which took account of programme quality factors (eg, was the programme
considered to be exemplary?), locational factors (eg, was the programme
geographically and financially accessible to case study?) and equity factors
(eg did the sample of 14 programmes selected include one or two that were
designed to meet the specific needs of targeted groups, such as females?).
The 14 programmes selected for case study were thus not intended to be
representative of the 200-odd programmes that existed in Australia. Rather
they were selected purposefully, so that critical instances (of exemplary
or interesting practices) were included in the sample to facilitate the
collection of the required qualitative data. In this way the sample was
neither random nor representative.

The project's methodological concerns for representativeness and validity
were accounted for by a population survey that was also undertaken as part
of the project. The survey part of the project provided evidence which
enabled the project team to draw conclusions about the scope and nature
of cooperative programmes in Australia. The case study part of the project
enabled the provision of qualitative data about their design and implementation
processes, as well as about the complex curriculum, organisational and
industrial issues involved in their effective design and delivery.

Ten of the case studies were classified as major studies; four were
classified as minor. This difference arose from the project team's judgement
concerning each case study writer's circumstance, availability and capacity
to undertake the study to the level of rigour that would be required to
accept the study as a major case study. Writers who were unable to deliver
a case study because of lack of time or expertise, for example, were encouraged
to submit a minor study. The four minor case studies received were consequently
less informative and comprehensive as research tools, but were nevertheless
still important in providing research evidence since they focused on programmes
that had characteristics unique to them. The classification of case studies
into major and minor groups was therefore one that arose from the circumstances
of the project. Minor studies were not discouraged because they lacked
the apparent rigour and design of the major studies; rather they were encouraged
because it was felt they would make a useful contribution to the research
purpose at hand. Indeed the project's overall research design was adapted
to permit the contribution of the minor studies. Indeed the project's overall
research design was adapted to permit the contribution of the minor studies.
In this way the case study guidelines did not serve to constrain the case
study approaches used. To achieve data that were useful to the project's
purpose, the case study requirements were administered and designed flexibly.

Case studies written by commissioned writers were negotiated in four
different ways. Four of the major studies were undertaken as formal commissioned
research. For these, direct negotiation between the writers and the project
team produced a contract which included a fee-for-service arrangement,
specifications about conditions of ownership of the case study and a time-frame
for delivery of the report. Two others writers were contracted less formally
to undertake their work on an honorarium basis - a nominal small fee was
paid. Others were undertaken following the granting of formal approval
by the writer's employing agency to undertake the work as part of their
normal duties. For these no fee was paid. The last kind of arrangement
adopted was quite informal. For these the writers undertook to complete
the work gratis, as an unendorsed service to the project.

Experience with these different modes has shown that the informal and
on-duty modes are clearly least costly to the project's budget. Successful
case study outcomes by these modes rely substantially upon the writer's
own level of commitment to and interest in the project. Their outcomes
are, however, also less reliable in terms of both timeliness and comprehensiveness.
Draft case studies provided by these modes were sometimes late and in some
cases deficient in content. By contrast the fee-for-service and honorarium
modes are more expensive and more reliable in delivery of outcomes.

In respect of quality of case study reporting, it was found that this
was dependent mainly on the case study writer's level of research experience
and expertise. Researchers who were engaged to write case studies generally
provided higher quality drafts than did writers who were selected because
of their participant position in the programme initiative (such as teachers).
This experience would therefore seem to have suggested a trade-off between
costs and standard of report delivery, which is a useful lesson for those
contemplating large scale research projects requiring case study work.

Summary observations

The writer's experiences in employing case study approaches to research
curriculum issues in TAFE in Australia have varied in terms of research
purposes and designs, data collection strategies, reporting styles, case
study management types, as well as modes (external and internal) of case
study contribution. These experiences have resulted in the belief that
case study approaches significantly enhance a researcher's capacity to
undertake useful and informative research in TAFE, particularly where fine-grained
research evidence is needed to help in the understanding and evaluation
of curriculum processes and outcomes.

An important lesson for the writer has been an appreciation of the need
for flexibility in designing case study methodologies. Case studies provide
a powerful means for gaining precise insights into the process or product
being studied. To reap the full potential by these means, it is important
to design the case study methodology to suit the circumstances of the research
object and the resources and capacities of the means for the research.
Case study research should not be straight-jacketed by theory or past practice.
It is an imaginative and powerful research strategy that will require the
researcher to employ every trick in the research text book (and some outside)
to be effective in delivering useful and credible research outcomes.

Endnotes

For the purposes of this paper, "case study research method"
in curriculum design, evaluation or implementation is taken to mean the
undertaking of curriculum research in design, evaluation or implementation
by the detailed study of a small number (sometimes one only) of critically
selected instances (or selected perceptions) of a curriculum activity,
product or process.

The writer is currently undertaking a study examining, among other
things, the reasons why researchers select the case study method.

A national core curriculum in Australian TAFE is a cooperative inter-State
curriculum development activity which usually results in the design of
a common or shared core curriculum. The core curriculum objectives are
determined by a process of agreement amongst designated curriculum representatives
of the participating States, generally involving a series of some five
or six task force meetings over a period of two to three years. The cooperative
activity also may result in agreements to design new, or share existing
curriculum materials such as teaching/learning resources, or assessment
materials.

A Schools/TAFE cooperative programme is a course, or subject of study
provided for students at the level of Year 11 and/or 12 secondary schooling
for which the curriculum (including resources and teaching inputs) and
accreditation of the course or subject have been negotiated between the
Education and TAFE authorities.

References

Anderson, A. & Jones, N. (1986). TAFE curriculum research: A
review of group process methods. Adelaide: TAFE National Centre for
Research and Development.

Broderick, J. (1981). An investigation into the curriculum development
processes in TAFE in Australia: A comparative analysis. Prepared by
the South Australian Department of Technical and Further Education for
the Technical and Further Education Council.

Chapman, G., Lappan, K. & Thorn, W. (1987). Life after PEP: A
series of case studies of students from the Participation and Equity Program.
Sydney: NSW Department of TAFE. Choice and Diversity Project. (1984). In
Better schooling: A national project in action. Victoria: Choice and Diversity
Project.

Jones, N. (1983). National core curricula: Development and implementation.
Centre Report 1, Adelaide: TAFE National Centre for Research and Development.

Jones, N. & Krzemionka, Z. (1987). Schools/TAFE cooperative
programs: A review of Australian practices. Volume V of a report prepared
for the Commonwealth Schools Commission and the TAFE Council of the Commonwealth
Tertiary Education Commission, Adelaide: TAFE National Centre for Research
and Development.

Kangan, M. (Chair) (1974). TAFE in Australia: Report on needs in
technical and further education. Canberra: Australian Government Printing
Services.

Poole, K. & Kuhl, D. (1983). Evaluation of the implementation
of the nationally common curricular content and practices in Plumbing,
Gas fitting and Draining. Adelaide: South Australia Department of Technical
and Further Education.