Issues Affecting On-line ESL Learning: A Singapore Case Study

Many studies suggest that on-line learning is positive for the learner, but our experience in running an on-line ESL writing course for Chinese students at the tertiary level in Singapore indicates that there are various challenges to meet in the areas of motivation, feedback, self-directed learning and computer technology. Particular mention will be made in this paper, of difficulties in assessment, differences in perception of feedback, problems in encouraging students to be more self-directed and the need to cope with the unreliability of technology. We will examine these factors in an attempt to share our experience, raise issues and make recommendations.

Background

"Singapore is well on its way towards an information society, with higher PC and Internet adoption rates than other IT savvy countries..."

- Straits Times, 21 Jan 2000

The Ministry of Education in Singapore provides enviable access to computers and financial support, as part of the huge thrust to incorporate information technology in education. Keeping in line with this the English Language Enhancement Programme (ELEP) at Temasek Polytechnic, Singapore includes an on-line component called the Test Taking Skills (TTS) as supplementary work. ELEP is an intensive language and learning skills programme which culminates in LATIS, a Language Assessment Test for International Students seeking admission to Temasek Polytechnic. The curriculum focuses on language and learning skills and runs for twelve weeks.

Recognising the student need for test-taking skills, we designed the on-line test-taking skills course, which has several advantages:

it helps to prepare students for the written essay component of LATIS

it gives students opportunities to apply the skills they learned in the classroom and practise essay writing

it exposes students to computers and helps them build up their on-line skills

it is a powerful medium for peer learning

it provides an alternative learning environment

it prepares students for e-learning

In this age of building intelligent classrooms, providing opportunities to hone a variety of skills is of essence. One of the many purposes of using the on-line component is to provide an alternative learning environment.

There were many concerns regarding our on-line component, particularly considering the profile of our students. All the students in this cohort were from China, with a background in teacher-directed learning with little exposure to computers. They appeared to possess certain common characteristics when they joined the programme, which at times had an impact on their learning. They were diligent and keen to please the teacher, yet hesitant to participate and highly competitive.

The on-line test-taking skills course builds on the writing skills learned in the classroom. Although on-line tasks were graded, students had the option of doing the tasks in any order. Activities included multiple-choice questions, short answer questions, rewriting given texts using appropriate styles, essay planning, identifying topic sentences, assessing sample essays and essay writing practice.

The information for the analysis was collated from various sources including, tutor observations, verbal and written comments from tutors and students, students' on-line work, cyber tutor feedback and evaluation forms. As student feedback was not completely reliable, we found that ongoing tutor observation was a more useful indicator since

"Observation is one of the most reliable tools for determining how students are progressing in class and can be purposefully employed to gather information about a wide variety of learners abilities, skills, and competencies."1

When we evaluated our online course, we found that four inter-related issues influenced the effectiveness: motivation, feedback, self-directed learning (SDL) and technology. The importance of motivation overrides them all, making it difficult to draw absolute conclusions about any one factor in isolation.

Motivation

Many questions arose regarding student motivation:

Are students motivated to work on the on-line course?

What type of motivation do the students feel (extrinsic, intrinsic etc)?

What are the factors affecting their motivation?

Do students use their competitive nature for positive or negative motivation?

Is it possible to "know" about students' motivation at all?

We have identified four stages of student motivation throughout the on-line component. The dominant feelings of the students varied from initial apprehension to curiosity followed by a peak in interest and ending in a decline in interest.

1. Apprehension Stage

"In our class there are some students not familiarize computer, maybe they do not like to use computer to learn." [sic] -- student comment

The initial apprehension was probably due to fear of the unknown. This was understandable considering that the majority of the students had little or no experience using computers. It is possible that the students felt that the on-line component would be an additional burden to their workload.

2. Curiosity Stage

"During these days I felt my English has improved because the TTS [online course] created a English atmosphere so that I can study English step by step." [sic] -- student comment

We were relieved to observe that after the introductory briefing and technical skills training, most students began to feel more comfortable in the new learning environment. The factors that contributed to their increase in interest included:

familiarisation provided by the training

awareness of the importance of the course for the LATIS preparation

the presence of a peer who was an experienced computer-user and willing to help

the novelty of using a computer to learn English

an increase in confidence with the hands-on experience

As Pennington has pointed out, incidental learning is especially effective when using a computer.2 We found this particularly true at this curiosity stage.

Other factors like lack of computer skills, perception that on-line work is not "real" work, because it is not tangible, and overwhelming workload possibly affected student motivation in a negative way.

3. Peak Stage

"When I using computer, I felt very interesting and pleasant. I know it useful." [sic] -- student comment

Pennington suggests, "One singularly motivating aspect of computers exploited by micro worlds is the challenge of figuring them out."3

We observed that by the time students reached this stage, they had 'figured out' how to use the computers and had had greater exposure to the on-line materials, which resulted in improved motivation. Positive feedback from the tutor may have also helped in motivating the students. The students' familiarisation with the TTS materials and the equipment created a higher comfort level for most students.

4. Fall Stage

"Too many similar exercises and some topics are not interesting enough." -- student comment

We noticed that student interest in the on-line tasks began to wane in the fourth week. This was evident from the decline in the number of submissions we received and student comments.

Student feedback mentions the following flaws in our on-line courseware:

too much scrolling required

too much text

lack of graphics

too many exercises

no clear instructions for paragraph formatting (even the tutors did not know how to do this)

Often the effects of instructional design on learner motivation are underestimated. Careful thought needs to be given to all aspects of courseware design including clarity of instructions, as suggested by Keng-Soon:

"The students may perceive the instruction as boring or difficult to comprehend. They become discouraged because the effort required to get at the task may exceed the effort required to carry it out; as a result of low motivation, less language learning may occur."4

On the other hand, we were amazed at how quickly the students' computer skills improved and with easy access to the Internet, students began to explore other avenues like e-mail and web surfing. With greater exposure, their interests widened and we observed that students were not always on task in the lab. It is a paradox that we wanted the students to gain computer expertise, but we were less appreciative of them using that expertise to surf the web during school hours.

One other important factor influencing student decline in motivation was the fact that no grades were awarded for their on-line coursework. We decided not to give grades but rather feedback in the form of detailed comments. We are convinced that "elaborated or cognitive feedback"5 is more beneficial for the learning process than "outcome feedback"6 (numerical or letter grades). This is supported by Egbert, who states:

"If feedback is seen as information that helps the learners understand just completed tasks, or assists with present or future tasks, then assessment that gives information to the learner is an important type of feedback in the learning classroom."7

By not giving grades, we could also be more fair to the majority of the students in this cohort who had very limited computer abilities. In this way we could avoid the problem of having to separate language skills from computer skills.

It is a generally accepted phenomenon that on-line learning environments are far from perfect technologically. Some of the technical difficulties that affected learner motivation in our course included server problems, faulty passwords, printer malfunctions and loss of data. A severe impediment to the on-line programme came when all of the student submissions were inadvertently deleted. It was a struggle both for the tutors and the students to overcome the frustration and redo the lost assignments.

During the various stages of the course the tutor took on many avatars from technical trainer to facilitator to the unenviable role of police officer (restraining the students from surfing and e-mailing). At the same time, the students moved from the "fear of the unknown" in the apprehension stage to the novice, tutor-dependent stage followed by an independent stage with a comfortable level of discovery learning, in a very short time span.

This may be surmised from the fact that students lingered on after school hours to work in the lab, or completed their on-line tasks early to make time to use the computer for their own personal purposes. These students did not own computers at home and had to rely on the facilities in the school.

Though it is believed that students understood the importance of completing the TTS course as preparation for the assessment test, whether they realised that it contributed to the "process of learning" is questionable. Students seemed to focus more on passing the exam rather than on improving their English. This implies that their interest in TTS was governed by their extrinsic motivation rather than their intrinsic motivation.

On the positive side, we observed in the computer lab that students were more motivated to share information. Contrary to this, in the classroom, students seldom took the initiative to seek help from peers. There could be various reasons for active peer-learning in the computer lab:

As the students were not being graded, they were unable to rank themselves against one another. Therefore there was no fear of competition.

The on-line environment was new to most students, so their eagerness to overcome the unknown outweighed their fear of losing face.

The teacher took over the role of facilitator so the students may have felt more freedom to interact.

Though the lack of marks observably had a positive affect on their motivation to interact, at the same time it is possible that the students were demotivated to put forth a great effort for the very same reason.

In addition, they were hesitant to share their knowledge publicly on the electronic discussion board. This could be a result of the fact that it was an optional activity and students had the advantage of speaking face to face in the lab. Interestingly, the face-to-face sharing focused on computer skills rather than language skills, by the students' own admission.

Recommendations

Though there is no foolproof method to ensure motivation, we believe that the following recommendations can avoid some of the pitfalls:

Computer skills training should be provided on an ongoing basis -- this could mean building on the basic computer skills like use of the keyboard to more advanced computer features like "hard returns" for paragraphing and skills relevant to the particular on-line materials being used -- like posting work on the discussion board.

Communication channels between tutors and students should be open and continuous. In circumstances where the tutors and students have the opportunities to meet face to face, the on-line course and the students' progress could be discussed.

Exploit the Internet medium while designing materials. The WELL project shows that the "commonest use of Web resources is as a source of materials which are downloaded by the lecturer and distributed to students. Instances of interactive use or exploitation of communicative potential are very limited."8

We should modify the TTS programme so that other computer activities such as e-mailing and surfing can be used in a 'purposeful way'. It is not sufficient to simply be aware of student interests. Designing materials and tasks within the framework of the syllabus that exploits student interests should be the key.

Redesign the pages regularly based on feedback, in an attempt to improve the effectiveness of the programme. For example, we need to reduce the amount of text, increase the size of the response boxes to avoid excessive scrolling, eliminate the "delete" icon.

A buddy system for the students would be useful for peer learning and support when they are faced with technical or task-related difficulties.

We could give grades.

Feedback

What is "good" feedback? Does tutor feedback contribute to the learning process? What effect does it have on student motivation? Do students follow-up?

What is meant by feedback?

This section deals with the tutor's comments on the student work. As mentioned earlier, we decided that this would be more valuable and useful to the learning process than giving grades. But we must confess that this created some difficulties.

"The exercises should be added up..." -- student comment.

Different perceptions of good feedback

A difficulty was getting students to agree with the tutor on the meaning of good feedback. Good feedback for the students meant identification and correction of all language and content errors. For the tutors, good feedback started with positive and encouraging comments, pointing out errors but not correcting them, using leading questions to help students to reflect on their work and make improvements.9 On many occasions, getting students to think critically helped them to find and correct their own mistakes.

"... feedback aids learning but students need a lot of encouragement to use the feedback in order to reflect on their work."10

While tutors thought that asking leading questions and getting students to think for themselves was "good" feedback, sometimes the students were frustrated with the perceived "lack of help".

Students also saw long comments from tutors as tantamount to a poor student answer, which could be due to their cultural background. . For some students, long, detailed feedback from the tutor, however positive, was equated with faultfinding and loss of face.11 How can we help students change this perception in an on-line environment?

Students felt that they did not know where they stood on a norm-referenced scale.

No matter how useful the cyber tutor thought the on-line feedback was to the learning process, the students did not place the same value on the feedback. They wanted to know how well they were doing relative to their peers, which could be a reflection of their competitive nature. For students, detailed feedback was not equivalent to marking.

Communication was not cyclical, just IRF (Initiation-Response-Feedback)

Having gone through fifteen years of teacher-directed learning, students cannot be blamed for having difficulties adapting to self-directed-learning. A Hong Kong study on self-access centres talks about a similar problem.

"With the practice of teacher directed learning over the years at primary and secondary levels...students are so used to this way of learning. Students may be able to appreciate the flexibility and the autonomy provided in the self-access system, however, they may also have psychological and practical problems in employing this new method of learning."12

We hoped that students would be more forthcoming with self-selection13 in participating on the electronic discussion board in comparison to the classroom. However they were used to the teacher having the last word and they transferred this practice to the online learning environment. For example, tutors hoped that the students would follow up on the tutor feedback by contributing to the discussion board and /or by seeking clarifications via e-mail to the tutor. But students did not exploit the discussion board or the e-mail facility as much as we would have liked. One student commented, "if we could get chance to add to something after teacher's review that will be better", but she was not motivated to translate this idea into action, when given the chance.

While we need to find ways and means to help the students adapt to the new learning environment and expectations, we need to be realistic because it can be an insurmountable task within the short time frame of the course. Hence we recognise the importance of expanding the communication beyond IRF both in the classroom and the on-line learning environment. Learning can be effective when skills are transferred across the curriculum. Also the need to integrate the on-line component with the classroom activities is crucial.

We do not know whether the students were motivated to read the comments and apply them to their work.

Students agreed with the following statements regarding the cyber tutor's on-line feedback.14

The feedback on the exercises was often easy to understand.

The tutor's on-line feedback encouraged me to learn.

Despite this positive response, there is no reliable evidence to prove that the feedback really contributed to student learning. In fact, there was no formal follow-up system to monitor whether or not the students reflected on their work and/or made corrections or improvements. Nor did they take any initiative to seek clarification or help regarding the feedback. Reasons contributing to this could be fear of appearing weak, apathy, low motivation, or uncertainty of procedures.

This is symptomatic of transferring traditional classroom expectations to the on-line learning environment in that students still perceive themselves to be only receivers of information. They need to become more active learners and take the process further by becoming enquiring learners. At the same time, tutors have the responsibility of creating an atmosphere conducive to enquiry learning.

Unresolved issues

Feedback as a form of assessment is acceptable to tutors but not students.

As there are no marks, should we expect students to do the tasks?

Links between motivation and feedback need to be investigated.

Time for writing quality feedback is limited. Cyber tutoring takes more time than expected.

Maybe there isSelf-directed Learning sufficient time for students to follow up on the feedback.

Recommendations

The questions raised above do not have simple answers. Yet, it is useful to reflect on them and consider some solutions:

Define terms like "good feedback" clearly and clarify expectations between tutors and students continually.

Try to develop an understanding of why tutors shouldSelf-directed Learning spoon feed but empower the students to think for themselves in the on-line learning environment. This can be strengthened through discussions in class and in consultations.

Provide guidance on developing strategies to learn from mistakes.

Continue to encourage reflection and critical thinking skills throughout the programme.

Have the tutors post student work on the discussion board more regularly and ask for student comments.

Follow up feedback with some sort of monitoring system e.g. reflection journal or resubmission.

Self-directed Learning (SDL)

What is SDL? Does the on-line learning environment, provide more opportunities to develop SDL than the classroom?

"Give some direct or teach me how to do it" [sic] -- student comment

In an instruction-led on-line course the degree of interaction between the computer and the learner is not balanced and is therefore dependent on how motivated the student is to be a self-directed learner. Hence the student should take greater responsibility for his learning but this does not mean that he/she has to learn alone.

We believe that self-directed learning includes:

knowing when to seek help from the tutor and/or peers

seeking other sources of help

setting goals

recognising strengths and weaknesses

understanding the importance of learning from mistakes

having an inquiring mind

learning through discovery

working at one's own pace and managing time effectively

making decisions

The on-line TTS course provides opportunities for students to apply and practise the skills learnt in the classroom. The materials were so designed that students presumably could work on their own, at their own pace within the time frame of the course. On the electronic survey students agreed that "most of the materials helped me learn on my own".

We thought that the instructions were clear and self-explanatory to allow self-directed learning. The tasks were graded, but students had the option of doing the tasks in any order. But one student complained, "some assignments are in wrong sequence". And another admitted "I made a mistake once because I misunderstood it [the instructions]."

We believed that features like Frequently Asked Questions (FAQs), the electronic discussion board and e-mail would promote self-directed learning, would be exploited in an attempt at becoming more self-directed learners. It was hoped that shy students would participate more actively in this less personal environment. The tutor assumed that the on-line environment would be a preferable working environment because the students could take ownership or control of their learning, without the teacher's obtrusive presence. The general tutor perception was that SDL would be a natural concomitant.

Tutors often expected students to adapt to the learning styles quickly and easily. In reality, students cannot be blamed for their preferred learning styles and need to be taught how to adapt to the new styles. Tutors did not always take into consideration that they must instruct students in how to adapt to new systems. Do we as tutors feel that students should agree with our value judgements regarding learning methods?

Students were required to complete 17 writing tasks. Working on the discussion list was optional. But we hoped that they would take advantage of it, as we had spent considerable effort in the class extolling the virtues of peer learning. We were disappointed to note that the students did not post their work on the discussion board or use e-mail regularly to clarify doubts. Though the students were trained to use these features and were using e-mail for personal reasons, they did not extend them to schoolwork. We caSelf-directed Learning, however, assume that because the features are accessible, they will be used. There may have been many factors involved:

Students did not know how (to work on the task, use the discussion list, e-mail...)

Students forgot how

Students were not motivated

Students simply wanted to meet the requirements, then move to surf the Internet

Students were apathetic

Students did not have enough time

Students were afraid to ask for help

Students may have been threatened by the medium

Students were afraid to post work because of fear of criticism/exposure

Students' competitive nature prevented them from sharing information

It took too much time to post answers and wait for comments.

Though we have attempted to list some factors that may influence SDL, it is not possible to come to definite conclusions, as the students' cognitive processes are not always clearly evident. Coming from a teacher-directed background, the students need more time and experience before they can become more confident self-directed learners.

One other issue remains. How much guidance should we provide? Where does instruction end and facilitation begin? Perhaps it is ironic that the students need some guidance in how to become a self-directed learner, but we found that it was crucial to teach some basic skills like time management, decision-making, and goal setting. Students also needed to be aware of the importance of learning from their mistakes.

We can create an environment that is as conducive as possible to SDL using the on-line medium, but we found that in the end it is the student who determines the degree of SDL that he or she is capable of at any given time. This is supported by Boud et al:

"...only learners themselves can learn and only they can reflect on their own experiences."16

Recommendations

Have tutors select and post student work regularly -- for students to learn on their own, we must provide enough examples.

Remind students to check the summary of their submissions regularly.

Continue to provide opportunities for SDL and encourage students as much as possible.

Technical Problems

"First teach some students how to use computer. For my case, I spend so much time to research how to use it. In fact most of our classmates didSelf-directed Learning use computer before." [sic] -- student comment

Do technical problems affect learning? Can we distinguish between computer skills and language skills? What if the student knows more than the tutor?

There were a range of computer abilities among the learners, but the majority of students had little or no experience, though it should be noted that one student knew more about computers than the lab facilitator and was always willing to help his classmates.

In the beginning, students were impressed with the facilities available and the novelty of using computers was motivating. As mentioned earlier, their initial apprehension was eased with hands-on experience and most students quickly mastered the basic skills like using the keyboard and mouse, going to the TTS course URLs etc. In a short period of time, with increased exposure to computers, students' expectations soared. For example, students started to complain about the printing facilities, inconsistent use of passwords, the school's slow intranet speed, (probably comparing it to the speed of YAHOO) etc. Kelly observes:

"With the abilities of the technology come certain user expectations of what a screen is supposed to show -- expectations that come from experience with the cinema, television and computer games."17

One of the challenges that the tutor faced was dealing with complaints. Students expected the facilitator to be a computer expert as well as a language expert. Frustration levels mounted when the facilitator couldSelf-directed Learning solve the problems. Although there was a technical support officer on the team, some of these problems were also beyond her control.

How much technical knowledge/training do the lab facilitators and cyber tutors need, in order to work effectively in an on-line learning environment? There are two schools of thought regarding on-line learning. One is that students need training in basic skills before the course starts. The other is that students will learn as they go along.18 From our experience with the last two cohorts, we found the need to provide some initial training in basic computer skills and then allow opportunities for self-directed learning.

The technical difficulties faced by the students directly affected their motivation and hence their performance.

Studies reveal that the "student's difficulty in the manipulation of the software usually undermine the student's interest in the class." [sic]19

As mentioned earlier, the loss of student submissions was the most serious technical problem we faced. The frustration was evident in the student comments such as, "The work I have done are lost. That experience is unforgettable."[sic] This is something that can be addressed. But a clear distinction needs to be made between technological gaffes such as these that can be easily avoided in the next run and problems involving the server that are beyond our control.

These problems exist for the tutor as well. In addition, providing detailed feedback on-line demands more time than traditional marking because it is necessary to log on to the network, scroll through the student responses and deal with the fact that it is not easy on the eyes looking at the monitor for long periods. In certain circumstances, when tutors work at home, they must pay to remain connected to the network.

One student commented, "All the test-taking could do on paper. Nothing special."[sic] From this we gather that not there is a need to "teach to the many different learning styles that exist in the class -- a difficult if not impossible task."20

Although we recognize that e-learning will be an integral part of future academic life in our school, and we provide opportunities and encourage on-line learning, perhaps we should be more flexible and think about the implications of offering a paper and pencil alternative for those students whose learning styles don't match the demands of our on-line course. (How this can be implemented is beyond the scope of this paper).

Recommendations

Make students aware of facilitator's role and lack of technical expertise.

Ensure the presence of a technical support officer.

Admit that technical problems will always be a factor -- there is no flawless system.

Redesign the pages to eliminate the scrolling and delete button.

Train the facilitator and cyber tutors in greater depth based on FAQs.

Monitor the students and provide additional training when necessary.

Conclusion

Overall, the survey results of the TTS courseware indicate that the students were generally positive regarding materials, feedback and preparation for LATIS (see Appendix A). We have to bear in mind however, that the students' responses may not be a valid representation of their true feelings as they were keen to provide answers that would please the tutor. Current research also suggests that students are inaccurate reporters of what they have learned, sometimes claiming to have learned things that did not occur in the lessons being evaluated.21

Regarding the four issues: motivation, feedback, self-directed learning and technology, several concerns remain.

The difficulty in assessing student motivation in an on-line environment.

The conflict between the purposes of providing useful feedback and awarding grades.

The difficulty in matching tutor and student expectations of self-directed learning. (e.g. participation on the electronic discussion board)

The danger of applying teacher-centred classroom practises to the medium of on-line learning.

The difficulty in bridging the gaps in perceptions between tutors and learners.

Warschauer points out that the "effectiveness of CALL cannot reside in the medium itself but only in how it is put to use."22 An awareness of the need for change in the pedagogical practices, expectations and behaviours of both students and tutors is only the first step. Throughout the process of curriculum planning and course implementation we need to:

explicitly communicate the purpose and importance of the on-line tasks

integrate the on-line tasks with classroom programme

exploit student interests in the instructional design

develop strategies to encourage active peer learning

ensure that the tutors have necessary technical expertise

Despite the several concerns that have been outlined, it is desirable to incorporate IT into our ESL programme. It should be remembered that the TTS course is not a course in isolation, but well integrated with classroom activities.

We hope that with greater exposure to on-line learning environments, students and tutors will become more effective in their roles as independent learners and facilitators.

Acknowledgements

We wish to thank Mr. Phillip Towndrow, Ms. Yeo Bee Gek, Ms. Jill Loke, Ms. Tiew Lee Ching and the ELEP students for their valuable contributions to this research study.

Appendix A

Feedback that employs leading questions:

Dear Student,

Thanks for your answer. The style of writing here is certainly narrative. Yes, it is difficult to follow the writer's thoughts because as you have pointed out, he did not support his opinions. As you have said, it seemed as if there isSelf-directed Learning any main idea. What do you think the writer should do to improve his style of writing?

You have revised the essay and made the style more formal. This is appropriate for writing this essay. But you must be careful with the grammar as well. Can you correct sentence 2? What about linking words? Look again at sentence 3. Otherwise, the flow of ideas in your corrected version is smoother.

Regards,

Cyber tutor

Appendix B

Detailed feedback: An extract

Hi Student,

The write-up is only one paragraph from the essay and yes, you are right in that he has not explained why his friend is "cool"; he has indeed digressed from his main point.

However, I think you have misinterpreted the next point. When he wrote "that's when we really started know each other", he meant to explain when it was that they started to get to know each other. He has also used some contractions such as "it's", "'til" (meaning 'until'), and "'cos" ('because') and this shows the informal style he has adopted.

What you have written is good in terms of content and you have made sure that each main idea was elaborated on - that's good, also, your write-up is definitely more interesting. However, there are several mistakes in your expressions. For example, "I have a very good friend, Harry. It was very odd that we could become good friends as we did not know each other until we were teenagers..." You have made several mistakes in spelling as well. Do check your spelling for words such as "bike" - sometimes you spelt it correctly and other times wrongly!

Most importantly, in choosing a style, think about your purpose for writing, which is the audience, how do you want him/her to feel after reading your essay?