NEW YORK INSTITUTE OF TECHNOLOGY

SCHOOL of EDUCATION Master of Science in Instructional Technology

Assessment & Evaluation

EDIT 690

Case Study Assignment

Afra Alzaabi & Rasha AlzaabiEDIT 690 Case Study Assignment

Introduction

The impulse of integrating technology in today’s classrooms became universal and this requires agood establish for introducing more effective professional development programs. Based on that,we went through three stages with specific group of teachers. Firstly, we have undertaken the needsanalysis since we wanted to determine the needs of using instructional technology for the teachersat Shaikh Zayed Center for Qur’an memorization. Utilizing surveys, interviews, and field-notesgave us a good indicator to the current participants’ needs. The gathered data showed thatparticipants need to learn ways of saving and sharing files, and submitting documents, so wedecided to introduce Google Drive and Google Classroom. After ensuring the needs ofparticipants, we proposed an instructional effectiveness plan for evaluating the results because itis very crucial to offer relevant training that meets participants’ needs. The next stage which cameafter was the implementation stage. After meeting with the senior manager and showing her thedata tools results (participants’ needs), we agreed on implementing the PD.

The main objectives for the PD were using Google Drive to save and share files and using GoogleClassroom to submit documents. There are two types of assessment to use with participants:formative and summative assessment. To ensure that participants have met the learning outcomesof the addressed goals, we used different tools to help us. From our experience, it is veryfundamental to see evidence of learners’ understanding and how they use their learning in newsituations and this occurs through using assessment tools. According to the Understanding byDesign framework, the second stage of backward design ensures that objectives are properlyassessed through using assessment tools (McTighe & Wiggins, 2012).

To start with summative assessment, adult learners are self-directed learners where they can usetheir critical thinking and connect between their experience and the new knowledge (Gutierrez,2015). An online survey created using Google Forms was distributed between participants toevaluate the learning process at the last day. Moreover, to evaluate participants’ learning at the endof PD sessions, we asked them to create folders (project-based) in Google Drive to save specificfiles shared through email and share them with their colleagues. After looking at participants’work, we were able to know which instructions would be adjusted. For the formative assessment,we focused on two tools; questioning strategies, and observations. To have successful sessions,

Afra Alzaabi & Rasha Alzaabi 2

EDIT 690 Case Study Assignment

questions were asked from instructors and participants which helped us to receive immediatefeedback on the learning process. While one of us was introducing a new idea, the other one waswalking around the room to make sure participants were on the right track and to write down notes.

Problem Statement

After analyzing the data that we got from the assessment tools, we determined that our participants’desire changed in some instructional technology areas, such as saving and sharing documentsbecause the survey showed that 85.5% of the participants agreed that they want to learn about itThey were using USB drives which is not effective as it spread viruses and cause documents lose.Therefore, we recommended the professional development course about using Google Drive inorder to help them to overcome the conditions.

Saving documents was one of the major issues that teachers confront. Based on the assessmenttools; one of the interviewed participants stated “As a result of having two duty-shits, the hugenumber of documents make me frustrated”. Moreover, another teacher commented on the surveythat “losing files from computer” was the main instructional technology issue that she used to have.Thus, it has been recommended to have a PD course with 3 sessions to teach them how to create agoogle drive account, create folders and save documents in those folders.

Sharing documents was another issue as most of the teachers agreed. One of the participantsemphasized in the interview that exchanging the teaching materials, such as power point and worddocuments by using USB devices is not practical at all especially with this big number of teachersat the center, and another one commented on the survey that “sharing teaching materials with otherteachers” was the main instructional technology issue that she used to have. Therefore, it has beenrecommended to have a PD course with 1 session to help the teachers to be able to make sharablelinks for the desired documents and send it among each other by using the Google Drive app.

Moreover, as the observation/field-notes showed that submitting frequent documents, such as

students’ monthly reports and attendance sheet to the center’s administration was another issuethat teachers have; they used the same ‘traditional’ method of putting them in a USB drive or evenprinting them out to be submitted. So, it has been recommended that participants continue the PDwith two more sessions to learn about another Google app; which was the Google Classroom. An

Afra Alzaabi & Rasha Alzaabi 3

EDIT 690 Case Study Assignment

account has been created for the center and all the teachers has been invited to that classroom tosubmit their documents easily.

Literature ReviewFormative and summative assessments are two types of educational evaluations with differentpurposes. These evaluation techniques can be used at various periods across the educational cycle.

Formative assessment is a continuous process which provides coaches with information to improvetraining programs while it is happening. Based on our research, criteria and goal setting,questioning strategies, and observations were used as formative tools. These tools were usedduring the four sessions to ensure participants’ awareness of their learning and to collect morefeedback on the learning process for immediate improvement. According to Derrell (2015), areliable formative assessment tool is the one that offers both teachers and learners opportunity toinspect the results and apply them for future practices.

Questioning was one of the most formative tools used by coaches and participants. In fact, wefocused on raising the level of questions because participants are adults. We noticed that when weraised the level of questions, participants asked more thoughtful questions which raised the levelof understanding. Based on research done on formative assessment in adult learning by (Swain,Griffiths, and Stone, 2006), it is more effective to use higher level questions with adults to inquiremore thinking rather than recalling information.

The second formative tool used during the sessions was observations. We focused on observingparticipants during the training programs since we wanted to identify their learning difficultiesduring the learning process. Furthermore, we wrote down some notes, so we used them to comparetheir understanding level through all sessions. One of the advantages of using observations wasthat we introduced one more session to ensure participants’ understanding in using GoogleClassroom as they faced some difficulties in uploading files. Observing learners provides valuableand immediate information about participants’ progression and what strategies can be used toimprove their learning as emphasized by Regier (2012).

Summative assessment is an effective method that coaches use in instructional technology PD

courses to measure the end results. In our research, we used two tools to focus on the outcomes of

Afra Alzaabi & Rasha Alzaabi 4

EDIT 690 Case Study Assignment

the training; an evaluation online survey and a given task where the participants created Googleaccounts and signed in to the Google Drive to save and share folders with other teachers, as wellas a Google Classroom account to upload documents, they were supposed to do it by themselvesindividually to examine their performance. Although the course has been done in different weeks(one session per week), we were able to do the summative assessment to measure the effectivenessof the whole process even after a period of time of having the first session. Based on the researchof Schilling & Applegate (2012) about the best method for evaluating educational impact, theyemphasized that “Summative evaluation represents a point in time usually either immediately aftertraining or longitudinally”.

In adults’ learning situation, the survey method is an authentic summative assessment tool thatgives the participants a sense of being part of the development process which could show validresults as Wiggins (1998) highlighted from the Indiana University research. Moreover, it is veryeffective for learners to express themselves in a manner that highlights valid aspects about theirlearning to help the instructors measure the effectiveness of their teaching in order to make futurechanges to enhance the performance.

The second summative assessment tool (the task that participants should fulfil) is another exampleof using suitable method for adult learners because their learning is goal-oriented and the task wasdirect to their goal of the instructional technology area of need which is saving and sharingdocuments. Therefore, the results of the summative assessment of this method was direct and clear.We found this so relevant to Pullagurla (2014) when he stated that “Once the learning goals havebeen identified, it is now imminent to align the learning activities such that these objectives arefulfilled within a certain period of time”.

Case Study

1. Description of Assessments used

Both summative and formative assessments showed results that helped us as coaches to determinewhat went well and what needs changes in the PD training. For example, the observation in theformative assessment indicated that few participants needed more time to finish the required taskwhereas the majority were able to finish the task on-time. Therefore, we implemented animmediate intervention and paired the participants to work collaboratively and help each other.

Afra Alzaabi & Rasha Alzaabi 5

EDIT 690 Case Study Assignment

The questioning method was effective to a certain point as some of the participants were able tointeract with the high level ones while the rest of them could respond to the basic questions only.

We think that the immediate intervention of action that we did during the PD based on theformative assessment affected the summative assessments results positively. The survey showeda very high level of satisfaction about the course as the participants wrote complements as well assuggestions for the next courses. Furthermore, the participants responded very well to the task-based tool at the end of the training which indicated a clear success for meeting the objectives ofthe course.

2. Data Generated by Assessments

Utilizing results of assessments is an important stage because it assists us as coaches to evaluate

the teaching content of training programs. We came up with some questions like, have theparticipants met the learning outcomes? Are the assessment tools relevant to the learningoutcomes? Do they effectively reflect what is being asked of participants? In fact, the data analysisgave us an answer to these questions. Qualitative and quantitative are two different types of datawhich was required for the results analysis.

Quantitative Data

Quantitative data refers to collecting information through numbers and it can be presented throughgraphs or tables. In this PD, we created an online survey using Google Forms application as aquantitative data source. The purpose of the survey was to collect more data about participants’understanding of the course content. To clarify, it showed that most of participants are able to saveand share files in Google Drive whereas some of them are struggling using Google Classroom.Using assessment tools that showed qualitative data was a good choice because we were able todetermine what the next step to take with participants is and it gave us an overall picture of theircurrent level of understanding as highlighted by Surbhi (2016). This tool was not hard

Qualitative Data

On the other hand, qualitative data is the type of information that cannot be measured. We focusedon observations as a qualitative data tool to obtain more reliable information. Our participants areadults, so they were more self-conscious when they were observed. This tool was not easy to

Afra Alzaabi & Rasha Alzaabi 6

EDIT 690 Case Study Assignment

implement because we had to select which areas to focus with participants during the session aswell as writing down some notes. Although observing participants was not easy, but it gave us aninsight into the bigger picture and it helped us to develop the rest of the training as mentioned inthe University of Surrey research (n.d).

Interpreting Data

Combining qualitative and quantitative data tells us the effectiveness of applying the trainingprogram and it also refers to the effectiveness of assessment tasks. In our case, we insisted on usingdifferent types of assessment tools because we wanted to get more accurate evaluations.Furthermore, putting interpretations and data together showed that we were able to meet the targetobjectives through using various assessment tools. Additionally, using various types of data raisedthe level of validity as we were able to depend on the data to evaluate participants’ understanding.

Decisions about Teaching and Learning based on Assessments

After reviewing the results of the evaluation tools we decided that the Google Drive will beimplemented officially in the center for the teachers to start using it after getting the permissionfrom the administration and then they will start using the Google Classroom, and the reason weput them in this sequence is based on the responses of the online survey (summative assessmenttool) after the PD course. Most teachers agreed that Google Drive is most important for them andthey want to start using it immediately because their priority is to keep their files safe and protectedas well as share them with each other without the risk of losing them. Consequently, GoogleClassroom will be used after the teachers become more familiar with the first google app and afterthe center creates an official account to be circulated to all teachers and used to submit all therequired documents.

In addition, the assessments did not only lead to current discussions, but they also informed otherindicators for future plans, such as using the Google apps from mobile devices and not beinglimited to the use of PC or desktop computers. Moreover, several teachers highlighted in the post-training survey that would be so helpful to use those apps with the learners and parents (of the KGsection) as well as to enhance the teaching and learning and collaborate by using the instructionaltechnology.

Afra Alzaabi & Rasha Alzaabi 7

EDIT 690 Case Study Assignment

Self-reflection of Collaborative Research

Collaboration in a research has so many advantages for both; researchers and the target group. Inour case, we found it very effective to collaborate in this project because we constructed ourknowledge about evaluation research by sharing and discussing the concepts to come up withinnovative ideas. Belcher (2010) highlighted that “When you have different people collaboratingon a project, then you get a greater sense of creative input”. We really found it useful to discuss asubject and see it from different points of view to expand the perspective of our thinking towardthe desired objective.

On the other hand, collaborative research could be challenging in many areas, such as timeconsumed. The time spent in negotiating the subject sometimes may cause a delay on the researchprocess. As SivaKumar (2016) emphasized, collaborative work can take much longer thananticipated. In addition, using similar language and following a logical sequence was anotherchallenge we faced during writing the report. However, we concluded that collaboration was agreat method that aimed at gaining powerful results throughout our success in planning, havingopen communication, respecting opinions and appreciating the effort of the other side.

Conclusions & Recommendations

To sum up, the main purpose of making professional development courses is to make changes inspecific and determined areas that participants need in order to enhance their performance. Ascoaches of 21 st century; the ‘instructional technology’ is a major target to focus on and to beharnessed for a very wide variety of fields, such as schools, hospitals, companies, etc. In our case,teachers of Shaikh Zayed Center was the focus group because we believe that it is very essentialfor educators of this era to use technology to adapt their teaching and learning. As stated in theHanover Research (2014) “Professional development provides educators the opportunity tounderstand new advancements and adapt their teaching styles and pedagogy to make effective useof available educational enhancements”.

After going through the process of evaluation research; starting from conducting the needsassessments tools moving to data gathering then analysis, we determined the instructionaltechnology area of need that our participants desired. Consequently, we made the PD course

Afra Alzaabi & Rasha Alzaabi 8

EDIT 690 Case Study Assignment

recommendation to train the teachers on how to use two tools of Google apps; the Google Driveand the Google Classroom in order to meet the main objectives of this intervention which aresharing, saving and submitting documents.

As coaches, we need to use evaluation tools to measure the effectiveness of the training course inorder to recognize the pros and cons of the process and to know what changes should be done inthe future to get better results. In our research, we used an online survey, as a summativeassessment tool to get feedback from the participants about the PD and to know their opinion aswell as their recommendations for next courses. We also used another summative assessment toolwhere teachers had to complete a task individually to assess their ability and to determine whetherthey met the objectives of the PD training or not. Furthermore, we used two formative assessmentinstruments; the questioning strategies and the observation method to monitor the learning processand to provide ongoing feedback. According to Carnegie Mellon University (2016), in adultslearning setting; the feedback of formative assessments can be used by coaches to improve theirteaching and by learners to improve their learning.

To ensure continues improvement of participants’ performance, coaches should always make

structured interventions to meet the variable needs. In instructional technology setting, theinterventions cannot be ended at any point because the technology is developing constantly.

Carnegie Mellon University. (2016). What is the difference between formative and summativeassessment. Retrieved on (January 21, 2018) from:https://www.cmu.edu/teaching/assessment/basics/formative-summative.html