TOPIC 1: Evaluation, assessment and testing.
Formative and summative
assessment. Critarion referencing and norm referencing assessment
Handout 1
What is testing?
What is assessment?
What is evaluation?
Elicit answers from the Ps and sum up:
EVALUATION thinking carefully about something before making
judgment about its value, importance or quality
ASSESSMENT the evaluation of a students achievement or proficiency; a
process which involves gathering and documenting information about the
achievement, knowledge, skills and abilities of an individual.
TESTING an examination of somebodys knowledge or ability, consisting of
questions for them to answer or activities for them to carry out.
(Macmillan English Dictionary for Advanced Learners)
Handout 2
Assessment and testing
Assessment is the systematic collection, review, and use of information about
educational programs undertaken for the purpose of improving student
learning and development.
- Today's students need to know not only the basic reading and learning skills,
but also skills that will allow them to face a world that is continually changing.
They must be able to think critically, to analyze, and to make inferences
Assessment and testing considerably differ from each other. While
testing is formal and often standardized, assessment is based on a collection of
information about what students know and what they are able to do. In other
words, students are given the exact procedures for administering and scoring in
testing. In assessment, on the other hand, there are multiple ways and
methods of collecting information at different times and contexts (Law and
Eckes, 1995, p.29).
Dietel, Herman, and Knuth (1991) define assessment as any method
used to better understand the current knowledge that a student possesses
(online document). According to Mitchell (1992, in Law and Eckes, 1995, p. 29)
testing can be defined as single-occasion, unidimensional, timed exercise,
usually in multiple choice or short-answer form. For a long time, student
learning was measured only by testing in traditional school settings. Currently,
it is realized that there is not only one way of gathering information about
student learning. Furthermore, testing is seen as only one part of
assessment and a broader concept of assessment is being widely used (Kulieke,
Bakker, Collins, Fennimore, Fine, Herman, Jones, Raack, and Tinzman, 1990).
Handout 3 What Is Assessment?
Definitions
Various definitions of assessment and the role it plays in teaching and learning:
Assessment involves the use of empirical data on student learning to refine
programs and improve student learning. (Assessing Academic Programs in
Higher Education by Allen 2004)
Assessment is the process of gathering and discussing information from
multiple and diverse sources in order to develop a deep understanding of what
students know, understand, and can do with their knowledge as a result of
their educational experiences; the process culminates when assessment results
are used to improve subsequent learning. (Learner-Centered Assessment on
College Campuses: shifting the focus from teaching to learning by Huba and
Freed 2000)
Assessment is the systematic basis for making inferences about the learning
and development of students. It is the process of defining, selecting,
designing, collecting, analyzing, interpreting, and using information to increase
students learning and development. (Assessing Student Learning and
Development: A Guide to the Principles, Goals, and Methods of Determining
College Outcomes by Erwin 1991)
Assessment is the systematic collection, review, and use of information about
educational programs undertaken for the purpose of improving student
learning and development. (Assessment Essentials: planning, implementing,
and improving assessment in higher education by Palomba and Banta 1999)
Handout 1
The role of testing and assessment in Education
Testing and Assessment are often used as measures of quality of learning,
teaching, and education. Some purposes include
Yardstick of individual progress in classrooms
Placement/filter of educational opportunity
Barometer of national educational condition
Evaluation of quality of teaching
Diagnosis of learning problems
Agents of school reform
Activity 1. Tell the Ps to discuss the following questions in groups of four:
What is the role of assessment in teaching languages?
How do you assess? What assessment tools do you use?
Who develops language tests/assessment criteria for your students?
Activity 2 Invite groups to plenary discussion.
Distribute Handout 1 with different selection of assessment types and ask Ps to
categorize them according to the four language skills.
Handout 2
Role-play, critical response to websites or logs, letter of explanation or
complaint, discussions, summary of the story, review report on the article,
review of a film/book, CV writing, feedback to a peer, presentation, a letter to
the editor, debates, logs, round-table discussions, multiple choice tests,
true/false/not given statements, essay, report on a research writing,
simulations, a story writing.
Writing
Reading
Listening/Speaking
Handout 1
Role-play, critical response to websites or logs, letter of explanation or
complaint, discussions, summary of the story, review report on the article,
review of a film/book, CV writing, feedback to a peer, presentation, a letter to
the editor, debates, logs, round-table discussions, multiple choice tests,
true/false/not given statements, essay, report on a research writing,
simulations, a story writing.
Writing
Reading
Listening/Speaking
Handout 2. Matching assessment criteria to the assessment types
Write assessment criteria next to each assessment type in the table
Note: there may be more than one assessment criterion for each assessment
type and one criterion can be used for more than one type of assessment
Turn-taking, ability to give a brief summary of , accuracy, content and task
response, fluency, recommendation for another reader, organization, style,
ability to express/justify ones opinion, comment on authors use of language,
variety of vocabulary and structure, mechanics, asking and answering
questions, coherence and cohesion, ability to give own opinion about with
reasons, adequate support with examples, design, completeness, evidence of
regular listening, ability to detect authors point of view, layout, evidence to
critical thinking, ability to catch specific information from , accuracy of facts,
Accuracy of vocabulary and grammatical structures, appropriate register,
originality, clear structure, appropriate use of language in a formal setting,
correct answer
Assessment types
Assessment criteria
Role-play
2
Critical response to website or blogs
Letter of explanation or complaint
Discussions
Summary of the story
Review report on the article
7
Review of a film/book
CV writing
Feedback to a peer
10
Presentation
11
Letter to the editor
12
Debates
13
Logs
14
Round-table discussions
15
Debates
16
Multiple-choice tests
17
True/false/not given statements
18
Essay
19
Report on a research writing
20
Simulation
Fundamental Components of Assessment
Four fundamental elements of learner-centered assessment:
Formulating Statements of Intended Learning Outcomes statements describing
intentions about what students should know, understand, and be able to do
with their knowledge when they graduate.
Developing or Selecting Assessment Measures designing or selecting data
gathering measures to assess whether or not our intended learning outcomes
have been achieved. Includes
Direct assessments projects, products, papers/theses, exhibitions,
performances, case studies, clinical evaluations, portfolios, interviews, and oral
exams which ask students to demonstrate what they know or can do with their
knowledge.
Indirect assessments self-report measures such as surveys in which
respondents share their perceptions about what graduates know or can do
with their knowledge.
Creating Experiences Leading to Outcomes ensuring that students have
experiences both in and outside their courses that help them achieve the
intended learning outcomes.
Discussing and Using Assessment Results to Improve Teaching and Learning
using the results to improve individual student performance.
Summative assessment
The goal of summative assessment is to evaluate student learning at the end of
an instructional unit by comparing it against some standard or benchmark.
Summative assessments are often high stakes, which means that they have a
high point value. Examples of summative assessments include:
a midterm exam
a final project
a paper
a senior recital
Information from summative assessments can be used formatively when
students or faculty use it to guide their efforts and activities in subsequent
courses.
Summative assessment:
refers to the assessment of participants, and summarizes their development at
a particular time.
used to evaluate student learning, skill acquisition, and academic achievement
at the conclusion of a defined instructional periodtypically at the end of a
project, unit, course, semester, program, or school year.
Assessment of learning
Focuses on the outcome
provide teachers and students with information about the attainment of
knowledge
The goal is to evaluate student learning at the end of an instructional unit by
comparing it against some sort of standard or benchmark
Assess whether the results of the object being evaluated met the stated goals.
High stakes or high point value
Examples of Summative Assessment:
Final projects (Portfolio, Research paper, Travelogue)
Chapter and Unit tests
Semester tests (midterm and final tests)
Standardized tests
College admissions test (CFAT)
End-of-course evaluation (Advanced Placement or International Baccalaureate
exams).
Assessment types relate to the purpose of assessment i.e. why assessment is
carried out. Five frequently used assessment types are:
initial
diagnostic
holistic
formative
summative
Initial assessment Relevant initial assessment activities will give
information regarding learners, for example, any specific assessment
requirements or needs they may have, their learning style, or any further
training and support they may require. This should take place prior to, or when
learners commence a particular programme.
Diagnostic assessment (a part of initial) Diagnostic assessments help identify
(i.e. diagnose) a learners current skills, knowledge and understanding towards
a particular subject area. They can also identify English, maths and information
and communication technology (ICT) skills.
Holistic assessment This is a method of assessing several aspects of a unit,
qualification, programme, or job at the same time. It should be a more efficient
system, plus save time, as one piece of good quality evidence, or a carefully
planned observation would cover several areas. Holistic assessment enables
learners to demonstrate their performance and knowledge in a more
appropriate whole job context, than is the case with task by task assessment,
for example, as part of a job the assessor observes the learner carrying a range
of individual tasks, which occur concurrently, or consecutively. This assessment
can be much more efficient and cost effective if planned correctly, for both the
assessor and learner.
HANDOUT 2:
Formative assessment Formative assessment should take place continually.
Simply asking questions, and observing actions can help assess ongoing
progress. Giving developmental feedback can enable learners to progress
further, before a summative assessment takes place. Formative assessment is
usually informal and often called assessment for learning as it prepares
learners for formal assessment. It is also used by the teacher to identify if the
planned learning activity is achieving the expected outcomes, so allows for
during programme amendments Summative assessment Summative
assessments usually occur at the end of a session, programme, topic, unit or
full qualification.
Summative assessment is usually formal and often called assessment of
learning as it assesses that learning has taken place. Different subjects will
require different types of assessment. You need to consider what type of
assessment you need to use and why. You can then plan when, where and how
this will take place. For example: What - an initial assessment Why - to
ascertain subject knowledge & skills When - at the beginning of the
programme Where - at the organisation How - by questions and discussion
Whatever type of assessment you use with your learners, you need to ensure
you are assessing only what needs to be assessed, i.e. the requirements of a
unit from a qualification, aspects of a work role, or progress towards particular
topics.
Formative assessment
First, let's define the term assessment. Assessment is the process of observing
a sample of a student's behavior and drawing inferences about the student's
knowledge and abilities. Yes, many synonyms exist for assessment, such as test,
exam, etc. The use of the word 'assessment' promotes more
positive connotationswith students in the classroom. It should be used in place
of terms that indicate possible failure and negative outcome, and may cause
additional anxiety among students.
What is the difference between formative and summative assessment?
Formative assessment
The goal of formative assessment is to monitor student learning to provide
ongoing feedback that can be used by instructors to improve their teaching and
by students to improve their learning. More specifically, formative
assessments:
help students identify their strengths and weaknesses and target areas that
need work
help faculty recognize where students are struggling and address problems
immediately
Formative assessments are generally low stakes, which means that they have
low or no point value. Examples of formative assessments include asking
students to:
draw a concept map in class to represent their understanding of a topic
submit one or two sentences identifying the main point of a lecture
turn in a research proposal for early feedback
Summative assessment
The goal of summative assessment is to evaluate student learning at the end of
an instructional unit by comparing it against some standard or benchmark.
Summative assessments are often high stakes, which means that they have a
high point value. Examples of summative assessments include:
a midterm exam
a final project
a paper
a senior recital
Information from summative assessments can be used formatively when
students or faculty use it to guide their efforts and activities in subsequent
courses.
Formative assessment:
Assessment for learning
Focuses on the process
Monitor student learning to provide ongoing feedbacks that can be used by
instructors to improve their teaching and by the students to improve their
learning
Helps to identify students strengths and weaknesses and target areas that need
work.
Help faculties recognize where students are struggling and address problems
immediately.
Foster development and improvement within an on-going activity
Low stakes low or no point value
Examples of Formative Assessment:
Projects and performances
Writing assignments
Tests and quizzes
Asking questions
Traditional View on Assessment
Traditionally, it is believed that assessment (i.e. quizzes, final examinations, pop
tests, etc.) is the best way to motivate the students.
Because of this, the term assessment or testing created a negative image to the
students. It has become the greatest intimidator of students.
Thus, anxiety arose from the students when the word assessment is said, or
when assessments are given the teacher.
Assessment is seen as a competition with teaching,than a tool to improve
teaching (Heritage, 2007).
What is Formative Assessment?
WHAT IS NOT FA?
Formative Assessment is not an instrument, or event.
It is not used for grading!
It is not used as a punishment for students if they misbehave.
Definition:
Collection of ; practices that all leads to student learning improvement.
Tool for the teachers to determine what they need to do to move the learner
forward.
A technique to help the students optimize learning
Formative assessment is an assessment done during the instructional process
for the purpose of improving teaching or learning (Black & William, 2003)
What makes formative assessment formative is that it is immediately used to
make adjustments to help students learn the lessons better.
Delivering Formative Assessment
Handout 2
Extension Activity I
Read about Mr. Knotts first exam experience and identify areas for
improvement this year.
As you read, circle or highlight problem areas. Then, compare your findings
with those of Mr.Knott`s colleague on page 15. Mr. Knotts First Exam
Experience
Mr. Knott, now in his second year of teaching, found his first year quite
challenging.
Last year, he taught in a different school where assessment did not receive
much attention. In college, Mr. Knott took a testing course, but he still felt
out of his element with assessment. He worried that if he admitted he didn`t
understand things like specifications that he would get a poor evaluation. He
avoided assigning portfolios and projects since he didn`t see the point of
multiple measures of assessment. He based students` grades solely on the
midterm and final exams. As the date of the midterm exam approached, Mr
Knott still hadn`t checked where he should be in the syllabus because he
advocated
A just-in-time approach.
He wanted to see what he actual had covered by the midterm exam date. A
few
Days before the test date, Mr. Knott put together a midterm exam consisting of
a reading from a well-known TOEFL practice book, a written expression section
from a grammar
book that he used in college, and a listening activity from a recent unit in the
students` textbook. He left out writing because he thought that the written
expression section would suffice. He didn`t want to leak details of the test, so
he didn`t supply students with information about the exam. Students were
surprised by several new formats they had never encountered. Mr. Knott
made all formats objective to make grading easier. He had students swap
papers in classes to grade some sections, and his wife helped him grade the
rest. He did not use an answer key. Instead, when many students had
difficulty with an item, he gave everyone full credit. The scores were high, so
Mr. Knott concluded that the test was good. He didn`t do any analysis or
reflect on how the test could give him feedback on his teaching. He was too
busy to give student feedback or make notes about the exam. Besides, why
would he want to change something that worked so we?
Handout 3. One of Mr Knott`s new colleagues had these comments and
suggestions.
1: The Process of Developing Assessment 15
Mr. Sanchez`s comments and suggestions
I was a new teacher not so long ago, and I remember being overwhelmed by all
the things
I was supposed to know about but really didn`t Mr. Knott is a talented teacher
and open to constructive feedback, so I hope these suggestions are helpful to
him.
Comments
He didn't understand specifications. He didn't see the point of muitiple
measures of assessment and based grades on two exams.
He didn't plan ahead 0! Keep up the syllabus. He used material intended for
different purposes and different course outcomes. He used a listening activity
that students had already heard and processed. He left out assessment of
writing. He failed to provide information to students who were subsequently
surprised by new formats. He didn't prepare an answer key in advance.
Students and his Wife graded the exam. He didn't conduct analysis because of
high scores. He didn't give students feedback. He didn`t have time for
reflection, nor did he make any notes about the experience. Suggestions
Specifications are really important in our assessment program, so I hope he
WILL attend a workshop and get some experience using them. Again, in out
program we use many types of assessment, not just tests and quizzes. Feel
free to ask your colleagues to show you examples of alternative assessment
that work for them. This is a disserwce to students because some will miss
out on Important material. In our lesson plans we have to show that we are
on target. We schedule a whole range of regular assessments so our students
get regular formative feedback. Without following specifications or mapping
the actual course
material, it is unlikely that these materials were relevant. He didn`t really test
listening as a skill. Instead, he tested students` memories. In our program, the
development of writing skills is very important. However, we assess it in many
ways, so he needs to consult the
writing specifications for details.
In our department, transparency is very imponant we inform students about
what to expect. We always introduce new formats in a teaching context before
usnng them in exams. Good assessment reflects actual teaching both what was
taught and how it was taught. No wonder some items didn`t work!
There are some real security and reliability issues here, made worse by the
absence of an answer key. The scores were inflated because of grading
procedures. The students` difficulty with certain items indicated a real need for
analysis. Assessment is part of the teaching/learning cycle, and students
should learn from each assessment experience. They need feedback presented
to them in a usable form. Fortunately, Mr. Knott seems to realize that this was
not the way assessment should be developed. In our department. We set a
time to talk about what we've learned from major assessments. I hope Mr.
Knott WILL participate.
