ASSESSING SPEAKING

The paper is written to fulfill the assignment of

“English Instructional Evaluation”

Lecturer :
Pryla Rochmawati, M.Pd

Compiled by:
Iis Rulianawati (204180036)
Khuriyatul Mala (204180044)
Mumtazza Nur Aini .R. (204180053)

English Education B / 5th Semester

ENGLISH EDUCATION DEPARTMENT
FACULTY OF TARBIYAH AND TEACHER TRAINING
STATE INSTITUTE OF ISLAMIC STUDIES PONOROGO
2020/2021
ASSESING SPEAKING

A. Basic Types of Speaking

We cited four categories of listening performance assessment tasks. A
similar taxonomy emerges for oral production.
1. Imitiative
At one end of a continuum of types of speaking performance is the
ability to simply parrot back (imitate) a word or phrase or possibly a
sentence. While this is a purely phonetic level of oral production, a
number of prosodic, lexical, and grammatical properties of language may
be included in the criterion per- formance.
2. Intensive
A second type of speaking frequently employed in assessment
contexts is the production of short stretches of oral language designed to
demonstrate competence in a narrow band of grammatical, phrasal, lexical,
or phonological relationships (such as prosodic elements-intonation, stress,
rhythm, juncture). The speaker must be aware of semantic properties in
order to be able to respond, but interaction with an interlocutor or test
administrator is minimal at best.
3. Reponsive
Responsive assessment tasks include interaction and test
comprehension but at the somewhat limited level of very short
conversations, standard greetings and small talk, simple requests and
comments, and the like.
4. Interactive
Interaction can take the two forms of transactional language,which
has the purpose of exchanging specific information, or interpersonal
exchanges, which have the purpose of maintaining social relationships. (In
tfie three dialogues cited above, A and B were transactional, and C was
interpersonal.) In interpersonal exchanges, oral production can become
pragmatically complex with the need to speak in a casual register and use
colloquial language, ellipsis, slang, humor, and other sociolinguistic
conventions.

1
5. Extensive (monologue)
Extensive oral production tasks include speeches, oral presentations,
and story-telling, during which the opportunity for oral interaction from
listeners is either highly limited (perhaps to nonverbal responses) or ruled
out all together.1

B. Micro and Macroskills of Speaking

A similar list of speaking skills can be drawn up for the same purpose
to serve as a taxonomy of skills from which we will select one or several
that will become the objective(s) of an assessment task. As same as
listening, speaking also have two types of skills. They are:
1. Microskills
The microskills refer to producing the smaller chunks of language such as
phonemes, morphemes, words, collocations, and phrasal units.
2. Macroskils
The macroskills imply thespeaker's focus on the larger elements fluency,
discourse, function, style, cohesion, nonverbal communication, and
strategic.

As we consider designing tasks for assessing spoken language, these

skills can act as a checklist of objectives. While the macroskills have the
appearance of being more complex than the microskills, both contain
ingredients of difficulty, depending on the stage and context of the test-taker.2

C. Designing Assessment Tasks: Imitative Speaking

Endless repeating of words, phrases, and sentences was the province of
the long-since-discarded Audiolingual Method, and in an era of
communicative language teaching. An occasional phonologically focused
repetition task is warranted as long as repetition tasks are not allowed to
occupy a dominant role in an overall oral production assessment, and as long
as you artfully avoid a negative washback effect. Such tasks range from word

1
H. Douglas Brown, Language Assessment: Principle and Classroom Practice, (San
Francisco, California: Longman, 2003), 141-142.
2
Ibid., 142.

2
level to sentence level, usually with each item focusing on. a specific
phonological criterion. In a simple repetition task, test-takers repeat the
stimulus, whether it is a pair ofwords, a sentence, or perhaps a question (to test
for intonation production).

D. Phone Pass
PhonePass widely used, commercially available speaking test in many
countries. Among a number of speaking tasks on the test, repetition of
sentences (of 8 to 12 words) occupies a prominent role. The Phone Pass test
elicits computer-assisted oral production over a telephone. Test-takers read
aloud, repeat sentences, say words, and answer questions. With a
downloadable test sheet as a reference, test-takers are directed to telephone a
designated number and listen for directions. The test has five sections:
1. Part A:
Test-takers read aloud selected sentences from among those printed on the
test sheet. Examples: a) Traffic is a huge problem in Southern California,
b) The endless city has no coherent mass transit system. c) Sharing rides
was going to be the solution to rush-hour traffict, d) Most people still want
to drive their own cars, though.
2. Part B:
Test-takers repeat sentences dictated over the phone. Examples: "Leave
town on the next train."
3. Part C:
Test-takers answer questions with a single word or a short phrase of two or
three words. Example: "Would you get water from a bottle or a
newspaper?"
4. Part D:
Test-taker hear three word groups in random order and must link them in a
correctly ordered sentence. Example: was reading/my mother/a magazine.

3
5. Part E:
Test-takers have 30 seconds to talk about their opinion about some topic
that is dictated over the phone. Topics center on family, preferences, and
choices.3

E. Designing Assessment Tasks: Intensive Speaking

At the intensive level, test-takers are prompted to produce short stretches
of discourse (no more than a sentence) through which they demonstrate
linguistic ability at a specified level of language. Many tasks are "cued" tasks
in that they lead the testtaker into a narrow band of possibilities.
1. Directed Response Tasks
In this type of task, the test administrator elicits a particular
grammatical form or a transformation of a sentence. Such tasks are clearly
mechanical and not communicative,but they do require minimal
processing ofmeaning in order to produce the correct grammatical output.
2. Read-Aloud Tasks
Intensive reading-aloud tasks include reading beyond the sentence
level up to a paragraph or two. This technique is easily administered by
selecting a passage that incorporates test specs and by recording the test-
taker's output; the scoring is relatively easy because all of the test~taker's
oral production is controlled. Because of the results of research on the
PhonePass test, reading aloud may actually be a surprisingly strong
indicator of overall oral production ability.
3. Sentenced Dialogue Completion Tasks and Oral Questionnaires
Another technique for targeting intensive aspects of language requires
test-takers to read dialogue in which one speaker's lines have been omitted.
Test-takers are first given time to read through the dialogue to get its gist
and to think about appropriate lines to fill in. Then as the tape, teacher, or
test administrator produces one part orally, the test-taker responds.

3
Ibid., 147-159.

4
4. Picture-Cued Tasks
One of the more popular ways to elicit oral language performance at
both intensive and extensive levels is a pictl1re-cued stimulus that requires
a description from the testtaker. Pictures may be very simple, designed to
elicit a word or a phrase; somewhat more elaborate and "busy"; or
composed of a series that tells a story or incident.
5. Translation (of Limited Stretches of Discourse)
Translation methods of teaching are certainly passe in an era of direct
approaches to creating communicative classrooms. But we should
remember that in countries where English is not the native or prevailing
language, translation is a meaningful communicative device in contexts
where the English user is. called on to be an interpreter. Also, translation is
a well-proven communication strategy for learners of a second language.4

F. Designing Assessment Tasks: Responsive Speaking

Assessment of responsive tasks involves brief interactions with an
interlocutor, d~ffering from intensive tasks in the increased creativity given to
the test-taker and from interactive tasks by the somewhat limited length of
utterances.
1. Question and Answer
Question-and-answer tasks can consist of one or two questions
from an interviewer, or they can make up a portion of a whole battery of
questions and prompts in an oral interview. In designing such questions for
test-takers, ifs important to make s~e that you know why you are asking
the question. Are you simply trying to elicit strings of language output to
gain a general sense of the test-taker's discourse competence? Are you
combining discourse and grammatical competence in the same question?
Is each question just one in a whole set of related questions? Responsive
questions may take the following forms:
a) What do you think about the weather today?
b) What do you like about the English language?

4
Ibid., 14

5
c) Why did you choose your academic major?
d) What kind of strategies have you used to help you learn English?
Oral interaction with a test administrator often involves the latter
forming all the questions. The flip side of this usual concept of question-
and-answer tasks is to elicit questions from the test-taker. To assess the
test-taker's ability to produce questions.5
2. Giving Instructions and Directions
We are all called on in our daily routines to read instructions on
how to operate an appliance, how to put a bookshelf together, or how to
create a delicious clam chowder. Somewhat less frequent is the mandate to
provide such instructions orally, but this speech act is still relatively
common. Using such a stimulus in an assessment context provides an
opportunity for the test-taker to engage in a relatively
extended stretch of discourse, to be very clear and specific, and to use
appropriate discourse markers and connectors. The technique is Simple:
the administrator poses the problem, and the test-taker responds. Scoring is
based primarily on comprehensibility and s~condari1y on other specified
grammatical or discourse categories. Here are some possibilities.
a) Describe how to make a typical dish from your country.
b) What's a good recipe for making ___?
c) How do you access email on a ·PC computer?6
3. Paraphrasing
Another type of assessment task that can be categorized as
responsive asks the test-taker to read or hear a limited number of sentences
(perhaps two to five) and-produce a paraphrase of the sentence. A more
authentic context for paraphrase is aurally receiving and orally relaying a
message. The advantages of such tasks are that they elicit short stretches of
output and perhaps tap into test-takers' ability to practice the
conversational art of conciseness by reducing the output/input ratio. Yet
you have to question the criterion being assessed. Is it a listening task
more than production? Does it test short-term memory rather than
5
Ibid., 160
6
Ibid., 161

6
linguistic ability? And how does the teacher determine scoring of
responses? If you use short paraphrasing tasks as an assessment procedure,
it's important to pinpoint the objective of the task clearly. In this case, the
integration of listening and speaking is probably more at stake than simple
oral production alone.7

G. Test of Spoken English

Somewhere straddling responsive, interactive, and extensive speaking
tasks lies another popular commercial oral production assessment, the Test of
Spoken English (TSE). The TSE is a 20-minute audiotaped test of oral
language ability within an academic or professional environment. TSE scores
are used by many North American institutions of higher education to select
international teaching assistants. The scores are also used for selecting and
certifying health professionals such as physicians, nurses, pharmacists,
physical therapists, and veterinarians.
The tasks on the TSE are designed to elicit oral production in various
discourse categories rather than in selected phonogical, grammatical or lexical
targets.8 The following content specifications for the TSE represent the
discourse and pragmatic contexts assessed in each administration:
1. Describe something physical.
2. Narrate from presented material.
3. Summarize information of the speaker's own choice.
4. Give directions based on visual materials.
5. Give instructions.
6. Give an opinion.
7. Support an opinion.
8. Compare/Contrast.
9. Hypothesize.
10. Function "interactively".
11. Define.
H. Designing Assessment Tasks: Interactive Speaking
7
Ibid., 162
8
Ibid., 163

7
The final two categories of oral production assessment (interactive and
extensive speaking) include tasks that involve relatively long stretches of
interactive discourse (interviews, role plays, discussions, games) and tasks. of
equally long duration but that involve less interaction (speeches, telling longer
stories, and extended explanations and translations).The obvious difference
between the two sets of tasks is the degree of interaction with an interlocutor.
Also, interactive tasks are what some would describe as interpersonal, while
the final category includes more transactional speech events.
1. Interview
Oral interview is a test administrator and a test-taker sit down, In a
direct face-to-face exchange and proceed through a protocol of questions
and directives. The interview, which may be tape-recorded for re-listening,
is then scored on one or more parameters such as accuracy in
pronunciation and/or grammar, vocabulary usage, fluency,
sociolinguistic/pragmatic appropriateness, task accomplishment, and even
comprehension.
Every effective interview contains a number of mandatory stages.
Two decades ago, Michael Canale (1984) proposed a framework for oral
proficiency testing that has withstood the test of time. He suggested that
test-takers will perform at their best if they are led through four stages:
a) Warm-up
In a minute or so of preliminary small talk, the interviewer directs
mutual introductions, helps the test-taker become comfortable with the
situation, apprises the test-taker of the format, and allays anxieties. No
scoring of this phase takes place.
b) Level check.
Through a series of preplanned questions, the interviewer stimulates
the test-taker to respond using -expected or predicted forms and
functions.

c) Probe.

8
Probe questions and prompts challenge test-takers to go to the heights
of their ability, to extend beyond the limits of the interviewer's
expectation through increasingly difficult questions.
d) Wind-down.

This final phase of the interview is simply a short period of time

during which the interviewer encourages the test-taker to relax with
some easy questions, sets the test-taker's mind at ease, and provides
information about when and where to obtain the results of the
interview.

A variation on the usual one-on-one format with one interviewer

and one test-taker is to place two test-takers at a time with the interviewer.
An advantage of a two-on-one interview is the practicality of scheduling
twice as many candidates in the same time frame, but more significant is
the opportunity for student-student interaction. By deftly posing questions,
problems, and role plays, the interviewer can maximize the output of the
test-takers while lessening the need for his or her own output. A further
benefit is the probable increase in authenticity when two test-takers can
actually converse with each other. Disadvantages are equalizing the output
between the two test-takers, discerning the interaction effect of unequal
comprehension and production abilities, and scoring two people
simultaneously.9
2. Role Play
Role playing is a popular pedagogical activity in communicative
language-teaching classes. Within constraints set forth by the guidelines, it
frees students to be somewhat creative in their linguistic output. In some
versions, role play allows some rehearsal time so that students can map out
what they are going to say. And it has the effect of lowering anxieties as
students can, even for a few moments, take on the persona of someone
other than themselves. While role play can be controlled or "guided" by
the interviewer, this technique takes test-takers beyond simple intensive

9
Ibid., 167

9
and responsive levels to a level of creativity and complexity that
approaches real-world pragmatics.10
3. Discussions and Conversations
As formal assessment devices, discussions and conversations with
and among students are difficult to specify and even more difficult to
score. But as informal techniques to assess learners, they offer a level of
authenticity and spontaneity that other assessment techniques may not
provide. discussion is an integrative task, and so it is also advisable to give
some cognizance to comprehension performance in evaluating learners.11
4. Games
Among informal assessment devices are a variety of games that
directly involve language production. Consider the following types:
"Tinkertoy" game, Crossword puzzles, City maps are distributed to class
members and etc. As assessments, the key is to specify a set of criteria and
a reasonably practical and reliable scoring method. The-benefit -ofsuclIan
informal assessment may not be as much in a summative evaluation as in
its formative nature, with cashback for the students.12

I. Oral Proficiency Interview

The best-known oral interview format is one that has gone through a
considerable metamorphosis over the last half-century, the Oral Proficiency
Interview (OPI). Originally known as the Foreign Service Institute (FSI) test,
the OPI is the result of a historical progression of revisions under the auspices
of several agencies, including the Educational Testing Service and the
American Council on Teaching Foreign Languages (ACTFL). The latter, a-
professional society for research on foreign language instruction and
assessment, has now become the principal body for promoting the use of the
OPI." The OPI is widely used across dozens of languages around the world.
Only certified examiners are authorized to administer the OP!; certification

10
Ibid., 174
11
Ibid., 175
12
Ibid., 176

10
workshops are available, at costs of around $700 for ACTFL members,
through ACTFL at selected sites and conferences throughout the year.
Specifications for the OPI approximate those delineated above under the
discussion of oral interviews in general In a series of structured tasks, the OPI
is carefully designed to elicit pronunciation, fluency and integrative ability,
sociolinguistic and cultural knowledge, grammar, and vocabulary.
Performance is judged by the examiner to be at one of ten possible levels on
the ACTFL-designated proficiency guidelines for speaking: Superior;
Advanced-high, mid, low; Intermediate-high, mid, low; Novice-high, mid,
low.
OPI relies on an administrative network that mandates certified examiners,
who pay a significant fee to achieve examiner status. This systemic control of
the OPI adds test reliability to the procedure and assures test-takers that
examiners are specialists who have gone through a rigorous training course.
All these safeguards discourage the appearance of" outlaw" examiners who
might render unreliable scores. On the other hand, the whole idea of an oral
,interview up-der the control oral interviewer has come under harsh criticism
from a number of language-testing specialists.13

J. Designing Assessment: Extensive Speaking

Extensive speaking tasks involve complex, relatively lengthy
stretches of discourse. They are frequently variations on monologues,
usually with minimal verbal inter action.
1. Oral Presentations
In the academic and professional arenas, it would not be uncommon
to be called on to present areport-,a-paper;a marketing plan,a-sales-idea, a
design of a new product, or a method. A summary of oral assessment
techniques would therefore be incomplete without some consideration of
extensive speaking tasks. Once again the rules for effective assessment
must be invoked:
a) Specify the criterion

13
Ibid., 177

11
b) Set appropriate tasks
c) Elicit optimal output
d) Establish practical, reliable scoring procedures.
And once again scoring is the key assessment challenge. " For oral
presentations, a checklist or grid is a common means of scoring.or evalu-
ation. Holistic scores are tempting to use for their apparent practica:lity,
but they may obscure the variability of performance across several
subcategories, especially the two major components of content and
delivery. Following is an example of a checklist for a prepared oral
presentation at the intermediate or advanced level of English.14
2. Picture-Cued Story-Telling
One of the most common techniques for eliciting oral production is
through visual pictures, photographs, diagrams, and charts. We have
already looked at this' elicitation device for intensive tasks, but at this level
we consider a picture or a series of pictures as a stimulus for a longer story
or description.15 For the example (with a little humor added!), are you
testing for oral vocabulary (girl, alarm, coffee, telephone, wet, cat, etc.),
for time relatives (before, after, when), for sentence connectors (then, and
then, so), for past tense of irregular verbs (woke, drank, rang), and/or for
fluency in general? Ifyou are eliCiting specific grammatical or discourse
features, you might add to the directions something like "Tell the story that
these pictures describe. Use the past tense of verbs." Your criteria for
scoring need to be clear about what it is you are hoping to assess. Refer
back to some of the guidelines suggested under the section on oral
interviews, above, or to the OPI for some general suggestions on scoring
such a narrative.16
3. Retelling a Story, News Event
In this type of task, test-takers hear or read a story or news event that
they are asked to retell. This differs from the paraphrasing task discussed
it is a longer stretch of discourse and a different genre. The objectives in

14
Ibid., 179
15
Ibid., 180
16
Ibid., 181

12
assigning such.a task vary from listening comprehension of the original to
production of a number of oral discourse features (communicating
sequences and relationships 01 events, stress and emphasis patterns, .
"expression" in the case of a dramatic story), fluency, and interaction with
the hearer. Scoring should of course meet the intended criteria.
4. Translation (of Extended Prose)
Translation of words, phrases, or short sentences was mentioned
under the category of-intensive speaking. Here, longer texts are presented
for the test-taker to read in the native language and then translate into
English.Those texts could come in many forms: dialogue, directions for
assembly of a product, a synopsis of a story or play or movie, directions on
how to find something on a map, and other genres. The advantage of
translation is in the control of the content, vocabulary, and, to some extent,
the grammatical and discourse features. The disadvantage is that
translation of longer texts is a highly specialized skill for which some
individuals obtain post-baccalaureate degrees! To judge a nonspecialist's
oral language ability on such a skill may be completely invalid, especially
if the test-taker has not engaged in translation at this level. Criteria for
scoring should therefore take into account not only the purpose in
stimulating a translation but the possibility of errors that are unrelated to
oral production ability.
One consequence of our being articulate mammals is an
extraordinarily complex system of vocal communication that has evolved
over the millennia of human existence. This chapter has offered a
relatively sweeping overview of some of the ways we have learned to
assess our wonderful ability to produce sounds, words, and sentences, and
to string them together to make meaningful texts. This chapter's limited
number of assessment techniques may encourage your imagination to
explore a potentially limitless number of possibilities for assessing oral
production.17

17
Ibid., 182

13
REFERENCES

Brown, H., Douglas. Language Assessment: Principles and Classroom Practice.

San Francisco, California: Longman, 2003.

14