Lenguax is proud to offer a high-quality online rater course for those interested in
obtaining or renewing a certificate in applying the ICAO Language Proficiency
Requirements to Aviation English test speech samples.
The course is a mixture of online group discussion sessions and self-access reading
and listening activities.
Course Content
Overall there are 8 modules of self-access study, filled with real-world examples of
candidate performances and commentary from expert ELE and SME raters. Each
module consists of approximately 2 hours of content.
In addition, we will meet for 5 sessions on Zoom where we will listen to more tests
and discuss, as a group, our rating.
The timings of these meetings will depend on the time zones of everyone who
enrolled for the course: we will do our best to accommodate everyone. If you
cannot attend a session we can arrange to record the session for you.
The timings of these meetings will depend on the time zones of everyone who
enrolled for the course: we will do our best to accommodate everyone. If you
cannot attend a session we can arrange to record the session for you.
Our course is accepted by the LBA Germany
Upon enrolling you will have access to three modules (see below for details). These
will form the basis of discussion during the first discussion session.
After this, on consecutive days you will receive two more modules each day, and
these will be the focus of the next daily session. Each session involves listening to
real-life test performances from candidates from a range of countries, and assessing
them for their performances in the areas that have been covered to date.
In the final sessions you will be able to confidently assess performances across the
range of abilities in all 6 ICAO language areas.
You can download a brochure here.
Rating Task
Lastly, you will listen to a number of tests, and submit the score you think the
candidates should be awarded in each area. You will be given individualized
feedback in your rating in the form of comments about the scoring from our team
of experienced ELE and SME raters and language assessors, and a statistical
analysis of your "harshness" or "leniency" in the form of a Rasch analysis.
This feedback will, we believe, help to guide you in your future rating career.
What’s in the rater course?
Here you can read about the contents of the self-access modules.
1
Module 1: ICAO Language Proficiency Requirements & Language Testing
In this module we explore the actual requirements from ICAO by looking at
Document 9835. We also consider some of the issues involved in language testing
by considering the topics of reliability and validity.
Module 2: ICAO Descriptors - Pronunciation
In this module we think about Pronunciation and how to assess fairly using the
ICAO language descriptors. We consider issues of intelligibility, English as an
International Language and some of the intelligibility issues that speakers from
different language backgrounds may experience. We listen to samples of speakers
from around the world and consider how their pronunciation does or does not
"interfere with meaning".
Module 3: ICAO Descriptors - Structure
This module considers Structure and how we can assess it using the ICAO
language descriptors. We think about the notions of "basic" and "complex"
structure and interference with meaning, and some of the difficulties faced by
speakers of other languages in learning English grammar. We then listen to some
examples of different levels of performance.
Module 4: ICAO Descriptors - Vocabulary
In this module we think about the "range and accuracy" of different examples of
Vocabulary performance and how to assess these with respect to the ICAO
descriptors. We also consider what "idiomatic" language might mean in this
context.
Module 5: ICAO Descriptors - Fluency
Fluency involves both "flow" and the use of discourse markers - two quite different
concepts. We will examine both then listen to some examples of different levels of
ability.
6
Module 6: ICAO Descriptors - Comprehension
We will consider Comprehension in the abstract: how can we know what someone
understands in a reliable and fair way? We will use the listening tasks from the
TEAC as our instrument, and listen to different ability levels attempting the tasks,
and learn how to map these performances to scores.
Module 7: ICAO Descriptors - Interactions
The interactions descriptor can be quite confusing for new raters: we will try to
demystify it and show how different test performances exemplify different degrees
of proficiency in this pragmatic skill.
Module 8: Examiner Behaviour
For valid, reliable testing examiner behaviour is critical: each candidate should
receive a similar testing experience, and the way an interlocutor behaves can
radically affect the performance that the candidate is able to deliver. We will read
about best practice, and consider some examples of.. less good practice!
Core Subjects
Principles of Verbal Language Testing
The ICAO Language Proficiency Scale
ICAO Language Proficiency categories
Design of ICAO Language Proficiency Tests
Administration of Tests
Marking of Tests
Rating of Oral Performance
Practicals: test preparation and administration
Practicals: rating recorded and live candidates on the ICAO Scale
THE FOLLOWING AREAS WILL NORMALLY BE COVERED:
Evaluating and comparing different test types (eg. for ab-initio trainees or trained
professionals, direct or semi-direct, including computerised tests)
Identifying the criteria for ICAO levels in: Listening (general comprehension, interpretation
and discrimination) Speaking (accuracy, intelligibility and communicative competence)
Defining and evaluating appropriate levels of competence in communication skills, and the 6
criteria of the ICAO Rating Scale.
Practical exercises in rating trainees’ performance at various levels of the ICAO Rating Scale,
with emphasis on Levels 2 to 5
Identifying areas for development in failed candidates’ performance, and guiding them
towards improving their language proficiency.
Blind rating assessments for standardisation purposes and to assess the course participants’
level of competency.
This course is for linguistic or operational experts involved in developing,
administering, managing, and rating Oral Proficiency Interview
assessments for the aviation industry.
Through this course, participants will gain an understanding of:
An introduction to the International Civil Aviation Organisation (ICAO),
including background to aviation language testing policies and a
familiarisation with relevant ICAO documentation, especially
Document 9835 (2010)
A description of aviation safety issues and discussion of historical
accidents with a communication component
A description and detailed analysis of the ICAO Language Proficiency
Rating Scale including listening to expert rated sample speech files
A lesson in Aviation English: phraseology, vocabulary and sentence
structures used in the aviation context; distinguishing between
phraseology and ‘plain language’
An exploration of the various Aviation English proficiency tests
Extensive practical rating examples, group calibration and standard
setting
Practice and instruction in interlocution, conducting different elements
of a test, using other raters and also ‘mock’ candidates
Detailed exploration of the rating scale commentary in Doc 9835
(2010) and its implications for effective and accurate rating
An in-depth examination of each of the six skill elements in the rating
scale and their implications.
Participants should:
be at ICAO Level 5 or above
have an English language teaching or testing background, or have
significant aviation operational background.
Rater and Test Development Course
The Aviation English Rater and Test Development Course develops participants’ skills in the context of
testing aviation English. Each day is divided into five sessions with an opening plenary and five
complementary sessions. Each specific session has a set focus to target participants’ needs precisely and the
duration of lessons ensures lessons are fast-paced and clear objectives are achieved.
This is a closed group course and runs on demand in Oxford.
By the end of the one-week course participants will have:
• Understood and applied the ICAO English Language Proficiency Descriptors
• Raised their awareness of the key issues in language testing
• Developed their skills in assessing oral / aural language
• Attended off-site activities to further immerse themselves in aviation English
• Set clear and measurable targets for themselves and developed on-going learning strategies
Click here for the Oxford information sheet
Rater and Test Development Course
09.00-10.00 Key issues
10.15-11.15 Rating tasks
11.30-12.30 Feedback & analysis
13.30-13-30 Lunch break
13.30-14.30 Test design
14.45-15.45 Workshop
Key Issues
This 1 hour session addresses the key issues and concepts in the context of aviation English assessment.
Rating Tasks
The next 1 hour session of the day provides practice in rating according to the requirements of the ICAO
language proficiency requirements. Participants apply the rating scale to speech samples of pilots and air
traffic controllers from around the world. Participants develop their abilities to rate and discuss associated
issues of security and administration.
Feedback & Analysis
Through feedback and analysis participants hone their skills and so develop their abilities to rate effectively
and reliably both on the course and on an on-going basis.
Test Design
This session focuses on test design, development and evaluation. Speaking and listening task types are
analysed.
Workshop
The workshop session provides an opportunity for further practice in required areas throughout the course.
Designed for the initial and recurrent training of LPR raters and for anyone involved
with the selection or development of aviation language tests, this comprehensive
course provides in-depth training on aviation language assessment with reference to
ICAO document 9835, literature in the field of language testing and current industry
guidance on best practice.
Delivered as a blend of self-access e-learning and Virtual Instructor-Led Training
(VILT) sessions taught by experts in the field, the course comprises eight modules
which:
Provide background to the ICAO LPRs and the current status of implementation
Introduce the key concepts in specific purpose language testing including test
construction and evaluation
Raise awareness of current issues and initiatives in the field
Explore the construct of aeronautical radiotelephony communication
Analyse and define the ICAO rating scale and holistic descriptors
Explore the ICAO Rated Speech Samples Training Aid and evaluate the
usefulness of a variety of aviation English speaking test tasks
Provide guided practice on the application of the rating scale to speech samples
generated from the English Test for Aviation, the first test in the world to
receive an endorsement from ICAO*
Assess participants’ ability to rate to an agreed standard including personal
feedback on rater reliability and severity
Introduce the use of statistics for monitoring rater reliability and procedures for
maintaining standards in language rating
Explore quality (reliability and validity) in language testing, including how test
quality manifests in aviation language testing practice and the evidence of
quality that service providers might supply in the process of regulatory approval
and/or ICAO recognition
Accessible on all devices, each module of web-based e-learning is designed to be
completed in advance of the corresponding VILT session. The practical VILT sessions
are delivered in our highly interactive online classroom and provide presentations, pair
and group work activities, discussion and analysis.
Contents of the initial and recurrent rater
training courses
Legal aspects
Introduction into LP legal requirements and regulations.
Rating scale
Learn how to apply the ICAO rating scale correctly
Examples
Numerous examples and ratings to standardise your assessment
Interviewing
Learn how to conduct language testing interviews
Administration
Get familiar with the administrative procedures
OJT
On-the-job training completes the training course
