Assessing the Candidate

Assessing the Candidate

Assessors are required to assess candidates based on the ICAO Language Rating Table, as shown in Annex X.

There are 6 levels, ranging from 1 (zero proficiency) to 6 (expert proficiency).

Only levels 4, 5 and 6 can be added to a pilot’s licence. Any level below 4 is considered a fail.

Level 6 - Expert

PRONUNCIATION

Pronunciation, stress, rhythm, and intonation, though possibly influenced by the first language or regional variation, almost never interfere with ease of understanding.

STRUCTURE

Both basic and complex grammatical structures and sentence patterns are consistently well controlled.

VOCABULARY

Vocabulary range and accuracy are sufficient to communicate effectively on a wide variety of familiar and unfamiliar topics. Vocabulary is idiomatic, nuanced, and sensitive to register.

FLUENCY

Able to speak at length with a natural, effortless flow. Varies speech flow for stylistic effect, e.g. to emphasize a point. Uses appropriate discourse markers and connectors spontaneously.

COMPREHENSION

Comprehension is consistently accurate in nearly all contexts and includes comprehension of linguistic and cultural subtleties.

INTERACTIONS

Interacts with ease in nearly all situations. Is sensitive to verbal and non-verbal cues and responds to them appropriately.

Level 5 - Extended

PRONUNCIATION

Pronunciation, stress, rhythm, and intonation, though influenced by the first language or regional variation, rarely interfere with ease of understanding.

STRUCTURE

Basic grammatical structures and sentence patterns are consistently well controlled. Complex structures are attempted but with errors which sometimes interfere with meaning.

VOCABULARY

Vocabulary range and accuracy are sufficient to communicate effectively on common, concrete, and work-related topics. Paraphases consistently and successfully. Vocabulary is sometimes idiomatic.

FLUENCY

Able to speak at length with relative ease on familiar topics but may not vary speech flow as a stylistic device. Can make use of appropriate discourse markers or connectors.

COMPREHENSION

Comprehension is accurate on common, concrete, and work-related topics and mostly accurate when the speaker is confronted with a linguistic or situational complication or an unexpected turn of events. Is able to comprehend a range of speech varieties (dialect and/or accent) or registers.

INTERACTIONS

Responses are immediate, appropriate, and informative. Manages the speaker/listener relationship effectively.

Level 4 - Operational

PRONUNCIATION

Pronunciation, stress, rhythm, and intonation are influenced by the first language or regional variation but only sometimes interfere with ease of understanding.

STRUCTURE

Basic grammatical structures and sentence patterns are used creatively and are usually well controlled. Errors may occur, particularly in unusual or unexpected circumstances, but rarely interfere with meaning.

VOCABULARY

Vocabulary range and accuracy are usually sufficient to communicate effectively on common, concrete, and work-related topics. Can often paraphrase successfully when lacking vocabulary in unusual or unexpected circumstances.

FLUENCY

Produces stretches of language at an appropriate tempo. There may be occassional loss of fluency on transition from rehearsed or formulaic speech to spontaneous interaction, but this does not prevent effective communication. Can make limited use of discourse markers or connectors. Fillers are not distracting.

COMPREHENSION

Comprehension is mostly accurate on common, concrete, and work-related topics when the accent or variety used is sufficiently intelligible for an international community of users. When the speaker is confronted with a linguistic or situational complication or an unexpected turn of events, comprehension may be slower or require clarification strategies.

INTERACTIONS

Responses are usually immediate, appropriate, and informative. Initiates and maintains exchanges even when dealing with an unexpected turn of events. Deals adequately with apparent misunderstandings by checking, confirming, or clarifying.

Level 3 - Pre-operational

PRONUNCIATION

Pronunciation, stress, rhythm, and intonation are influenced by the first language or regional variation and frequently interfere with ease of understanding.

STRUCTURE

Basic grammatical structures and sentence patterns associated with predictable situations are not always well controlled. Errors frequently interfere with meaning.

VOCABULARY

Vocabulary range and accuracy are often sufficient to communicate on common, concrete, or work-related topics, but range is limited and the word choice often inappropriate. Is often unable to paraphrase successfully when lacking vocabulary.

FLUENCY

Produces stretches of language, but phrasing and pausing are often inappropriate. Hesitations or slowness in language processing may prevent effective communication. Fillers are sometimes distracting.

COMPREHENSION

Comprehension is often accurate on common, concrete, and work-related topics when the accent or variety used is sufficiently intelligible for an international community of users. May fail to understand a linguistic or situational complication or an unexpected turn of events.

INTERACTIONS

Responses are sometimes immediate, appropriate, and informative. Can initiate and maintain exchanges with reasonable ease on familiar topics and in predictable situations. Generally inadequate when dealing with an unexpected turn of events.

Level 2 - Elementary

PRONUNCIATION

Pronunciation, stress, rhythm, and intonation are heavily influenced by the first language or regional variation and usually interfere with ease of understanding.

STRUCTURE

Shows only limited control of a few simple memorized grammatical structures and sentence patterns.

VOCABULARY

Limited vocabulary range consisting only of isolated words and memorized phrases.

FLUENCY

Can produce very short, isolated, memorized utterances with frequent pausing and a distracting use of fillers to search for expressions and to articulate less familiar words.

COMPREHENSION

Comprehension is limited to isolated, memorized phrases when they are carefully and slowly articulated.

INTERACTIONS

Response time is slow and often inappropriate. Interaction is limited to simple routine exchanges.

Level 1 - Pre-elementary

PRONUNCIATION

Performs at a level below the Elementary level.

STRUCTURE

Performs at a level below the Elementary level.

VOCABULARY

Performs at a level below the Elementary level.

FLUENCY

Performs at a level below the Elementary level.

COMPREHENSION

Performs at a level below the Elementary level.

INTERACTIONS

Performs at a level below the Elementary level.

As a general principle, assessors should mark positively, giving credit for the competence candidates demonstrate, rather than writing down every error and – consciously or not – “deducting points” for errors made. Noting some errors or error types may be useful when debriefing the candidate with concrete examples. In this way credit may be given to candidates who speak fluently and at length on a subject but who make a number of minor and / or local errors. Conversely, candidates who say a few simple phrases with one only error may be performing well in terms of structural accuracy but less so in terms of structural complexity and so will not necessarily be demonstrating high level competence for structure. assessors should however make themselves familiar with ICAO doc 9835 Appendix B part IV which contains a glossary of what can be described as basic and complex structures. They should also be aware of ICAO doc 9835 paragraph 2.3.4.1 which describes the differences between local and global errors.

ICAO doc 9835, chapter 4, paragraph 4.6 gives a very detailed explanation of the rating scale descriptors and should always be referred to by assessors where clarification is required.

The role of the assessor is also to facilitate communications and elicit discourse, especially at the beginning of the examination where the candidate is probably feeling stressed. assessors should use their judgement as to how and when to intervene to facilitate the flow of communication, avoiding both excessive intervention as well as long and unnatural pauses. In the latter case, assessors should attempt to restart the conversation through simple prompting and encouragement. If this prompting and facilitation continues throughout the examination, then of course this must be taken into account when assessing the overall level for interaction and fluency, but the technique can be used sympathetically at the beginning of the examination to help the candidate “warm up”.

Where one assessor is physically present and the other is listening remotely the perception of the examination may sometimes be quite different. The assessor present will often have more of a “feeling” about interaction and more readily perceive any non-verbal communication cues. The remote assessor will probably pick up on structural errors more easily as they are not involved in managing the interaction process with the candidate.

A poor connection and variations in volume can also adversely affect the remote assessor’s perception of the candidate’s performance, notably in terms of pronunciation. Remote assessors should ensure that they are operating in a low noise environment conducive to listening for detail. Use of headsets is recommended. assessors should also ensure that their microphone is switched off in order to avoid any feedback problems or extraneous distractions to the candidate. All equipment for remote communication and filming must be fully charged and assessmented before the examination. A procedure should be established between both assessors of what to do in case of connection loss during the exam.