Striving for fairness

Fairness is an overriding concern in all aspects of all CAA International assessments and provides a context for the principles of good practice discussed previously.

The design, development and administration of the CAAi EALTS, based upon these sound principles of good practice in assessment, allow for the achievement of the high standards of quality and fairness expected in high-stakes testing.

Within the Testing System, every practical and reasonable measure has been taken to ensure test effectiveness and reliability. Below is a list of some of the features of the CAAi EALT that help to ensure fairness to all test stakeholders. 1

Pilots and air traffic controllers

Alternative adaptations of the EALT can be used to assess the English for Aviation language proficiency of both pilots and air traffic controllers. This is a significant benefit to licensing authorities and national regulatory bodies in that candidates from both populations are assessed by a similar means, ensuring parity in the assessment process across the two operational areas. 2

Customisable

The EALT is a test which can be easily customised to provide individual test-takers with familiar aviation-related contexts. Thus, pilots are able to take tests using or referring to the environments of fixed or rotary wing, heavy or light aircraft, and controllers are able to take tests using or referring to a tower, approach or en route environment. These alternative adaptations are seen as adaptations in the interest of the comfort of the test-taker, not as specialised tests of distinct varieties of language proficiency. 3

Simplified notional airspace

The EALT is a test of language proficiency and does not seek to assess either operational skills or specific technical knowledge of operations of the test-taker. In the interests of fairness, a generic, simplified, notional airspace has been designed for the listening tasks of the EALT. 4

Recording of interviews

The EALT Test of Speaking is recorded as a matter of procedure, allowing for assessment in both real time and subsequent to the test. These recordings provide a safeguard against unsupported subjective judgements of examiners and against individual or institutional unfairness or bias against a candidate. Additionally these recordings form the basis of any investigation following an appeal. 5

Multiple assessments

Each candidate’s performance in the EALT Test of Speaking is assessed by a minimum of four ‘primary’ qualified and certified assessors, reducing the possibility of rater error and helping to ensure a comprehensive evaluation of proficiency. Where the assessments of the four primary assessors are divergent, a further assessment by two senior assessors will be used in the final determination of ratings. 6

Central EALTS Administration provides that each candidate will be assessed by at least one assessor with aviation operational expertise and two assessors with language specialist expertise. The EALT assessment procedure ensures the multiple complementary perspectives of a candidate’s performance recommended by ICAO.

This multiple assessment of performance is both a further safeguard of the fairness, reliability and security of the test results and an effective measure in the monitoring of the standardisation of the assessment and the assessors.

Appeals

Candidates or sponsoring organisations may apply for an enquiry on the CAAi EALTS assessment procedure at the local test centre at which they took their test. This application should be made within four weeks of issue of result. The candidate’s test material is then re-assessed.

The appeals procedure is at once transparent, appropriate and efficient.

There is a fee for this procedure which is refunded should the candidate’s overall rating be amended. 7

Standardisation

The standardisation of design, development, administration, delivery and assessment of the EALT is a key feature in the striving for fairness by the EALTS Test Partners in the assessment process.

1 c.f. ICAO Doc 9835 Ch.6 Aviation Language Testing 6.7.4

2 c.f. Section CAA International EALTS Candidature

3 c.f. Section CAA International EALTS

4 c.f. Section EALTS listening items and notional airspace

5 c.f. Section EALT Test of Speaking - General Description

6 c.f. Section EALT Test of Speaking - Assessment – General Principles

7 c.f. Section Appealing an EALTS Result