Computer Adaptive Testing (CAT) - Why and What

In a computer adaptive test, a student responds to assessment tasks presented by the computer. According to those responses the computer updates an estimate of the student's ability to complete the task. The updated estimate of the ability of a student is used to select the next appropriate task from a bank of tasks in a database. When a bank has sufficient tasks, with a wide range of difficulty levels and text types, CAT can be a more efficient method of assessing student’s abilities than a traditional paper-and-pencil test.

Paper-and-pencil tests are typically "fixed-task" tests in which a student answers the same set of tasks printed in a test booklet. Since all students are required to take every task, a student will probably do tasks that are either too easy or too difficult. These wide ranging tasks provide relatively little information about a student's ability level. Consequently, a large number of tasks have to be taken in order to obtain a modest degree of precision regarding the ability level of a student.

With CAT, the student's ability level is continuously updated during the testing process and an appropriate task, in terms of difficulty level, is selected by the computer. Students are provided with tasks that determine the maximum information about their ability levels, taking into consideration other factors, i.e. the balance of the test content. Thus, students receive few tasks that are either too easy or too difficult to do. This tailored task selection results in greater precision and can be accurate with just a handful of properly selected tasks.

The CAT process is usually an iterative one with the following steps:

  1. All tasks that have not yet been selected are evaluated to determine which will be the best one to be presented according to the current ability level of a student
  2. The "best" next task is presented and the student responds
  3. A new ability estimate of the student is concurrently computed based on responses to all of the administered tasks
  4. Steps 1 through 3 are repeated until a precise estimate of student ability is obtained or other criterion to stop is met, i.e. time is up.
Hong Kong Examinations and Assessment Authority. All Rights Reserved.