Home Site

Welcome to PC AI Online

( This Site works best with IE 5.5+ or Netscape 6.0+ )

    Page 54
16.4 Table of Contents Bottom of Page Site Map
historical student test records such as how each student answered each question and the score awarded can be used to generate decision trees for determining an optimal question sequence for profiling a successful or unsuccessful student. In these tests, one uses a database of test information to determine decision trees to drive other tests. With this technique, the system identifies successful students with a small fraction of the questions given on the original test. Students that passed the test answered a subset of the questions the same, and students that failed, answered a small subset of the questions the same. Some questions do not influence whether the student passed or failed so it is possible to cull out these questions and make the test more efficient. The simplest case is when almost everyone answered them correctly, or almost everyone answered them wrong.

Automatic Generation of Decision Trees from Test Scores

Following is a very simple example with hypothetical
test scores that represent a simplistic Pass/Fail three-question test taken by 64 students (see Figure 4). A real test would probably use scores ranging from 0 to 100% and have many more questions.
Here is the resulting decision tree, drawn by the
Knowledge Builder system using its automatic inductive knowledge acquisition facility, and not drawn by a developer.
This also helps with a perennial issue in multiple choice
testing: students who guess. After all, in a four-answer test protocol, students are likely to be correct 25% of the time. A simple approach, recommended by many testing authorities, is to use five or more distracters. On a more individual level, you simply throw out tests that have very low scores. The knowledge based induction engine helps with this in generating a decision tree by effectively ignoring or grouping all such low scorers and other outliers. Also, as the student's score is being monitored continuously as the test proceeds, it is possible to include specific question groups that assess whether or not extensive guessing is occurring.
Notice that every student that answered Q1 with 'c' or
'd' failed the test. Therefore, if this tree were to drive an adaptive test, then the students that answered Q1 with 'c' or 'd' would fail the test immediately, and not see any other questions. This is extremely efficient! In a real situation, this usually happens after answering approximately 20% of the test questions. The current student's profile fits a failing student's profile as determined from past test records. Notice that Q3 is never asked to pass a student if

Figure 3: Twenty-Four of Sixty-Four Test Data Examples.

Figure 4: Decision Tree Result of Inductive Knowledge Acquisition on Test Data Examples

To Page 53

16.4 Table of Contents
Top of Page

To Page 55

16.4 2002

PC AI Magazine - PO Box 30130 Phoenix, AZ 85046 - Voice: 602.971.1869 Fax: 602.971.2321
e-mail: info@pcai.com - Comments? webmaster@pcai.com