Variation of Restricted Optimal Item Calibration for Computerized Adaptive Test

Item calibration is the process of estimating the parameters of test items or questionswhen item response theory (IRT) is used to analyze test response data, provide a score for each test-taker, and/or facilitate computerized adaptive testing (CAT). CAT is a modern technology for test administration where test items are selected on the fly for each examinee to tailor the test individually. The adaptive item selection algorithms in CAT depend on known item parameters. Hence item calibration is a critical component in CAT operations. The statistical process of estimating item parameters depends on observed data generated by examinees responding to the items. The most basic way to obtain response data for item calibration is by conducting a standalone pretesting event where test-takers are recruited to take the test solely for the purpose of collecting data for item calibration. Because that process can be costly and time-consuming, online calibration techniques are invented to insert several pretest items in a live test, so that response data are collected on the go. Given a pool of pretest items to be calibrated, different approaches have been proposed to select pretest items for each examinee, such as random selection, examine-centered selection, and various item-centered selection methods. A common goal of most pretest item selection approaches is to optimize the efficiency of the statistical estimation of the item parameters. Hence the theories and techniques of optimal design in statistics have been applied and extended for this application. Optimal design, in general, involves identifying the most important variables that influence the outcome of the experiment, and determining the optimal values of these factors that provides the most information for estimating statistical model parameters. The main focus of this dissertation is applying, extending, and evaluating variations of optimal restricted design for online calibration for the 2-parameter logistic model. A simulation study is conducted to compare the existing methods with variations of restricted optimal design using different criteria.