Comparing Classical and Item-Response Theories

Comparing Classical and Item-Response Theories

Item analysis is one of the most important elements of test construction. Statistical techniques can be used to carefully examine how a test item functions. This analysis can reveal how easy or difficult an item is and how well the item discriminates between test-takers. Item analysis can also reveal whether an item functions similarly when administered to different populations or when it is translated into another language. Classical test theory (CTT) and item-response theory (IRT) are the two commonly used methods of assessing test item characteristics.

Classical test theory focuses on an individual’s observed score of an entire instrument, not just individual items. It assumes that test scores have two major influences: consistency factors and inconsistency factors. In other words, there are characteristics of the test taker that are consistent and reflect the amount of whatever is being measured that is possessed by the individual. The inconsistency factors are those error factors from the individual or environment that have nothing to do with what is actually being measured. According to CTT then, a person’s observed score is the true score plus or minus error from these inconsistencies.

Item-response theory, on the other hand, is a set of models that establishes a person’s location on a latent variable using the observed data from a test item. This means that according to IRT, item responses on tests are explained by latent traits, or a trait that is presumed to exist in some quantity but cannot be directly assessed (e.g., intelligence or anxiety). Once you know where a person is on a latent variable scale (for instance, high anxiety or low anxiety), you can compare test takers to each other by seeing where each is on the continuum. According to IRT, each test item has an associated function, showing the probability that an individual with a certain quantity of the latent trait will pass the item.

When a new test is developed, test developers typically begin by creating more items than are needed. Item analysis techniques such as CTT and IRT are then used to “diagnose” items, and bad ones are discarded. IRT techniques can also be used to develop item banks. Items that function similarly can be used interchangeably, and computerized test administration can present a test tailored to the examinee’s ability level.

With these thoughts in mind:

Post by Day 4 an explanation of advantages and disadvantages of CTT and IRT. Describe two specific circumstances for which IRT might provide meaningful benefits if applied to the development or implementation of the test that you are proposing for your Final Project, and explain why. Finally, describe two specific circumstances for which CTT might provide meaningful benefits to your test, and explain why. Support your response using the Learning Resources and the current literature.