Using Testlets in Education: eTIMSS 2019 as an Example

Abstract

A testlet refers to groups or clusters of items that are linked to a common stimulus such as a text, graphic or table. Due to the shared stimulus among these items, there is a high likelihood of inter-item dependency within the responses, which violates the assumption of local independence in Item Response Theory (IRT). This violation results in local dependence among the items within the testlets. Therefore, this study employed IRT and the Testlet Response Theory (TRT) models to assess the impact of local dependence stemming from testlets on item and ability parameter estimations, classification accuracy, and Differential Item/Bundle Functioning (DIF/DBF), and compared the findings obtained from these models. Responses to three testlets that were both in booklets 13 and 14 in the eTIMSS 2019 mathematics subtest were analysed using the mirt package in R software. The analysis revealed a moderate degree of local dependence in the testlets. Additionally, a very high correlation was observed between the item and ability parameter estimations derived from both models. Regarding classification accuracy, the IRT and TRT models demonstrated equivalent performance. When items were analysed both independently and as part of testlets, no items exhibited evidence of DIF/DBF based on gender. The findings indicate that IRT can tolerate the effects of testlets when the degree of local dependence is low to moderate.