High Profile Talk: Detection of Inattentiveness in Questionnaire or Survey Data
Welcome to CEMO's high profile talk. Associate Professor Ying Cheng from University of Notre Dame in the United States will give a presentation about Detection of Inattentiveness in Questionnaire or Survey Data.
Careless or inattentive responding is frequently observed in questionnaire or survey data, which jeopardizes test validity and more broadly the replicability and generalizability of research findings. It is therefore very important to detect such response behavior. The most frequently encountered type of careless response behavior is back random responding (BRR). Literature suggests that BRR is challenging to detect, with reported power of detection around .5 or lower. Change point analysis (CPA), which is a widely used statistical process control method, can be applied to item response or item response time data or both to detect if aberrant behavior exists in a response pattern. In this talk I will first review existing CPA methods that have been applied in psychometrics to detect intra-individual change, and then introduce a new method that can be applied to item response or response time data to detect BRR. Simulation results indicated that the proposed new method is able to detect BRR behavior with much higher power than existing methods, while keeping the Type-I error rate well under control.
Dr. Ying (“Alison”) Cheng is Associate Professor of Psychology at University of Notre Dame. Her research focuses on methodological issues in measurement, in particular development and applications of item response theory, such as computerized adaptive testing, differential item functioning (DIF), and cognitive diagnostic modeling. Her work has appeared in premier journals, such as Psychometrika, Psychological Methods, and British Journal of Mathematical and Statistical Psychology (BJMSP). In 2009 she received the Bradley Hanson Award for Contributions to Educational Measurement, and in 2012 the Jason Millman Promising Measurement Scholar Award from the National Council of Measurement in Education. In 2014 she received the faculty CAREER award from the National Science Foundation. In 2017 she was elected to the Society of Multivariate and Experimental Psychology. She currently serves as the Editor of BJMSP.