I am currently a Professor of Educational Measurement in the Centre for Educational Measurement at University of Oslo. I joined UiO in September of 2015, where I enjoy conducting basic and applied research in international large-scale assessment. Prior to my appointment in Oslo, I was an Assistant Professor of Inquiry Methodology at Indiana University - Bloomington, where I taught courses in advanced quantitative methods. Prior to that, I was a research associate for the International Association for the Evaluation of Educational Achievment (IEA) - the folks that bring you TIMSS & PIRLS.
My experience with the IEA and my doctoral work at University of Illinois Urbana-Champaign has inspired my work in the field of international large-scale assessment (LSA). In particular, I am interested in critically analyzing the methods used to gather data and to produce achievement scores. I am also interested in methods and models for accommodating heterogeneous populations in latent variable models. I have a number of peer-reviewed and forthcoming papers in these areas. I regularly consult for international organizations including the IEA and the OECD and provide workshops on quantitative methods around the world.
I have published an edited volume: Rutkowski, L. von Davier, M. & Rutkowski, D. (2014). Handbook of International Large-Scale Assessment. Boca Raton, FL: Chapman & Hall/CRC Press.
- Embracing heterogeneity in international surveys: Optimal test design and parameter estimation (funded by the Norwewgian Research Council, 2016 - 2020)
- Covariance structure analysis
- Multilevel modeling
- Categorical data analysis
- Longitudinal data analysis
- Multivariate statistics
- Ph.D., Educational Psychology, 2007, University of Illinois at Urbana-Champaign
- M.S., Statistics, 2005, University of Illinois at Urbana-Champaign
- B.A., Economics, 2003, University of New Mexico, Albuquerque
- 2015 - Present Professor of Educational Measurement, University of Oslo, Norway
- 2010 - 2015 Assistant Professor of Inquiry Methodology, Counseling and Educational Psychology, Indiana University, Bloomington, IN, USA
- 2007 - 2009 Research Associate, Research and Analysis Unit, IEA-DPC, Hamburg, Germany
- 2006 - 2007 Statistical Research Consultant, Women's Health Sciences Division, National Center for Post-Traumatic Stress Disorder, Boston, MA
- 2003 - 2007 Graduate Instructor, Educational Psychology, University of Illinois, Champaign, IL, USA
Rutkowski, L. & Zhou, Y. (2015). Errors in variables models for population estimation: Including measurement error in the conditioning model. Journal of Educational Measurement, 52(4), 359-375.
Rutkowski, L., & Rutkowski, D., & Zhou, Y. (2015). Parameter estimation methods and the stability of achievement estimates and system rankings: Another look at the PISA model. International Journal of Testing. doi: 10.1080/15305058.2015.1036163
Rutkowski, L. & Zhou, Y. (2015). The impact of missing and error-prone auxiliary information on sparse-matrix sub-population parameter estimates. Methodology, 11(3), 89-99.
Rutkowski, D., Rutkowski, L., & Plucker, J. (Dec. 2014/Jan. 2015). Should individual schools participate in PISA? Some cautions and recommendations for interested participants. Phi Delta Kappan, 96(4), 68-73.
Svetina, D., & Rutkowski, L. (2014). Detecting differential item functioning using generalized logistic regression in the context of large-scale assessments. Large-Scale Assessments in Education, 2(4), 1-17.
Rutkowski, D., Rutkowski, L. & Engel, L. (Published online, 19 June 2014). Inclusive schooling: Fostering citizenship among immigrant students in Europe. Intercultural Education. doi: 10.1080/14675986.2014.926144
Wong, Y. J., Brownson, C., Rutkowski, L., Nguyen, C. P., & Becker, M. (2014). A mediation model of professional psychological help seeking for suicide ideation among Asian American and White American college students. Archives of Suicide Research, Online First. doi: 10.1080/13811118.2013.824831.
Engel, L., Rutkowski, D., & Rutkowski, L. (2014). Global mobility and rising inequality: A cross-national study of immigration, poverty, and social cohesion. Peabody Journal of Education. 89(1), 123-140.
Rutkowski, L. (2014). Sensitivity of achievement estimation to conditioning model misclassification. Applied Measurement in Education, 27(2), 115-132. (http://www.tandfonline.com/eprint/tZ8m2cdPR6nrEP8qm2PV/full)
Rutkowski, L. & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale assessments. Educational and Psychological Measurement. 74(1), 31-57.
Rutkowski, D., & Rutkowski, L. (2013). Measuring socioeconomic background in PISA: One size might not fit all. Research in Comparative and International Education. 8(3), 259-278.
Rutkowski, L., Rutkowski, D., & Engel, L. (2013). Sharp contrasts at the boundaries: School violence and educational outcomes internationally. Comparative Education Review, 57(2), 232-259.
Rutkowski, D., Rutkowski, L., & Plucker, J. (2012). International trends in excellence gaps: A growth modeling perspective. High Ability Studies, 23(2), 143-166.
Rutkowski, L., Rutkowski, D, & Plucker, J. (2012). International determinants of private school attendance. Educational Research and Evaluation, 18(4), 375-391.
Steinfeldt, J., Rutkowski, L., & Orr, T. (2012). Moral atmosphere and masculine norms in American college football. The Sport Psychologist, 36, 341-358.
Rutkowski, D., Rutkowski, L. & Langfeldt, G. (2012). Reading economics, thinking education: The relevance – and irrelevance – of economic theory for curriculum research. Journal of Curriculum Studies, 44(2), 165-192.
Rutkowski, L. (2011). The impact of missing background data on subpopulation estimation. Journal of Educational Measurement, 48(3), 293-312.
Steinfeldt, J., Rutkowski, L., Vaughn, E., & Steinfeldt, M. (2011). Masculinity, moral atmosphere, and moral functioning of high school football players. Journal of Sport and Exercise Psychology, 33(2), 215-234.
Rutkowski, D., Rutkowski, L., & Sparks, J. (2011). Information and Communication Technologies Support for 21st Century Teaching: An International Analysis. Journal of School Leadership, 21(2), 190-215.
Gonzalez, E., & Rutkowski, L. (2010). Principles of multiple matrix booklet designs and parameter recovery in large-scale assessments. IEA-ETS Research Institute Monograph, 3, 125-156.
Rutkowski, L., & Rutkowski, D. (2010). Getting it ‘better’: The importance of improving background questionnaires in international assessment. Journal of Curriculum Studies, 42(3), 411-430.
Rutkowski, L., Gonzalez, E., Joncas, M., & von Davier, M. (2010). International large-scale assessment data: Recommendations for secondary analysis and reporting. Educational Researcher, 39(2), 142-151.
Rutkowski, L., Vasterling, J.J. & Proctor, S.P., Anderson, C. J. (2010). Posttraumatic stress disorder and standardized test-taking ability. Journal of Educational Psychology. 102(1), 223-233.
Engel, L., Rutkowski, D. &, Rutkowski, L. (2009). The harsher side of globalization: Violent conflict and academic achievement. Globalisation, Societies and Education, 7(4). 433-456.
Barth, J., Rutkowski, L., Neuschmidt, O., & Gonzalez, E. (2009). Curriculum coverage and scale correlation on TIMSS 2003. IEA-ETS Research Institute Monograph, 2, 85-112.
Rutkowski, L., & Rutkowski, D. (2009). Trends in TIMSS responses over time: Evidence of global forces in education? Educational Research and Evaluation, 15(2), 137-152.
Books and Book Chapters
Rutkowski, L., von Davier, M, & Rutkowski, D. (Eds.). (2014). Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis. London: Chapman & Hall/CRC Press.
Rutkowski, L., von Davier, M. Gonzalez, E., & Zhou, Y. (2014). Assessment design for international large-scale assessment. In L. Rutkowski, M. von Davier, & D. Rutkowski (Eds.), Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis. London: Chapman & Hall/CRC Press.
Rutkowski, L. & Zhou, Y. (2014). Structural models for international large-scale assessment data. In L. Rutkowski, M. von Davier, & D. Rutkowski (Eds.), Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis. London: Chapman & Hall/CRC Press.
Neuschmidt, O., Barth, J., Rutkowski, L., & Rutkowski, D. (2010). Effective schools in the Arab region: A Multi-level approach using TIMSS 2003 data. In D. Sharpes (Ed.). Handbook on International Studies in Education. Charlotte, NC: Information Age Publishing.
Anderson, C.J., & Rutkowski, L. (2008). Logistic regression models for multicategory variables. In J. Osborne (ed.) Best Practices in Quantitative Methods. Thousand Oaks, CA: Sage.
Reports and Other Published Work
Carney, C., Rutkowski, D., & Rutkowski, L. (2015). Indiana educational policy context: The case of the Rules for Educator Preparation and Accountability (REPA). Paper commissioned by Indiana University School of Education Policy Council and Long Range Planning Committee.
Rutkowski, D., Rutkowski, L. & Plucker, J. (2014). What matters is what’s measured— not who measures it. PDK Blog: Learning on the edge. https://pdkassociation.org/blogs/learning-on-the-edge/matters-whats-measured-measures/
Plucker, J., Rutkowski, L., & Rutkowski, D. (2014, December 2). Should local districts participate in international assessments? Education Week.
Rutkowski, L., & Rutkowski, D. (2014, March 20). So how overblown were No. 1 Shanghai’s PISA results? Washington Post. Retrieved from http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/03/20/so-how-overblown-were-no-1-shanghais-pisa-results/
OECD. (2013). The TALIS 2013 conceptual framework. Document commissioned by the Organisation for Economic Cooperation and Development. Paris: OECD.
Rutkowski, D., & Rutkowski, L. (2013, October 2). Schools good, schools bad: Reducing the American education debate to absurdity. Education Week, pp. 28, 32.
Rutkowski, D., Wild, J., & Rutkowski, L. (2013). Indiana’s TIMSS 2011 performance: Outperforming much of the world in math and science, but issues remain for gender achievement and high performers. CEEP Education Policy Brief, 11(1), 1-8.
Rutkowski, L., & Svetina, D. (2013). Evaluation of distributional assumptions and hypotheses of measurement invariance in the 2013 Teaching and Learning International Survey. Paper commissioned by the Organisation for Economic Co-operation and Development.
OECD. (2012). Analysis of TALIS 2013 field trial data – summary report. Document commissioned by the Organisation for Economic Cooperation and Development. Paris: OECD.
Rutkowski, L., Rutkowski, D., Plucker, J., & Prusinski, E. (2011). Differential impacts of dual and advanced placement credits on higher education outcomes. Report commissioned by the Office of the Vice President for University Regional Affairs, Planning, and Policy, Indiana University Bloomington.
OECD. (2009). Chapter five: School evaluation, teacher appraisal and feedback and its impact upon schools and teachers. Creating Effective Teaching and Learning Environments: First Results from TALIS. Paris: OECD.
Rutkowski, L. (2009). Analyzing the TALIS 2008 data using WesVar. TALIS User Guide for the International Database. Paris: OECD.
Rutkowski, L. (2007). Review of Clustering for data mining: A data recovery approach by Boris Mirkin. Psychometrika, 72(1), 109-110.
- Rutkowski, Leslie; Svetina, Dubravka & Liaw, Yuan-Ling (2019). Collapsing Categorical Variables and Measurement Invariance. Structural Equation Modeling. ISSN 1070-5511. . doi: 10.1080/10705511.2018.1547640 Show summary
- Svetina, Dubravka; Liaw, Yuan-Ling; Rutkowski, Leslie & Rutkowski, David (2019). Routing Strategies and Optimizing Design for Multistage Testing in International Large-Scale Assessments. Journal of Educational Measurement. ISSN 0022-0655. 56(1), s 192- 213 . doi: 10.1111/jedm.12206
- Liaw, Yuan-Ling; Wu, Yuanyue; Rutkowski, David & Rutkowski, Leslie (2018). Evaluating PISA scales across Chinese economies. Asia Pacific Journal of Education. ISSN 0218-8791. 38(3), s 432- 451 . doi: 10.1080/02188791.2018.1491388
- Matta, Tyler; Rutkowski, Leslie; Rutkowski, David & Liaw, Yuan-Ling (2018). lsasim: an R package for simulating large‑scale assessment data. Large-scale Assessments in Education. ISSN 2196-0739. 6:15(15), s 1- 33 . doi: 10.1186/s40536-018-0068-8 Full text in Research Archive. Show summary
- Rutkowski, David; Rutkowski, Leslie & Liaw, Yuan-Ling (2018). Measuring Widening Proficiency Differences in International Assessments: Are Current Approaches Enough?. Educational Measurement: Issues and Practice. ISSN 0731-1745. 37(4), s 40- 48 . doi: 10.1111/emip.12225 Show summary
- Rutkowski, Leslie & Rutkowski, David (2018). Improving the Comparability and Local Usefulness of International Assessments: A Look Back and A Way Forward. Scandinavian Journal of Educational Research. ISSN 0031-3831. 62(3), s 354- 367 . doi: 10.1080/00313831.2016.1261044
- Rutkowski, David; Rutkowski, Leslie; Wild, J. & Burroughs, N. (2017). Poverty and educational achievement in the U.S.: A less biased estimate using PISA 2012 data. Journal of Children and Poverty. ISSN 1079-6126. 24(1), s 47- 67 . doi: 10.1080/10796126.2017.1401898 Full text in Research Archive.
- Rutkowski, Leslie (2017). Design considerations for planned missing auxiliary data in a latent regression context. Psychological Test and Assessment Modeling. ISSN 2190-0493. 59(1), s 55- 70
- Rutkowski, Leslie & Svetina, Dubravka (2017). Measurement Invariance in International Surveys: Categorical Indicators and Fit Measure Performance. Applied Measurement in Education. ISSN 0895-7347. 30(1), s 39- 51 . doi: 10.1080/08957347.2016.1243540
- Svetina, Dubravka & Rutkowski, Leslie (2017). Multidimensional measurement invariance in an international context: Fit measure performance with many groups. Journal of Cross-Cultural Psychology. ISSN 0022-0221. 48(7), s 991- 1008 . doi: 10.1177/0022022117717028
- Rutkowski, Leslie & Rutkowski, David (2016). A call for a more measured approach to reporting and interpreting PISA results. Educational Researcher. ISSN 0013-189X. 45(4), s 252- 257 . doi: 10.3102/0013189X16649961
- Rutkowski, Leslie; Rutkowski, David & Zhou, Yan (2016). Item Calibration Samples and the Stability of Achievement Estimates and System Rankings: Another Look at the PISA Model. International Journal of Testing. ISSN 1530-5058. 16(1), s 1- 20 . doi: 10.1080/15305058.2015.1036163
- Rutkowski, Leslie & Zhou, Yan (2015). Correcting Measurement Error in Latent Regression Covariates via the MC-SIMEX Method. Journal of Educational Measurement. ISSN 0022-0655. 52(4), s 359- 375 . doi: 10.1111/jedm.12090
- Rutkowski, Leslie & Zhou, Yan (2015). The impact of missing and error-prone auxiliary information on sparse-matrix sub-population parameter estimates. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences. ISSN 1614-1881. 11(3), s 89- 99 . doi: 10.1027/1614-2241/a000095
- Rutkowski, David & Rutkowski, Leslie (2016). What teachers can’t and can learn from international assessments. CEMO white paper.
- Rutkowski, Leslie (2016). A look at the most pressing methodological issues in international assessment.. Show summary
- Rutkowski, Leslie (2016). Causal inferences with cross-sectional large scale assessment data. Large-scale Assessments in Education. ISSN 2196-0739. 4 Show summary
- Rutkowski, Leslie (2016). Introduction to special issue on quasi-causal methods. Large-scale Assessments in Education. ISSN 2196-0739. 4, s 1- 6 . doi: 10.1186/s40536-016-0023-5
- Rutkowski, Leslie & Rutkowski, David (2016). International surveys, educational ISBN: 9780470670590 DOI: 10.1002/9780470670590.wbeog949. Show summary
- Rutkowski, Leslie & Rutkowski, David (2016). The relation between students’ perception of InQua and bullying victimization, In Trude Nilsen & Jan-Eric Gustafsson (ed.), Teacher Quality, Instructional Quality and Student Outcome. Relationships Across Countries, Cohorts and Time.. Springer. ISBN 978-3-319-41251-1. Chapter. s 103 - 120