rogerska

• Kaliisa, Rogers; Damsa, Crina; Eagan, Brendan; Esterhazy, Rachelle; Langfold, Malcolm & Misiejuk, Kamila (2026). Do modes of presence matter? regulation and participation in graduate students’ hybrid synchronous group-based collaboration. Learning and Instruction. ISSN 0959-4752. 105. doi: 10.1016/j.learninstruc.2026.102405. Full text in Research Archive
• Mahdi, Reza; Keykha, Ahmad; Kaliisa, Rogers & Darabi, Fatemeh (2025). Exploring applications of artificial intelligence in enhancing the quality of medical education: a mixed methods research synthesis. Journal of Medical Education Development. 18(4), p. 119–139. doi: 10.61882/edcj.18.4.119. Full text in Research Archive
• Qushem, Umar Bin; Christopoulos, Athanasios; Kaliisa, Rogers; Khalil, Mohammad; Salakoski, Tapio & Laakso, Mikko Jussi (2025). Technology-enhanced Learning and Learning Analytics for personalized STEM learning: A scoping review. International Journal of Educational Research. ISSN 0883-0355. 134. doi: 10.1016/j.ijer.2025.102827. Full text in Research Archive Show summary

  Background The increasing focus on Personalized STEM Learning (PSL) highlights the need to understand how Technology-Enhanced Learning (TEL) and Learning Analytics (LA) can be effectively integrated to support adaptive learning. While TEL-LA interventions have shown promise in optimizing learning pathways, an in-depth review is needed to evaluate their technological, pedagogical, and analytical characteristics, as well as their impact and implementation challenges. Aim and method The present study constitutes a systematic scoping review of 31 empirical intervention studies published between 2020 and 2024, analyzing recent developments in TEL-LA for PSL. The review examines the key characteristics of these interventions, their impact on learning outcomes, and the challenges in their implementation. Results The findings indicate that Intelligent Tutoring Systems (ITS) were the most widely applied technology in K–12 (mathematics), while Virtual Reality (VR) was utilized for immersive educational experiences in Higher Education (engineering). LA techniques, such as regression analysis, exploratory data analysis, and clustering, were crucial in monitoring engagement and providing personalized feedback. Self-regulated learning strategies were frequently embedded in TEL-LA interventions, with studies reporting improvements in student motivation, problem-solving skills, and academic performance. Conclusions The present study provides a robust foundation for understanding how TEL-LA for PSL can be effectively implemented to achieve Precision Education (PE), thereby offering evidence-based insights for educators, practitioners, and policymakers seeking to enhance personalized learning experiences in STEM education. It also expands the corpus of knowledge on how TEL-LA interventions are determining the learning outcomes and measuring the learning impact across education contexts.

• Kaliisa, Rogers; Baker, Ryan Shaun; Wasson, Barbara & Prinsloo, Paul (2025). The Coming but Uneven Storm: How AI Regulation Will Impact AI and Learning Analytics Research in Different Countries: How AI Regulation Will Impact AI and Learning Analytics Research in Different Countries. Journal of Learning Analytics. 12(2), p. 140–157. doi: 10.18608/jla.2025.8495. Full text in Research Archive Show summary

  This article investigates the state of AI regulations from diverse geopolitical contexts including the European Union, the United States, China, and several African nations, and their implications for learning analytics (LA) and AI research. We used a comparative analysis approach of 11 AI regulatory documents and applied the OECD framework to classify core priorities. The findings showed that the European Union and China have adopted the most comprehensive and strict AI regulations, potentially setting global standards once developed into legal instruments. In contrast, the United States has experienced a significant policy shift: the previously established AI Bill of Rights and Executive Order 14110 were revoked in early 2025 and replaced by a new Executive Order focused on economic competitiveness and deregulation. While the EU, UK, and China prioritize safety and avoiding algorithmic bias, Japan and some African countries emphasize AI systems’ economic and societal potential. Certain data types (e.g., Biometric sensors) might become challenging to use for education due to strict guidelines (particularly within the EU), necessitating careful consideration of data nature and privacy risks before use and when transferring data across jurisdictions. As regulations shift, researchers and practitioners must adapt to changing regulations across international boundaries, the legal benefits and risks of using open-source licenses, and regulatory sandboxes to minimize the risks for learners and developers and ensure ethical and legally compliant AI-driven LA systems.

• Kaliisa, Rogers; Misiejuk, Kamila; López-Pernas, Sonsoles & Saqr, Mohammed (2025). How does artificial intelligence compare to human feedback? A meta-analysis of performance, feedback perception, and learning dispositions. Educational Psychology. ISSN 0144-3410. p. 32–32. doi: 10.1080/01443410.2025.2553639. Full text in Research Archive Show summary

  This exploratory meta-analysis synthesises current research on the effectiveness of Artificial Intelligence (AI)-generated feedback compared to traditional human-provided feedback. Drawing on 41 studies involving a total of 4813 students, the findings reveal no statistically significant differences in learning performance between students who received AI-generated feedback and those who received human-provided feedback. The pooled effect size was small and statistically insignificant (Hedge’s g = 0.25, CI [−0.11; 0.60]), indicating that AI feedback is potentially as effective as human feedback. A separate meta-analysis focusing exclusively on studies in the domain of language and writing confirmed similar findings, with high heterogeneity persisting (I2 = 95%). The study further explored differences in feedback perception and found a small, negative, and statistically insignificant effect size (Hedge’s g = −0.20, CI [−0.67; 0.27]). The study advocates for a hybrid approach, leveraging the scalability of AI while retaining the deep, empathetic, and contextual features of human feedback.

• Esterhazy, Rachelle; Kaliisa, Rogers; Sanchez, Daniel; Langford, Malcolm & Damsa, Crina I. (2025). Advancing Multimodal Collaboration Analytics. Journal of Learning Analytics. 12(2), p. 105–124. doi: 10.18608/jla.2025.8625. Full text in Research Archive
• Misiejuk, Kamila; López-Pernas, Sonsoles; Kaliisa, Rogers & Saqr, Mohammed (2025). Mapping the Landscape of Generative Artificial Intelligence in Learning Analytics: A Systematic Literature Review. Journal of Learning Analytics. 12(1), p. 12–31. doi: 10.18608/jla.2025.8591. Full text in Research Archive Show summary

  Generative artificial intelligence (GenAI) has opened new possibilities for designing learning analytics (LA) tools, gaining new insights about student learning processes and their environment, and supporting teachers in assessing and monitoring students. This systematic literature review maps the empirical research of 41 papers utilizing GenAI and LA and interprets the results through the lens of the LA/EDM process cycle. Currently, GenAI is mostly implemented to automate discourse coding, scoring, or classification tasks. Few papers used GenAI to generate data or to summarize text. Classroom integrations of GenAI and LA mostly explore facilitating human–GenAI collaboration, rather than implementing automated feedback generation or GenAI-powered learning analytics dashboards. Most papers use Generative Adversarial Network models to generate synthetic data, BERT models for classification or prediction tasks, BERT or GPT models for discourse coding, and GPT models for tool integration. Although most studies evaluate the GenAI output, we found examples of using GenAI without the output validation, especially when its output feeds into an LA pipeline aiming to, for example, develop a dashboard. This review offers a comprehensive overview of the field to aid LA researchers in the design of research studies and a contribution to establishing best practices to integrate GenAI and LA.

• Lopez-Pernas, Sonsoles; Misiejuk, Kamila; Kaliisa, Rogers & Saqr, Mohammed (2025). Capturing the Process of Students' AI Interactions when Creating and Learning Complex Network Structures. IEEE Transactions on Learning Technologies. 18, p. 556–568. doi: 10.1109/TLT.2025.3568599. Full text in Research Archive Show summary

  Despite the growing use of large language models (LLMs) in educational contexts, there is no evidence on how these can be operationalized by students to generate custom datasets suitable for teaching and learning. Moreover, in the context of network science, little is known about whether LLMs can replicate real-life network properties. This study addresses these gaps by evaluating the use of generative artificial intelligence (AI), specifically LLMs, to create synthetic network datasets for educational use. The analyzed data include students’ AI-generated network datasets, their interactions with the LLMs, and their perceptions and evaluations of the task's value. The results indicate that the LLM-generated networks had properties closer to real-life networks, such as higher transitivity, network density, and smaller mean distances compared to randomly generated networks. Thus, our findings show that students can use LLMs to produce synthetic networks with realistic structures while tailoring to the individual preferences of each student. The analysis of students’ interactions (prompts) with the LLMs revealed a predominant use of direct instructions and output specifications, with less emphasis on providing contextual details or iterative refinement of the LLM's responses, which highlights the need for AI literacy training to optimize students’ use of generative AI. Students’ perceptions of the use of AI were overall positive; they found using LLMs time saving and beneficial, although opinions on output relevance and quality varied, especially for assignments requiring replication of specific networks.

• Kaliisa, Rogers; López-Pernas, Sonsoles; Misiejuk, Kamila; Damsa, Crina I.; Sobocinski, Márta & Järvelä, Sanna [Show all 7 contributors for this article] (2025). A Topical Review of Research in Computer-Supported Collaborative Learning: Questions and Possibilities. Computers & Education. ISSN 0360-1315. 228. doi: 10.1016/j.compedu.2025.105246. Full text in Research Archive Show summary

  This study maps the evolution and state of Computer-Supported Collaborative Learning (CSCL) research, analyzing 6388 documents published between 1990 and 2022. The findings highlight the sustained engagement of a core group of scholars and the field's geographic concentration in Western countries, particularly the USA and Europe. While the field remains productive and diverse, recent trends reflect a growing emphasis on integrating emerging technologies such as learning analytics, augmented and virtual reality, and artificial intelligence (AI) into collaborative learning contexts. The study uncovers a tension within CSCL between its epistemological and methodological diversity and the need for theoretical coherence. This diversity has allowed the field to adapt and innovate but raises concerns about fragmentation and the risk of losing a unified identity. For example, while scripting remains a foundational topic, debates persist on balancing instructional guidance with learner agency to foster productive collaboration. Looking ahead, the study underscores the need for CSCL to develop integrative theoretical frameworks that bridge its rich historical foundations with the challenges posed by large-scale, distributed, and technology-mediated collaboration. Addressing these challenges, such as aligning AI innovations with existing theories and ensuring cumulative knowledge-building, will be critical for the field's ability to sustain its relevance and influence in understanding collaborative learning in complex educational environments.

• Misiejuk, Kamila; Samuelsen, Jeanette; Kaliisa, Rogers & Prinsloo, Paul (2025). Idiographic learning analytics: Mapping of the ethical issues. Learning and Individual Differences. ISSN 1041-6080. 117. doi: 10.1016/j.lindif.2024.102599. Full text in Research Archive Show summary

  This paper examines the ethical implications of idiographic learning analytics (LA), a personalized approach to learning where individual student data serves as the primary reference frame for analysis. By focusing on the unique characteristics of each learner, idiographic LA offers tailored educational experiences that could potentially enhance learning outcomes. This personalized approach offers new possibilities to mitigate common ethical challenges in nomothetic LA. This study explores the ethical dimensions of idiographic LA by integrating theories and lessons learned from implementing idiographic approaches in the fields of quantified self, recommender systems, intelligent tutoring systems, and artificial intelligence personalization. We employ a consequentialist ethical perspective to assess the implications of idiographic practices in education, highlighting potential benefits and risks and mapping the main ethical issues differentiating idiographic LA from nomothetic LA. The paper provides several points of consideration to ensure ethical implementations of idiographic LA.

• Kaliisa, Rogers; López-Pernas, Sonsoles; Misiejuk, Kamila; Saqr, Mohammed & Damsa, Crina I. (2024). Exploring the Dynamics and Trends of Knowledge Exchange: A Structured Topic Modeling Approach of the CSCL Conference Proceedings. Proceedings of the 17th International Conference on Computer-Supported Collaborative Learning. p. 261–264. doi: 10.22318/cscl2024.105178. Full text in Research Archive
• Sanchez, Daniel; Kaliisa, Rogers; Esterhazy, Rachelle & Damsa, Crina I. (2024). Unpacking the Elicitation Process for Multimodal Learning Analytics based Feedback. Proceedings 14th International Conference on Learning Analytics & Knowledge. Full text in Research Archive
• Misiejuk, Kamila; Kaliisa, Rogers & Scianna, Jennifer (2024). Augmenting assessment with AI coding of online student discourse: A question of reliability. Computers & Education: Artificial Intelligence (CAEAI). 6. doi: 10.1016/j.caeai.2024.100216. Full text in Research Archive Show summary

  Currently, many generative Artificial Intelligence (AI) tools are being integrated into the educational technology landscape for instructors. Our paper examines the potential and challenges of using Large Language Models (LLMs) to code student-generated content in online discussions based on intended learning outcomes and how instructors could use this to assess the intended and enacted learning design. If instructors were to rely on LLMs as a means of assessment, the reliability of these models to code the data accurately is crucial. Employing a diverse set of LLMs from the GPT family and prompting techniques on an asynchronous online discussion dataset from a blended-learning bachelor-level course, our research examines the reliability of AI-supported coding in educational research. Findings reveal that while AI-supported coding demonstrates efficiency, achieving substantial, moderate agreement with human coding for specific nuanced and context-dependent codes is challenging. Moreover, the high cost, token limits, and the advanced necessary skills needed to write API scripts might limit the usability of AI-driven coding. Finally, implementation would require specific parameterization techniques based on the class and may not be feasible for widespread implementation. Our study underscores the importance of transparency in AI coding methodologies and the need for a hybrid approach that integrates human judgement to ensure data accuracy and interpretability. In addition, it contributes to the knowledge base about the reliability of LLMs to code real, small datasets using complex codes that are common in the instructor's practice and explores the potential and challenges of using these models for assessment purposes.

• Prinsloo, Paul & Kaliisa, Rogers (2024). Dimensions of privacy and its implications for learning analytics. Preliminary insights from/for African higher education. Learning, Media and Technology. ISSN 1743-9884. doi: 10.1080/17439884.2024.2321437. Full text in Research Archive
• Kaliisa, Rogers; Dane, Jai Oni; Sanchez, Daniel; Pratt, Jade; Damsa, Crina I. & Scianna, Jennifer (2023). Understanding Group Dynamics During Synchronous Collaborative Problem-Solving Activities: An Epistemic Network Approach. In Ruis, Andrew R & Seung, Lee. B (Ed.), Advances in Quantitative Ethnography. Springer Nature. ISSN 9783030677886. p. 81–95. doi: 10.1007/978-3-031-47014-1_6. Full text in Research Archive
• Qushem, Umar Bin; Oyelere, Solomon Sunday; Akçapınar, Gökhan; Kaliisa, Rogers & Laakso, Mikko-Jussi (2023). Unleashing the Power of Predictive Analytics to Identify At-Risk Students in Computer Science. Technology, Knowledge and Learning. ISSN 2211-1662. doi: 10.1007/s10758-023-09674-6. Full text in Research Archive
• Scianna, Jennifer & Kaliisa, Rogers (2023). ‘SSEEN’: a networked approach to uncover connections between sentiment, social, and epistemic elements of student online forum discourse. Educational Technology Research and Development. ISSN 1042-1629. doi: 10.1007/s11423-023-10310-4. Full text in Research Archive Show summary

  Educational researchers have pointed to socioemotional dimensions of learning as important in gaining a more nuanced description of student engagement and learning. However, to date, research focused on the analysis of emotions has been narrow in its focus, centering on affect and sentiment analysis in isolation while neglecting how emotions potentially interact with social elements and course objectives in learning environments. In this paper, we present a case study analysis of seven asynchronous online discussions delivered as part of a blended-learning bachelor level course; we demonstrate the utility of a novel analysis workflow and visualization method which we refer to as SSEEN (social sentiment embedded epistemic networks) to uncover insights into the connection between social networks, course content, and detected student sentiment. The findings show that negative sentiment was most often associated with course content, but contrary to insights from prior research, negative sentiment served as a marker of engagement as students connected content to their own personal experiences. By simultaneously considering the social network alongside the sentiment-colored edges, we note that negative sentiment is not consistent within the discourse of particular students or between pairs indicating that sentiment did not appear to be an indicator of peer conflict or breaches in social contracts. The findings demonstrate how the proposed approach (SSEEN) can support educational researchers to gain a more nuanced understanding of the social, emotional, and epistemic dimensions of learning in asynchronous online discussions.

• Kaliisa, Rogers; Jivet, Ioana & Prinsloo, Paul (2023). A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboards. International Journal of Educational Technology in Higher Education. doi: 10.1186/s41239-023-00394-6. Full text in Research Archive Show summary

  Higher education institutions are moving to design and implement teacher-facing learning analytics (LA) dashboards with the hope that instructors can extract deep insights about student learning and make informed decisions to improve their teaching. While much attention has been paid to developing teacher-facing dashboards, less is known about how they are designed, implemented and evaluated. This paper presents a systematic literature review of existing studies reporting on teacher-facing LA dashboards. Out of the 1968 articles retrieved from several databases, 50 articles were included in the final analysis. Guided by several frameworks, articles were coded based on the following dimensions: purpose, theoretical grounding, stakeholder involvement, ethics and privacy, design, implementation, and evaluation criteria. The findings show that most dashboards are designed to increase teachers’ awareness but with limited actionable insights to allow intervention. Moreover, while teachers are involved in the design process, this is mainly at the exploratory/problem definition stage, with little input beyond this stage. Most dashboards were prescriptive, less customisable, and implicit about the theoretical constructs behind their designs. In addition, dashboards are deployed at prototype and pilot stages, and the evaluation is dominated by self-reports and users’ reactions with limited focus on changes to teaching and learning. Besides, only one study considered privacy as a design requirement. Based on the findings of the study and synthesis of existing literature, we propose a four-dimensional checklist for planning, designing, implementing and evaluating LA dashboards.

• Kaliisa, Rogers; Jivet, Ioana & Prinsloo, Paul (2023). A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboards. International Journal of Educational Technology in Higher Education. 20(1). doi: 10.1186/s41239-023-00394-6. Full text in Research Archive
• Kaliisa, Rogers; Palmer, Edward & Miller, Julia (2022). Lecturers’ Perspectives on Mobile Learning in Higher Education: Experiences and Implementation Barriers. Information Technology, Education and Society. ISSN 1037-616X. doi: 10.7459/ites/18.1.02. Full text in Research Archive Show summary

  This paper reports the experiences of university lecturers towards mobile learning from two points of view: (i) lecturers’ needs and experiences in relation to mobile learning design and implementation, and (ii) possible barriers to their adoption in higher education classrooms. The findings from a qualitative case study that conducted semi-structured interviews with 12 lecturers from a developed and developing country contexts revealed; the need for strong commitment from institutional leadership, presence of an institutional and national mobile learning policy, a culture of innovation and willingness to revise the traditional curriculum to incorporate the new mobile pedagogies.

• Prinsloo, Paul & Kaliisa, Rogers (2022). Data privacy on the African continent: Opportunities, challenges and implications for learning analytics. British Journal of Educational Technology (BJET). ISSN 0007-1013. p. 1–20. doi: 10.1111/bjet.13226. Full text in Research Archive Show summary

  Whilst learning analytics is still nascent in most African higher education institutions, many African higher education institutions use learning platforms and analytic services from providers outside of the African continent. A critical consideration of the protection of data privacy on the African continent and its implications for learning analytics in African higher education is there-fore needed. In this paper, we map the current state of legal and regulatory environments and frameworks on privacy to establish their implications for learning analytics. This scoping review of privacy regulations in 32 African countries, complemented by 15 scholarly papers, revealed that there are numerous national and regional legislation and regulatory frameworks, provid-ing clear pointers pertaining to (student) data privacy to governments, higher education institutions and re-searchers. As such, the findings of this research have implications for African higher education to ensure not only legal compliance but also to oversee and safe-guard student data privacy as part of their fiduciary duty. This research provides crucial insights regarding the importance of context for thinking about the expansion and institutional adoption of learning analytics.

• Kaliisa, Rogers & Dolonen, Jan Arild (2022). CADA: a teacher-facing learning analytics dashboard to foster teachers’ awareness of students’ participation and discourse patterns in online discussions. Technology, Knowledge and Learning. ISSN 2211-1662. doi: 10.1007/s10758-022-09598-7. Full text in Research Archive Show summary

  Despite the potential of learning analytics (LA) to support teachers’ everyday practice, its adoption has not been fully embraced due to the limited involvement of teachers as co-designers of LA systems and interventions. This is the focus of the study described in this paper. Following a design-based research (DBR) approach and guided by concepts from the socio-cultural perspective and human-computer interaction (HCI), we design, test, and evaluate a teacher-facing LA dashboard, the Canvas Discussion Analytics Dashboard (CADA), in real educational settings. The goal of this dashboard is to support teachers’ roles in online environments through insights into students’ participation and discourse patterns. We evaluate CADA through 10 in-depth interviews with university teachers to examine their experiences using CADA in seven blended undergraduate and graduate courses over a one-year period. The findings suggest that engaging teachers throughout the analytics tool design process and giving them control/agency over LA tools can favour their adoption in practice. Additionally, the alignment of dashboard metrics with relevant theoretical constructs allows teachers to monitor the learning designs and make course design changes on the fly. The teachers in this study emphasise the need for LA dashboards to provide actionable insights by moving beyond what things are toward how things should be. This study has several contributions. First, we make an artefact contribution (e.g. CADA), an LA dashboard to support teachers with insights into students’ online discussions. Second, by leveraging theory, and working with the teachers to develop and implement a dashboard in authentic teaching environments, we make an empirical, theoretical and methodological contribution to the field of learning analytics and technology-enhanced learning. We synthesise these through practical design and implementation considerations for researchers, dashboard developers, and higher education institutions.

• Prinsloo, Paul & Kaliisa, Rogers (2022). Learning Analytics on the African Continent: An Emerging Research Focus and Practice. Journal of Learning Analytics. 9(2), p. 218–235. doi: 10.18608/jla.2022.7539. Full text in Research Archive
• Kaliisa, Rogers; Rienties, Bart; Mørch, Anders & Kluge, Anders (2022). Social learning analytics in computer-supported collaborative learning environments: A systematic review of empirical studies. Computers and Education Open (CAEO). doi: 10.1016/j.caeo.2022.100073. Full text in Research Archive Show summary

  Social learning analytics (SLA) is a promising approach for identifying students’ social learning processes in computer-supported collaborative learning (CSCL) environments. To identify the main characteristics of SLA, gaps and future opportunities for this emerging approach, we systematically identified and analyzed 36 SLA related studies conducted between 2011 and 2020. We focus on SLA implementation and methodological characteristics, educational focus, and the studies’ theoretical perspectives. The results show the predominance of SLA in formal and fully online settings with social network analysis (SNA) a dominant analytical technique. Most SLA studies aimed to understand students’ learning processes and applied the social constructivist perspective as a lens to interpret students’ learning behaviours. However, (i) few studies involve teachers in developing SLA tools, and rarely share SLA visualizations with teachers to support teaching decisions; (ii) some SLA studies are atheoretical; and (iii) the number of SLA studies integrating more than one analytical approach remains limited. Moreover, (iv) few studies leveraged innovative network approaches (e.g., epistemic network analysis, multimodal network analysis), and (v) studies rarely focused on temporal patterns of students’ interactions to assess how students’ social and knowledge networks evolve over time. Based on the findings and the gaps identified, we present methodological, theoretical and practical recommendations for conducting research and creating tools that can advance the field of SLA.

• Misiejuk, Kamila; Scianna, Jennifer; Kaliisa, Rogers; Vachuska, Karl & Shaffer, David Williamson (2021). Incorporating Sentiment Analysis with Epistemic Network Analysis to Enhance Discourse Analysis of Twitter Data. In Ruis, Andrew R & Seung, Lee. B (Ed.), Advances in Quantitative Ethnography. Springer Nature. ISSN 9783030677886. p. 375–389. doi: 10.1007/978-3-030-67788-6_26. Full text in Research Archive Show summary

  While there has been much growth in the use of microblogging platforms (e.g., Twitter) to share information on a range of topics, researchers struggle to analyze the large volumes of data produced on such platforms. Established methods such as Sentiment Analysis (SA) have been criticized over their inaccuracy and limited analytical depth. In this exploratory methodological paper, we propose a combination of SA with Epistemic Network Analysis (ENA) as an alternative approach for providing richer qualitative and quantitative insights into Twitter discourse. We illustrate the application and potential use of these approaches by visualizing the differences between tweets directed or discussing Democrats and Republicans after the COVID-19 Stimulus Package announcement in the US. SA was integrated into ENA models in two ways: as a part of the blocking variable and as a set of codes. Our results suggest that incorporating SA into ENA allowed for a better understanding of how groups viewed the components of the stimulus issue by splitting them by sentiment and enabled a meaningful inclusion of data with singular subject focus into the ENA models.

• Kaliisa, Rogers; Misiejuk, Kamila; Irgens, Golnaz Arastoopour & Misfeldt, Morten (2021). Scoping the Emerging Field of Quantitative Ethnography: Opportunities, Challenges and Future Directions. In Ruis, Andrew R & Seung, Lee. B (Ed.), Advances in Quantitative Ethnography. Springer Nature. ISSN 9783030677886. p. 3–17. doi: 10.1007/978-3-030-67788-6_1. Full text in Research Archive Show summary

  Quantitative Ethnography (QE) is an emerging methodological approach that combines ethnographic and statistical tools to analyze both Big Data and smaller data to study human behavior and interactions. This paper presents a methodological scoping review of 60 studies employing QE approaches with an intention to characterize and establish where the boundaries of QE might and should be in order to establish the identity of the field. The key finding is that QE researchers have enough commonality in their approach to the analysis of human behavior with a strong focus on grounded analysis, the validity of codes and consistency between quantitative models and qualitative analysis. Nonetheless, in order to reach a larger audience, the QE community should attend to a number of conceptual and methodological issues (e.g. interpretability). We believe that the strength of work from individual researchers reported in this review and initiatives such as the recently established International Society for Quantitative Ethnography (ISQE) can present a powerful force to shape the identity of the QE community

• Damsa, Crina-Ioana; Muukkonen, Hanni; Leeuwen, Anouschka van; Gašević, Dragan; Janssen, Jeroen & Esterhazy, Rachelle [Show all 10 contributors for this article] (2021). Analyzing and Conceptualizing Collaborative Learning with Digital Knowledge Objects. Proceedings of the annual meeting of the ISSS. p. 41–44. Full text in Research Archive
• Kaliisa, Rogers; Mørch, Anders Irving & Kluge, Anders (2021). ‘My Point of Departure for Analytics is Extreme Skepticism’: Implications Derived from An Investigation of University Teachers’ Learning Analytics Perspectives and Design Practices. Technology, Knowledge and Learning. ISSN 2211-1662. doi: 10.1007/s10758-020-09488-w. Full text in Research Archive Show summary

  The literature until 2020 has forecasted a significant uptake of learning analytics (LA) to support learning design in higher education. However, there remain only a few investigations into teachers’ course design practices and their perspectives on LA as a tool to support their design practices. This paper presents findings from an examination of 16 university teachers’ design practices and perspectives on LA at two Norwegian universities (The University of Oslo and Oslo Metropolitan University). On one hand, findings identified situational factors, feedback sources and teachers’ intuition as key influencers of teachers’ course design decisions. On the other hand, guided by principles of the technology acceptance model, this study identified mixed reactions amongst teachers regarding the awareness, understanding and potential use of LA to support course design practices. In particular, most teachers appreciated the formative and normative value of LA to provide more objective evidence about students’ learning patterns and to shape learning trajectories, but some were skeptical about the evaluative role where LA is used to evaluate teachers’ and students’ performance based on unnuanced data (e.g. no theory guidance) with limited depth in observation. This article contributes to the understanding of factors fundamental to linking LA to teachers’ course design practices by synthesizing findings to propose a ‘bi-directional LA-course design’ conceptual framework that clarifies key elements that influence teachers’ design practices and highlighting their implications for LA integration.

• Kaliisa, Rogers; Kluge, Anders & Mørch, Anders (2021). Overcoming Challenges to the Adoption of Learning Analytics at the Practitioner Level: A Critical Analysis of 18 Learning Analytics Frameworks. Scandinavian Journal of Educational Research. ISSN 0031-3831. doi: 10.1080/00313831.2020.1869082. Full text in Research Archive Show summary

  Learning analytics (LA) is a fast-growing field but adoption by teachers remain limited. This paper presents the results of a review of 18 LA frameworks and discusses how they have tried to address prominent challenges in LA adoption. The results show that researchers have made significant advances in developing appropriate frameworks to conceptualise LA adoption among teachers, and have advanced considerably in connecting LA and learning theory. However, few frameworks are concretised into technological artefacts and concrete data streams. Moreover, there is a need to empirically validate and put into use the most promising existing frameworks. We hope that this review will be informative for teachers who have little LA experience but are interested in adopting LA in authentic practice.

• Kaliisa, Rogers; Kluge, Anders & Mørch, Anders Irving (2020). Combining Checkpoint and Process Learning Analytics to Support Learning Design Decisions in Blended Learning Environments. Journal of Learning Analytics. 7(3), p. 33–47. doi: 10.18608/JLA.2020.73.4. Full text in Research Archive Show summary

  Learning analytics (LA) constitutes a key opportunity to support learning design (LD) in blended learning environments. However, details as to how LA supports LD in practice and information on teacher experiences with LA are limited. This study explores the potential of LA to inform LD based on a one-semester undergraduate blended learning course at a Norwegian university. Our findings indicate that creating valuable connections between LA and LD requires a detailed analysis of student checkpoints (e.g., online logins) and process analytics (e.g., online content and interaction dynamics) to find meaningful learning behaviour patterns that can be forwarded to teachers in retrospect to support the redesign of courses. Moreover, the teachers in our study found the LA visualizations to be valuable for understanding student online learning processes, but they also requested the timely sharing of aggregated LA visualizations in a simple, easy-to-interpret format, yet detailed enough to be informative and actionable. We conclude the paper by arguing that the potential of LA to support LD is improved when multiple levels of LA are considered.

• Oyelere, Solomon Sunday; Bouali, Nacir; Kaliisa, Rogers; Obaido, George; Yunusa, Abdullahi Abubaker & Jimoh, Ebunayo R. (2020). Exploring the trends of educational virtual reality games: a systematic review of empirical studies. Smart Learning Environments. 7, p. 1–22. doi: 10.1186/s40561-020-00142-7. Full text in Research Archive Show summary

  Virtual Reality (VR) and educational games are emerging technologies mediating a rapid transformation in the educational world. However, few studies have systematically analyzed Educational Virtual Reality Games (EVRGs) and how they have been applied in educational settings. This study reviewed 31 articles published in high impact journals and educational conference proceedings to unravel the technological, pedagogical, and gaming characteristics of contemporary EVRGs. The results show the predominance of Oculus Rift headsets and HTC Vive as the main technology used in EVRGs. Moreover, the analysis revealed that the pedagogical application of the majority of EVRGs was developed for all levels of education (e.g. tertiary, K-12, lifelong learning), with the specific target audience of each game based on the desired learning outcome. Furthermore, the application of EVRGs has primarily focused on out of class use, with healthcare education topics dominating the topics taught using EVRGs. Based on our findings, we highlight some key implications and suggestions to advance the field of EVRGs.

• Kaliisa, Rogers; Mørch, Anders Irving & Kluge, Anders (2019). Exploring Social Learning Analytics to Support Teaching and Learning Decisions in Online Learning Environments. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11722 LNCS, p. 187–198. doi: 10.1007/978-3-030-29736-7_14. Full text in Research Archive Show summary

  Most teachers to date have adopted summative assessment items as a benchmark to measure students’ learning and for making pedagogical decisions. However, these may not necessarily provide comprehensive evidence for the actual learning process, particularly in online learning environments due to their failure to monitor students’ online learning patterns over time. In this paper, we explore how social learning analytics (SLA) can be used as a proxy by teachers to understand students’ learning processes and to support them in making informed pedagogical decisions during the run of a course. This study was conducted in a semester-long undergraduate course, at a large public university in Norway, and made use of data from 4 weekly online discussions delivered through the university learning management system Canvas. First, we used NodeXL a social network analysis tool to analyze and visualize students’ online learning processes, and then we used Coh-Metrix, a theoretically grounded, computational linguistic tool to analyze the discourse features of students’ discussion posts. Our findings revealed that SLA provides insight and an overview of the students’ cognitive and social learning processes in online learning environments. This exploratory study contributes to an improved conceptual understanding of SLA and details some of the methodological implications of an SLA approach to enhance teaching and learning in online learning environments.

• Kaliisa, Rogers & Picard, Michelle (2019). Mobile learning policy and practice in Africa: Towards inclusive and equitable access to higher education. Australasian Journal of Educational Technology (AJET). ISSN 1449-3098. 35(6), p. 1–14. doi: 10.14742/ajet.5562. Full text in Research Archive Show summary

  This article presents the results of a review of practice and policy in relation to mobile learning and its potential to enhance inclusive and equitable access to higher education in Africa. We reviewed academic literature on potential barriers. Then, we explored the current state of the mobile learning policy environment in 10 African countries through an analysis of how these policies have tried to address the prominent challenges in the adoption of mobile learning as identified in the literature. The findings reveal that significant resourcing inequalities and epistemological, sociocultural, and institutional barriers remain and affect mobile learning adoption. The analysis also reveals that there is still a policy vacuum in relation to mobile learning specific policies within African higher education institutional and governmental policies. Thus, the formal integration of mobile learning in higher education to facilitate equitable access is very much in its infancy. This article suggests a strong need for institutional, cross-institutional, national and African-wide mobile learning specific policies to ensure better implementation of mobile learning. As interest in mobile learning continues to grow, this review will provide insights into policy and strategic planning for the adoption of mobile learning to achieve inclusive and equitable access to higher education.

View all works in NVA

• Kaliisa, Rogers; Anna, Gillespie; Herodotou, Christothea; Kluge, Anders & Rienties, Bart (2021). Teachers’ Perspectives on the Promises, Needs and Challenges of Learning Analytics Dashboards: Insights from Institutions Offering Blended and Distance Learning. Springer Nature. ISBN 9783030812225. 20 p. Full text in Research Archive

View all works in NVA

• Kaliisa, Rogers & Damsa, Crina I. (2025). Understanding the impact of physical, online, and hybrid modalities on self and co-regulation during collaborative problem solving. Full text in Research Archive
• Damsa, Crina I. & Kaliisa, Rogers (2025). Towards Supporting Group Learning through Design-based Research and Multimodal Data-based Tools. Proceedings of the 18th International Conference on Computer-Supported Collaborative Learning - CSCL 2025. Full text in Research Archive
• Damsa, Crina I.; Esterhazy, Rachelle; Sanchez, Daniel & Kaliisa, Rogers (2024). Teamwork analytics for training collaborative problem solving in professional higher education. Full text in Research Archive
• Damsa, Crina I.; Esterhazy, Rachelle; Sanchez, Daniel & Kaliisa, Rogers (2024). Teamwork analytics for training collaborative problem solving in professional higher education. Full text in Research Archive
• Kaliisa, Rogers; Esterhazy, Rachelle; Damsa, Crina I. & Dolonen, Jan Arild (2024). The temporality of epistemic, procedural, and social dynamics in collaborative problem-solving. Full text in Research Archive
• Sanchez, Daniel; Kaliisa, Rogers; Esterhazy, Rachelle & Damsa, Crina I. (2023). From multimodal data to meaning: Developing interpretations of learning in team-based simulations. Full text in Research Archive
• Kaliisa, Rogers; Esterhazy, Rachelle; Sanchez, Daniel & Damsa, Crina I. (2023). Towards a conceptual framework for assessing collaborative problem solving in co-located settings using multimodal analytics. Full text in Research Archive
• Kaliisa, Rogers & Damsa, Crina I. (2023). Supporting teachers in hybrid learning environments: The role of learning analytics. Full text in Research Archive
• Esterhazy, Rachelle; Kaliisa, Rogers; Sanchez, Daniel; Langford, Malcolm & Damsa, Crina I. (2023). Multimodal collaboration analytics in collaborative problem solving – a scoping review. Full text in Research Archive
• Kaliisa, Rogers; Mørch, Anders & Kluge, Anders (2021). Correction to: ‘My Point of Departure for Analytics is Extreme Skepticism’: Implications Derived from An Investigation of University Teachers’ Learning Analytics Perspectives and Design Practices (Technology, Knowledge and Learning, (2021), 10.1007/s10758-020-09488-w). Technology, Knowledge and Learning. ISSN 2211-1662. doi: 10.1007/s10758-021-09574-7. Full text in Research Archive
• Mørch, Anders Irving; Andersen, Renate; Kaliisa, Rogers & Kristina, Litherland (2020). Mixed methods with social network analysis for networked learning: Lessons learned from three case studies. Full text in Research Archive
• Mørch, Anders Irving; Andersen, Renate; Kaliisa, Rogers & Litherland, Kristina (2020). Mixed methods with social network analysis for networked learning: Lessons learned from three case studies. Full text in Research Archive
• Kaliisa, Rogers (2020). Unpacking the Bi-directional relationship between learning analytics and learning design in blended learning environments. Full text in Research Archive Show summary

  This paper suggests a technology-supported teacher-led approach that includes leveraging learning analytics (LA) to support data-informed learning design (LD) decisions in blended learning environments. The context of this study is three to five blended Bachelors courses using the Canvas learning management system (LMS) at two public universities in Norway. This is a design-based research study, employing quantitative ethnography approaches. Data will be collected from multiple sources, i.e. course analytics, discussion forums, interviews, teachers LD representations, and in-class observations. The analysis will be conducted using social and epistemic network analysis, automated discourse analysis, inferential statistics and inductive thematic analysis. This PhD project is anticipated to contribute towards an empirically based theoretical discussion about the potential affordances of LA to transform LD from a craft into a sounder and more evidence-based field of research and practice. (4) (PDF) Unpacking the Bi-directional relationship between learning analytics and learning design in blended learning environments. Available from: https://www.researchgate.net/publication/340133140_Unpacking_the_Bi-directional_relationship_between_learning_analytics_and_learning_design_in_blended_learning_environments [accessed Mar 30 2020].

• Morten, Misfeldt; Spikol, Daniel; Kaliisa, Rogers; Jesper, Bruun; Saqr, Mohammed & Andrew, Ruis [Show all 7 contributors for this article] (2020). Quantitative ethnography as a framework for network analysis – a discussion of the foundations for network approaches to leaning analysis. Full text in Research Archive
• Kaliisa, Rogers (2019). LEVERAGING SOCIAL LEARNING ANALYTICS TO SUPPORT AND EVALUATE LEARNING DESIGN AND ADAPTIVE LEARNING. Full text in Research Archive
• Kaliisa, Rogers; Kluge, Anders & Mørch, Anders Irving (2018). How Learning Analytics supports Teachers to make good design choices and provide timely personalized feedback to students. Full text in Research Archive

View all works in NVA