Fall 2025 AI Research Study

During Fall 2025, faculty in the College of Education in collaboration with the Division of Information Technology, conducted a randomized control trial of the UMD Virtual Study Assistant (VSA) - a chatbot that answers student questions using the instructor's own ELMS-Canvas course materials (including Panopto recordings) to support student learning and engagement.

The study results were recently released: Effects of AI Tutoring on Student Performance and Engagement: A Randomized University Trial. Researchers found that access to UMD’s course-integrated Generative Artificial Intelligence (GenAI) VSA was associated with lower final grades and substantially lower ELMS-Canvas participation, with larger estimated effects for first-generation students. These findings held even for students who had access to the tool, but did not use it. 

These findings do not mean that GenAI tutoring tools are inherently harmful; rather, they underscore that simply providing access to a GenAI tool is not, by itself, an instructional intervention. Effective use depends on intentional design, clear expectations, and ongoing monitoring of how the tool is integrated into teaching and learning practices.

Instructors who are embedding the UMD VSA tool within their ELMS-Canvas course spaces should use it as a structured supplement to, not a replacement for, students’ direct engagement with course materials, problem-solving, office hours, or meetings with instructional staff or peers. Get expanded faculty guidance on use of the VSA. Detailed documentation is also available to support integration of the tool into course spaces:

• Faculty Guide
• Virtual Study Assistant: Admin Console Basics
• UMD Virtual Agent: Admin Console Intermediate Guide
• Student Guide

As we continue to use an evidence-based approach to enhancing instruction on our campus, it’s important that GenAI-supported learning is purposeful, intentional, and grounded in effective teaching practices. How a tool is designed, configured, introduced, and embedded in course activities, and the active learning behaviors that students’ use of these tools may potentially displace, are likely just as important to consider as the integration of the tool itself.