Purpose: In this project, the team will fully develop and test TaylorAI, an artificial intelligence (AI) formative feedback and assessment system for hands-on science investigations. Research demonstrates that STEM interventions that provide formative results back to students can build competence as they engage in laboratory activities. However, educators and students often have difficulty measuring and applying data from inquiry activities to further ideas, skills, and knowledge of STEM. Project Activities: During Phase I in 2020 the team developed a prototype of TaylorAI, including an AI algorithm and training data set, and a physical science activity for generating, analyzing, and interpreting graphs of motion to investigate relationships between variables, slope, and velocity. Researchers conducted a pilot study with six middle school science teachers and 120 students to assess the usability, feasibility, and promise of the prototype to offer feedback on the experiments to support student learning. Results demonstrated that the prototype AI mechanism generated results that students and educators were able to view, and student self-reported that the prototype increased their ability to correctly analyze graphs generated from their own experiments. Teachers reported that students were engaged by and used the formative results generated by the prototype. In Phase II of the project, the team will fully develop and validate the AI mechanism, a teacher dashboard, and lesson content for eight physical science activities. After development concludes, researchers will conduct a pilot study to assess the feasibility and usability, fidelity of implementation, and the promise of TaylorAI for improving student learning. The team will collect data from 30 middle school classes, with half randomly assigned to use TaylorAI and the other half to use business as usual laboratory activities to teach the same course content. Researchers will compare pre-and-post scores for student learning of NGSS-based outcomes, including analyzing and interpreting data and constructing explanations and designing solutions. Researchers will gather cost information using the "ingredients method" and will include all expenditures on things such as personnel, facilities, equipment, materials, and training. Product: TaylorAI will be an AI-formative feedback and assessment system for hands-on laboratory science activities that provides information and scaffolded hints to students as they collect and analyze experimental data. TaylorAI will operate in the background of an existing cloud-based software platform, called PocketLab, which includes a small wireless device that transmits scientific data using Bluetooth, and a user-interface notebook for students to keep track of their work. TaylorAI will employ an AI algorithm that analyzes real-world experimental data that students collect using PocketLab. TaylorAI will be used by students in and out of classrooms to provide feedback and assist in the assessment of key science and engineering practices from the Next Generation Science Standards.
