SBIR-STTR Award

Through a prior IES-funded SBIR project in 2020, the developer created a prototype of an augmented reality application with audio feedback as blind and visually impaired students engage with tactile molecular models. In this Phase I project, the team will develop a new prototype of an augmented reality system that provides audio generated information on scientific images and diagrams used for teaching and learning chemistry. The prototype will include tactile physical manipulatives and software to provide students a more comparable experience of the visualization to their sighted peers. At the end of Phase I, in a pilot study with three special education chemistry educators and 10 high school students who are blind or visual impaired, the researchers will examine whether the prototype functions as planned and is engaging for users, if the prototype shows promise for supporting student learning of chemistry concepts, and whether educators believe that the full product would be feasible to integrate into classroom practice.

Purpose: This project team will fully develop a product to support students who are blind or visually impaired as they are learning chemistry. The Kasi system provides audio generated information when scientific images and diagrams are presented during instruction. Diagrams and images are essential for practicing scientists and chemistry, in particular, is a visual science. In fact, visual model comprehension has been shown to predict exam scores in introductory chemistry courses. However, for students who are blind or visually impaired, many science materials are not accessible. Project Activities: In the prior 2021 Phase I project, the team developed a prototype of an augmented reality system that provides audio generated information on scientific images and diagrams used for teaching and learning chemistry. The prototype, called Kasi, also included tactile physical manipulatives and software to provide students a more comparable experience of the visualization to that of their sighted peers. At the end of Phase I, a pilot study was conducted with three special education teachers and 10 high school students who are blind or visual impaired. Results demonstrated that the prototype functioned as planned, all students reported that they were engaged and learned chemistry concepts, and educators believed that the full product would be feasible to integrate into classroom practice. To fully develop Kasi in Phase II, the team will add chemistry content, optimize the calibration system to accommodate differing camera angles and lighting conditions, and finalize the digital interactive pedagogy, tactile pieces, and algorithms that generate audio feedback. After development concludes, researchers will conduct a pilot study to assess the feasibility and usability, fidelity of implementation, and promise of the product to support high school students who are blind or visual impaired in learning chemistry. The project team will collect data from a new group of 10 high school students who are blind or visually impaired to examine whether they can use the product independently and their chemistry content knowledge is improved. Chemistry knowledge will be measured by items from the American Chemical Society (ACS) High School exam and state assessments. Researchers will also conduct a cost analysis using the CostOut tool to determine the cost per student needed to implement and use Kasi. Product: The project team will fully develop Kasi, an augmented reality product that provides audio generated information on scientific images and diagrams used for teaching and learning chemistry. The product will include digital interactives with tactile pieces and computer vision algorithms to produce a multisensory augmented reality method for students who are blind or visually impaired to use as they learn chemistry diagrams and content. A suite of five digital interactives and their corresponding tactile pieces will be developed and combined with a pedagogical system of audio-based augmented reality learning prompts to teach concepts throughout the secondary chemistry curriculum.