Recent headlines predict that generative AI will revolutionize education by replacing human teachers with superior "artificial" pedagogues. But what if these AI teachers hurt their students? In this presentation, Dr. Kevin Salfen discusses a series of experiments conducted with several Large Language Models, where the task was to teach a specific passage of piano music. The findings confirm generative AI's current limitations, as of February 2025, in reading and analyzing sheet music. These limitations, coupled with an inability to integrate music analysis with the physical aspects of piano playing, frequently result in recommendations that would cause harm, underscoring the urgent need for developers of generative AI to establish standards for the prevention of injury. These standards could be adapted from principles as familiar as Asimov's Three Laws of Robotics, envisioned for a science fiction world that, in the age of autonomous drone warfare, increasingly mirrors our reality. Kevin Salfen is a composer, scholar, filmmaker, and producer compelled by intersections of art, science, and social justice. His music has been performed in Asia, Europe, and throughout the U.S. He is a member of international performing ensemble Theatre Nohgaku, and two of his projects with that ensemble have received awards from the National Endowment for the Arts. His music has been released on Centaur and Broadway Bird Records. He curates and writes for the blog "Sound Trove." This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at https://www.ted.com/tedx