Multimodal speech and audio user interfaces for K-12 outreach
Mark Hasegawa‐Johnson, Camille Goudeseune, Jennifer Cole, Hank Kaczmarski, Heejin Kim, Sarah King, Tim Mahrt, Jui Ting Huang, Xiaodan Zhuang, Kai Lin, Harsh Vardhan Sharma, Zhen Li, Thomas S. Huang · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference · 2011
Elementary school children have short attention spans. This paper describes three multimodal speech and audio user interfaces that captured and held the attention of a few dozen elementary-school and high-school children during the course of a two-day university open house. The Speech Recognition Game demonstrated an isolated word recognizer with a rapidly-won game, in which children were challenged to get ten words in a row correctly recognized. The Audio Easter Egg Hunt demonstrated our timeliner multimedia analytics platform with a faster-than-real-time search through orchestral music for audio anomalies (cuckoo clocks, motorcycles, etc). Finally, at the Intonation Station, children had to pick the pitch contour that would help a friendly troll to successfully hunt dragons in the city of Champaign. Results suggest that competition, collaboration, and other forms of social interaction may motivate children more than prizes.