2017 pilot open speech analytic technologies evaluation (2017 NIST pilot OpenSAT):
Fred Byers, Omid Sadjadi · 2019
NIST conducted the 2017 Pilot Open Speech Analytic Technologies Evaluation (OpenSAT) Evaluation as the first in a new type of evaluation series.This new series is being designed to combine data domains and speech-analytic tasks including the public-safetycommunications domain.This report summarizes the set up and results of the Pilot.The report includes 1) anonymous, composite plots, 2) an analysis of systems output, and 3) the dataset challenges in the evaluation.The methods used in the Pilot OpenSAT Evaluation included speech-activity detection; keyword search; automatic, speech recognition; and, three data domains.Those domains were low-resource language, public-safety communications, and YouTube videos.For the speech-analytics task, participants in the evaluation could choose one or all of the methods and one or all of the data domains.