Say the Right Thing Right: Ethics Issues in Natural Language Generation Systems
Charese Smiley, Frank Schilder, Vassilis Plachouras, Jochen L. Leidner · 2017
We discuss the ethical implications of Natural Language Generation systems.We use one particular system as a case study to identify and classify issues, and we provide an ethics checklist, in the hope that future system designers may benefit from conducting their own ethics reviews based on our checklist. QUESTION EXAMPLE RESPONSE SECTION Human consequencesAre there ethical objections to building the application?No objections anticipated 4.3 How could a user be disadvantaged by the system?No anticipated disadvantages to user 4.4-4.7 Does the system use any Personally Identifiable Information?No PII collected or used 4.5 Data issues How accurate is the underlying data?*Data is drawn from trusted source 4 Are there any misleading rankings given?Yes, detected via data validation 4.1 Are there (automatic) checks for missing data?Yes, detected via data validation 4.2 Does the data contain any outliers?Yes, detected via data validation 4.2 Generation issues Can you defend how the story is written?*Yes via presupposition checks and disclosure 5 Does the style of the automated report match your style?*Yes, generations reviewed by domain experts 5 Who is watching the machines?*Conducted internal evaluation and quality control 5 Provenance Will you disclose your methods?*Disclosure text 4.4