How to Make Your AI Fireproof
Andrea Schiel · 2024
Most of the time we know when our AI is not doing well, and we may be quite adept at patching – either in a frantic scramble before shipping or as an ongoing live support practice. Sometimes we do not even remember how we got into the state we are in. After reviewing many different AI implementations, a pattern started to emerge. The same mistakes and decisions were being made repeatedly – by some very experienced teams. At the heart of much of this was some very natural cognitive bias that can make any of us blind to how poorly our AI is doing and why we stick with an AI that is not doing well. The first step is knowing that there are systemic issues and identifying which are applicable to your AI. There are some best practices on how to avoid these problems in the first place, but often the second step is how to course correct an existing AI. This chapter will cover some of the more common issues (or fires), how to identify which apply, and then cover how to avoid them in the first place or what to do if they are occurring in an existing AI. Whether you are a senior AI lead or just starting out, this chapter should help you to identify some best practices.