Applying Computer Vision to Track Tool Movement in an Automotive Assembly Plant
Emily Turner, Landon Newberry, Sheridan Santinga, Jeff Gray, Sandeep Gopu, Jeffery Peoples, Jon Hobbs, Sanford White · 2019
In the context of automobile assembly, torque tools are used by technicians to mount bolts to parts of a vehicle. Different bolts require varying torque levels to be fastened correctly. An intelligent tool can be programmed to deliver torque levels in a specified order that must be strictly followed by the technician as they secure the bolts. Because of the pre-specified order, it is imperative that the technician follows the correct order precisely. This paper describes our investigation into the use of computer vision to identify, correct, and document the human error involved in the bolt securing process (i.e., incorrect ordering) on an automotive factory assembly line. We built a computer vision application to select a desired order of bolts, detect visitation of a torque tool to each bolt location, and report errors made in the sequence of actions in a vehicle assembly line. The application is used to determine the accuracy of error detection for varying degrees of distance between bolt locations and sizes in order to determine which assembly line stations are appropriate for this type of monitoring. The context for application of our project is a large automotive manufacturing facility in the Southeastern United States -- the Mercedes-Benz US International (MBUSI) factory in Vance, Alabama. The limitations of computer vision for determining errors in assembly order using the torque tool are reported with a discussion of lessons learned.