Accelerator Multi-Threading
Nic McDonald, Subhasis Das, Milad Mohammadi · 2012
Special purpose accelerators are being used widely in most embedded and server processors to gain orders of magnitude gains in terms of efficiency as well as performance. Currently most accelerators are used such that a single application gets exclusive access to the accelerator at a time. This can cause the accelerator to become unfair to small jobs since they can be waiting for a longer job to complete. However granting exclusive access ensures that a single job operates at the maximum throughput possible since there are no overheads for context switching. In this work, we propose a hardware framework to enable an accelerator to process multiple jobs simultaneously and compare it to a software based multithreading approach. We show that this achieves much throughput similar to no threading while maintaining fairness levels as fine grained switching. It is able to do so by decreasing the switching overhead by 150x.