Continual Inference: A Library for Efficient Online Inference with Deep Neural Networks in PyTorch
Lukas Hedegaard, Alexandros Iosifidis · Lecture notes in computer science · 2023
We present Continual Inference, a Python library for implementing Continual Inference Networks (CINs), a class of Neural Networks designed for redundancy-free online inference. This paper offers a comprehensive introduction and guide to CINs and their implementation, as well as best-practices and code examples for composing basic modules into complex neural network architectures that perform online inference with an order of magnitude less floating-point operations than their non-CIN counterparts. Continual Inference provides drop-in replacements of PyTorch modules and is readily downloadable via the Python Package Index and at www.github.com/lukashedegaard/continual-inference .