Resilient training of neural network classifiers with approximate computing techniques for a hardware-optimized implementation

• Autor(a) principal: Vitor Angelo Maria Ferreira Torres
• Data de Publicação: 2019
• Tipo de documento: Tese
• Idioma: eng
• Título da fonte: Repositório Institucional da UFMG
• Texto Completo: http://hdl.handle.net/1843/32399
• topic: Approximate computing; Artificial neural networks; Hardware implementation; Engenharia elétrica; Redes neurais (Computação); Computação aproximativa

As Machine Learning applications drastically increase their demand for optimized implementations, both in embedded environments and in high-end parallel processing platforms, the industry and research community have been responding with different approaches to provide the required solutions. This work presents approximations to arithmetic operations and mathematical functions that, associated with adaptive Artificial Neural Networks training methods and an automatic precision adjustment mechanism, provide reliable and efficient implementations of classifiers, without depending on mixed operations with higher precision or complex rounding methods that are commonly proposed only with highly redundant datasets and large networks. This research investigates Approximate Computing concepts that simplify the design of classifier training accelerators based on hardware with Application Specific Integrated Circuits or Field Programmable Gate Arrays (FPGAs). The goal was not to find the optimal simplifications for each problem but to build a method, based on currently available technology, that can be used as reliably as one implemented with precise operations and standard training algorithms. Reducing the number of bits in the Floating Point (FP) format from 32 to 16 has an immediate effect of dividing by half the memory requirements and is a commonly used technique. By not using mixed precision and performing further simplifications to the smaller format, this thesis reduces the implementation complexity of the FP software emulation by 53%. Exponentiation and division by square root operations are also simplified, without requiring Look-Up Tables and with implicit interpolation. A preliminary migration of the design to an FPGA has confirmed that the area optimizations are also relevant in this environment, even when compared to other optimized implementation which lack the mechanism to adapt the FP representation range. A logical resource reduction of 64% is achieved when compared to mixed-precision approaches.