MSE Analysis of 24-Bit Fixed-Point Format in a Wide Variance Range
Bojan Denić, Zoran Perić, Milan R. Dincic · 2023
In this paper, we consider a fixed-point format which is one of the commonly used binary format for data representation. In particular, we focus on the 24-bit fixed-point format (FXP24) and the analysis provided here can be observed as our further research in this area. Using the relationship that exists between the structure of the fixed-point format and uniform scalar quantization, we actually use SQNR (signal-to-quantization noise ratio) to measure the performance of mentioned binary format. Assuming data modelled with a Laplacian distribution, we theoretically investigate the performance of the FXP24 format across the wide dynamic range of variances under the constraint that the number of bits in the integer part$\boldsymbol{n}$changes. We have shown that with an adequate selection of$\boldsymbol{n}$, it is possible to find the range of variances in which FXP24 provides better SQNR (that is, a higher accuracy) than its floating-point counterpart FP24.