Implementation of JPEG XS entropy encoding and decoding on FPGA

Journal of Real-Time Image Processing (2024) 21:34 https://doi.org/10.1007/s11554-023-01410-8 RESEARCH Implementation of JPEG XS entropy encoding and decoding on FPGA Shuang Tian1 · Qinghua Song1 · Jialin He1 · Yihan Wang1 · Kai Nie1 · Gang Du1 · Ling Bu1 Received: 1 September 2023 / Accepted: 28 December 2023 / Published online: 19 February 2024 © The Author(s) 2024 Abstract JPEG XS is the latest international standard for shallow compression fields launched by the International Organization for Standardization (ISO). The coding standard was officially released in 2019. The JPEG XS standard can be encoded and decoded on different devices, but there is no research on the implementation of JPEG XS entropy codec on FPGAs. This paper briefly introduces JPEG XS encoding, proposes a modular design scheme of encoder and decoder on FPGA for the entropy encoding and decoding part, and parallelizes the algorithm in JPEG XS coding standard according to the characteristics of FPGA parallelization processing, mainly including low-latency optimization design, storage space optimization design. The optimized scheme in this paper scheme enables encoding speeds of up to 4 coefficients/clock and decoding speeds of up to 2 coefficients/clock, with a 75% reduction in encoding and decoding time. The maximum clock frequency of the entropy encoder is about 222.6 MHz, and the maximum clock frequency of the entropy decoder is about 127 MHz. The design and implementation of the FPGA-based JPEG XS entropy encoding and decoding algorithm is of great significance and provides ideas for the subsequent implementation and optimization of the entire JPEG XS standard on FPGAs. This work is the first in the world to propose the design and implementation of an algorithm that can implement the JPEG XS entropy encoding and decoding process on FPGA. It creates the possibility for the effective application of JPEG XS standard in more media. Keywords DWT · Entropy codec · FPGA · JPEG XS · Shallow compression 1 Introduction In 2016, the Joint Photographic Experts Group (JPEG), jointly established by the International Organization for Standardization and the International Telegraph and * Gang Du * Ling Bu Shuang Tian Qinghua Song Jialin He Yihan Wang Kai Nie 1 School of Information Engineering, China University of Geosciences, Beijing 100083, China Telephone Advisory Committee (CCITT) under the International Telecommunication Union (ITU), launched a project for low-complexity, low-latency image compression codecs [1], the JPEG XS international standard, to support fields requiring low latency and high quality, such as autonomous driving, virtual reality (VR), and broadcast television. JPEG XS is the latest international standard in the shallow compression domain, called ISO/IEC 21122: JPEG XS lowlatency lightweight image coding system [2, 3], the coding standard was officially released in 2019. In 1991, the Joint Photographic Experts Group released the still image compression standard JPEG, which has been around for a long time. But is still widely used which is one of the most widely used digital image compression standards in the world. Moreover, the Joint Photographic Experts Group has been promoting the development of image compression standards for nearly 30 years. The group has continuously introduced JPEG-LS, JPEG 2000 [4], JPEG XR [5], and other better codec standards, but JPEG still dominates the market, which shows that the performance of JPEG can meet the demand compared to other more complex codec standards. Low complexity and easy implementation are Vol.:(0123456789) 34 Page 2 of 17 more important [6, 7]. With the development of technology, in some scenarios, in addition to low complexity, strict bandwidth, and latency requirements need to be met, but low-complexity codecs such as JPEG [8, 9] and JPEG-LS [10, 11] cannot ensure compression to a given target bitrate while keeping latency below a single frame. As a result, new standards need to be developed to meet low-complexity, low-latency coding at high bit rates. JPEG XS is formulated to meet the above requirements, with visually lossless quality, multi-generation robustness, multi-platform operability (CPU, GPU, ASIC, FPGA), low complexity, and low latency [12]. The JPEG XS standard was developed with the possibility of running on a variety of platforms, including FPGA platforms. There have been studies of decoding on GPU [13]. There has also been a lot of research on implementing JPEG2000 on FPGA [14, 15]. A. Legrand et al. also discussed the deployment of JPEG XS in lossless packet networks and proposed an unequal error protection scheme with optimal rate distortion [16]. However, there is currently no research into FPGA implementations of JPEG XS entropy codecs. FPGA is becoming particularly popular as hardware accelerators and is known for their programmability, reconfigurability, and massive parallelism through large numbers of configurable logic blocks (CLBS) [17]. In this paper, the JPEG XS algorithm is implemented on the FPGA, and its encoding and decoding process is optimized in parallel. Four encoding types can be completed when the data is read once, and the encoding speed can reach 4 coefficients/clock, reducing the encoding time by 75%. In the entropy decoding process, parallel decoding optimization is carried out for unary decoding, and the decoding speed reaches 8 bits per clock. Then, the memory structure of the decoding is optimized. The decoding of the magnitude with uncertain delay is fixed to 1 clock cycle and the 4 clock cycles of symbol decoding are reduced to 1 clock cycle, reducing the decoding time by 75%. Finally, by adjusting the decoding timing, the memory space multiplexing optimization is completed, which can be synchronously decoded, so that a large amount of intermediate data does not need to be stored, and part of the storage space can be multiplexed, and finally, the decoding speed reaches 2 coefficients/clock. 2 JPEG XS standard The encoding and decoding process of JPEG XS is shown in Fig. 1, The main steps of the encoder are image preprocessing, 5/3 DWT(Discrete Wavelet Transform) [18], quantization, entropy coding, and output code stream. The main steps of the decoder are code stream decomposition, entropy Journal of Real-Time Image Processing (2024) 21:34 Fig. 1  The figure shows the JPEG XS encoding and decoding step. The image is converted into a code stream after processing and restored to an image after decoding Fig. 2  The image structure after DWT, 3840 × 2160 image was used in the experiment, 1 vertical and 5 horizontal decomposition level DWT) decoding, inverse quantization, discrete inverse wavelet transformation, and output image. 2.1 Image preprocess To obtain more accurate intermediate results in subsequent transformations, after receiving the input image, the bit precision is first extended and the bit depth of the image data is expanded to 20 bits. The subsequent oper (...truncated)