Search

CN-115601228-B - Image processing method, system, electronic device and medium

CN115601228BCN 115601228 BCN115601228 BCN 115601228BCN-115601228-B

Abstract

The invention discloses an image processing method, an image processing system, electronic equipment and a medium, wherein the image processing method comprises the steps of dividing an image to be processed into a plurality of image blocks, carrying out shaping FFT (fast Fourier transform) on the image blocks to obtain image block frequency domain data, carrying out frequency domain processing on the image block frequency domain data to obtain image block frequency domain processed data, carrying out IFFT (inverse fast Fourier transform) on the image block frequency domain processed data to obtain an image block IFFT conversion result, and obtaining a result image according to the image block IFFT conversion result corresponding to each image block. The invention effectively improves the parallelism of operation and the processing speed, and effectively saves resources while ensuring the operation precision through reasonably setting the data bit width in the operation process.

Inventors

  • ZHANG XIAOMENG
  • LIU XIAOWEI
  • CHEN HUAN

Assignees

  • 展讯通信（上海）有限公司

Dates

Publication Date
20260512
Application Date
20200628

Claims (20)

  1. 1. An image processing method, characterized by comprising the steps of: Dividing an image to be processed into a plurality of image blocks; Shaping FFT conversion is carried out on the image block so as to obtain image block frequency domain data; Performing frequency domain processing on the image block frequency domain data to obtain image block frequency domain processed data; Performing IFFT on the image block frequency domain processed data to obtain an image block IFFT result; obtaining a result image according to the IFFT conversion result of the image block corresponding to each image block; the step of performing IFFT on the image block frequency domain processed data comprises: performing conjugate processing on the image block frequency domain processed data to obtain conjugate data; lifting the bit width of the numerical part of the conjugate data to a second target bit width to form expanded conjugate data; performing butterfly operation on the extended conjugated data; The second target bit width is n_in_ifft+n_iup, where n_iup=32-2-n_in_ifft, n_in_ifft=n_in+2×k, n_in represents the bit width of the numerical portion of the original data of the image block, and k represents the number of butterfly operation stages required by IFFT.
  2. 2. The image processing method according to claim 1, wherein the step of dividing the image to be processed into image blocks comprises: Dividing the image to be processed into a plurality of image units, wherein each image unit consists of adjacent pixels, and any two image units are not overlapped, each image unit comprises a first type image unit and a second type image unit, each first type image unit comprises N rows and N columns of pixels, N is an integer power of 2, and each second type image unit is the image unit except for the first type image unit; expanding the second type image unit into an expanded unit with N rows and N columns of pixels; and taking the first type image unit and the expanded unit as the image block.
  3. 3. The image processing method of claim 2, wherein said step of shaping FFT transforming said image block comprises: Constructing a first shaping array and a second shaping array, wherein the first shaping array comprises N first elements, each first element is used for storing a real part of input data, the second shaping array comprises N second elements, and each second element is used for storing an imaginary part of the input data; lifting the bit width of the numerical value part of the input data to a first target bit width to form expanded data; and performing butterfly operation based on the expansion data.
  4. 4. The image processing method according to claim 3, wherein N is 64 or less.
  5. 5. The image processing method of claim 3, wherein the first target bit width Wt = n_in_fft + n_up, wherein n_in_fft represents the bit width of the numeric portion of the input data, n_up = 32-1-n_in_fft-2 x k, k = log2 (N), k represents the number of butterfly stages required by the FFT.
  6. 6. The image processing method of claim 3, wherein the step of performing a butterfly operation based on the extension data comprises: and shaping the weight factors used for butterfly operation.
  7. 7. The image processing method of claim 3, wherein the step of performing a butterfly operation based on the extension data comprises: firstly performing butterfly operation in the row direction, and then performing butterfly operation in the column direction; Or, the butterfly operation is performed in the column direction and then in the row direction.
  8. 8. The image processing method according to claim 7, wherein butterfly operations corresponding to each row are performed in parallel during butterfly operations in a row direction, and butterfly operations corresponding to each column are performed in parallel during butterfly operations in a column direction.
  9. 9. The image processing method of claim 1, wherein the step of performing a shaping FFT on the image block to obtain image block frequency domain data comprises: shaping FFT conversion is carried out on each image block in parallel to obtain the image block frequency domain data corresponding to each image block; The step of performing IFFT on the image block frequency domain processed data to obtain an image block IFFT result comprises: and performing IFFT on the data after the frequency domain processing of each image block in parallel to obtain the IFFT result of the image block corresponding to each image block.
  10. 10. The image processing method of claim 1, wherein the step of performing a butterfly operation on the conjugate data comprises: the butterfly operation is performed in the row direction and then in the column direction.
  11. 11. The image processing method as claimed in claim 10, wherein the step of performing the butterfly operation in the row direction first includes: When the butterfly operation corresponding to one stage is completed, the result of the butterfly operation of the stage is shifted by one bit to the right until the butterfly operations of k stages are completed, or the butterfly operation corresponding to each stage is completed in turn until the butterfly operations of k stages are completed, and then the result obtained by the butterfly operation is shifted by k bits to the right; The step of performing butterfly operation in the column direction comprises the following steps: and (3) right shifting the result of the butterfly operation of each stage by one bit every time the butterfly operation corresponding to each stage is completed until the butterfly operation of each stage is completed, or sequentially completing the butterfly operation corresponding to each stage until the butterfly operation of each stage is completed, and then right shifting the result obtained by the butterfly operation by k bits.
  12. 12. The image processing method as claimed in claim 1, wherein the step of obtaining a result image from the IFFT transformation result of the image block corresponding to each of the image blocks comprises: Splicing the IFFT conversion results of the image blocks corresponding to the image blocks into an intermediate image; and carrying out smoothing filtering processing on the intermediate image to obtain the result image.
  13. 13. The image processing method of claim 3, wherein the step of performing a butterfly operation based on the extension data comprises: Performing butterfly operation on each row in parallel according to a row direction, performing butterfly operation on a plurality of target columns in parallel according to a column direction to obtain local frequency domain data, and obtaining the image block frequency domain data according to the conjugate symmetry of Fourier transform based on the local frequency domain data, wherein the plurality of target columns are [1, N/2+1] columns of the image block; Or, the step of performing butterfly operation based on the extension data includes: Performing butterfly operation on each column in parallel according to the column direction, performing butterfly operation on a plurality of target rows in parallel according to the row direction to obtain local frequency domain data, and obtaining the image block frequency domain data according to the conjugate symmetry of Fourier transformation based on the local frequency domain data, wherein the target rows are [1, N/2+1] rows of the image block.
  14. 14. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the image processing method of any of claims 1-13 when executing the computer program.
  15. 15. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the image processing method of any one of claims 1-13.
  16. 16. An image processing system is characterized by comprising a dividing unit, a shaping FFT conversion unit, a frequency domain processing unit, an IFFT conversion unit and a result image generating unit; The dividing unit is used for dividing the image to be processed into a plurality of image blocks; The shaping FFT conversion unit is used for carrying out shaping FFT conversion on the image block to obtain image block frequency domain data; The frequency domain processing unit is used for performing frequency domain processing on the image block frequency domain data to obtain image block frequency domain processed data; the IFFT conversion unit is used for performing IFFT conversion on the image block frequency domain processed data to obtain an image block IFFT conversion result; The result image generating unit is used for obtaining a result image according to the IFFT conversion result of the image block corresponding to each image block; the IFFT conversion unit is also used for carrying out conjugate processing on the data after the image block frequency domain processing so as to obtain conjugate data; the IFFT transformation unit is further configured to boost a bit width of the numerical portion of the conjugate data to a second target bit width to form extended conjugate data; The IFFT conversion unit is also used for performing butterfly operation on the extended conjugated data; The second target bit width is n_in_ifft+n_iup, where n_iup=32-2-n_in_ifft, n_in_ifft=n_in+2×k, n_in represents the bit width of the numerical portion of the original data of the image block, and k represents the number of butterfly operation stages required by IFFT.
  17. 17. The image processing system of claim 16, wherein the dividing unit is further configured to divide the image to be processed into a plurality of image units, the image units being composed of adjacent pixels, and any two of the image units not overlapping, the image units including a first type of image unit having N rows and N columns of pixels, N being an integer power of 2, and a second type of image unit being the image unit other than the first type of image unit; The dividing unit is further used for expanding the second type of image units into expanded units with N rows and N columns of pixels; the dividing unit is further configured to use the first type image unit and the expanded unit as the image block.
  18. 18. The image processing system of claim 17, wherein the shaping FFT transformation unit is further configured to construct a first shaping array and a second shaping array, the first shaping array comprising N first elements, each of the first elements being configured to store a real part of the input data, the second shaping array comprising N second elements, each of the second elements being configured to store an imaginary part of the input data; the shaping FFT unit is further configured to boost a bit width of a numerical portion of the input data to a first target bit width to form spread data; the shaping FFT unit is also used for performing butterfly operation based on the expansion data.
  19. 19. The image processing system of claim 18, wherein N is 64 or less.
  20. 20. The image processing system of claim 18, wherein the first target bit width Wt = n_in_fft + n_up, wherein n_in_fft represents the bit width of the numeric portion of the input data, n_up = 32-1-n_in_fft-2 x k, k = log2 (N), k represents the number of butterfly stages required by the FFT.

Description

Image processing method, system, electronic device and medium The invention relates to a method, a system, an electronic device and a medium for image processing, which are classified in China patent application with the application date of 2020, the application number of 2020105982401 and the invention name of 2020, 28 and 28. Technical Field The present invention relates to the field of image processing technologies, and in particular, to an image processing method, an image processing system, an electronic device, and a medium. Background The most common fast implementation method of fourier transform is the FFT (Fast Fourier Transform ) algorithm of Cooley and Tukey, and the basic idea of FFT is to decompose the original N-point sequence into two or more shorter sequences and recombine into DFT (discrete fourier transform) of the original sequence, so that the number of operation times of indirect algorithm is much less than that of direct computation DFT, thereby improving the computation speed of DFT. On the one hand, the decomposition operation of FFT/IFFT (inverse fast fourier transform) can be basically divided into two types, one type is to successively decompose the time series X (n) called time-decimation algorithm (Decimation In Time, DIT), and the other type is to decompose the fourier transform series X (k) called frequency-decimation algorithm (Decimation In Frequency, DIF). The FFT/IFFT algorithm can be divided into several types according to the number M of short sequences decomposed by the FFT, which is generally called the base-M algorithm. For example, the radix-2 FFT algorithm may decompose an N-point DFT operation into N/2 log2N 2-point DFT operations, and the radix-4 FFT algorithm may decompose an N-point DFT operation into N/4 log4N 4-point DFT operations. At present, a plurality of optimizations related to FFT arithmetic units exist, the optimization direction is mainly aimed at optimizing a hardware implementation structure, and the goal is to optimize the aspects of occupying hardware resources, power consumption aging and the like. Conventional FFT software operates at a slow speed compared to a hardware FFT processor. SRFFT (split-radix FFT) algorithm is a typical representation of FFT optimization, which reduces the number of multiplications and additions required for FFT computation further significantly. However, when a plurality of FFT operations are required to be performed, the serial processing time still cannot meet the needs of practical applications. The image processing algorithm developed for the product-level application is highly probable to be processed based on the spatial pixel domain and less involved in the frequency domain processing, which is greatly affected by time consumption and computational complexity. The reason is that the algorithm processing complexity of the frequency domain is not high, but the two processes of Fourier forward transformation and inverse transformation involved in the time-frequency space conversion are too time-consuming. To address this problem, many algorithms utilize characteristics of the time-frequency space to bypass frequency domain processing, e.g., to replace the dot product operation of the frequency domain with convolution deconvolution of the time domain. However, for frequency domain filters such as adaptive, this approach may be prone to failure because the equivalent spatial filter order is too large or complex to apply, so that the frequency domain processing cannot be reasonably replaced by the time domain processing. Another solution is to hardware the process of frequency domain processing to increase speed. However, the hardware is limited by hardware area, etc., and the development period is long, the flexibility is poor, and the method is not suitable for rapid development and update. The mobile terminal application, taking a mobile phone as an example, emphasizes real-time performance and requires less occupation of hardware resources. These conditions have largely limited the adoption of frequency domain processing techniques for images, mainly because the implementation of time-frequency space transformation based on software is too time-consuming to meet the time requirements. Disclosure of Invention The invention aims to overcome the defect of long time consumption of image processing in the prior art, and provides an image processing method, an image processing system, electronic equipment and a medium. The invention solves the technical problems by the following technical scheme: the invention provides an image processing method, which comprises the following steps: Dividing an image to be processed into a plurality of image blocks; shaping FFT conversion is carried out on the image block so as to obtain image block frequency domain data; Performing frequency domain processing on the image block frequency domain data to obtain image block frequency domain processed data; Performing IFFT on the data a