CN-121640250-B - Image recognition method, product, equipment and medium based on data multiplexing

CN121640250BCN 121640250 BCN121640250 BCN 121640250BCN-121640250-B

Abstract

The invention discloses an image recognition method, a product, equipment and a medium based on data multiplexing, which are applied to an image recognition circuit comprising a plurality of groups of recognition sub-circuits; the method comprises the steps of storing initial image data of an original image to be identified in a preset global buffer, controlling convolution windows of an image identification CNN to slide in the initial image data, enabling all identification sub-circuits to correspond to different convolution windows respectively and comprise a selector, a data register and a memristor array, enabling the selector to respectively obtain non-multiplexing and multiplexing target data from the preset global buffer and the data register corresponding to the last convolution window, storing the target data serving as data to be convolved of a current convolution window in the data register, enabling the memristor array to convolve the data to be convolved to obtain convolution results of the current convolution window, and generating feature images of the original image to be identified according to the convolution results of all the convolution windows to conduct image identification. And the transmission delay and the hardware power consumption in the image recognition process are reduced.

Inventors

YUAN XINXIN
LI ZHONGHUA
LI TUO
WANG CHANGHONG
XI XIN

Assignees

山东云海国创云计算装备产业创新中心有限公司

Dates

Publication Date: 20260508
Application Date: 20260205

Claims (15)

1. The image recognition method based on data multiplexing is characterized by being applied to an image recognition circuit, wherein the image recognition circuit comprises a plurality of groups of recognition sub-circuits arranged according to a two-dimensional grid, and the method comprises the following steps: Preprocessing an original image to be identified to generate initial image data in a two-dimensional matrix form, and storing the initial image data in a preset global cache; The convolution window of the target convolution neural network model is controlled to slide in the initial image data, wherein the target convolution neural network model is used for image recognition, each recognition sub-circuit corresponds to different convolution windows, each recognition sub-circuit comprises a selector, a data register and a memristor array, and the selector is connected with the preset global cache through a bus; Acquiring first target data of a current convolution window from the preset global cache and second target data of the current convolution window from a data register corresponding to a previous convolution window through a bus by using a selector corresponding to the current convolution window, wherein the first target data and the second target data are respectively non-multiplexing data and multiplexing data between the current convolution window and the previous convolution window; the first target data and the second target data are used as data to be convolved of a current convolution window and stored in a data register corresponding to the current convolution window, and the memristor array corresponding to the current convolution window is used for convolving the data to be convolved to obtain a convolution result of the current convolution window; Generating a feature map of the original image to be identified according to convolution results of all convolution windows, and carrying out image identification based on the feature map to obtain an image identification result; obtaining second target data of the current convolution window from a data register corresponding to the previous convolution window, wherein the second target data comprises: And adopting run-length coding to code and compress continuously repeated image data in second target data of a current convolution window in a data register corresponding to the previous convolution window, transmitting the compressed data through an internal data path, transmitting the compressed data to a selector corresponding to the current convolution window, and finishing data restoration through a quick decompression logic to obtain the second target data of the current convolution window.
2. The method for data multiplexing based image recognition according to claim 1, wherein the preprocessing of the original image to be recognized to generate the initial image data in the form of a two-dimensional matrix comprises: adjusting the image size of the original image to be identified based on a preset size requirement to obtain a first preprocessed image; and normalizing each pixel value in the first preprocessed image to generate initial image data in a two-dimensional matrix form based on the obtained normalized pixel values.
3. The data multiplexing-based image recognition method according to claim 1, further comprising: Storing convolution kernel weights of all layers in a target convolution neural network model in the memristor array in a conductivity mode; Correspondingly, the convolving the data to be convolved by using the memristor array corresponding to the current convolution window includes: loading the data to be convolved stored in the data register corresponding to the current convolution window to two ends of a memristor array corresponding to the current convolution window, and convolving the data to be convolved by utilizing the convolution kernel weight in the memristor array.
4. The method for identifying an image based on data multiplexing as defined in claim 3, wherein the performing convolution processing on the data to be convolved by using the memristor array corresponding to the current convolution window to obtain a convolution result of the current convolution window includes: and clamping and controlling a column line of the memristor array corresponding to the current convolution window on a preset reference level, generating current by utilizing the effect of the data to be convolved and the convolution kernel weight in the memristor array, collecting the current on the column line of the memristor array, and determining a convolution result of the current convolution window according to the magnitude of the collected current on the column line.
5. The method for data multiplexing based image recognition according to claim 4, wherein the determining a convolution result of a current convolution window according to the magnitude of the current collected on the column line comprises: Determining the current collected on the column line as a multiply-accumulate result, wherein the multiply-accumulate result is a current signal; and converting the multiply-accumulate result into a digital signal to obtain a convolution result of the current convolution window.
6. The method for identifying an image based on data multiplexing according to claim 1, wherein said generating a feature map of the original image to be identified from the convolution results of each convolution window comprises: determining the feature map size of the original image to be identified according to the size of the initial image data and the convolution kernel size of the target convolution neural network model; Establishing a mapping relation between the position of each convolution window and the characteristic image pixel coordinates of the original image to be identified based on the characteristic image size; and splicing convolution results of all convolution windows in a two-dimensional matrix form according to the mapping relation to obtain a feature map of the original image to be identified.
7. The data multiplexing-based image recognition method according to claim 1, wherein a convolution window of the control-target convolutional neural network model slides in the initial image data, comprising: setting the size and sliding step length of a convolution window of a target convolution neural network model; And controlling the convolution window to alternately reversely slide in the initial image data according to the horizontal direction.
8. The method for image recognition based on data multiplexing as recited in claim 7, wherein the number of data registers is in correspondence with a convolution kernel size of the target convolutional neural network model, wherein storing the first target data and the second target data as data to be convolved of a current convolution window in the data registers corresponding to the current convolution window, comprises: moving the first target data and the second target data to corresponding target positions in the current convolution window according to the sliding direction of the current convolution window relative to the previous convolution window so as to obtain data to be convolved of the current convolution window; and storing the data to be convolved of the current convolution window in a data register corresponding to the current convolution window.
9. The method for identifying an image based on data multiplexing as claimed in claim 8, wherein said moving the first target data and the second target data to the corresponding target positions in the current convolution window according to the sliding direction of the current convolution window with respect to the previous convolution window comprises: if the sliding direction of the current convolution window relative to the previous convolution window is a rightward horizontal sliding direction, shifting the second target data leftwards according to the sliding step length so as to move to the left array position in the current convolution window, and moving the first target data to the vacant position in the current convolution window; And if the sliding direction of the current convolution window relative to the previous convolution window is a left horizontal sliding direction, shifting the second target data to the right according to the sliding step length so as to move to the right array position in the current convolution window, and moving the first target data to the vacant position in the current convolution window.
10. The method for identifying an image based on data multiplexing as claimed in claim 8, wherein said moving the first target data and the second target data to the corresponding target positions in the current convolution window according to the sliding direction of the current convolution window with respect to the previous convolution window comprises: If the sliding direction of the current convolution window relative to the previous convolution window is a downward vertical sliding direction, the second target data is shifted upwards according to the sliding step length so as to move to the upper array position in the current convolution window, and the first target data is moved to the vacant position in the current convolution window; And if the sliding direction of the current convolution window relative to the previous convolution window is an upward vertical sliding direction, shifting the second target data downwards according to the sliding step length so as to move to the lower array position in the current convolution window, and moving the first target data to the vacant position in the current convolution window.
11. The method for identifying an image based on data multiplexing according to claim 1, wherein the obtaining, by using a selector corresponding to a current convolution window, first target data of the current convolution window from the preset global cache and obtaining second target data of the current convolution window from a data register corresponding to a previous convolution window through a bus includes: If the current convolution window is the first convolution window, acquiring first target data of the current convolution window from the preset global cache through a bus by using a selector corresponding to the current convolution window, and setting second target data of the current convolution window to be null; if the current convolution window is a non-first convolution window, acquiring first target data of the current convolution window from the preset global cache through a bus by using a selector corresponding to the current convolution window, determining position overlapping data between the current convolution window and a last convolution window as second target data of the current convolution window, and acquiring the second target data of the current convolution window from a data register corresponding to the last convolution window by using the selector corresponding to the current convolution window.
12. The image recognition method based on data multiplexing as claimed in claim 1, wherein the performing image recognition based on the feature map to obtain an image recognition result comprises: Screening effective feature areas in the feature images based on a preset threshold value, and eliminating edge invalid feature values in the feature images to obtain cut feature images; And inputting the cut feature map into a classifier in the target convolutional neural network model to determine the category of the original image to be identified, and outputting an image identification result.
13. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the data multiplexing based image recognition method of any one of claims 1 to 12.
14. An electronic device, comprising: A memory for storing a computer program; A processor for executing the computer program to implement the steps of the data multiplexing based image recognition method according to any one of claims 1 to 12.
15. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the steps of the data multiplexing based image recognition method according to any one of claims 1 to 12.

Description

Image recognition method, product, equipment and medium based on data multiplexing Technical Field The present invention relates to the field of image processing technologies, and in particular, to an image recognition method, product, device, and medium based on data multiplexing. Background In the field of image recognition, convolutional neural networks are widely used because of excellent feature extraction and recognition capability, in order to improve the operation efficiency, a special hardware circuit is often adopted to realize hardware acceleration of convolutional operation, a memristor array becomes an important carrier for realizing the convolutional operation hardware by virtue of the characteristic of integration of memory operation, an image recognition circuit is often constructed by combining a memristor array and a Complementary Metal Oxide Semiconductor (CMOS) process in the prior art, and the convolutional operation is completed on the preprocessed initial image data through a convolutional window so as to extract the image features, thereby realizing image recognition. When the image recognition circuit executes convolution operation, all window data needs to be loaded, so that the bus data transmission quantity is huge, the bus load is overhigh, and meanwhile, the data transmission delay and the hardware power consumption are increased due to frequent global cache access. It can be seen that how to reduce the transmission delay and hardware power consumption in the image recognition process is a problem that needs to be solved by those skilled in the art. Disclosure of Invention The embodiment of the invention aims to provide an image recognition method, device, equipment and medium based on data multiplexing, which reduce transmission delay and hardware power consumption in the image recognition process. The specific scheme is as follows: In a first aspect, the invention discloses an image recognition method based on data multiplexing, which is applied to an image recognition circuit, wherein the image recognition circuit comprises a plurality of groups of recognition sub-circuits arranged according to a two-dimensional grid, and the method comprises the following steps: Preprocessing an original image to be identified to generate initial image data in a two-dimensional matrix form, and storing the initial image data in a preset global cache; The convolution window of the target convolution neural network model is controlled to slide in the initial image data, wherein the target convolution neural network model is used for image recognition, each recognition sub-circuit corresponds to different convolution windows, each recognition sub-circuit comprises a selector, a data register and a memristor array, and the selector is connected with the preset global cache through a bus; Acquiring first target data of a current convolution window from the preset global cache and second target data of the current convolution window from a data register corresponding to a previous convolution window through a bus by using a selector corresponding to the current convolution window, wherein the first target data and the second target data are respectively non-multiplexing data and multiplexing data between the current convolution window and the previous convolution window; The first target data and the second target data are used as data to be convolved of a current convolution window and stored in a data register corresponding to the current convolution window, and the data to be convolved is convolved by using a memristor array corresponding to the current convolution window to obtain a convolution result of the current convolution window; And generating a feature map of the original image to be identified according to convolution results of all convolution windows, and carrying out image identification based on the feature map to obtain an image identification result. Optionally, the preprocessing the original image to be identified to generate initial image data in a two-dimensional matrix form includes: adjusting the image size of the original image to be identified based on a preset size requirement to obtain a first preprocessed image; and normalizing each pixel value in the first preprocessed image to generate initial image data in a two-dimensional matrix form based on the obtained normalized pixel values. Optionally, the image recognition method based on data multiplexing further includes: Storing convolution kernel weights of all layers in a target convolution neural network model in the memristor array in a conductivity mode; Correspondingly, the convolving the data to be convolved by using the memristor array corresponding to the current convolution window includes: loading the data to be convolved stored in the data register corresponding to the current convolution window to two ends of a memristor array corresponding to the current convolution window, and convolving the data to be co