Search

KR-102963144-B1 - Sparse data based convolution calculate method and apparatus using artificial neural network

KR102963144B1KR 102963144 B1KR102963144 B1KR 102963144B1KR-102963144-B1

Abstract

A convolution operation method based on sparse data using an artificial neural network according to one embodiment may include: an input data collection step of collecting information on valid data related to a row of output data that is output by performing a convolution operation on input data, divided by row of the output data; an extended row information generation step of generating extended row information and input index information for the valid data based on column information where the valid data is located within the range of input data corresponding to the movement path of the kernel; an operation rule generation step of generating location information of the output data based on the extended row information and generating a convolution operation rule based on the input index information, the extended row information, and the location information; and a convolution operation step of performing a convolution operation based on the operation rule.

Inventors

  • 최정욱
  • 이민재

Assignees

  • 한양대학교 산학협력단

Dates

Publication Date
20260511
Application Date
20221222

Claims (15)

  1. In a method for performing convolution operations using a processor and memory, An input data collection step in which information regarding valid data related to rows of output data, which is output by performing a convolution operation on input data by a plurality of input data collection modules, is collected by dividing it by row of said output data; Extended row information generation step, which generates extended row information and input index information for the valid data based on column information where the valid data is located within the range of input data corresponding to the kernel's movement path: A step for generating a convolution rule, which generates location information of output data based on the above-mentioned expanded row information and generates a convolution operation rule based on the above-mentioned input index information, the above-mentioned expanded row information and the above-mentioned location information; and It includes a convolution operation step that performs a convolution operation based on the above operation rules; and The above input data collection step is, A step of collecting input data for duplicate rows by utilizing already collected data, taking into account the location information of the row for which input data is to be collected and the row for which input data has already been collected; and The method comprises the step of sequentially collecting and storing input data for the above duplicate rows through a pipeline in which the plurality of input data collection modules are serially connected; Sparse data-based convolution operation method using artificial neural networks.
  2. delete
  3. delete
  4. In paragraph 1, It includes an index information extraction step performed prior to the above input data collection step, and The above index information extraction step is, A step comprising extracting index information, which is location information for valid data where data exists and invalid data where data does not exist within input data. Sparse data-based convolution operation method using artificial neural networks.
  5. In paragraph 4 The above index information extraction step is, Extracting index information using CSR FORMAT information, Sparse data-based convolution operation method using artificial neural networks.
  6. In paragraph 1, The above step of generating extended row information is, The method includes the step of generating extended row information sequentially at corresponding column positions, starting from the valid data located in the smallest column among the valid data existing within the range of input data corresponding to the movement path of the kernel. Sparse data-based convolution operation method using artificial neural networks.
  7. In paragraph 6, The above step of generating extended row information is, The method includes the step of collecting index information for valid data located in the smallest column among valid data existing within the range of input data corresponding to the movement path of the kernel, divided by row. Sparse data-based convolution operation method using artificial neural networks.
  8. In paragraph 1, The above operation rule generation step is, A step of generating output index information that generates corresponding reference output index information based on input index information included in the above-mentioned extended row information, Sparse data-based convolution operation method using artificial neural networks.
  9. In Paragraph 8, The above output index information generation step is, A step including generating output index information by expanding it left and right based on the kernel size, Sparse data-based convolution operation method using artificial neural networks.
  10. In Article 9, The above kernel includes a matrix of size 3x3, 4x4, or 5x5, Sparse data-based convolution operation method using artificial neural networks.
  11. An index information extraction module that extracts index information, which is location information for valid data that exists and invalid data that does not exist within the input data; An input data collection module that collects information regarding valid data related to rows of output data, which is produced by performing a convolution operation on the input data, divided by row of the output data; A row information generation module that generates extended row information and input index information for valid data based on column information where the valid data is located within the range of input data corresponding to the kernel's movement path: and A convolution operation module that generates location information of output data based on the above-mentioned extended row information, generates a convolution operation rule based on the above-mentioned input index information, the above-mentioned extended row information and the above-mentioned location information, and performs a convolution operation based on the generated operation rule; The above input data collection module includes a plurality of input data collection modules, and Considering the location information between the row for which input data is to be collected and the row for which input data has already been collected, input data for duplicate rows is collected by utilizing the already collected data, and Input data for the above duplicate rows is sequentially collected and stored through a pipeline in which the plurality of input data collection modules are serially connected, Sparse data-based convolution operation device using artificial neural networks.
  12. delete
  13. delete
  14. An index information extraction module that extracts index information, which is location information for valid data that exists and invalid data that does not exist within the input data; An input data collection module that collects information regarding valid data related to rows of output data, which is produced by performing a convolution operation on the input data, divided by row of the output data; A row information generation module that generates extended row information and input index information for valid data based on column information where the valid data is located within the range of input data corresponding to the kernel's movement path: and A convolution operation module that generates location information of output data based on the above-mentioned extended row information, generates a convolution operation rule based on the above-mentioned input index information, the above-mentioned extended row information and the above-mentioned location information, and performs a convolution operation based on the generated operation rule; The above kernel corresponds to a 3x3 kernel, and The above row information generation module is, A zero-row collected data storage module, a first-row collected data storage module, and a second-row collected data storage module are connected in series, Sparse data-based convolution operation device using artificial neural networks.
  15. In Paragraph 14, The above-mentioned zero-row collected data storage module, the above-mentioned first-row collected data storage module, and the above-mentioned second-row collected data storage module are, Connected serially through a pipeline, capable of sequentially transmitting, receiving, and storing data, Sparse data-based convolution operation device using artificial neural networks.

Description

Sparse data based convolution calculate method and apparatus using artificial neural network The present invention relates to a method and apparatus for convolution operations based on sparse data using an artificial neural network, and more specifically, to a technology for performing convolution operations using an artificial neural network, wherein the relationship between input data and output data is analyzed using kernel characteristics, and the convolution operation is performed more quickly using a rule generated based on the analysis result. Artificial Intelligence (AI) technology refers to technology that realizes human learning ability, reasoning ability, perception ability, and natural language understanding ability through computer programs, and unlike conventional rule-based smart systems, it refers to a system in which machines learn, make judgments, and become smarter on their own. Artificial intelligence technology consists of machine learning (deep learning) and component technologies utilizing machine learning. Machine learning is an algorithmic technology that classifies and learns the characteristics of input data on its own, and component technologies are technologies that mimic the functions of the human brain, such as cognition and judgment, by utilizing machine learning algorithms such as deep learning; these technologies are composed of fields such as linguistic understanding, visual understanding, reasoning/prediction, knowledge representation, and motion control. Artificial intelligence technology is applied and utilized in various fields, including the field of language understanding, which recognizes, applies, and processes human language and text; the field of visual understanding, which perceives objects like human vision to handle object tracking, person recognition, spatial understanding, and scene understanding; and the field of reasoning and prediction, which evaluates information to logically infer and predict. With the advancement of artificial intelligence technology, AI is being applied in the field of autonomous driving to recognize objects surrounding moving vehicles. Specifically, object recognition methods based on Lidar/RGB-D sensors are predominant; these methods utilize point cloud data to distinguish the location and type of objects around the vehicle, and extract features capable of classifying objects from the point cloud data by repeatedly performing convolution operations across multiple layers. However, due to the nature of object recognition methods, point cloud data exists sparsely in space, so convolution operations are also performed on sparsely distributed data. Consequently, features extracted per layer are stored in memory irregularly, which necessitates irregular access to retrieve the feature data used for convolution operations from memory. Therefore, when performing convolution operations according to conventional techniques, there is a problem in that the time required to complete the entire process increases significantly. A brief description of each drawing is provided to help to better understand the drawings cited in the detailed description of the invention. FIG. 1 is a diagram illustrating the process of performing a sparse data-based convolution operation according to the prior art. FIG. 2 is a block diagram illustrating some components of a sparse data-based convolution operation device using an artificial neural network according to an embodiment of the present invention. FIG. 3 is a diagram illustrating an example of input data and an example of a kernel as an embodiment of the present invention. FIGS. 4 to 7 are drawings illustrating the process of an input data collection module generating row information for input data according to an embodiment of the present invention. FIGS. 8 and 9 are drawings for explaining a method in which a row information generation module according to one embodiment generates zero-row extended row information corresponding to zero row of output data. FIGS. 10 to 13 are drawings for explaining a method in which a row information generation module according to one embodiment generates first row extended row information corresponding to a first row of output data. FIGS. 14 to 16 are drawings for explaining a method in which a row information generation module according to one embodiment generates second row extended row information corresponding to a second row of output data. FIGS. 17 and 18 are diagrams illustrating a method in which a convolution operation module according to one embodiment generates an index rule related to the output information of the 0th row of output data for each input index. FIGS. 19 to 22 are diagrams illustrating a method in which a convolution operation module according to one embodiment generates an index rule related to the output information of a first row of output data for each input index. FIGS. 23 to 25 are drawings for explaining the process of a convolution operation m