Search

EP-3779805-B1 - PROCESSING METHOD AND PROCESSING DEVICE USING SAME

EP3779805B1EP 3779805 B1EP3779805 B1EP 3779805B1EP-3779805-B1

Inventors

  • KAMADA, Shohei
  • HORII, Toshihide

Dates

Publication Date
20260506
Application Date
20190325

Claims (4)

  1. A processing device (100) for counting a number of a plurality of patterns (20) included in an image (10) for each type of pattern (20), comprising: an input unit (120) that receives an input of an image (10), the image including a plurality of patterns (20), wherein types of the plurality of patterns (20) having similar shapes are aggregated in families, wherein a number of the families is smaller than a number of the types of the patterns (20); a first processing unit (122) that detects each of the plurality of patterns (20) included in the image (10) as one of the families and detects a position of the family, by subjecting the image (10) input to the input unit (120) to a process in a neural network, the neural network having been trained with images corresponding to the families as training data; an extraction unit (124) that extracts a plurality of parts, that include the patterns (20) corresponding to the families, from the image (10), based on the positions of the families detected by the first processing unit (122); a second processing unit (126) that subjects the plurality of parts extracted by the extraction unit (124) to the neural network process and that acquires intermediate data (148) from an intermediate layer that precedes an output of the second processing unit (126) by one or two steps; a clustering unit (128) that subjects the intermediate data (148) acquired by the second processing unit (126) to clustering in accordance with the number of the types of the pattern (20) and generates clusters (150); and a calculation unit (130) that calculates a number of the patterns (20) included in each of the clusters (150) that results from clustering in the clustering unit (128).
  2. The processing device (100) according to claim 1, wherein the second processing unit (126) uses the neural network used in the first processing unit (122).
  3. The processing device (100) according to claim 2, wherein the neural network used in the first processing unit (122) and the second processing unit (126) includes a convolutional layer (142) and a pooling layer (144) and is a convolutional neural network in which a fully connected layer is excluded, and a filter in the convolutional layer (142) in the convolutional neural network is trained to learn a processing result having a 1×1 spatial dimension.
  4. A processing method for counting a number of a plurality of patterns (20) included in an image (10) for each type of pattern (20), comprising: receiving an input of an image (10), the image including a plurality of patterns (20), wherein types of the plurality of patterns (20) having similar shapes are aggregated in families, wherein a number of the families is smaller than a number of the types of the patterns (20); detecting each of the plurality of patterns (20) included in the image (10) as one of the families and detecting a position of the family, by performing a processing of a neural network on the image (10) input to the process, the neural network having been trained with images corresponding to the families as training data; extracting a plurality of parts, that include the patterns (20) corresponding to the families, from the image (10), based on the positions of the families detected; performing a processing of the neural network on the plurality of parts extracted and acquiring intermediate data (148) from an intermediate layer that precedes an output of the second processing unit (126) by one or two steps; subjecting the intermediate data (148) acquired to clustering in accordance with the number of the types of the pattern (20) and generates clusters (150); and calculating a number of the patterns (20) included in each of the clusters (150) that results from clustering.

Description

[TECHNICAL FIELD] The present disclosure relates to a processing technology and, more particularly, to a processing method for processing an image and a processing device that uses the processing method. [BACKGROUND ART] In recent years, construction drawings are easily created by using a computer-aided design (CAD) system. However, data for construction drawings created in different systems are not compatible with each other, and it may not be possible to use the data when a period elapses or the contractor is changed. This will require quite a lot of effort to extract electric symbols from construction drawings already created. To extract electric symbols drawn on a construction drawing, a process is performed to create outline data for an image based on run data of image data produced by reading an image of the construction drawing and to define outline data, for which the degree of circularity of the outline loop for the outline data is equal to or greater than a threshold value, as an electric symbol (see, for example, patent literature 1). [Patent Literature 1] JP10-111937 LUOTING FU ET AL. "From engineering diagrams to engineering models: Visual recognition and applications", (COMPUTER-AIDED DESIGN 43 (2011)278-292) discloses a computational recognition approach to convert network-like, image-based engineering diagrams into engineering models. [SUMMARY OF INVENTION] [TECHNICAL PROBLEM] A neural network can be used to extract a pattern such as that of electric symbols from an image such as a construction drawing. Extraction of patterns makes it possible to count a number of a plurality of patterns included in the image for each type of pattern. Because there are a large number of types of pattern, however, it is difficult to cause a neural network to learn all patterns. Meanwhile, insufficient learning reduces the accuracy of the process. The present invention addresses the above-described issue, and an illustrative purpose thereof is to provide a technology for inhibiting reduction in the accuracy of the process, while inhibiting an increase in the volume of work required for learning at the same time. [SOLUTION TO PROBLEM] A processing device according to the present invention is provided by claim 1. A processing method according to the present invention is provided by claim 4. Optional combinations of the aforementioned constituting elements, and implementations of the present disclosure in the form of devices, systems, computer programs, recording mediums recording computer programs, etc. may also be practiced as additional modes of the present invention. [ADVANTAGEOUS EFFECTS OF INVENTION] According to the present disclosure, it is possible to inhibit reduction in the accuracy of the process, while inhibiting an increase in the volume of work required for learning at the same time. [BRIEF DESCRIPTION OF DRAWINGS] Fig. 1 shows an image subject to the process of the embodiment;Fig. 2 shows patterns and families that could be included in the image of Fig. 1;Figs. 3A-3B shows a configuration of the processing device according to the embodiment;Fig. 4 shows an outline of a process in the processing unit of Figs. 3A-3B;Fig. 5 shows a data structure of training data input to the first input unit of Fig. 3A;Figs. 6A-6B show results of the process in the processing device of Fig. 3B;Fig. 7 shows an outline of the process in the clustering unit of Fig. 3B; andFig. 8 is a flowchart showing a sequence of steps performed in the processing device of Fig. 3B. [DESCRIPTION OF EMBODIMENTS] A summary will be given before describing the embodiment of the present disclosure in specific details. The embodiment relates to a processing device that refers to a construction drawing showing a plurality of electric symbols and counts the number for each type of electric symbol. It is possible to use a neural network to improve the precision of extraction of electric symbols from a construction drawing. However, there are as many as several tens of types of electric symbol, and there are also electric symbols having similar shapes. For this reason, it is difficult to cause a neural network to learn all electric symbols, and a false determination may be obtained when a new electric symbol occurs. A false determination may be obtained due also to insufficient learning. To address this, this embodiment takes advantage of the fact that a plurality of patterns are organized into groups fewer than the number of types of electric symbol, by aggregating electric symbols having similar shapes into a group. Hereinafter, electric symbols are referred to as "patterns", a group is referred to as a "family", and a construction drawing is referred to as an "image". For example, patterns are organized into three families including intercom family, differential smoke detector family, and photoelectric smoke detector family. In this condition, the neural network in the processing device is configured to learn these families. When an image is