JP-7854839-B2 - Drawing recognition device and drawing recognition program

JP7854839B2JP 7854839 B2JP7854839 B2JP 7854839B2JP-7854839-B2

Inventors

鈴木利久
福地賢
宗野勇

Assignees

三菱電機株式会社
三菱電機エンジニアリング株式会社

Dates

Publication Date: 20260507
Application Date: 20220405

Claims (7)

A drawing acquisition unit that acquires drawing data, A first recognition unit recognizes information about symbols in the drawing data based on the drawing data acquired by the drawing acquisition unit and a first machine learning model that takes the drawing data as input and outputs recognition results regarding symbols on the drawing data. A second recognition unit recognizes information regarding attributes in the drawing data based on the drawing data acquired by the drawing acquisition unit and a second machine learning model that takes the drawing data as input and outputs recognition results regarding attributes on the drawing data. A third recognition unit recognizes information about connecting lines in the drawing data based on the drawing data acquired by the drawing acquisition unit and a third machine learning model that takes the drawing data as input and outputs recognition results regarding connecting lines that connect symbols on the drawing data. A fourth recognition unit recognizes information regarding the correspondence between attributes, symbols, and connecting lines in the drawing data, based on information regarding symbols recognized by the first recognition unit using the first machine learning model, information regarding attributes recognized by the second recognition unit using the second machine learning model, and information regarding connecting lines recognized by the third recognition unit using the third machine learning model, and information regarding attributes recognized by the second recognition unit as input, and a fourth machine learning model that outputs information regarding the correspondence between attributes and symbols in the drawing data, and information regarding the correspondence between attributes and connecting lines in the drawing data, A drawing recognition device equipped with the following features.
The fourth machine learning model described above is: The model is trained to output symbols and connecting lines associated with a given primary attribute, which is the attribute whose correspondence to symbols and connecting lines in the drawing shown in the drawing data is inferred, and peripheral attributes, which are attributes that exist in a specific range centered on the primary attribute in the drawing shown in the drawing data, as input to a column of attributes. The fourth recognition unit is, Based on the attribute information recognized by the second recognition unit, the primary attribute is identified, and a column of attributes generated based on the identified primary attribute and peripheral attributes which are attributes that exist in a specific range centered on the primary attribute in the drawing shown in the drawing data is input to the fourth machine learning model, thereby obtaining information on the symbol and connecting lines associated with the primary attribute. The drawing recognition device according to claim 1, characterized in that it recognizes information regarding the correspondence between attributes, symbols, and connecting lines in the drawing data by repeatedly generating a column of attributes and inputting the generated column of attributes into the fourth machine learning model, while changing the main attribute based on the attribute information recognized by the second recognition unit.
The fourth machine learning model described above is: This model is trained to output, in response to input of a numerical feature obtained by transforming the aforementioned attribute column, the column of attributes to be transformed into the numerical feature, and the symbols and connecting lines associated with the principal attributes included in the column of attributes to be transformed into a numerical feature that approximates the said numerical feature. The fourth recognition unit is, By rearranging the surrounding attributes in order of proximity to the main attribute in the drawing shown in the drawing data and then linking them to the main attribute, a column of attributes is generated. The drawing recognition device according to claim 2, characterized in that it converts the generated column of attributes into numerical features, and inputs the numerical features obtained by the conversion into the fourth machine learning model, thereby obtaining information about the column of attributes to be converted into numerical features, and the symbols and connecting lines to which the principal attributes included in the column of attributes to be converted into numerical features that approximate the numerical features are associated.
The fourth recognition unit is, When inputting the generated column of attributes into the fourth machine learning model, the corresponding number, which is set in advance for each category to which the symbol belongs and indicates the number of primary attributes that can be associated with the symbol, is referenced. The drawing recognition device according to claim 2, characterized in that, if the number of primary attributes included in the generated attribute column does not match the corresponding number, the number of primary attributes included in the generated attribute column matches the corresponding number, and then inputs this to the fourth machine learning model.
A drawing recognition device according to any one of claims 1 to 4, further comprising a determination unit that determines whether the information relating to the correspondence between attributes, symbols, and connecting lines in the drawing data, recognized by the fourth recognition unit, is appropriate, based on determination information set in advance for determining whether the information relating to the correspondence between attributes, symbols, and connecting lines in the drawing data is appropriate.
The determination unit records the information regarding the correspondence relationship that was determined to be negative by the determination, along with the information regarding the symbol, attribute, and connection line that formed the basis of the information regarding the correspondence relationship, as undetermined data. The drawing recognition device according to claim 5, characterized in that the fourth recognition unit recognizes again information regarding the correspondence between attributes, symbols and connecting lines in the drawing data based on the information recorded as undetermined data by the determination unit and the fourth machine learning model.
Computers, A drawing acquisition unit that acquires drawing data, A first recognition unit recognizes information about symbols in the drawing data based on the drawing data acquired by the drawing acquisition unit and a first machine learning model that takes the drawing data as input and outputs recognition results regarding symbols on the drawing data. A second recognition unit recognizes information regarding attributes in the drawing data based on the drawing data acquired by the drawing acquisition unit and a second machine learning model that takes the drawing data as input and outputs recognition results regarding attributes on the drawing data. A third recognition unit recognizes information about connecting lines in the drawing data based on the drawing data acquired by the drawing acquisition unit and a third machine learning model that takes the drawing data as input and outputs recognition results regarding connecting lines that connect symbols on the drawing data. A drawing recognition program for functioning as a fourth recognition unit that recognizes information regarding the correspondence between attributes, symbols, and connecting lines in drawing data , based on information regarding symbols recognized by the first recognition unit using the first machine learning model, information regarding attributes recognized by the second recognition unit using the second machine learning model, and information regarding connecting lines recognized by the third recognition unit using the third machine learning model, and information regarding attributes recognized by the second recognition unit as input, and a fourth machine learning model that outputs information regarding the correspondence between attributes and symbols in the drawing data, and information regarding the correspondence between attributes and connecting lines in the drawing data.

Description

This disclosure relates to a drawing recognition device and a drawing recognition program. In drawings containing a mixture of characters and symbols, drawing recognition technologies are known that recognize the characters or symbols present in the drawing and determine their corresponding relationships. For example, Patent Document 1 discloses a drawing recognition technology that, in drawing data containing a mixture of characters, symbols, and pipes, determines which symbols or pipes each individually recognized character corresponds to, based on pre-prepared knowledge of the relative positions of the characters, symbols, or pipes. Japanese Patent Publication No. 2004-234424 This figure shows an example of the configuration of the drawing recognition device according to Embodiment 1.This is a diagram illustrating an example of an image drawing in Embodiment 1.This figure illustrates an example of a determination result output by the determination unit in Embodiment 1.This is a diagram illustrating an example of component information in Embodiment 1.This figure shows a detailed configuration example of the recognition unit in Embodiment 1.In Embodiment 1, Figure 6A shows an example of the symbol recognition result acquired by the symbol recognition unit, the attribute recognition result acquired by the attribute information recognition unit, and the connection line recognition result acquired by the connection line recognition unit. Figure 6B shows an image in which the symbol recognition result, attribute recognition result, and connection line recognition result are mapped onto an image drawing acquired by the drawing acquisition unit, and Figure 6B shows an image in which the contents of the symbol recognition result, attribute recognition result, and connection line recognition result are shown in text.This figure illustrates a specific example of inference performed by the association recognition unit in Embodiment 1.This figure illustrates another specific example of inference by the association recognition unit in Embodiment 1.This figure illustrates an example of how the association recognition unit in Embodiment 1 absorbs variations in attribute notation.This figure illustrates a specific example of the determination process performed by the determination unit in Embodiment 1.This is a flowchart illustrating the operation of the drawing recognition device according to Embodiment 1.This is a flowchart illustrating the details of the correspondence recognition process performed by the association recognition unit in step ST5 of Figure 11.Figures 13A and 13B show an example of the hardware configuration of the drawing recognition device according to Embodiment 1.This figure shows an example of the configuration of a learning device according to Embodiment 1.This figure shows a specific example of learning when the association model generation unit generates the fourth machine learning model in Embodiment 1.This is a flowchart illustrating the operation of the learning device according to Embodiment 1.This is a flowchart illustrating the details of the generation process of the fourth machine learning model performed by the association model generation unit in step ST25 of Figure 16.Figures 18A and 18B show an example of the hardware configuration of the learning device 400 according to Embodiment 1. The embodiments of this disclosure will be described in detail below with reference to the drawings. Embodiment 1. Figure 1 is a diagram showing an example of the configuration of a drawing recognition device 100 according to Embodiment 1. The drawing recognition device 100 is connected to the operation input device 200 and the display device 300. The operation input device 200 includes, for example, a keyboard 201 and a mouse 202. The display device 300 includes a display 301. The display 301 is, for example, a liquid crystal display or an organic EL (Electroluminescence) display. The display device 300 may be mounted on the operation input device 200. The operation input device 200 receives operations from the user of the drawing recognition device 100. For example, the user inputs information to the drawing recognition device 100 by operating the keyboard 201 or mouse 202 provided on the operation input device 200. Specifically, the user operates the keyboard 201 or mouse 202 to display a list of image drawings 1 on the display 301 of the display device 300, and then selects an image drawing 1. More specifically, for example, multiple image drawings 1 are pre-recorded in a drawing recording unit (not shown) located in a place accessible to the drawing recognition device 100 and the operation input device 200. The user, for example, operates the keyboard 201 or mouse 202 to cause the drawing recognition device 100 to display a list of multiple image drawings 1 recorded in the drawing recording unit on the display 301. Then, the user, for example, operates the keyboard 201 or mouse 202 to select a desi