Search

KR-102961801-B1 - WORK TASK RISK DETERMINATION SYSTEM AND METHOD THEREOF USING VISION AI AND LANGUAGE MODEL

KR102961801B1KR 102961801 B1KR102961801 B1KR 102961801B1KR-102961801-B1

Abstract

A work risk determination system according to one embodiment of the present invention may provide a work risk determination system comprising: a video object recognition unit that recognizes video objects in a captured video; a safety measure sentence model including a plurality of entities, relationships, and labels constructed based on language units extracted from safety measure sentences; a video object matching unit that matches video objects with a plurality of entities and analyzes relationships between the matched objects; a risk determination AI model that determines a dangerous situation in a captured video based on relationships between the matched objects and the safety measure sentence model; and a continuity determination unit that analyzes video objects flowing in the captured video to determine additional dangerous situations.

Inventors

  • 최대현
  • 양재군

Assignees

  • 주식회사 엔소프트

Dates

Publication Date
20260508
Application Date
20240508

Claims (11)

  1. An image object recognition unit that recognizes image objects in a captured image; A safety statement model comprising multiple entities, relationships, and labels constructed based on language units extracted from safety statement; An image object matching unit that matches the image object with the plurality of entities and analyzes the relationships between the matched objects; A risk judgment AI model that determines a dangerous situation in the captured image based on the relationship between the matched objects and the safety measure sentence model; and A continuity determination unit that analyzes a moving image object in the above-described image to determine an additional risk situation; is included, The above image object matching unit is, A first entity - first object matching part that matches a first entity and a first object; A second entity - second object matching part that matches a second entity and a second object; It includes a two-dimensional positional relationship analysis unit that analyzes separation, proximity, distance, and relative positional relationships between a first object in an image and a second object in an image, and It further includes a 3D positional relationship analysis unit that analyzes the relationship between a first object and a second object within a 3D image, and Extract safety measure text related to the first entity and the second entity above, and Based on the above safety measure text, verify the relationship between the first entity and the first object in the image and the second entity and the second object in the image. Labeling as 'risk' or 'safe' based on the above verified relationship and the above safety measure statement Work Risk Assessment System.
  2. In paragraph 1, The above safety measure statements are collected from at least one model among workplace safety guides, work rules, specifications, JSA documents, laws, and enforcement decrees, and The above entity includes a first entity corresponding to the subject of the above safety measure sentence and a second entity corresponding to the object. Work Risk Assessment System.
  3. In paragraph 1, The above relationship and label are extracted from the predicate of the above safety measure statement. Work Risk Assessment System.
  4. delete
  5. delete
  6. In paragraph 1, A work risk judgment system in which the attributes of the above location include longitudinal and transverse information and direction information of the above image object.
  7. In a method for determining work risk using a work risk determination system, a) A step of recognizing an image object from a captured image; b) providing a safety statement model including multiple entities, relationships, and labels constructed based on language units extracted from safety statement; c) a step of matching the recognized image object with a plurality of entities of the safety measure sentence model and analyzing the relationships between the matched objects; d) a step of providing a risk judgment AI model that determines a dangerous situation in the captured image based on the relationship between the matched objects and the safety measure sentence model; e) a step of determining risk using the above-mentioned risk judgment AI model and outputting the determined result; and f) including a step of analyzing moving image objects in the above-captured video to determine additional risk situations The above step c) is a step of matching a first entity with a first object in the image and matching a second entity with a second object in the image; Analyze the separation, proximity, distance, and relative positional relationships between a first object and a second object within a 2D image. A step of analyzing the relationship between a first object and a second object within a 3D image. It includes, The above step d) includes the step of matching objects recognized from the captured image with the first entity and the second entity of the safety measure sentence model; and A step of extracting safety measure text related to the first entity and the second entity; A step of verifying the relationship between the first entity and the first object in the image and the second entity and the second object in the image based on the above safety measure text; and A method for determining work risk comprising the step of labeling as 'risk' or 'safe' based on the above-mentioned verified relationship and the above-mentioned safety measure statement.
  8. In Paragraph 7, The above safety measure statements are collected from at least one model among workplace safety guides, work rules, specifications, JSA documents, laws, and enforcement decrees, and The above entity includes a first entity corresponding to the subject of the above safety measure sentence and a second entity corresponding to the object, and The above relationship and label are extracted from the predicate of the above safety measure statement. Method for assessing work risk.
  9. In Paragraph 7, The above step b) is, Step of analyzing the above safety measure sentence; A step of extracting entities from the above safety measure statement; A step of distinguishing subject entities and object entities from the extracted entities above; A step of generating relationships between entities based on the predicates of the above safety measure sentences; and The step of creating a dataframe of a safety statement model using the generated data regarding the first entity, second entity, relationship, and label. A method for determining work risk that includes
  10. delete
  11. In Paragraph 7, The relationship between the first entity and the second entity includes at least one of state, behavior, wearing, installation, location, and use. Method for assessing work risk.

Description

Work Task Risk Determination System and Method Using Vision AI and Language Model The present invention relates to a system and method for determining risks occurring in a workplace. More specifically, the invention relates to a system and method for determining work risks using image artificial intelligence and a language model, which can determine work risks by extracting language from sentences of safety measure documents to generate a language model, and matching objects detected in the workplace and the relationships between objects with safety measures through the generated language model. In industrial production sites, work radii are divided according to the specific tasks within each industrial sector. These radii are comprised of diverse personnel and equipment, and consequently, various hazardous situations exist. In particular, since large-scale production sites, including construction, have a very high potential for danger, sensors are installed and monitored at the work radius level. However, since this risk observation method requires the installation of various sensors in equipment and unit zones, a large number of sensors are required, and the cost of installing and building the sensor system is substantial. Moreover, the larger the work site, the more work managers are required, and the reality is that it is difficult to individually check and direct the numerous unit tasks performed on-site. In this regard, there has recently been an increase in technologies that use artificial intelligence to assess hazardous situations at work sites. Generally, AI requires a training process using learning data to make situational judgments. Furthermore, an AI model is required to assess the diverse situations occurring within various radius of the work site. For example, there is a problem in that work videos must be filmed and collected for each situation occurring at various radii, image processing must be performed, and a large training dataset must be constructed for each situation. Korean Registered Patent Publication No. 2126498 (June 18, 2020) is disclosed as a prior art addressing these problems. The aforementioned prior art is an artificial intelligence risk judgment technology that efficiently determines dangerous situations by extracting objects from images and analyzing the movement of objects. According to the aforementioned conventional technology, artificial intelligence primarily determines risk by extracting detailed movements of objects from images, and secondarily determines risk by re-collecting and analyzing images of objects determined to be at risk. In other words, time and cost are incurred for image preprocessing and re-collecting images to recognize the detailed movements of objects. Therefore, there is a need for AI risk assessment technology capable of efficiently performing risk judgments without collecting large amounts of training data or re-collecting candidate risk images for various situations occurring at the work site. FIG. 1 is a block diagram illustrating the overall configuration of a work risk judgment system according to the present invention. FIG. 2 is a flowchart illustrating the process of generating a safety measure sentence model according to an embodiment of the present invention. FIG. 3 is a reference diagram for explaining the data frame of a safety measure sentence model according to an embodiment of the present invention. FIG. 4 is a detailed block diagram of an image object matching unit according to an embodiment of the present invention. FIG. 5 is a flowchart illustrating a method for generating a risk judgment AI model according to an embodiment of the present invention. FIG. 6 is a flowchart illustrating the process of determining work risk according to an embodiment of the present invention. FIG. 7 is a flowchart illustrating the steps of matching entities and image objects according to an embodiment of the present invention in detail. The present invention will be described below with reference to the attached drawings. However, the present invention may be implemented in various different forms and is therefore not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification have been given similar reference numerals. Throughout the specification, when it is stated that a part is "connected (connected, in contact, combined)" with another part, this includes not only cases where they are "directly connected," but also cases where they are "indirectly connected" with other members interposed between them. Furthermore, when it is stated that a part "includes" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but rather allows for the inclusion of additional components. The terms used herein are merely for describing specifi