Search

KR-20260066543-A - METHOD AND SYSTEM FOR DETECTING DEFECTS IN ARTIFICIAL INTELLIGENCE MODELS

KR20260066543AKR 20260066543 AKR20260066543 AKR 20260066543AKR-20260066543-A

Abstract

A defect detection method for an artificial intelligence model according to the present invention may include: a step of collecting input data of an artificial intelligence model to be a defect detection target; a step of collecting output data of the artificial intelligence model for the input data; a step of generating a prompt of a defect detection model for detecting defects in the artificial intelligence model using the input data and the output data; a step of processing the prompt as an input of the defect detection model; and a step of obtaining a defect detection result of the output data for the input data from the defect detection model.

Inventors

  • 김세종
  • 신재훈
  • 심상진
  • 한형동
  • 유승학
  • 김용범

Assignees

  • 네이버 주식회사

Dates

Publication Date
20260512
Application Date
20241104

Claims (18)

  1. A step of collecting input data for an artificial intelligence model that is subject to defect detection; A step of collecting output data of the artificial intelligence model for the above input data; A step of generating a prompt for a defect detection model that detects defects in the artificial intelligence model using the above input data and the above output data; The step of processing the above prompt as input to the defect detection model; and A defect detection method of an artificial intelligence model comprising the step of obtaining a defect detection result of the output data for the input data from the defect detection model.
  2. In paragraph 1, The above defect detection results are, A method for detecting defects in an artificial intelligence model, characterized by including information regarding at least one of the reason for the defect, the type of defect, and the method for improvement of the artificial intelligence model related to the output data.
  3. In paragraph 2, The above defect detection model is, A method for detecting defects in an artificial intelligence model, characterized in that the output data output by the artificial intelligence model is configured to generate a reason for defects that explains what defects are included in relation to the input data.
  4. In paragraph 3, The above defect detection model is, A method for detecting defects in an artificial intelligence model, characterized by generating the reason for the defect based on defect description data related to the artificial intelligence model included in the above prompt.
  5. In paragraph 1, The above prompt is, A first type of data including descriptive data on the functions of the artificial intelligence model and descriptive data on defects related to the artificial intelligence model, A second type of data including defect sample data of the above artificial intelligence model, and A method for detecting defects in an artificial intelligence model, characterized by generating a prompt that includes at least one of a third type of data including the input data and the output data.
  6. In paragraph 5, The defect description data related to the artificial intelligence model included in the above-mentioned first type of data is, A method for detecting defects in an artificial intelligence model, characterized by including a plurality of different defect types that may occur in the artificial intelligence model and a description of the plurality of different defect types.
  7. In paragraph 6, The defect sample data of the artificial intelligence model included in the above second type of data is, Sample input data and sample output data of the artificial intelligence model for the sample input data, and A method for detecting defects in an artificial intelligence model, characterized by including data regarding the reason for the sample defect, the type of the sample defect, and the method for improving the sample of the artificial intelligence model for the sample output data.
  8. In Paragraph 7, A defect detection method for an artificial intelligence model characterized in that the above sample defect type is any one of the above multiple different defect types.
  9. In Paragraph 7, The defect sample data of the artificial intelligence model included in the above second type of data is, A defect detection method of an artificial intelligence model characterized by including multiple defect sample data corresponding to each of the above-mentioned multiple different defect types.
  10. In paragraph 6, The step of obtaining the defect detection result of the above output data is, A step of classifying which of the multiple different defect types the output data corresponds to with respect to the input data, and A defect detection method for an artificial intelligence model characterized by including a step of generating defect reasons for classified defect types.
  11. In Paragraph 10, In the above classification step, A method for detecting defects in an artificial intelligence model, characterized by defining a new defect type of the artificial intelligence model for the output data when there is no defect type related to the output data among the above-mentioned multiple different defect types.
  12. In Paragraph 11, A method for detecting defects in an artificial intelligence model, characterized by further including the step of updating the first type of data so that data related to the new defect type is included in the first type of data.
  13. In Paragraph 12, In conjunction with the updating of the above-mentioned first type data to include data related to the above-mentioned new defect type, A method for detecting defects in an artificial intelligence model, characterized by further including the step of updating the second type of data such that the input data, the output data, and data related to the new defect type are included in the second type of data as sample defect data of the artificial intelligence model.
  14. In paragraph 5, The above-mentioned first type of data and the above-mentioned second type of data are, A method for detecting defects in an artificial intelligence model, characterized by being configured differently according to the functions of the artificial intelligence model.
  15. In paragraph 6, A method for detecting defects in an artificial intelligence model, characterized by further including the step of generating statistical information on defect types of a plurality of input data input to the artificial intelligence model and a plurality of output data corresponding to each of the plurality of input data, using the defect detection results above.
  16. In paragraph 15, A method for detecting defects in an artificial intelligence model, characterized by further including the step of generating priority information regarding which defect type among the multiple different defect types should be prioritized for improvement in order to improve defects in the artificial intelligence model based on the above statistical information.
  17. In a defect detection system for an artificial intelligence model, The above system includes memory and at least one processor, The above memory and the above processor cooperate, Collect input data for the artificial intelligence model subject to defect detection, and Collect output data of the artificial intelligence model for the above input data, and Using the above input data and the above output data, a prompt for a defect detection model that detects defects in the above artificial intelligence model is generated, and Process the above prompt as input to the above defect detection model, and A defect detection system of an artificial intelligence model characterized by obtaining a defect detection result of the output data for the input data from the above defect detection model.
  18. A program that is executed by one or more processes in an electronic device and stored on a computer-readable medium, The above program is, A step of collecting input data for an artificial intelligence model that is subject to defect detection; A step of collecting output data of the artificial intelligence model for the above input data; A step of generating a prompt for a defect detection model that detects defects in the artificial intelligence model using the above input data and the above output data; The step of processing the above prompt as input to the above defect detection model; and A program stored on a computer-readable medium characterized by including instructions for performing a step of obtaining a defect detection result of the output data for the input data from the defect detection model.

Description

Method and System for Detecting Defects in Artificial Intelligence Models The present invention relates to a method and system for detecting defects in an artificial intelligence model. The dictionary definition of artificial intelligence is a technology that realizes human learning, reasoning, perception, and natural language understanding abilities through computer programs. This artificial intelligence has achieved rapid development through deep learning. In particular, driven by the advancement of artificial intelligence, various language models have been developed. These models have reached a level where they not only recognize text and understand its meaning but also extract and classify information from vast amounts of text-based data, such as documents, and even generate text directly. These language models are actively utilized in various fields and exist in diverse areas where text-based operations can be performed, such as search engines, document creation (e.g., resume writing, report writing, posting, etc.), free conversation on various topics, data parsing from given text (e.g., data summarization, classification, etc.), provision of expertise, programming, and converting given sentences into sentences of an appropriate style. Additionally, Korean Published Patent No. 10-2024-0004054 discloses a method for generating marketing text for a target to be advertised using a language model. While various studies are being conducted to ensure the task quality of AI for the provision of AI-based services, they still rely on human judgment. Since human judgment has limitations in terms of cost, time, manpower, and consistency, there is a need for automated methods to guarantee AI quality. FIG. 1 is a block diagram illustrating a defect detection system of an artificial intelligence model according to the present invention. FIGS. 2a and FIGS. 2b are flowcharts illustrating data processing of an artificial intelligence model according to the present invention. Figure 3 is a flowchart illustrating a method for detecting defects in an artificial intelligence model in the present invention. FIGS. 4 and FIGS. 5 are conceptual diagrams illustrating a method for detecting defects in an artificial intelligence model in the present invention. Figures 6 and 7 are flowcharts illustrating a method for updating a new defect type of an artificial intelligence model in the present invention. FIG. 8 is a conceptual diagram illustrating the defect detection results of an artificial intelligence model in the present invention. Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components are assigned the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not have distinct meanings or roles in themselves. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention. Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. The present invention presents an automated methodology for detecting defects in an artificial intel