KR-20260064492-A - METHOD AND SYSTEM FOR BAYESIAN OPTIMIZATION

Abstract

The present invention relates to a Bayesian optimization method and system. More specifically, the present invention relates to a Bayesian optimization method and system using kernel density estimation (KDE) of individual parameters in a mixed variable environment.

Inventors

• 임태윤
• 어문정
• 오연주
• 서민국
• 임우형

Assignees

• 주식회사 LG 경영개발원

Dates

Publication Date

20260507

Application Date

20250813

Priority Date

20241031

Claims (20)

1. In a computerized method including the following, A step of specifying multiple parameters having different characteristics; A step of estimating a prior distribution for each of the plurality of parameters using a pre-established probability distribution estimation technique; A step of constructing an initial sample set based on the prior distribution estimated for each of the plurality of parameters; and A Bayesian optimization method characterized by including the step of training a prediction model using the initial sample set to produce an optimal solution for the plurality of parameters.
2. In paragraph 1, In the step of estimating the prior distribution mentioned above, For each of the plurality of parameters, the pre-set probability distribution estimation technique is applied to estimate the prior distribution for each of the plurality of parameters, and The above initial sample set is, A Bayesian optimization method characterized by being constructed by performing sampling from the estimated prior distribution for each of the plurality of parameters.
3. In paragraph 2, The step of configuring the above initial sample set is, A step of sampling the value of each of the plurality of parameters from a prior distribution for each of the plurality of parameters; A step of generating a plurality of initial samples using the values of each of the sampled plurality of parameters; and A Bayesian optimization method characterized by including the step of constructing an initial sample set using a plurality of initial samples to train the prediction model.
4. In paragraph 3, In the step of configuring the above initial sample set, An evaluation is performed on each of the plurality of initial samples sampled from the prior distribution for each of the plurality of parameters, and Based on the above evaluation, an evaluation result for each of the plurality of initial samples is obtained, and A Bayesian optimization method characterized by constructing the initial sample set using the plurality of initial samples and the evaluation results for each of the plurality of initial samples.
5. In paragraph 4, In the step of training the above prediction model, A Bayesian optimization method characterized by training the prediction model using the plurality of initial samples included in the initial sample set and the evaluation results.
6. In paragraph 4, The above prediction model is, A Bayesian optimization method characterized by learning an approximation of a function to be optimized in an environment where the plurality of parameters are mixed, using a prior distribution for each of the plurality of parameters.
7. In paragraph 6, A Bayesian optimization method characterized in that the learning of the approximation of the function to be optimized is a process of learning to approximate the plurality of initial samples and the evaluation results for each of the plurality of initial samples.
8. In paragraph 3, The step of generating the above plurality of initial samples is, A Bayesian optimization method characterized by the step of generating a plurality of initial samples including different parameter combinations using the values of each of the plurality of parameters sampled from a prior distribution for each of the plurality of parameters.
9. In paragraph 1, The above prediction model is, A Bayesian optimization method characterized by selecting an optimization evaluation point by learning the interaction between multiple parameters having different characteristics.
10. In paragraph 1, A step of sampling candidate evaluation points from a prior distribution for each of the plurality of parameters; and A Bayesian optimization method characterized by further including the step of selecting at least one specific evaluation point among the candidate evaluation points to be evaluated using the above-mentioned learned prediction model and specific function.
11. In Paragraph 10, The step of sampling the above candidate evaluation points is, A Bayesian optimization method characterized by the step of sampling candidate evaluation points containing different parameter combinations from a prior distribution for each of the plurality of parameters.
12. In Paragraph 10, In the step of selecting the specific evaluation point mentioned above, Using the above-mentioned learned prediction model, predict a predicted value for each of the above-mentioned candidate evaluation points, and A Bayesian optimization method characterized by selecting a specific evaluation point using a predicted value for each of the above candidate evaluation points and a specific function.
13. In Paragraph 12, In the step of selecting the specific evaluation point mentioned above, Calculate the specific function value for each of the candidate evaluation points using at least one of the optimal values serving as a criterion for selecting the above predicted value and the above specific evaluation point, and A Bayesian optimization method characterized by selecting, among the above candidate evaluation points, at least one evaluation point in which the calculated specific function value satisfies a preset criterion as the specific evaluation point.
14. In Paragraph 13, A Bayesian optimization method characterized in that the above-mentioned specific evaluation point includes a point that maximizes the above-mentioned specific function value.
15. In Paragraph 10, The above-mentioned learned prediction model is, A Bayesian optimization method characterized by predicting at least one of a predicted value and a predicted variance for each of the above-mentioned candidate evaluation points.
16. In Paragraph 10, A step of performing an evaluation on the specific evaluation point selected above; A step of updating the learned prediction model based on the evaluation results for the specific evaluation point; and A Bayesian optimization method characterized by further including the step of calculating an optimal solution for a plurality of parameters based on the update to the learned prediction model.
17. In Paragraph 16, A Bayesian optimization method characterized in that the optimal solution includes an optimal combination of parameters calculated from the plurality of parameters.
18. In paragraph 1, The plurality of parameters having the above different characteristics are, It includes at least one of a first parameter having a first characteristic and a second parameter having a second characteristic, and In the step of estimating the prior distribution mentioned above, By applying the pre-set probability distribution estimation technique to each of the first parameter and the second parameter, a prior distribution for each of the first parameter and the second parameter is estimated, and The above prediction model A Bayesian optimization method characterized by being learned based on the prior distribution estimated for each of the first parameter and the second parameter using the above-mentioned probability distribution estimation technique.
19. In a system comprising memory configured to store executable instructions and one or more processors configured to perform operations by executing one or more instructions, The above system is, Specifying multiple parameters having different characteristics, and Using a pre-established probability distribution estimation technique, a prior distribution for each of the plurality of parameters is estimated, and Based on the prior distribution estimated for each of the above multiple parameters, an initial sample set is constructed, and A Bayesian optimization system characterized by training a prediction model using the initial sample set to produce an optimal solution for the plurality of parameters.
20. A program that is executed by one or more processes in an electronic device and stored on a computer-readable recording medium, The above program is, A step of specifying multiple parameters having different characteristics; A step of estimating a prior distribution for each of the plurality of parameters using a pre-established probability distribution estimation technique; A step of constructing an initial sample set based on the prior distribution estimated for each of the plurality of parameters; and A program stored on a computer-readable recording medium characterized by including instructions for performing a step of training a prediction model using an initial set of samples to calculate an optimal solution for the plurality of parameters.

Description

Bayesian Optimization Method and System The present invention relates to a Bayesian optimization method and system. More specifically, the present invention relates to a Bayesian optimization method and system using kernel density estimation (KDE) of individual parameters in a mixed variable environment. Bayesian optimization is a technique that utilizes prior information to efficiently search for an optimal solution while minimizing high-cost function evaluations. This type of optimization is applied in various fields, including hyperparameter tuning, process control, and autonomous driving system optimization. However, in real-world industrial environments, optimization problems frequently occur in environments involving a mixture of continuous and discrete variables (or parameters). For example, in mixed-variable environments where continuous and discrete variables are combined, accurately estimating the joint probability distribution of each variable becomes difficult as the number of variables increases and their ranges widen, and optimization performance can degrade rapidly due to the curse of dimensionality. In this regard, conventional Bayesian optimization methods have limitations in that they use a method of modeling the joint distribution by integrating all variables, which increases computational costs and makes it difficult to finely reflect the characteristics of each variable. Furthermore, in high-dimensional and mixed-variable environments, the distribution characteristics of each variable differ, so applying the same estimation technique or parameters can reduce optimization efficiency. Conventional Bayesian optimization methods are difficult to apply to real-time processing or large-scale system optimization due to the rapid increase in computational load, and often fail to reflect the characteristics of different variable types, which may limit the accuracy and interpretability of optimization results. Accordingly, there is still a need for a method that enables the precise estimation of the individual distributions of each variable, even in environments where continuous and discrete variables are mixed, and allows for stable and efficient Bayesian optimization based on this. FIG. 1 is a conceptual diagram illustrating a Bayesian optimization system according to the present invention. FIGS. 2, FIGS. 3 and FIGS. 4 are flowcharts illustrating a Bayesian optimization method according to the present invention. Figures 5 and 6 are formulas related to the Bayesian optimization method according to the present invention. FIG. 7 shows an example of an algorithm of a Bayesian optimization method according to the present invention. FIG. 8 is a table showing an example of experimental results of a Bayesian optimization method and system according to the present invention. Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components are assigned the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not have distinct meanings or roles in themselves. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention. Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. A singular expression includes a plural expression unless the context clearly indicates otherwise. In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification