Search

CN-122018320-A - Self-adaptive multi-order cooperative non-convex optimization control method, device and system

CN122018320ACN 122018320 ACN122018320 ACN 122018320ACN-122018320-A

Abstract

The invention provides a self-adaptive multistage cooperative non-convex optimization control method, device and system. The method adopts a local optimization strategy based on gradient information to realize rapid convergence, automatically introduces second-order curvature analysis after reaching a first-order stable point, activates higher-order information as required if a second-order stable condition is met, and constructs directional disturbance in the identified escapable ion space to break through stagnation. The above process is iteratively performed until a valid escape direction cannot be identified under the current information. The invention can obviously improve the capability of the algorithm to escape from various flat saddle points (including the degraded saddle point and the weak positive curvature saddle point) and converge to a better local solution on the premise of not obviously increasing the calculation cost of the main optimization stage, and effectively balances the calculation efficiency and the solution quality.

Inventors

  • ZHU XIHUA
  • CHEN KE

Assignees

  • 上海商学院

Dates

Publication Date
20260512
Application Date
20260304

Claims (10)

  1. 1. The self-adaptive multi-order cooperative non-convex optimization control method is characterized by comprising the following steps of: step S101, acquiring an objective function, and performing iterative search on a non-convex curved surface of the objective function until a first-order stable point meeting a preset first-order stable condition is obtained; step S102, calculating second-order curvature information corresponding to the first-order stable point according to the first-order stable point, and judging second-order stable conditions to obtain a second-order stable point meeting preset second-order stable conditions; Step 103, according to a preset saddle point discrimination model, saddle point evadability discrimination is carried out on the second-order stable points, and when the discrimination result is evadability, a corresponding evadability ion space is determined; step S104, responding to the escape judgment result of the saddle point as an escape, and generating and verifying directional disturbance in the escape space based on the escape space and a third derivative tensor at the second-order stable point; And step 105, if the directional disturbance is verified to be effective, taking a new point after the disturbance as input, and repeatedly executing the steps S101 to S104 until the result of the escape judgment of the saddle point is non-escape.
  2. 2. The adaptive multi-order collaborative non-convex optimization control method according to claim 1, wherein the steps of obtaining an objective function and performing iterative search on a non-convex curved surface of the objective function until a first-order stable point satisfying a preset first-order stable condition is obtained comprise: Performing iterative search on a non-convex definition domain of an objective function based on a preset search model until a preset first-order stability condition is met to obtain a first-order stability point, wherein the search model is internally provided with any local optimization algorithm based on gradient information, and the local optimization algorithm comprises, but is not limited to, a gradient descent method, a momentum method, a self-adaptive moment estimation method and a trust domain algorithm; The first-order stable condition comprises that the gradient norm of the current iteration point is smaller than or equal to a first preset termination threshold value, or when a trust domain algorithm is adopted, the estimated descent quantity of the first-order Taylor model at the current iteration point in the trust domain is smaller than or equal to the product of the first preset termination threshold value and the radius of the trust domain.
  3. 3. The adaptive multi-order collaborative non-convex optimization control method according to claim 1 or 2, wherein the step of calculating second order curvature information corresponding to the first order stabilization point according to the first order stabilization point, and performing second order stabilization condition judgment to obtain a second order stabilization point satisfying a preset second order stabilization condition comprises: Calculating a Hessian matrix at the first-order stable point as second-order curvature information; Judging whether a preset second-order stability condition is met; if the second order stable condition is satisfied, determining the current point as a second order stable point, and executing step S103; If any second order stable condition is not satisfied, determining whether to update the current iteration point and/or algorithm parameters based on the second order curvature information, and feeding back the current iteration point and the corresponding algorithm parameters to the step S101 to continue to execute the iteration process of the steps S101 to S102 until a point satisfying the second order stable condition is obtained; The second-order stability condition comprises that the minimum eigenvalue of the Hessian matrix at the first-order stability point is larger than or equal to a negative second preset termination threshold value, or when a trust domain algorithm is adopted, the estimated descent quantity of the second-order Taylor model at the first-order stability point in the trust domain is smaller than or equal to the product of the second preset termination threshold value and the square of the radius of the trust domain.
  4. 4. The adaptive multi-order cooperative non-convex optimization control method according to claim 1, wherein the step of performing saddle point evadability discrimination on the second-order stable point according to a preset saddle point discrimination model comprises: Performing eigenvalue decomposition on the Hessian matrix at the second-order stable point to obtain a plurality of unit orthogonal eigenvectors which are ordered from large to small according to eigenvalues; sequentially taking a space formed by the i-th to n-th eigenvectors as a candidate subspace, wherein i=1, 2, & gt, n is a Hessian matrix dimension; Calculating the projection norm of the third-order derivative tensor in the current candidate subspace, and judging whether the evasive condition is satisfied; If yes, judging the current point as an escapable saddle point, recording the subspace as an escapable space, outputting the projection norms of the escapable space and the third derivative tensor in the escapable space, and terminating the judgment; if all the candidate subspaces are not satisfied after the traversal, judging the current point as a local optimal point which cannot escape, and terminating the optimization process.
  5. 5. The adaptive multi-order collaborative non-convex optimization control method according to claim 4, wherein the evacuable condition includes a square of the projection norm being greater than or equal to an evacuable threshold collectively determined by a third-order Lipschitz constant estimate, a eigenvalue of the current candidate subspace starting eigenvector, and a preset control parameter, the third-order Lipschitz constant estimate having a preset initial value.
  6. 6. The adaptive multi-order cooperative non-convex optimization control method according to claim 4, wherein the step of generating and verifying directional disturbance in the evasive ion space includes: Searching a unit vector in the escapable ion space, so that a third-order action value of a third-order derivative tensor on the unit vector is larger than or equal to the ratio of the projection norm to a preset control parameter; Calculating a directional disturbance step length and constructing a third-order disturbance step based on the unit vector, the projection norm, the third-order Lipschitz constant estimated value and the preset control parameter; Calculating the actual function descent quantity after disturbance execution based on the third-order disturbance step; calculating a theoretical third-order descent reference amount based on the projection norm, the third-order Lipschitz constant estimation value and the preset control parameter; calculating the ratio of the actual function descent quantity to the theoretical third-order descent reference quantity; If the ratio is greater than or equal to a preset acceptance threshold, accepting the new point after disturbance, and returning the new point as input to the step S101; If the ratio is smaller than the preset acceptance threshold, keeping the current second-order stable point unchanged, increasing the third-order Lipschitz constant estimated value, and returning to step S103.
  7. 7. The adaptive multi-order collaborative non-convex optimization control method according to claim 6, wherein the step S101 is implemented by using an adaptive reliability domain framework, and the framework dynamically selects a first-order taylor model or a second-order taylor model for constructing reliability domain sub-problems according to current local geometric characteristics in an iterative process, and always uses the first-order stability condition as a termination criterion.
  8. 8. The utility model provides a non-protruding optimal control device of self-adaptation multistage cooperation which characterized in that includes: The first-order solver is used for acquiring an objective function, and performing iterative search on a non-convex curved surface of the objective function until a first-order stable point meeting a preset first-order stable condition is obtained; the second-order solver is used for receiving the first-order stable point, calculating second-order curvature information corresponding to the first-order stable point, and judging second-order stable conditions to obtain a second-order stable point meeting preset second-order stable conditions; The saddle point discriminator is used for discriminating saddle point evadability of the second-order stable point according to a preset saddle point discrimination model, and determining a corresponding evadability space when the discrimination result is evadability; The third-order perturbator comprises a perturbation generating unit and a perturbation verifying unit, wherein the perturbation generating unit is used for generating a directional perturbation step length based on a third-order derivative tensor at the second-order stable point in the escapable ion space; Wherein the perturbed new point is configured to return to the first-order solver to iteratively execute the foregoing modules until the saddle point evacuable discrimination results in non-evacuable.
  9. 9. The self-adaptive multi-order cooperative non-convex optimization control system is characterized by comprising a main controller, a cross-order information multiplexing bus and the self-adaptive multi-order cooperative non-convex optimization control device as claimed in claim 8, wherein the main controller comprises a state trigger for controlling state switching, data flow and iteration loops among a first-order solver, a second-order solver, a saddle point discriminator and a third-order perturber to form closed-loop optimization control logic of low-order optimization, high-order analysis, intelligent escape and feedback loops, the self-adaptive multi-order cooperative non-convex optimization control device is in communication connection with the main controller, and the cross-order information multiplexing bus is used for data transmission.
  10. 10. A method for solving a non-convex optimization problem, comprising: An objective function of the non-convex optimization problem is optimized to obtain a solution of the problem using the adaptive multi-order collaborative non-convex optimization control method according to any one of claims 1-7.

Description

Self-adaptive multi-order cooperative non-convex optimization control method, device and system Technical Field The invention relates to the technical field of computational mathematics and artificial intelligence, in particular to a self-adaptive multi-order cooperative non-convex optimization control method, a device and a system and a solving method of a non-convex optimization problem. Background The non-convex optimization problem widely exists in the fields of artificial intelligence, scientific calculation, engineering decision and the like, such as deep neural network training, molecular conformational search, financial combination optimization and the like. The objective function of such problems is typically structurally complex involving a large number of saddle points, and a significant portion of them are difficult to escape effectively by low-order optimization methods due to the high flatness of the local curvature (e.g., degenerate saddle points or weak positive curvature saddle points where the Hessian is all positive but third order effects are significant). Specifically, the first-order method (such as gradient descent and Adam) relies on gradient information only, cannot sense the further optimized direction in a flat area with the gradient close to zero, is easy to fall into long-term stagnation, the second-order method (such as newton method and trust area) can recognize the negative curvature direction by using the Hessian matrix to escape from a typical saddle point, but at the flat saddle point, hessian lacks effective negative characteristic values, so that a secondary model of the method cannot provide a reliable descent path, and the change trend of the curvature of a function can be captured by introducing higher derivative information (such as third-order tensor) to reveal the high-order escape direction hidden in the flat area, so that the limitation of the low-order method is broken through to find a solution with better quality. However, if the high-order tensor is continuously calculated and stored in the whole optimization process, the calculation and memory overhead which are difficult to bear are brought, and the application of the high-order tensor in the actual scene is limited. In summary, how to effectively identify and escape from flat saddle points (including degraded saddle points and saddle points with weak positive curvature) which are difficult to process by the low-order method while ensuring the calculation efficiency is a problem to be solved in the present day. Disclosure of Invention The technical problem to be solved by the invention is that the low-order optimization method in the related technology is difficult to effectively escape from saddle points with flat local curvature (such as degraded saddle points or weak positive curvature saddle points with complete positive Hessian and obvious third-order effect), so that the convergence accuracy and reliability are insufficient, and the higher-order optimization method has the problems of higher discrimination capability, high calculation complexity, high resource consumption and difficult efficient application in actual scenes. In order to solve the technical problems, the invention provides a self-adaptive multi-order cooperative non-convex optimization control method, a device and a system and a non-convex optimization problem solving method. According to a first aspect of the present invention, the adaptive multi-order cooperative non-convex optimization control method provided by the present invention includes: step S101, acquiring an objective function, and performing iterative search on a non-convex curved surface of the objective function until a first-order stable point meeting a preset first-order stable condition is obtained; Step S102, calculating second-order curvature information corresponding to the first-order stable point according to the first-order stable point, and judging second-order stable conditions to obtain a second-order stable point meeting preset second-order stable conditions; Step 103, according to a preset saddle point discrimination model, saddle point evadability discrimination is carried out on the second-order stable points, and when the discrimination result is evadability, a corresponding evadability space is determined; step S104, responding to the escape judgment result of the saddle point as an escape, and generating and verifying directional disturbance in the escape space based on the escape space and a third derivative tensor at the second-order stable point; And step 105, if the directional disturbance is verified to be effective, taking a new point after the disturbance as input, and repeatedly executing the steps S101 to S104 until the result of the escape judgment of the saddle point is non-escape. Optionally, the step of obtaining the objective function and performing iterative search on the non-convex curved surface of the objective function until a first-