CN-121981296-A - Model uncertainty optimization method, device, processing equipment, product and medium

CN121981296ACN 121981296 ACN121981296 ACN 121981296ACN-121981296-A

Abstract

The embodiment of the application provides a model uncertainty optimization method, a model uncertainty optimization device, processing equipment, a product and a medium, which are applied to the technical field of artificial intelligence. The method comprises the steps of carrying out feedforward operation on training data aiming at an industrial detection scene through a quantized deep learning model, obtaining first moment and second moment of an activation variable of each hidden layer in the quantized deep learning model, determining feedforward loss and uncertainty estimated values of the quantized deep learning model according to the first moment and the second moment of the activation variable of each hidden layer, and updating quantization weights of the quantized deep learning model according to the feedforward loss and the uncertainty estimated values to obtain a target quantized deep learning model, wherein the target quantized deep learning model is used for detecting related equipment or products in the industrial detection scene. By adopting the method, the problem of poor detection effect caused by inaccurate uncertainty estimation of the quantized deep learning model facing the industrial detection scene can be solved.

Inventors

LIU ZIANG
ZHANG SHAOQUN
WANG ZHAO
LIANG SHUANG
LV YAN
MA XU

Assignees

中国移动紫金(江苏)创新研究院有限公司
中国移动通信集团江苏有限公司
中国移动通信集团有限公司
南京大学

Dates

Publication Date: 20260505
Application Date: 20260126

Claims (10)

1. A method of model uncertainty optimization, the method comprising: Performing feedforward operation on training data aiming at an industrial detection scene through a quantized deep learning model, and acquiring a first moment and a second moment of an activation variable of each hidden layer in the quantized deep learning model; determining feedforward loss and uncertainty estimation values of the quantized deep learning model according to the first moment and the second moment of the activation variable of each hidden layer; And updating the quantization weight of the quantized deep learning model according to the feedforward loss and the uncertainty estimation value to obtain a target quantized deep learning model, wherein the target quantized deep learning model is used for detecting related equipment or products in the industrial detection scene.
2. The model uncertainty optimization method of claim 1, wherein the activation variables comprise a pre-activation variable and a post-activation variable; obtaining a first moment and a second moment of an activation variable of each hidden layer in the quantized deep learning model comprises: According to the interlayer sequence of each hidden layer in the quantized deep learning model, aiming at each hidden layer, performing linear transformation on the first moment and the second moment of the first variable received by the hidden layer to obtain the first moment and the second moment of the variable before activation of the hidden layer, and performing nonlinear transformation on the first moment and the second moment of the variable before activation of the hidden layer to obtain the first moment and the second moment of the variable after activation of the hidden layer; Wherein, in the case that the hidden layer is a first hidden layer, the first variable is an input variable, and the first hidden layer is a first hidden layer in the quantized deep learning model; and under the condition that the hidden layer is a second hidden layer, the first variable is an activated variable of the last hidden layer of the second hidden layer, and the second hidden layer is any hidden layer except the first hidden layer in the quantized deep learning model.
3. The model uncertainty optimization method according to claim 1, wherein updating the quantization weights of the quantized deep learning model according to the feedforward loss and the uncertainty estimation value to obtain a target quantized deep learning model comprises: based on the feedforward loss and gradient descent algorithm, updating the quantization weight of the quantized deep learning model; Updating a cut-off threshold value of the quantized deep learning model according to the quantization weight; obtaining a target cutoff threshold by solving an uncertainty optimization problem, the uncertainty optimization problem targeting at least minimizing an uncertainty estimate of the quantized deep learning model; updating the quantization weight of the quantized deep learning model according to the target cut-off threshold; repeatedly executing the first step to the second step until a preset condition is met, and obtaining the target quantized deep learning model; Wherein the first step is the step of updating the quantization weights of the quantized deep learning model based on the feedforward loss and gradient descent algorithm; the second step is a step of updating the quantization weight of the quantized deep learning model according to the target cutoff threshold; the preset conditions include that the quantized deep learning model tends to converge and/or reaches a maximum number of iterations.
4. A model uncertainty optimization method according to claim 3, characterized in that the uncertainty optimization problem is constructed based on a first moment alignment function and a second moment alignment function; the first moment alignment function is used for minimizing the difference between the quantized deep learning model and the average value of the last layer of activation variables of the full-precision model; The second moment alignment function is used for minimizing the difference between variances of the quantized deep learning model and the last layer of activation variables of the full-precision model; The quantized deep learning model is a model obtained by performing weight quantization processing on the full-precision model.
5. A model uncertainty optimization method according to claim 3, wherein said obtaining a target cutoff threshold by solving said uncertainty optimization problem comprises: The target cutoff threshold is determined within a predefined search range using a grid search method.
6. A model uncertainty optimization method as claimed in claim 3, wherein said updating a cutoff threshold of said quantized deep learning model according to said quantization weights comprises: According to the formula Updating a cut-off threshold of the quantized deep learning model; Wherein, the Representing a truncated threshold of the quantized deep learning model after updating, A model parameter matrix representing the quantized deep learning model prior to the s+1st training, Representation of The number of elements in the list.
7. A model uncertainty optimization apparatus, the apparatus comprising: The first processing module is used for carrying out feedforward operation on training data aiming at an industrial detection scene through a quantized deep learning model and obtaining a first moment and a second moment of an activation variable of each hidden layer in the quantized deep learning model; The second processing module is used for determining feedforward loss and uncertainty estimation values of the quantized deep learning model according to the first moment and the second moment of the activation variable of each hidden layer; And the third processing module is used for updating the quantization weight of the quantized deep learning model according to the feedforward loss and the uncertainty estimation value to obtain a target quantized deep learning model, and the target quantized deep learning model is used for detecting related equipment or products in the industrial detection scene.
8. A processing device comprising a processor, a memory and a program stored on the memory and executable on the processor, the program when executed by the processor implementing the model uncertainty optimization method of any of claims 1 to 6.
9. A computer program product comprising computer instructions which, when executed by a processor, implement the steps of the model uncertainty optimization method according to any one of claims 1 to 6.
10. A readable storage medium, characterized in that it has stored thereon a program, which when executed by a processor, implements the steps in the model uncertainty optimization method according to any of claims 1 to 6.

Description

Model uncertainty optimization method, device, processing equipment, product and medium Technical Field The application relates to the technical field of artificial intelligence, in particular to a model uncertainty optimization method, a model uncertainty optimization device, a model uncertainty optimization processing device, a model uncertainty optimization product and a model uncertainty optimization medium. Background With the continuous advancement of industrial intelligent and digital manufacturing, deep learning technology has been widely applied to the field of industrial detection, including key links such as product appearance defect detection, quality classification, equipment state monitoring, and abnormal behavior recognition. Compared with manual detection, the automatic detection system based on deep learning has remarkable advantages in detection precision, efficiency and consistency, and has gradually become a core technical support in an industrial production line. Industrial detection scenarios generally place more stringent requirements on the real-time performance, stability and interpretability of the system, and are often deployed on edge devices or embedded platforms with limited computational effort, storage and energy consumption, which constitutes a significant constraint on the scale and computational complexity of the deep learning model. Weight quantization is one of the main technologies for realizing the lightweight deployment of deep learning at present, and researches show that the quantized deep learning realizes model compression at least 16 times and reasoning acceleration 4 times, and can meet the requirements of industrial sites on low time delay and low resource consumption. However, the quantization process inevitably introduces discrete errors and numerical approximation deviations, resulting in problems of output fluctuations, predictive confidence distortion, and increased sensitivity to noise and distribution variations in the model during reasoning. Currently, in high-reliability applications such as industrial detection, the uncertainty of the model is effectively described simply by relying on the model prediction result, so that false detection, missing detection or risk decision is easily caused, and the production safety and the product quality are further affected. Disclosure of Invention The application aims to provide a model uncertainty optimization method, a device, processing equipment, a product and a medium, which are used for solving the problem that a quantized deep learning model oriented to an industrial detection scene is poor in detection effect due to inaccurate uncertainty estimation. One embodiment of the present application provides a model uncertainty optimization method, which includes: Performing feedforward operation on training data aiming at an industrial detection scene through a quantized deep learning model, and acquiring a first moment and a second moment of an activation variable of each hidden layer in the quantized deep learning model; determining feedforward loss and uncertainty estimation values of the quantized deep learning model according to the first moment and the second moment of the activation variable of each hidden layer; And updating the quantization weight of the quantized deep learning model according to the feedforward loss and the uncertainty estimation value to obtain a target quantized deep learning model, wherein the target quantized deep learning model is used for detecting related equipment or products in the industrial detection scene. Optionally, the activation variables comprise a pre-activation variable and a post-activation variable; obtaining a first moment and a second moment of an activation variable of each hidden layer in the quantized deep learning model comprises: According to the interlayer sequence of each hidden layer in the quantized deep learning model, aiming at each hidden layer, performing linear transformation on the first moment and the second moment of the first variable received by the hidden layer to obtain the first moment and the second moment of the variable before activation of the hidden layer, and performing nonlinear transformation on the first moment and the second moment of the variable before activation of the hidden layer to obtain the first moment and the second moment of the variable after activation of the hidden layer; Wherein, in the case that the hidden layer is a first hidden layer, the first variable is an input variable, and the first hidden layer is a first hidden layer in the quantized deep learning model; and under the condition that the hidden layer is a second hidden layer, the first variable is an activated variable of the last hidden layer of the second hidden layer, and the second hidden layer is any hidden layer except the first hidden layer in the quantized deep learning model. Optionally, the updating the quantization weight of the quantized deep lea