CN-116972764-B - Multi-level deep learning neural network self-adaptive global deformation vision measurement method and system

CN116972764BCN 116972764 BCN116972764 BCN 116972764BCN-116972764-B

Abstract

The invention provides a multi-level deep learning neural network self-adaptive global deformation vision measurement method and system, which comprise the steps of S1, vertically aligning optical axes of a camera and a lens with the surface of a sample, recording information of the sample in different states, S2, carrying out global gridding formation function on a region of interest in an image based on the multi-level deep learning neural network so as to realize global deformation measurement, S3, optimizing parameters and node displacement of the neural network by using Adam by taking a zero-mean normalized minimum distance square standard as a loss function, wherein weights and deviations are functions of node positions, realizing grid self-adaptation by optimizing parameters of the neural network, and S4, deriving a global displacement field and a strain field according to an optimization result, so as to realize high-precision self-adaptive global deformation measurement. The invention realizes global deformation measurement, and can solve the problem of incompatibility of displacement fields in the traditional method, thereby improving the accuracy of visual deformation measurement.

Inventors

HE JI
Ren Enzhen
QIAN CHANGMING

Assignees

上海交通大学

Dates

Publication Date: 20260512
Application Date: 20230721

Claims (6)

1. A multi-level deep learning neural network self-adaptive global deformation vision measurement method is characterized by comprising the following steps: s1, vertically aligning the optical axes of a camera and a lens with the surface of a sample, and recording information of the sample in different states; Step S2, performing global gridding shaping function on the region of interest in the image based on a multi-level deep learning neural network, so as to realize global deformation measurement; S3, using a zero-mean normalized minimum distance square standard as a loss function, optimizing parameters and node displacement of the neural network by using Adam, wherein the weights and the deviations are functions of the node positions, and realizing grid self-adaption by optimizing the parameters of the neural network; s4, deriving a global displacement field and a strain field according to an optimization result to realize high-precision self-adaptive global deformation measurement; in step S2, global gridding forming functions are carried out on the interested areas in the images based on the multi-level deep learning neural network, and displacement of all nodes is determined by simultaneously matching all units, so that continuity of a displacement field is ensured; the form of the shape function is: Wherein, the Is a displacement field; The number of nodes is the grid unit; Is displacement field coordinates; coordinates of grid node i; Displacing for the grid node; Is that A network structure of a shape function of (a); the form function of the multi-level deep learning neural network is as follows: Wherein, the Weighting the middle layer of the neural network; Is neural network intermediate layer deviation; The displacement fields between the adjacent units meet the consistency requirement by sharing the nodes and the boundaries between the units, and the displacement of the adjacent units is the same on the unit boundaries and the nodes, so that the global continuity of the displacement fields is ensured; Weights for middle layers of neural networks And deviation of Is the grid node coordinates Is a function of: continuously optimizing parameters by training a neural network The weights and deviations are grid node coordinates Is optimized by a function of The coordinates of the grid nodes are optimized; The multi-level deep learning neural network is composed of three basic modules, namely a linear function module, a multiplication module and an inversion module, and different types of shape functions are generated through the three modules so as to adapt to different materials and deformation characteristics, and for bilinear element shape functions, the formula is as follows: Wherein, the Represent the first Fourth node unit (four-node unit) Of individual nodes Coordinates; Represent the first Fourth node unit (four-node unit) Of individual nodes Coordinates.
2. The method for measuring the self-adaptive global deformation vision of the multi-level deep learning neural network according to claim 1, wherein in the step S3, a zero-mean normalized minimum distance square standard ZNSSD is selected as a loss function, so that errors caused by exposure intensity changes at different moments are avoided, and the expression is as follows: In the formula, For all pixel numbers in a cell; And Respectively representing the gray value of the ith pixel of the unit in the reference image and the deformed image; And Respectively representing the average gray values of all pixels in the reference image and the target image; And Respectively representing standard deviation of gray values of all pixels in the reference image and the target image, wherein ZNSSD is 0 when the two images are completely matched; for all finite element units, ZNSSD is taken as a loss function, expressed as: Wherein, the Representing the number of finite element units; Represent the first ZNSSD of individual units.
3. The method for measuring the self-adaptive global deformation vision of the multi-level deep learning neural network according to claim 1, wherein the method is characterized in that the method is used for reducing the order in a regular tensor decomposition mode so as to improve the solving efficiency, and comprises the following specific processes: Wherein, the Coordinates of the corresponding coordinate axes of the displacement field; The order of the tensor decomposition; Is a function decomposed according to coordinate axes; For the x-axis direction: Wherein, the Representing the number of nodes along the x-axis, Is a decomposition coefficient; the form function form based on the multi-level deep learning neural network is as follows: 。
4. a multi-level deep learning neural network adaptive global deformation vision measurement system, comprising: the module M1 is used for vertically aligning the optical axes of the camera and the lens with the surface of the sample and recording the information of the sample in different states; the module M2 performs global gridding shaping function on the region of interest in the image based on a multi-level deep learning neural network, so as to realize global deformation measurement; the module M3 takes a zero-mean normalized minimum distance square standard as a loss function, optimizes parameters and node displacement of the neural network by using Adam, and realizes grid self-adaption by optimizing the parameters of the neural network, wherein the weights and the deviations are functions of the node positions; the module M4 derives a global displacement field and a strain field according to the optimization result to realize high-precision self-adaptive global deformation measurement; in a module M2, performing global gridding forming functions on the region of interest in the image based on a multi-level deep learning neural network, and determining all node displacements by simultaneously matching all units, so that the continuity of a displacement field is ensured; the form of the shape function is: Wherein, the Is a displacement field; The number of nodes is the grid unit; Is displacement field coordinates; coordinates of grid node i; Displacing for the grid node; Is that A network structure of a shape function of (a); the form function of the multi-level deep learning neural network is as follows: Wherein, the Weighting the middle layer of the neural network; Is neural network intermediate layer deviation; The displacement fields between the adjacent units meet the consistency requirement by sharing the nodes and the boundaries between the units, and the displacement of the adjacent units is the same on the unit boundaries and the nodes, so that the global continuity of the displacement fields is ensured; Weights for middle layers of neural networks And deviation of Is the grid node coordinates Is a function of: continuously optimizing parameters by training a neural network The weights and deviations are grid node coordinates Is optimized by a function of The coordinates of the grid nodes are optimized; The multi-level deep learning neural network is composed of three basic modules, namely a linear function module, a multiplication module and an inversion module, and different types of shape functions are generated through the three modules so as to adapt to different materials and deformation characteristics, and for bilinear element shape functions, the formula is as follows: Wherein, the Represent the first Fourth node unit (four-node unit) Of individual nodes Coordinates; Represent the first Fourth node unit (four-node unit) Of individual nodes Coordinates.
5. The adaptive global deformation vision measurement system of a multi-level deep learning neural network according to claim 4, wherein in the module M3, a zero-mean normalized minimum distance square standard ZNSSD is selected as a loss function, so as to avoid errors caused by exposure intensity changes at different moments, and the expression is: In the formula, For all pixel numbers in a cell; And Respectively representing the gray value of the ith pixel of the unit in the reference image and the deformed image; And Respectively representing the average gray values of all pixels in the reference image and the target image; And Respectively representing standard deviation of gray values of all pixels in the reference image and the target image, wherein ZNSSD is 0 when the two images are completely matched; for all finite element units, ZNSSD is taken as a loss function, expressed as: Wherein, the Representing the number of finite element units; Represent the first ZNSSD of individual units.
6. The adaptive global deformation vision measurement system of a multi-level deep learning neural network according to claim 4, wherein the reduction is performed by a regular tensor decomposition method, so as to improve the solving efficiency, and the specific process is as follows: Wherein, the Coordinates of the corresponding coordinate axes of the displacement field; The order of the tensor decomposition; Is a function decomposed according to coordinate axes; For the x-axis direction: Wherein, the Representing the number of nodes along the x-axis, Is a decomposition coefficient; the form function form based on the multi-level deep learning neural network is as follows: 。

Description

Multi-level deep learning neural network self-adaptive global deformation vision measurement method and system Technical Field The invention relates to the technical field of computer vision and surveying, in particular to a multi-level deep learning neural network self-adaptive global deformation vision measurement method and system. Background Visual deformation measurement is an image-based optical measurement technique that can be used to measure displacement and strain fields of an object surface. The heart of visual deformation measurement is an image matching algorithm, which can be divided into two main types, subset-based local methods and finite element-based global methods. Subset-based local methods, which were the earliest and most commonly used visual deformation measurement methods, divide a region of interest (Region of Interest, ROI) in an image into several subsets, and then perform individual image matching (also known as correlation) on each subset to determine the displacement of the center of each subset. The method has the advantages of simplicity, intuitiveness and flexibility, but the displacement continuity between subsets cannot be ensured. Patent document CN115345781a (application number: CN 202210956950.6) discloses a multi-view video stitching method based on deep learning, which includes the steps of firstly, utilizing a Airsim simulator to collect images and depth data at a set virtual common view point, generating a dataset for video stitching task, and preprocessing the images such as cylindrical projection. And then, respectively designing an artifact eliminating module and a smooth transition module by utilizing a convolutional neural network, wherein the artifact eliminating module considers the characteristic correlation of the overlapped areas, the overlapped areas are aligned by viewpoint regression to eliminate the fused artifact, and the smooth transition module transmits the obtained deformation rule of the overlapped areas to the non-overlapped areas according to the characteristic information of the images to guide the smooth transition among the areas so as to improve the visual impression. And finally, the original viewpoint image is distorted and transformed according to the predicted displacement field, and weighted linear fusion is carried out to obtain a splicing result. The finite element-based global method is a visual deformation measurement method developed in recent years, which discretizes a region of interest (ROI) in an image into several finite element units, and then performs simultaneous search and matching on all the units to determine displacements of all nodes. The method has the advantages of explicitly ensuring displacement continuity, reducing image registration errors and improving measurement accuracy, but also has the disadvantages that the manual setting of grid node positions cannot be changed, for example, so that the measurement accuracy and stability are difficult to ensure when the non-uniformly deformed object is measured. Disclosure of Invention Aiming at the defects in the prior art, the invention aims to provide a visual measurement method and a visual measurement system for self-adaptive global deformation of a multi-level deep learning neural network. The invention provides a multi-level deep learning neural network self-adaptive global deformation vision measurement method, which comprises the following steps: s1, vertically aligning the optical axes of a camera and a lens with the surface of a sample, and recording information of the sample in different states; Step S2, performing global gridding shaping function on the region of interest in the image based on a multi-level deep learning neural network, so as to realize global deformation measurement; S3, using a zero-mean normalized minimum distance square standard as a loss function, optimizing parameters and node displacement of the neural network by using Adam, wherein the weights and the deviations are functions of the node positions, and realizing grid self-adaption by optimizing the parameters of the neural network; And S4, deriving a global displacement field and a strain field according to the optimization result, and realizing high-precision self-adaptive global deformation measurement. Preferably, in step S2, global gridding formation functions are performed on the region of interest in the image based on the multi-level deep learning neural network, and displacement of all nodes is determined by simultaneously matching all units, so that continuity of a displacement field is ensured; the form of the shape function is: Wherein u e (x) is a displacement field, n is the number of grid unit nodes, x is the coordinate of the displacement field, x i is the coordinate of grid node i, and u i is the grid node displacement; a network structure that is a function of the shape of (x, x i); the form function of the multi-level deep learning neural network is as follows: Wh