CN-115937724-B - Virtual soft testing method and system for machine vision training based on digital twin

CN115937724BCN 115937724 BCN115937724 BCN 115937724BCN-115937724-B

Abstract

The embodiment of the invention provides a virtual soft testing method and a virtual soft testing system for machine vision training based on digital twinning. The method comprises the steps of obtaining an image with edge characteristics of a target object from an image database through a vision twin master core, dynamically updating a virtual model of an industrial environment in the model database, collecting customized training data for an image processing model in the dynamically updated virtual model according to customized demand configuration information by the vision twin master core, sending the customized training data to an artificial intelligent training end, and reloading the image processing model trained by the artificial intelligent training end by an embedded end to realize dynamic virtual soft testing of the industrial environment. According to the embodiment of the invention, available training data close to a real scene can be generated in the virtual model according to different industrial environment requirements, the efficiency and reliability of machine vision construction are improved, and meanwhile, the data processing and data transmission burden of the vision twin main core is reduced by the embedded end introduced in the method.

Inventors

WANG XINHANG
WANG YUCHEN
LI PEISONG
ZHANG JINHUA

Assignees

西交利物浦大学

Dates

Publication Date: 20260512
Application Date: 20221202

Claims (10)

1. A virtual soft testing method for machine vision training based on digital twinning, comprising: Preprocessing an image dynamically captured in an industrial environment by utilizing an image processing model carried by an embedded end, and inputting the preprocessed image with the edge characteristics of a target object into an image database; the visual twin master kernel acquires the image with the edge characteristics of the target object from the image database, and dynamically updates the virtual model of the industrial environment in the model database; The vision twinning host core collects customized training data for an image processing model in the dynamically updated virtual model according to the customized demand configuration information, and sends the customized training data to an artificial intelligent training end, so that the collection of the customized training data and the training of the image processing model are synchronously carried out; The embedded end reloads the image processing model trained by the artificial intelligent training end so as to realize automatic control training of machine vision in the industrial environment.
2. The method of claim 1, wherein prior to the embedded end piggybacking an image processing model, the method comprises: The visual twin master check is used for carrying out image post-processing on training images of the industrial environment input by the embedded end, wherein the image post-processing comprises cleaning and/or image segmentation and/or color gamut conversion and/or image compression and/or edge detection; Identifying object types and object characteristics of objects in the training image after the image post-processing, determining a target object, and compiling the training image into a space image of a three-dimensional point cloud; Performing machine vision virtual mapping on the space image of the target object in the three-dimensional point cloud through the vision twin master check, constructing a virtual model of the industrial environment, and storing the virtual model into a model database; The visual twin host core collects customized training data in the virtual model according to the customized demand configuration information, and sends the customized training data to an artificial intelligent training end for training of an image processing model; the artificial intelligence training end carries the trained image processing model to the embedded end.
3. The method of claim 2, wherein the machine-vision virtual mapping of the target object at the aerial image of the three-dimensional point cloud by the vision twinning master check comprises: Determining spatial information of the industrial environment by compiling an image data type of a spatial image of the target object at the three-dimensional point cloud based on a visual twin master of a just-in-time rendering engine; and constructing a virtual model of the industrial environment based on the spatial information.
4. The method of claim 1, wherein the industrial environment comprises a monitored environment of a target object, a motion control environment of a target object; When the industrial environment is a monitoring environment of a target object, the preprocessing comprises cleaning and/or image compression and/or color gamut conversion and/or edge detection; when the industrial environment is a motion control environment for a target object, the preprocessing includes image segmentation and/or edge detection.
5. The method of claim 1, wherein the customization demand configuration information is entered by a user at an interactive interface of the visual twinning host core.
6. A digital twinning-based virtual soft testing system for machine vision training, comprising: the image processing program module is used for preprocessing an image dynamically captured in an industrial environment by utilizing an image processing model carried by the embedded end, and inputting the preprocessed image with the edge characteristics of the target object into the image database; The virtual model updating program module is used for acquiring the image with the edge characteristics of the target object from the image database through the vision twin main kernel and dynamically updating the virtual model of the industrial environment in the model database; The training program module is used for the vision twinning host core to collect customized training data for the image processing model in the dynamically updated virtual model according to the customized demand configuration information, and send the customized training data to an artificial intelligent training end to synchronously collect the customized training data and train the image processing model; And the dynamic soft measurement program module is used for reloading the image processing model trained by the artificial intelligent training end by the embedded end so as to realize automatic control training of machine vision in the industrial environment.
7. The system of claim 6, wherein the system further comprises a model building program module for: The visual twin master check is used for carrying out image post-processing on training images of the industrial environment input by the embedded end, wherein the image post-processing comprises cleaning and/or image segmentation and/or color gamut conversion and/or image compression and/or edge detection; Identifying object types and object characteristics of objects in the training image after the image post-processing, determining a target object, and compiling the training image into a space image of a three-dimensional point cloud; Performing machine vision virtual mapping on the space image of the target object in the three-dimensional point cloud through the vision twin master check, constructing a virtual model of the industrial environment, and storing the virtual model into a model database; The visual twin host core collects customized training data in the virtual model according to the customized demand configuration information, and sends the customized training data to an artificial intelligent training end for training of an image processing model; the artificial intelligence training end carries the trained image processing model to the embedded end.
8. The system of claim 7, wherein the model building program module is further configured to: Determining spatial information of the industrial environment by compiling an image data type of a spatial image of the target object at the three-dimensional point cloud based on a visual twin master of a just-in-time rendering engine; and constructing a virtual model of the industrial environment based on the spatial information.
9. An electronic device comprising at least one processor and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method of any one of claims 1-5.
10. A storage medium having stored thereon a computer program, which when executed by a processor performs the steps of the method according to any of claims 1-5.

Description

Virtual soft testing method and system for machine vision training based on digital twin Technical Field The invention relates to the field of machine vision, in particular to a virtual soft testing method and system for machine vision training based on digital twinning. Background With the advancement of technology, the application of machine vision has been expanded to industrial environments in numerous fields such as agricultural production, metal processing, chemical monitoring, and pharmaceutical manufacturing. Due to the new demands of industrial automation and intelligence, the importance of machine vision is increasing, and the performance and reliability of the machine vision are also increasing. Data acquisition and artificial intelligence training are two basic parts of building machine vision. For the machine vision building paradigm in the prior art, the two parts of data acquisition and artificial intelligence training are seriously coupled, namely training data must be acquired in advance and then sent to a training end, and the trained artificial intelligence is deployed in a camera to work. Therefore, data acquisition and artificial intelligence training cannot be performed simultaneously, and the construction efficiency of machine vision must be affected. Data acquisition is very time consuming and labor intensive in many scenarios. Taking product quality inspection as an example, machine vision for quality inspection requires a large number of defective product images as training data. With the existing quality engineering technology, defective products are often picked up in many thousands, and sufficient defective product images are difficult to obtain. Therefore, the collection of defective product data often undergoes mass production, and the construction of machine vision is necessarily affected. If the environment or the target is changed, retraining is needed, and the construction efficiency of machine vision is further affected. In the prior art, in order to cope with the problem that training data is difficult to acquire, data enhancement is generally used. The data enhancement means that the effect of multiplying the training data volume is achieved by partially modifying the real image data by shielding, changing the color gamut, overturning and the like. The enhanced training set can be subjected to subsequent processing such as parameter adjustment, model adaptation and the like to optimize the final training effect. In the process of implementing the present invention, the inventor finds that at least the following problems exist in the related art: Since the new data generated by data augmentation is not a real-world situation in the physical world, the training effect on machine vision tends to be unstable. The training set with the enhanced data often brings the problem of overfitting after training due to consistent scenes, namely, the accuracy of machine vision is reliable only in specific scenes. Therefore, data enhancement remains a key disadvantage for solving the building efficiency of machine vision. Even if the training problem of machine vision is solved by using a virtual soft testing tool in the prior art, the virtual soft testing tool provides the functions of analysis, simulation, verification and the like under different situations by constructing a virtual simulation environment. In the prior art, because test hardware is often unavailable or difficult to build, the virtual soft test tool effectively simplifies the development process. In the field of machine vision, the virtual soft measurement tool can rapidly and pertinently customized extended training data through simulation, so that the problem of excessive fitting of training is effectively solved. However, due to the limited amount of data, it is difficult for the virtual soft test tool to restore mechanisms in the physical world sufficiently truly. Meanwhile, the virtual soft testing tool is not directly related to the real scene, and the updating of the virtual soft testing tool depends on manual operation and is often not periodic, so that hysteresis of data in the virtual model is necessarily caused. The construction of the virtual soft measurement tool is generally based on the knowledge, experience and cognition of engineers, so that deviation from the actual situation is not avoided. Disclosure of Invention The method aims at least to solve the problem that training data obtained by machine vision training overfitting and virtual soft measurement and reality are not directly related in the prior art are not fit with reality. In a first aspect, an embodiment of the present invention provides a digital twinning-based virtual soft testing method for machine vision training, including: Preprocessing an image dynamically captured in an industrial environment by utilizing an image processing model carried by an embedded end, and inputting the preprocessed image with the edge characteristics of a t