Search

CN-121998858-A - Automatic 3D HMI (human machine interface) testing method based on comparison of AI (advanced technology attachment) video stream and frame by frame

CN121998858ACN 121998858 ACN121998858 ACN 121998858ACN-121998858-A

Abstract

The invention relates to the technical field of 3D HMI automatic test, in particular to a 3D HMI automatic test method based on AI video stream and frame-by-frame comparison, which is characterized in that an industrial camera or ADB is used for collecting pictures and video data of a tested 3D HMI, after denoising, enhancing and decoding pretreatment, 3D characteristics and objects are identified by using an adaptation algorithm and a training model of an AI video stream processing unit, the skin rendering effect is verified by quantifying the color ratio by an image identification module, the frames are aligned by a video frame-by-frame comparison unit, and the dynamic content is evaluated by setting a threshold value; the method solves the problem of misjudgment and large error of the traditional image comparison under 3D and dynamic scenes, has obvious advantages in the aspects of dynamic interaction, space verification and anti-interference capability, is suitable for complex scenes such as vehicle-mounted instruments and the like, and remarkably improves the test precision and efficiency.

Inventors

  • MA YINGHAO
  • WANG BOYU

Assignees

  • 沈阳东信创智科技有限公司

Dates

Publication Date
20260508
Application Date
20260128

Claims (10)

  1. 1. The automatic 3D HMI testing method based on the comparison of the AI video stream and the frame by frame is characterized by comprising the following steps: s1, acquiring screen pictures and video data of a tested 3D HMI through an image acquisition unit; s2, denoising, enhancing and decoding preprocessing is carried out on the acquired picture and video data through a preprocessing unit; S3, processing the preprocessed picture data through an AI video stream processing unit, training to generate a data model, and identifying 3D features of the detected 3DHMI based on the model; S4, comparing the preprocessed video data with the standard video frame by frame through a video frame by frame comparison unit, and evaluating the dynamic content effect; S5, realizing operation-display-response full-link closed-loop test by controlling the external execution component in linkage with the cooperative unit; S6, generating a multi-dimensional test report through a test report generating unit.
  2. 2. The method for automated testing of 3D HMI based on AI video stream versus frame-by-frame alignment of claim 1 wherein step S3 comprises: The method comprises the steps of carrying out algorithm adaptation processing on preprocessed picture data, wherein the algorithm adaptation processing comprises the steps of adopting a template matching algorithm when image similarity or positioning target change is required to be calculated, adopting an image segmentation method when a key region and other regions are required to be separated, adopting a high-precision matching algorithm when the position and the gesture of a part of a measured object are required to be identified, and adopting a color identification algorithm when a specific color region is required to be identified; integrating a plurality of preprocessed images processed by an algorithm into a data set, marking the positions of elements, adding element labels, inputting the data set into an AI training platform for training to generate a data model; By intercepting the pixel area of the identified object, the edge characteristics, the behavior characteristics and the color characteristics of the pixel area are analyzed, so that the 3D characteristics are accurately identified, and the effect of the angle of the identified object is avoided.
  3. 3. The automated test method of 3D HMI based on AI video streaming versus frame-by-frame according to claim 1, further comprising, after step S3, when the test scene involves 3D model skin rendering: The preprocessed image is converted into an HSV space or a Lab space from an RGB space through an image recognition module, color intervals are divided, the color duty ratio of each interval is calculated, the color duty ratio is subjected to difference comparison with the color duty ratio of a standard rendering effect, and the skin rendering effect is verified.
  4. 4. The AI video stream versus frame-by-frame alignment based 3D HMI automated test method of claim 1 wherein in step S4 the frame-by-frame alignment comprises: Aligning the video to be detected with the corresponding standard video in the standard video database according to the time stamp or the key frame; setting an independent similarity threshold or a comprehensive score threshold for the characteristics of each frame of the video; And (5) counting the proportion of continuous matching frames of the video to be tested and the standard video or analyzing the similarity of sequence modes, and evaluating the dynamic content effect.
  5. 5. A 3D HMI automation test system based on AI video stream and frame-by-frame alignment for implementing the AI video stream and frame-by-frame alignment based 3D HMI automation test method of any one of claims 1-4, comprising: The image acquisition unit is used for acquiring screen pictures and video data of the detected 3D HMI; The preprocessing unit is used for denoising, enhancing and decoding the picture and video data acquired by the image acquisition unit; the AI video stream processing unit is used for carrying out algorithm adaptation processing, data set construction and model training on the preprocessed picture data, and carrying out AI identification on the 3D characteristics of the detected 3D HMI based on the trained model; The image recognition module is used for verifying the skin rendering effect of the 3D model in the detected 3D HMI; The video frame-by-frame comparison unit is used for carrying out frame-by-frame comparison and effect evaluation on the dynamic content of the detected 3D HMI; The control and coordination unit is used for integrating test cases, and linking the units and the external execution component to realize full-link closed-loop test; And the test report generating unit is used for generating a test report containing the space effect, the dynamic effect and the interaction logic.
  6. 6. The AI video stream versus frame-by-frame comparison based 3D HMI automated test system of claim 5 wherein the image acquisition unit comprises an industrial camera and/or ADB screenshot module, the video data being synchronously acquired by a video stream capture module.
  7. 7. The AI video stream-based frame-by-frame comparison 3D HMI automated test system of claim 5 wherein the AI video stream processing unit comprises an algorithm adaptation module, a dataset construction and model training module, and a feature recognition module; The algorithm adaptation module is used for selecting a template matching algorithm, an image segmentation method, a high-precision matching algorithm or a color recognition algorithm according to the test scene; The data set construction and model training module is used for marking the positions of the elements of the image after the algorithm processing and adding labels to form a data set and then training to generate a data model; The feature recognition module is used for intercepting pixel areas of the recognized objects and analyzing edge features, behavior features and color features.
  8. 8. The AI video stream versus frame-by-frame comparison based 3D HMI automated test system of claim 5, wherein the image recognition module comprises a color space conversion sub-module, a color duty cycle statistics sub-module, and a difference comparison sub-module; The color space conversion sub-module is used for converting the preprocessed image from an RGB space to an HSV space or a Lab space; The color occupation ratio statistics sub-module is used for dividing color intervals and calculating the color occupation ratio of each interval; And the difference comparison submodule is used for carrying out difference calculation on the color duty ratio of the detected image and the color duty ratio of the standard rendering effect.
  9. 9. The AI video stream and frame-by-frame comparison based 3D HMI automation test system of claim 5, wherein the video frame-by-frame comparison unit comprises a standard video database, a frame alignment module, a threshold setting module, and a dynamic assessment module; the standard video database covers various dynamic scenes of 3 DHMI; The frame alignment module is used for aligning the video to be detected with the standard video according to the time stamp or the key frame; The threshold setting module is used for setting an independent similarity threshold or a comprehensive score threshold for the characteristics of the video frames; the dynamic evaluation module is used for counting the proportion of continuous matched frames or analyzing the similarity of sequence modes and evaluating the overall similarity of videos.
  10. 10. The AI video stream and frame-by-frame alignment based 3D HMI automated test system of claim 5 wherein the external execution unit comprises a robotic arm and a bus tool, the control and co-ordination unit is configured to co-operate with the robotic arm to simulate user operation and co-operate with an ADB command while controlling the bus tool to monitor bus signals.

Description

Automatic 3D HMI (human machine interface) testing method based on comparison of AI (advanced technology attachment) video stream and frame by frame Technical Field The invention relates to the technical field of 3D HMI automatic testing, in particular to a 3D HMI automatic testing method based on comparison of AI video streams and frame by frame. Background At present, the main method of HMI automatic test is to capture the picture of the display screen by a camera and analyze the difference before and after picture switching by using the image recognition technology. The method can effectively identify the static animation-free HMI screen, is an HMI automatic testing means widely applied in the industry at present, and provides a basic solution for HMI testing in static scenes. However, when the test scene relates to 3D effects or dynamic contents (such as a rotation model and a gradual change animation), the traditional image comparison method has obvious limitation due to dynamic changes of the picture contents, so that the identification error is obviously increased, even misjudgment occurs, the accurate test requirement of the 3D HMI in the complex scene cannot be met, and the problems of 3D effect deviation, dynamic content blocking or rendering error and the like are difficult to effectively identify. Disclosure of Invention The invention aims to provide a 3D HMI automatic test method based on AI video stream and frame-by-frame comparison, so as to solve the problems in the background technology. In order to achieve the above purpose, the present invention provides the following technical solutions: A3D HMI automatic test method based on AI video stream and frame-by-frame comparison comprises the following steps: s1, acquiring screen pictures and video data of a tested 3D HMI through an image acquisition unit; s2, denoising, enhancing and decoding preprocessing is carried out on the acquired picture and video data through a preprocessing unit; S3, processing the preprocessed picture data through an AI video stream processing unit, training to generate a data model, and identifying 3D features of the detected 3DHMI based on the model; S4, comparing the preprocessed video data with the standard video frame by frame through a video frame by frame comparison unit, and evaluating the dynamic content effect; S5, realizing operation-display-response full-link closed-loop test by controlling the external execution component in linkage with the cooperative unit; S6, generating a multi-dimensional test report through a test report generating unit. As a further scheme of the invention, the step S3 comprises the following steps: The method comprises the steps of carrying out algorithm adaptation processing on preprocessed picture data, wherein the algorithm adaptation processing comprises the steps of adopting a template matching algorithm when image similarity or positioning target change is required to be calculated, adopting an image segmentation method when a key region and other regions are required to be separated, adopting a high-precision matching algorithm when the position and the gesture of a part of a measured object are required to be identified, and adopting a color identification algorithm when a specific color region is required to be identified; integrating a plurality of preprocessed images processed by an algorithm into a data set, marking the positions of elements, adding element labels, inputting the data set into an AI training platform for training to generate a data model; By intercepting the pixel area of the identified object, the edge characteristics, the behavior characteristics and the color characteristics of the pixel area are analyzed, so that the 3D characteristics are accurately identified, and the effect of the angle of the identified object is avoided. As a further aspect of the present invention, when the test scene relates to 3D model skin rendering, the method further comprises, after step S3: The preprocessed image is converted into an HSV space or a Lab space from an RGB space through an image recognition module, color intervals are divided, the color duty ratio of each interval is calculated, the color duty ratio is subjected to difference comparison with the color duty ratio of a standard rendering effect, and the skin rendering effect is verified. As a further aspect of the present invention, in step S4, the frame-by-frame comparison includes: Aligning the video to be detected with the corresponding standard video in the standard video database according to the time stamp or the key frame; setting an independent similarity threshold or a comprehensive score threshold for the characteristics of each frame of the video; And (5) counting the proportion of continuous matching frames of the video to be tested and the standard video or analyzing the similarity of sequence modes, and evaluating the dynamic content effect. The 3D HMI automatic test system based on the compari