CN-122019359-A - Cross-platform automatic test system and method for multidimensional sensing analysis

CN122019359ACN 122019359 ACN122019359 ACN 122019359ACN-122019359-A

Abstract

The invention relates to the technical field of software testing and automatic control, and discloses a multi-dimensional perception analysis cross-platform automatic testing system and a multi-dimensional perception analysis cross-platform automatic testing method, wherein a cross-platform execution layer acquires operation interface data of the operation of an application to be tested; the multi-dimensional perception recognition analysis layer executes corresponding multi-dimensional perception analysis operation on the operation interface data according to the multi-dimensional perception analysis model to generate a candidate element set corresponding to the operation interface data, the comprehensive target positioning layer synthesizes the calculation model and the candidate element set according to the preset multi-dimensional feature similarity to determine a target element set corresponding to the operation interface data, and the cross-platform execution layer executes application test operation matched with the tested application according to the target element set and the pre-analyzed test instruction set. Therefore, the invention can improve the flexibility and the robustness of the cross-platform automatic test in a multi-mode manner.

Inventors

ZHOU ZHENBIN
LOU KE
LI JIANFENG
BIN CHAOLIN
ZHOU XIONG

Assignees

深圳鼎匠科技有限公司

Dates

Publication Date: 20260512
Application Date: 20251212

Claims (10)

1. A cross-platform automated testing system for multi-dimensional perceptual analysis, the system comprising: the cross-platform execution layer is used for acquiring the operation interface data of the operation of the tested application; The multidimensional sensing analysis layer is used for executing corresponding multidimensional sensing analysis operation on the operation interface data according to a multidimensional sensing analysis model to generate a candidate element set corresponding to the operation interface data, wherein the multidimensional sensing analysis operation comprises at least two operations of character recognition and analysis, semantic understanding and analysis, icon recognition and analysis, interface layout recognition and analysis and action intention prediction and analysis; The comprehensive target positioning layer is used for determining a target element set corresponding to the operation interface data according to a preset multidimensional feature similarity comprehensive calculation model and the candidate element set; the cross-platform execution layer is further configured to execute an application test operation adapted to the tested application according to the target element set and the pre-parsed test instruction set.
2. The cross-platform automated testing system of multi-dimensional perceptual analysis of claim 1, wherein the multi-dimensional perceptual analysis model comprises at least a multi-dimensional feature recognition analysis model, a multi-dimensional semantic understanding analysis model, and an operation intention prediction analysis model, the multi-dimensional perceptual analysis model is matched with the operation, the multi-dimensional perceptual recognition analysis layer performs a corresponding multi-dimensional perceptual analysis operation on the operation interface data according to the multi-dimensional perceptual analysis model, and a specific manner of generating the candidate element set corresponding to the operation interface data comprises: Performing interface element recognition and image denoising processing on the operation interface data according to the multidimensional feature recognition analysis model to obtain a primary element recognition result; Performing multi-mode semantic understanding operation on the primary element recognition result according to the multi-dimensional semantic understanding analysis model so as to integrate icon detection, layout analysis and semantic partition characteristics and generate a semantic enhancement element set; And according to the operation intention prediction analysis model and action intention instructions in the pre-analyzed test instruction set, carrying out intention matching and confidence calculation on the semantic enhancement element set to generate a candidate element set corresponding to the operation interface data, wherein each candidate element in the candidate element set comprises element attributes, element position coordinates and confidence scores corresponding to the candidate element set.
3. The multi-dimensional perceptual analysis cross-platform automated test system of claim 1, wherein the determining, by the comprehensive target positioning layer, the specific manner of the target element set corresponding to the operation interface data according to the preset multi-dimensional feature similarity comprehensive calculation model and the candidate element set comprises: Extracting, for each candidate element in the candidate element set, similarity information between the candidate element and target element description data in a test instruction set analyzed in advance in a plurality of feature dimensions, wherein the feature dimensions at least comprise text features, local image features and semantic features, the similarity information in the text features is used for representing the matching degree between the candidate element and the target element description data in a text content dimension, the similarity information in the local image features is used for representing the matching degree between the candidate element and the target element description data in an element visual appearance dimension, and the similarity information in the semantic features is used for representing the semantic association degree between the candidate element and the target element description data in an interface context dimension; Inputting all the extracted similarity information into a preset multidimensional feature similarity comprehensive calculation model to carry out weighted fusion calculation, wherein the weighted fusion calculation dynamically adjusts weight distribution of each feature dimension based on interface context so as to generate a comprehensive similarity score of each element; And determining a target element set corresponding to the operation interface data according to the comprehensive similarity score and a preset confidence threshold.
4. The multi-dimensional perceptual analysis cross-platform automated test system of claim 1, wherein the cross-platform execution layer performs application test operations adapted to the application under test according to the target element set and the pre-parsed test instruction set in a specific manner comprising: converting element position coordinates in the target element set into normalized coordinate representations irrelevant to equipment, wherein the normalized coordinate representations adapt to different screen resolutions and special-shaped screen display characteristics through safe area correction processing; And calling a platform adaptation component matched with the tested application running platform according to operation types and parameters defined in a pre-analyzed test instruction set, mapping the normalized coordinate representation into executable operation instructions of a specific platform corresponding to the tested application, so as to execute application test operation adapted to the tested application, wherein the operation types comprise at least one of clicking, sliding, long-pressing, double-clicking, dragging, zooming, text input and gesture operation.
5. The multi-dimensional, perceptive analytical, cross-platform automated test system of any of claims 1-4, further comprising: an assertion check layer, configured to capture, in real time, multi-dimensional feedback data during execution of the application test operation, where the multi-dimensional feedback data includes at least one of interface response state data, an operation track sequence, web request log data, system event log data, and performance index data; The assertion checking layer is further configured to compare and verify the multidimensional feedback data with preset assertion conditions in the test instruction set to obtain a multidimensional verification result; And the assertion checking layer is further used for generating a traceable test report of the application test operation according to the multi-dimensional verification result.
6. The cross-platform automated testing system of multi-dimensional perceptual analysis of claim 5, wherein the specific manner in which the assertion verification layer compares and verifies the multi-dimensional feedback data with preset assertion conditions in the test instruction set to obtain a multi-dimensional verification result comprises: performing cross-dimension association analysis on the multi-dimensional feedback data to obtain inter-dimension association rules, wherein the inter-dimension association rules are used for representing statistical dependence or time sequence relation among different dimension feedback data, and the inter-dimension association rules comprise at least one of causal relation rules, time sequence constraint rules and statistical correlation rules; generating a self-adaptive verification condition set according to the inter-dimension association rule and the preset assertion condition, wherein the self-adaptive verification condition set is used for dynamically adjusting the severity or priority of verification, and comprises at least one of a dynamic threshold condition, a verification priority rule and a condition combination rule; And carrying out layered verification on the multidimensional feedback data based on the self-adaptive verification condition set to generate a multidimensional verification result.
7. The multi-dimensional, perceptive-analyzed, cross-platform automated test system of claim 6, said system further comprising: the self-adaptive closed loop layer is used for generating a test adaptation strategy according to the multi-dimensional verification result and the test instruction set, wherein the test adaptation strategy is used for representing a self-adaptive adjustment rule aiming at a test execution process, the test adaptation strategy is obtained by analyzing the association relation between a verification abnormal event in the multi-dimensional verification result and the test instruction set, and the test adaptation strategy comprises at least one of a test step skipping condition, a retry triggering condition and an execution sequence optimizing rule; the self-adaptive closed loop layer is further used for dynamically reconstructing the test instruction set based on the test adaptation strategy to generate a self-adaptive test instruction sequence, and the dynamic reconstruction is realized by modifying execution logic, parameters or sequences of the test instructions; The self-adaptive closed loop layer is further used for re-executing the application test operation according to the self-adaptive test instruction sequence, and the application test operation adapts to the dynamic behavior change of the tested application through the closed loop control of the self-adaptive test instruction sequence and the real-time test feedback.
8. A cross-platform automated testing method for multidimensional sensing analysis, the method comprising: acquiring operation interface data of the operation of the tested application; According to a multidimensional sensing analysis model, corresponding multidimensional sensing analysis operation is carried out on the operation interface data, a candidate element set corresponding to the operation interface data is generated, and the multidimensional sensing analysis operation comprises at least two operations of character recognition and analysis, semantic understanding and analysis, icon recognition and analysis, interface layout recognition and analysis and action intention prediction and analysis; Determining a target element set corresponding to the operation interface data according to a preset multidimensional feature similarity comprehensive calculation model and the candidate element set; and executing application test operation matched with the tested application according to the target element set and the pre-analyzed test instruction set.
9. A cross-platform automated testing system for multi-dimensional perceptual analysis, the system comprising: a memory storing executable program code; a processor coupled to the memory; The processor invokes the executable program code stored in the memory to perform the cross-platform automated test method of multi-dimensional perceptual analysis of claim 8.
10. A computer storage medium having stored thereon computer instructions which, when invoked, are operable to perform the cross-platform automated test method of multi-dimensional perceptual analysis of claim 8.

Description

Cross-platform automatic test system and method for multidimensional sensing analysis Technical Field The invention relates to the technical field of software testing and automatic control, in particular to a cross-platform automatic testing system and method for multidimensional sensing analysis. Background With the rapid development of mobile internet and internet of things, various Application software (Application) needs to run stably on various hardware devices and operating systems. To ensure functional consistency, stability and user experience of applications in different platform environments, automated testing techniques have become an indispensable part of the software development lifecycle. Conventional automated testing methods, such as technologies based on control tree recognition (e.g., UI Automator of Android, XCUITest of iOS) or based on image recognition (e.g., sikuliX), face significant challenges in practical cross-platform testing. Control tree based methods rely heavily on the accessibility interface provided by a particular platform to obtain properties (e.g., ID, text, coordinates) of the interface element. However, in a cross-platform scenario, interface elements of the same application often have different internal identifiers on different platforms, and for games, applications using custom controls in large quantities, or partial Hybrid development (Hybrid) applications, control tree information may be incomplete or unavailable, resulting in test scripts that cannot be multiplexed across platforms, and high maintenance costs. And based on a fixed coordinate or static image template matching method, different screen resolutions, proportions and dynamically-changed interface contents are difficult to adapt, and the robustness is poor. It is important to provide a technical solution for improving flexibility and robustness of cross-platform automated testing. Disclosure of Invention The invention provides a multi-dimensional perception analysis cross-platform automatic test system and a multi-dimensional perception analysis cross-platform automatic test method, which can improve flexibility and robustness of cross-platform automatic test. To solve the above technical problems, a first aspect of the present invention discloses a cross-platform automated testing system for multidimensional sensing analysis, the system comprising: the cross-platform execution layer is used for acquiring the operation interface data of the operation of the tested application; The multidimensional sensing analysis layer is used for executing corresponding multidimensional sensing analysis operation on the operation interface data according to a multidimensional sensing analysis model to generate a candidate element set corresponding to the operation interface data, wherein the multidimensional sensing analysis operation comprises at least two operations of character recognition and analysis, semantic understanding and analysis, icon recognition and analysis, interface layout recognition and analysis and action intention prediction and analysis; The comprehensive target positioning layer is used for determining a target element set corresponding to the operation interface data according to a preset multidimensional feature similarity comprehensive calculation model and the candidate element set; the cross-platform execution layer is further configured to execute an application test operation adapted to the tested application according to the target element set and the pre-parsed test instruction set. In an optional implementation manner, in a first aspect of the present invention, the multidimensional sensing analysis model at least includes a multidimensional feature recognition analysis model, a multidimensional semantic understanding analysis model and an operation intention prediction analysis model, the multidimensional sensing analysis model is matched with the operation, the multidimensional sensing recognition analysis layer executes a corresponding multidimensional sensing analysis operation on the operation interface data according to the multidimensional sensing analysis model, and a specific manner of generating the candidate element set corresponding to the operation interface data includes: Performing interface element recognition and image denoising processing on the operation interface data according to the multidimensional feature recognition analysis model to obtain a primary element recognition result; Performing multi-mode semantic understanding operation on the primary element recognition result according to the multi-dimensional semantic understanding analysis model so as to integrate icon detection, layout analysis and semantic partition characteristics and generate a semantic enhancement element set; And according to the operation intention prediction analysis model and action intention instructions in the pre-analyzed test instruction set, carrying out intention matching and confi