US-12625858-B2 - Automated user interface testing with machine learning

Abstract

Systems and methods are provided for implementing automated user interface testing with integrated machine learning models. Systems and methods for detecting and preemptively correcting flow path errors are disclosed. Systems and methods for minimizing user input and optimizing testing efficiency are disclosed. A result dashboard is disclosed in which testing results and errors are displayed and a user may interact with interactive testing reports.

Inventors

• Harsh Sharma
• Yogendra Singh KATHERIA
• Seerajudeen Sheik AHAMED
• Rajiv RAMANJANI
• Shefali Garg

Assignees

• FIDELITY INFORMATION SERVICES, LLC

Dates

Publication Date

20260512

Application Date

20241028

Priority Date

20210820

Claims (20)

1. 1 . A system for implementing an automated user interface testing service, comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the system to perform operations comprising: training, by a computer, a neural network model based on input data to generate a trained neural network; receiving a uniform resource locator; extracting user interface data of a user interface element identified by the uniform resource locator; detecting an error in the extracted user interface data using the trained neural network; and based on detecting the error using the trained neural network, automatically correcting the error in the extracted user interface data, thereby updating the user interface element.
2. 2 . The system of claim 1 , wherein the extracting includes extracting user interface elements from a website identified by the uniform resource locator.
3. 3 . The system of claim 1 , wherein the extracting includes: retrieving Xpaths associated with the uniform resource locator.
4. 4 . The system of claim 3 , wherein the operations further comprise: determining a probability score based on a comparison of two or more Xpaths; and updating the training of the neural network model based on the probability score.
5. 5 . The system of claim 1 , wherein the detecting further includes detecting one or more changes in the user interface element using the trained neural network.
6. 6 . The system of claim 5 , wherein: the detecting further includes retrieving, from a database, one or more update attributes associated with the one or more changes in the user interface element; and the automatically correcting further includes updating the user interface element to incorporate each of the one or more update attributes.
7. 7 . The system of claim 1 , wherein: the detecting further includes detecting whether the uniform resource locator includes a flow path error; and the automatically correcting further includes correcting the flow path error.
8. 8 . The system of claim 1 , wherein the operations further comprise: testing the updated user interface element.
9. 9 . The system of claim 8 , wherein the operations further comprise: displaying results of testing the updated user interface element in a results dashboard on a user device.
10. 10 . The system of claim 9 , wherein the results dashboard includes: one or more interactive features which a user may manipulate; and one or more graphical result elements that are based on the user's selection of the one or more interactive features.
11. 11 . A method for implementing an automated user interface testing service, comprising: training, by a computer, a neural network model based on input data to generate a trained neural network; receiving a uniform resource locator; extracting user interface data of a user interface element identified by the uniform resource locator; detecting an error in the extracted user interface data using the trained neural network; and based on detecting the error using the trained neural network, automatically correcting the error in the extracted user interface data, thereby updating the user interface element.
12. 12 . The method of claim 11 , wherein the extracting includes extracting user interface elements from a website identified by the uniform resource locator.
13. 13 . The method of claim 11 , wherein the extracting includes: retrieving Xpaths associated with the uniform resource locator.
14. 14 . The method of claim 13 , further comprising: determining a probability score based on a comparison of two or more Xpaths; and updating the training of the neural network model based on the probability score.
15. 15 . The method of claim 11 , wherein the detecting further includes detecting one or more changes in the user interface element using the trained neural network.
16. 16 . The method of claim 15 , wherein: the detecting further includes retrieving, from a database, one or more update attributes associated with the one or more changes in the user interface element; and the automatically correcting further includes updating the user interface element to incorporate each of the one or more update attributes.
17. 17 . The method of claim 11 , wherein: the detecting further includes detecting whether the uniform resource locator includes a flow path error; and the automatically correcting further includes correcting the flow path error.
18. 18 . The method of claim 11 , further comprising: testing the updated user interface element; displaying results of testing the updated user interface element in a results dashboard on a user device, wherein the results dashboard includes: one or more interactive features which a user may manipulate; and one or more graphical result elements that are based on the user's selection of the one or more interactive features.
19. 19 . A non-transitory computer-readable medium including instructions executable by one or more processors to carry out operations comprising: training, by a computer, a neural network model based on input data to generate a trained neural network; receiving a uniform resource locator; extracting user interface data of a user interface element identified by the uniform resource locator; detecting an error in the extracted user interface data using the trained neural network; and based on detecting the error using the trained neural network, automatically correcting the error in the extracted user interface data, thereby updating the user interface element.
20. 20 . The non-transitory computer-readable medium of claim 19 , wherein the extracting includes extracting user interface elements from a website identified by the uniform resource locator.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. patent application Ser. No. 18/345,774, filed Jun. 30, 2023, which is a continuation of U.S. patent application Ser. No. 17/492,236, filed Oct. 1, 2021, currently patented as U.S. Pat. No. 11,741,074 issued August 29, 2023, which claims priority under 35 U.S.C. § 119 to Indian Patent Application No. 202111037812, filed on Aug. 20, 2021, the disclosures of which are expressly incorporated herein by reference in their entirety. TECHNICAL FIELD The present disclosure generally relates to systems and methods for implementing automated user interface testing. In particular, embodiments of the present disclosure relate to inventive and unconventional systems for integrating artificial intelligence and various service modules into a testing system. BACKGROUND Current automation testing tools are tightly coupled with particular automation testing frameworks and require scripts to be executed regularly using methodologies suitable only for a specific automation context or use. Likewise, automation frameworks often target only a single user base. As a consequence, current automation testing tools can be cumbersome, lead to cost and processing inefficiencies, require customized compatibility tools, and are often inaccessible except to the most experience users. Thus, testing tool implementers are currently forced to expend considerable time and resources to hire or train personnel with specialized programming knowledge to write automation scripts. Furthermore, even experienced personnel may be required to expend time and effort managing and operating automation testing tools, because analyzing an application user interface, identifying necessary locators, backtracking changes in a user interface to make corresponding script modification, and writing scripts for automation are each time-consuming tasks that may be necessary in modifying an automated testing tool. Although current automation testing tools implement graphical user interfaces that seek to alleviate some of these drawbacks, integrating automation testing tools with a user interface also has significant drawbacks. For example, even small modifications to an automation testing tool on the user-facing front end may require significant back-end framework and script modifications. Thus, automation testing tools with user interfaces may still be cumbersome and costly in ways that are undesirable, and still leave much to be desired in terms of overall user-friendliness, cost efficiency, compatibility, and processing efficiency. These drawbacks are compounded when they limit testing framework accessibility to the users who might actually most frequently interact with it, such as a company's employees untrained in specialized programming syntax and manual framework testers. In addition, persons or entities implementing automation testing across multiple databases must also implement multiple corresponding automation frameworks. Such implementations can be cumbersome, lead to cost and processing inefficiencies, require customized compatibility tools, and are often inaccessible except to the most experienced users. Whether implemented alone or in connection with other frameworks, current automation frameworks often require users to understand specialized programming syntax. This limits accessibility to the framework for the users who actually most frequently interact with it, such as a company's employees untrained in programming and manual framework testers. SUMMARY Embodiments of the present disclosure are directed to systems and methods for enabling autonomous automated user interface testing and services. An example method comprises receiving from a user a resource identifier associated with a resource, detecting one or more changes in a user interface of the resource, retrieving from a database update attributes associated with each of the one or more changes in the user interface, and updating the resource to incorporate each of the one or more update attributes. Systems and computer-readable media (such as non-transitory computer-readable media) that implement the above method are also provided. Additional objects and advantages of the embodiments will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice. The objects and advantages will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the claims. BRIEF DESCRIPTION OF DRAWINGS The drawings are not necessarily to scale or exhaustive. Instead, emphasis is generally placed upon illustrating the principles of the embodiments described herein. The accompanying drawings, which are incorporated in and constitute a part of thi