Search

EP-4736013-A1 - AUTOMATED SOFTWARE TESTING USING NATURAL LANGUAGE-BASED FORM COMPLETION

EP4736013A1EP 4736013 A1EP4736013 A1EP 4736013A1EP-4736013-A1

Abstract

Aspects of the disclosure include methods and systems for performing automated software testing. The method can include executing software under test and determining that the user interface of the software includes a textual input field. The method includes identifying a label of the textual input field, inputting into a natural language processing system the label as a query and receiving, from the natural language processing system in response to the query, a first input text. The method also includes inputting the first input text into the textual input field and recording a first response of the software to the first input text.

Inventors

  • BAKER, WILLIAM TIGARD
  • NALLAMALLI, Swamy V. P. L. N.
  • WARREN, Dallas Allen
  • GUPTA, PIYUSH
  • DIETRICH, AARON EDWARD

Assignees

  • Microsoft Technology Licensing, LLC

Dates

Publication Date
20260506
Application Date
20240615

Claims (20)

  1. 1. A method for automated software testing, the method comprising: executing software under test (112); determining that a user interface (114) of the software (112) includes a textual input field (202); identifying a label (204) of the textual input field (202); inputting into a natural language processing system (120) a query' comprising the label (204); receiving, from the natural language processing system (120) in response to the query, a first input text (304); inputting the first input text (304) into the textual input field (202); and recording a first response of the software (112) to the first input text (304).
  2. 2. The method of claim 1. further comprising: determining that the first response of the software does not include an error message related to the textual input field; and storing the label associated with the textual input field and the first input text in one or more of a testing log associated with the software and a textual input database.
  3. 3. The method of claim 1. further comprising identifying additional information regarding the textual input field and wherein the query further comprises the additional information.
  4. 4. The method of claim 1, further comprising determining that the first response of the software includes a first error message related to the textual input field.
  5. 5. The method of claim 4, wherein the determination that the first response of the software includes the first error message is based on a comparison of the user interface before the inputting the first input text into the textual input field and after inputting the first input text into the textual input field.
  6. 6. The method of claim 4. further comprising: inputting into the natural language processing system a second query comprising the label and information about the first error message; receiving, from the natural language processing system in response to the second query, a second input text; inputting the second input text into the textual input field; and recording a second response of the software to the second input text.
  7. 7. The method of claim 6, further comprising: determining that the second response of the software does not include a second error message related to the textual input field; and storing the label associated with the textual input field and the second input text in one or more of a testing log associated with the software and a textual input database.
  8. 8. The method of claim 6, wherein the second response of the software to the second input text includes a second error message related to the textual input field.
  9. 9. The method of claim 8, further comprising: determining that the first error message and the second error message are different; inputting, into the natural language processing system the label, the first error message, and information about the second error message as a third query; receiving, from the natural language processing system in response to the third query, a third input text; inputting the third input text into the textual input field; and recording a third response of the software to the third input text.
  10. 10. The method of claim 9. further comprising: determining that the first error message and the second error message are identical; and transmitting an error notification.
  11. 11. The method of claim 9, wherein the third response of the software includes a third error message related to the textual input field, and the method further comprises transmitting an error notification.
  12. 12. The method of claim 1, wherein the natural language processing system is a large language model.
  13. 13. A method for automated software testing, the method comprising: executing software under test (112); determining that a user interface (114) of the software includes a textual input field (202); identifying a label (204) of the textual input field (202); obtaining a first input text (402) from a textual input database (140) based on the label (204); inputting the first input text (402) into the textual input field (202); determining that a first response of the software (112) to the first input text (402) includes a first error message (502) related to the textual input field (202); inputting into a natural language processing system (120) the label (204) and information about the first error message (502) as a query (602); receiving, from the natural language processing system (120) in response to the query (602), a second input text (604); inputting the second input text (604) into the textual input field (202); and recording a second response of the software (112) to the second input text (604).
  14. 14. The method of claim 13, further comprising: determining that the second response of the software does not include an error message related to the textual input field; and updating the textual input database to include the second input text.
  15. 15. The method of claim 13, wherein the second response of the software to the second input text includes a second error message related to the textual input field.
  16. 16. The method of claim 15, further comprising: inputting, into the natural language processing system the label, the first error message, and information about the second error message as a second query; receiving, from the natural language processing system in response to the second query, a third input text; inputting the third input text into the textual input field; and recording a third response of the software to the third input text.
  17. 17. The method of claim 16, wherein the third response of the software includes a third error message related to the textual input field, the method further comprises comparing the third error message to one or more of the first error message and the second error message.
  18. 18. The method of claim 17, further comprising: determining that the third error message is identical to one or more of the first error message and the second error message; and transmitting an error notification to a user.
  19. 19. A system having a memory, computer readable instructions, and a processing system for executing the computer readable instructions, the computer readable instructions control the processing system to perform operations comprising: executing software under test (112); determining that a user interface (114) of the software includes a textual input field (202); identifying a label (204) of the textual input field (202); inputting into a natural language processing system (120) the label (204) as a first query (302); receiving, from the natural language processing system (120) in response to the first query (302), a first input text (304); inputting the first input text (304) into the textual input field (202); determining that a first response of the software (112) includes a first error message (502) related to the textual input field (202); inputting into the natural language processing system (120) the label (204) and information about the first error message (502) as a second query (602); receiving, from the natural language processing system (120) in response to the second query (602), a second input text (604); inputting the second input text (604) into the textual input field (202); and recording a second response of the software (112) to the second input text (604).
  20. 20. The system of claim 19, wherein the operations further comprise: determining that the second response of the software includes a second error message related to the textual input field; and comparing the first error message to the second error message.

Description

AUTOMATED SOFTWARE TESTING USING NATURAL LANGUAGE-BASED FORM COMPLETION INTRODUCTION [0001] The subject disclosure relates to software testing, and particularly to automated software testing using natural language-based form completion. [0002] Testing software prior to its release is often performed to ensure the quality and reliability of the software. Proper testing helps identify bugs, errors, and usability issues, allowing developers to fix them before the software reaches users. Traditionally, testing of new software was a manual task that that required software developers to spend significant resources to ensure proper operation of the software. Attempts to reduce the time and resources required for testing new software products led to the use of test scripts to test software. Test scripts are written in a programming or scripting language and are used to automate the execution of test cases. [0003] Automated testing with test scripts can significantly improve the efficiency of the software testing process. Scripts can execute tests much faster than manual testing, allowing for quicker feedback on the software's quality and reducing the time required for testing. In addition, test scripts ensure that the same set of tests are executed consistently, eliminating human errors and variations in test execution. [0004] While test scripts can greatly enhance the efficiency and effectiveness of software testing, test scripts require substantial effort to develop and maintain and certain aspects of testing still require manual intervention. For example, testing of a software product that includes a user interface with textual input fields often requires manual intervention because the textual input fields often have specific rules that may not be property programmed into the testing script. As a result, the testing scripts often fail to populate textual input fields with valid values. SUMMARY [0005] Embodiments of the present disclosure are directed to methods for automated testing of software under test. An example method includes executing software under test and determining that a user interface of the software under test includes a textual input field. The method also includes identifying a label of the textual input field, inputting into a natural language processing system the label as a query, and receiving, from the natural language processing system in response to the query, a first input text. The method also includes inputting the first input text into the textual input field and recording a first response of the software under test to the first input text. [0006] Embodiments of the present disclosure are directed to methods for automated testing of software under test. An example method includes executing software under test and determining that the user interface of the software under test includes a textual input field. The method also includes identifying a label of the textual input field, obtaining a first input text from a textual input database based on the label, and inputting the first input text into the textual input field. The method further includes determining that a first response of the software under test to the first input text includes a first error message related to the textual input field, inputting into a natural language processing system the label and the first error message as a query, and receiving, from the natural language processing system in response to the query' a second input text. The method also includes inputting the second input text into the textual input field and recording a second response of the software under test to the second input text. [0007] Embodiments of the present disclosure are directed to a system having a memory, computer readable instructions, and a processing system for executing the computer readable instructions. The computer readable instructions control the processing system to perform operations that include executing software under test, determining that user interface of the software under test includes a textual input field, and identifying a label of the textual input field. The operations also include inputting into a natural language processing system the label as a first query', receiving, from the natural language processing system in response to the first query, a first input text, and inputting the first input text into the textual input field. The operations also include determining that a first response of the software under test includes a first error message related to the textual input field, inputting into the natural language processing system the label and the first error message as a second query', and receiving, from the natural language processing system in response to the second query7, a second input text. The operations further include inputting the second input text into the textual input field and recording a second response of the software under test to the second input text. [0008] The above features and ad