US-12619520-B2 - Software engineering with machine-readable feature specifications

Abstract

Systems and methods are provided for assisted software feature (or “requirement”) generation and for the automated or semi-automated generation of test cases therefrom. Also included are improved systems and methods for associating test cases with requirements and for modifying the test cases as the requirements are modified, added, and/or removed. These embodiments allow the cost of generating and maintaining a set of requirements and test cases and the cost of validating such a set of test cases to be reduced.

Inventors

• Avishek Dey
• Aakanchha Trivedi

Assignees

• SERVICENOW, INC.

Dates

Publication Date

20260505

Application Date

20230420

Claims (18)

1. 1 . A method comprising: obtaining a text-based description of a software feature of a software application; parsing the text-based description into a software feature specification in a machine-executable format; based on the software feature specification, generating a test case, wherein the test case represents a machine-executable description of a test to evaluate whether the software feature has been implemented or has been improperly implemented; storing a representation of the software feature specification and a representation of the test case in a record that indicates that the test case is linked to the software feature; during testing of the software application, determining that the test case is linked to the software feature; executing the test case on the software feature as implemented in the software application to determine whether the software feature has been implemented or has been improperly implemented; obtaining a second text-based description of a second software feature of the software application; parsing the second text-based description into a second software feature specification in the machine-executable format; based on the second software feature specification, generating a second test case; determining that a level of similarity between the test case and the second test case exceeds a threshold level of similarity; and responsive to determining that the level of similarity between the test case and the second test case exceeds the threshold level of similarity, performing one of: updating a record to indicate that the test case is linked to the second software feature; or updating the record that indicates that the test case is linked to the software feature to indicate that the second test case is linked to the software feature.
2. 2 . The method of claim 1 , wherein parsing the text-based description into the software feature specification in the machine-executable format comprises: extracting a context for the software feature from the text-based description; extracting at least one actor for the software feature from the text-based description; segmenting the text-based description into two or more text segments; generating at least one action for the software feature specification based on a corresponding at least one text segment of the two or more text segments, wherein the at least one action includes at least one object selected from an enumerated set of objects that is associated with the context; and generating at least one result for the software feature specification based on a corresponding at least one text segment of the two or more text segments, wherein the at least one result includes at least one object selected from the enumerated set of objects that is associated with the context.
3. 3 . The method of claim 2 , wherein generating the test case based on the software feature specification comprises: generating a first task to be performed by the at least one actor to enter the context; based on the at least one action, generating at least one additional task to be performed by the at least one actor within the context; and based on the at least one result, generating at least one outcome that can be detected as a result of performance of the at least one additional task.
4. 4 . The method of claim 1 , further comprising: obtaining an update to the text-based description of the software application; parsing the update to the text-based description of the software application to update the software feature specification; and based on the software feature specification as updated and the record that indicates that the test case is linked to the software feature, updating the representation of the test case to reflect the software feature specification as updated.
5. 5 . The method of claim 4 , further comprising: obtaining a third text-based description of a third software feature of the software application; parsing the third text-based description into a third software feature specification in the machine-executable format; and storing a representation of the third software feature specification in a third record that indicates that the test case is linked to the third software feature.
6. 6 . The method of claim 5 , further comprising: during testing of the software application, determining that the test case is linked to the software feature and the third software feature; and executing the test case on the software feature and on the third software feature as implemented in the software application to determine whether the software feature has been implemented or has been improperly implemented and to determine whether the third software feature has been implemented or has been improperly implemented.
7. 7 . The method of claim 5 , further comprising: based on the third software feature specification, generating a third test case, wherein the third test case represents a third machine-executable description of a third test to evaluate whether the second software feature has been implemented or has been improperly implemented; determining that a level of similarity between the test case and the third test case exceeds a threshold level of similarity; and responsive to determining that the level of similarity between the test case and the third test case exceeds the threshold level of similarity, deleting the third test case, wherein storing the representation of the third software feature specification in the record that indicates that the test case is linked to the third software feature is performed responsive to determining that the level of similarity between the test case and the third test case exceeds the threshold level of similarity.
8. 8 . The method of claim 1 , further comprising: during testing of the software application, determining that the test case is linked to the software feature and the second software feature; and executing the test case on the software feature and on the second software feature as implemented in the software application to determine whether the software feature has been implemented or has been improperly implemented and to determine whether the second software feature has been implemented or has been improperly implemented.
9. 9 . The method of claim 1 , wherein the second test case represents a second machine-executable description of a second test to evaluate whether the second software feature has been implemented or has been improperly implemented, wherein the method further comprises, responsive to determining that the level of similarity between the test case and the second test case exceeds the threshold level of similarity, updating the record to indicate that the test case is linked to the second software features and deleting the second test case.
10. 10 . The method of claim 1 , wherein obtaining the text-based description of the software feature comprises: providing, via a user interface, a textual input field; receiving, via the textual input field, textual input representative of a first context of an enumerated set of contexts; receiving, via the textual input field, additional textual input that includes an escape symbol; responsive to receiving the additional textual input that includes the escape symbol, providing, via the user interface, a menu of object names, wherein each object name in the menu of object names is associated with the first context; receiving, via the user interface, a user selection of one of the object names in the menu; and responsive to receiving the user selection, adding the object name as selected to the textual input field.
11. 11 . The method of claim 1 , further comprising: obtaining a third text-based description of a third software feature of the software application; parsing the third text-based description into a third software feature specification in the machine-executable format; based on the third software feature specification, generating a third test case, wherein the third test case represents a third machine-executable description of a third test to evaluate whether the third software feature has been implemented or has been improperly implemented; determining that a level of similarity between the test case and the third test case exceeds a threshold level of similarity; and responsive to determining that the level of similarity between the test case and the third test case exceeds the threshold level of similarity: storing a representation of the third software feature specification and a representation of the third test case in a second record that indicates that the second test case is linked to the second software feature, and deleting the representation of the software feature specification and the representation of the test case.
12. 12 . The method of claim 1 , further comprising: obtaining a third text-based description of a third software feature of the software application; parsing the third text-based description into a third software feature specification in the machine-executable format; based on the third software feature specification, generating a third test case, wherein the third test case represents a third machine-executable description of a third test to evaluate whether the third software feature has been implemented or has been improperly implemented; determining that a level of similarity between the test case and the third test case exceeds a threshold level of similarity; responsive to determining that the level of similarity between the test case and the third test case exceeds the threshold level of similarity, providing, via a user interface, an indication of the software feature specification, an indication of the test case, an indication of the third software feature specification, and an indication of the third test case; receiving, via the user interface, a command; and responsive to the command, performing one of: storing a representation of the third software feature specification in a third record that indicates that the test case is linked to the third software feature; deleting the third software feature specification and the third test case; or storing a representation of the third software feature specification and a representation of the third test case in a fourth record that indicates that the third test case is linked to the third software feature, and deleting the representation of the software feature specification and the representation of the test case.
13. 13 . The method of claim 1 , further comprising: obtaining a third text-based description of a third software feature of the software application; parsing the third text-based description into a third software feature specification in the machine-executable format; based on the third software feature specification, generating a third test case, wherein the third test case represents a third machine-executable description of a third test to evaluate whether the third software feature has been implemented or has been improperly implemented; storing a representation of the third software feature specification and a representation of the third test case in a third record that indicates that the third test case is linked to the second software feature; obtaining an update to the text-based description of the software application; parsing the update to the text-based description of the software application to update the software feature specification; based on the software feature specification as updated and the record that indicates that the test case is linked to the software feature, generating an updated test case and updating the representation of the test case to reflect the updated test case; determining that a level of similarity between the updated test case and the third test case exceeds a threshold level of similarity; responsive to determining that the level of similarity between the updated test case and the third test case exceeds the threshold level of similarity, performing one of: deleting the representation of the third test case and updating the record that indicates that the third test case is linked to the second software feature to indicate that the updated test case is linked to the third software feature; deleting the representation of the third software feature specification and the representation of the third test case; or deleting the representation of the software feature specification and the representation of the test case.
14. 14 . The method of claim 13 , further comprising: responsive to determining that the level of similarity between the updated test case and the third test case exceeds the threshold level of similarity, providing, via a user interface, an indication of the software feature specification as updated, an indication of the updated test case, an indication of the third software feature specification, and an indication of the third test case; and receiving, via the user interface, a command to perform one of: deleting the representation of the third test case and updating the record that indicates that the third test case is linked to the third software feature to indicate that the updated test case is linked to the third software feature; deleting the representation of the third software feature specification and the representation of the third test case; or deleting the representation of the software feature specification and the representation of the test case.
15. 15 . The method of claim 1 , further comprising: providing, via a user interface, an indication of the software feature specification and an indication of the test case; and receiving, via the user interface, a user input, wherein storing the representation of the software feature specification and the representation of the test case in a record that indicates that the test case is linked to the software feature is performed responsive to the user input.
16. 16 . The method of claim 15 , wherein the user input comprises at least one of a modification to the software feature specification or a modification to the test case, and wherein storing the representation of the software feature specification and the representation of the test case in the record that indicates that the test case is linked to the software feature comprises at least one of storing a representation of the software feature specification as modified or storing a representation of the test case as modified.
17. 17 . A non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing system, cause the computing system to perform operations comprising: obtaining a text-based description of a software feature of a software application; parsing the text-based description into a software feature specification in a machine-executable format; based on the software feature specification, generating a test case, wherein the test case represents a machine-executable description of a test to evaluate whether the software feature has been implemented or has been improperly implemented; storing a representation of the software feature specification and a representation of the test case in a record that indicates that the test case is linked to the software feature, during testing of the software application, determining that the test case is linked to the software feature; executing the test case on the software feature as implemented in the software application to determine whether the software feature has been implemented or has been improperly implemented; obtaining a second text-based description of a second software feature of the software application; parsing the second text-based description into a second software feature specification in the machine-executable format; based on the second software feature specification, generating a second test case; determining that a level of similarity between the test case and the second test case exceeds a threshold level of similarity; and responsive to determining that the level of similarity between the test case and the second test case exceeds the threshold level of similarity, performing one of: updating a record to indicate that the test case is linked to the second software feature; or updating the record that indicates that the test case is linked to the software feature to indicate that the second test case is linked to the software feature.
18. 18 . A system comprising: one or more processors; and memory, containing program instructions that, upon execution by the one or more processors, cause the system to perform operations comprising: obtaining a text-based description of a software feature of a software application; parsing the text-based description into a software feature specification in a machine-executable format; based on the software feature specification, generating a test case, wherein the test case represents a machine-executable description of a test to evaluate whether the software feature has been implemented or has been improperly implemented; storing a representation of the software feature specification and a representation of the test case in a record that indicates that the test case is linked to the software feature, during testing of the software application, determining that the test case is linked to the software feature; executing the test case on the software feature as implemented in the software application to determine whether the software feature has been implemented or has been improperly implemented; obtaining a second text-based description of a second software feature of the software application; parsing the second text-based description into a second software feature specification in the machine-executable format; based on the second software feature specification, generating a second test case; determining that a level of similarity between the test case and the second test case exceeds a threshold level of similarity; and responsive to determining that the level of similarity between the test case and the second test case exceeds the threshold level of similarity, performing one of: updating a record to indicate that the test case is linked to the second software feature; or updating the record that indicates that the test case is linked to the software feature to indicate that the second test case is linked to the software feature.

Description

BACKGROUND The process of software development can be organized by representing features of the software to be implemented as “requirements.” From each requirement, one or more tests may be created, which can be evaluated (e.g., by a quality control professional, or via an automated process) to determine whether the requirement has been fully and accurately implemented. As software development continues, the set of requirements, and their associated tests, may expand and may also be modified or removed (e.g., as requirements are changed or deprecated). This can lead to increased storage requirements as the set of requirements and tests expands, as well as increased computation requirements to perform analyses or other tasks on the set of requirements and tests. Further, validation of the tests increases in cost (computational cost for those validated by computer execution, human effort and time for those validated by human quality control professionals) as the number of tests increases. SUMMARY During software development (or the development of some other product, service, or system), the set of requirements can expand and be modified. Correspondingly, the set of test cases used by quality control or other individuals or systems to evaluate whether those requirements have been fully and accurately implemented may also expand significantly. Embodiments described herein reduce the amount of human effort and the computational cost (e.g., storage, cycles) needed to generate and organize representations of the requirements and of their associated test cases and to modify such requirements and test cases. This is accomplished, in part, by parsing human-generated narratives that describe a new (or updated) requirement into a machine-readable format. Such a machine-readable software feature specification can then be used to generate one or more test cases for the underlying requirement, and/or to update such test cases in response to modifications to their corresponding requirement(s). Human input may be used to approve such automatically-generated test cases and/or to modify such test cases. Additionally, embodiments described herein can reduce the cost to maintain (e.g., to store, to update) and validate (e.g., via automated test case validation) test cases for a set of software feature specifications by linking the test cases to their associated software feature specifications. Such linking can allow the test cases to be automatically updated as the underlying requirements (via the associated software feature specification) are modified and/or to delete or otherwise deprecate test cases, in an automated manner, as the underlying requirements are deprecated or otherwise removed from the set of requirements. Embodiments provided herein also allow the total number of test cases to be reduced by linking individual test cases to multiple software feature specifications (e.g., where the operations to validate two or more software feature specifications are sufficiently similar to be described by a single test case). Such embodiments, in linking test cases to corresponding software feature specifications, can reduce the total number of test cases. Such embodiments can also reduce the total number of software feature specifications, e.g., by deprecating older software feature specifications when newly-added software feature specifications are sufficiently similar and/or by rejecting newly-added software feature specifications when older software feature specifications are sufficiently similar. Such a reduction in the number of test cases and/or software feature specifications can reduce the memory, interconnect bandwidth, processor cycles, or other computational costs necessary to update and maintain the set of test cases and software feature specifications. Such a reduction can also reduce the cost to validate whether a set of requirements (represented by the set of software feature specifications) has been fully and accurately implemented by reducing the total number of test cases to be evaluated in order to validate the set of requirements. In embodiments where some or all of the test cases are validated automatically (e.g., as opposed to being validated by the efforts of a human quality control professional), such a reduction in the total number of test cases as is provided by the embodiments herein can reduce the total computational cost (e.g., processor cycles, server time) necessary to perform such automated validation. Accordingly, a first example embodiment may involve a method that includes: (i) obtaining a text-based description of a software feature of a software application; (ii) parsing the text-based description into a software feature specification in a machine-executable format; (iii) based on the software feature specification, generating a test case, wherein the test case represents a machine-executable description of a test to evaluate whether the software feature has been implemented or has been imp