US-12619431-B2 - Assessing network features through selective execution of software tests

US12619431B2US 12619431 B2US12619431 B2US 12619431B2US-12619431-B2

Abstract

Systems and methods for assessing network features by selectively executing software tests to enhance a customer's experience with a telecommunications service network are disclosed. The method includes accessing, from a database, a suite of test profiles corresponding to software tests for assessing a network feature, wherein each test profile comprises (1) at least a portion of a script, (2) conditions for triggering execution, (3) power consumption availability threshold required for execution, and (4) metrics to be measured. The system may receive, from a device, data indicative of power consumption availability of the device and values corresponding to conditions for triggering execution for the software tests. The system identifies software tests that the device is capable of executing. The system transmits, to the mobile device, at least one portion of a script and receiving an indication of successful execution and metrics indicating network performance of the device during the execution.

Inventors

Madhav Ram NUSETTY
Bhanu Pant
Eaian Lorrie Alcanices
Jay Jitendrabhai Patel

Assignees

T-MOBILE USA, INC.

Dates

Publication Date: 20260505
Application Date: 20230908

Claims (20)

1 . A computer-implemented method for assessing network features by selectively executing software tests to enhance a customer's experience with a telecommunications service network, the method comprising: accessing, from a database, a suite of test profiles corresponding to software tests for assessing a network feature, wherein each test profile comprises (1) at least a portion of a script to be executed on a mobile device, (2) a set of conditions for triggering execution on the mobile device, (3) power consumption availability threshold required for execution, and (4) one or more metrics to be measured during execution; receiving, from the mobile device, data indicative of power consumption availability of the mobile device and a set of values corresponding to one or more conditions for triggering execution for the software tests; identifying a plurality of software tests that the mobile device is capable of executing based on the power consumption availability, the set of values corresponding to one or more set of conditions of one or more associated test profiles, and a plurality of user preferences; transmitting, to the mobile device, at least one portion of a script from the test profiles corresponding to the plurality of software tests; and receiving, from the mobile device, via a telecommunications service network, an indication of successful execution of the at least one portion of the script and one or more metrics indicating network performance of the mobile device during the execution.
2 . The method of claim 1 , wherein each test profile further comprises resources to be transferred responsive to an indication of successful completion of execution and wherein the method further comprises: responsive to the indication of successful execution, generating an instruction configured to cause transmission of the resources corresponding to the test profiles executed by the mobile device to an account associated with the mobile device.
3 . The method of claim 2 , wherein the method further comprises: transmitting, for display on an interface of the mobile device, options for selecting one or more software tests from the identified plurality of software tests and resources corresponding to each; and receiving, from the mobile device, a user selection for one or more software tests.
4 . The method of claim 1 , wherein identifying the plurality of software tests that the mobile device is capable of executing further comprises: calculating, for each software test, a first projected fuel cost for measuring the one or more metrics using the mobile device; calculating, for each software test, a second projected fuel cost for measuring the one or more metrics using an alternate tester device; and determining, whether the second projected fuel cost exceeds the first projected fuel cost.
5 . The method of claim 1 , wherein the set of conditions may include whether or not the mobile device is in a specified region, whether a current time is within a specified time during which a script may be executed, and a rate of movement of the mobile device.
6 . The method of claim 2 , wherein the plurality of user preferences comprises preferences for power consumption availability for executing a script, number of software tests for which scripts can be executed during a time interval, and minimum amount of resources associated with a software test.
7 . The method of claim 1 , wherein the mobile device is configured to execute a first version of a network feature and the software tests comprise a software test for assessing performance of a second version of the network feature, and wherein the method further comprises: retrieving one or more baseline metrics indicating network performance of the mobile device during execution of the first version of the network feature; determining a difference between the one or more baseline metrics and the one or more metrics indicating network performance of the mobile device during execution of the second version of the network feature; and responsive to determining that the difference exceeds a threshold, generating a recommendation for an action to improve network performance.
8 . The method of claim 7 , wherein the recommendation for the action to improve network performance comprises a recommendation for terminating execution of the second version of the network feature and recommending execution of the first version of the network feature.
9 . The method of claim 7 , further comprising: responsive to determining that the difference exceeds a threshold, automatically terminating execution of the second version of the network feature and recommencing execution of the first version of the network feature.
10 . One or more non-transitory computer-readable media containing instructions which when executed by a processor, perform a method for assessing network features by selectively executing software tests, the method comprising: accessing, from a database, a suite of test profiles corresponding to software tests for assessing a network feature, wherein each test profile comprises (1) at least a portion of a script to be executed on a mobile device, (2) a set of conditions for triggering execution on the mobile device, (3) power consumption availability threshold required for execution, and (4) one or more metrics to be measured during execution; receiving, from the mobile device, data indicative of power consumption availability of the mobile device and a set of values corresponding to one or more conditions for triggering execution for the software tests; identifying a plurality of software tests that the mobile device is capable of executing based on the power consumption availability, the set of values corresponding to one or more set of conditions of one or more associated test profiles, and a plurality of user preferences; transmitting, to the mobile device, a set of test script indicators corresponding to one or more of the plurality of software tests; and receiving, from the mobile device via a telecommunications service network, one or more metrics indicating network performance of the mobile device during execution of at least one of the one or more of the plurality of software tests at the mobile device.
11 . The one or more non-transitory computer-readable media of claim 10 , wherein the method further comprises: transmitting, for display on an interface of the mobile device, options for selecting one or more software tests from the plurality of software tests; and receiving, from the mobile device, a user selection for one or more software tests.
12 . The one or more non-transitory computer-readable media of claim 10 , wherein receiving the one or more metrics indicating network coverage strength comprises receiving a set of records associated with the mobile device, wherein the set of records includes: latency, signal speed, and/or jitter.
13 . The one or more non-transitory computer-readable media of claim 10 , wherein the set of conditions may include whether or not the mobile device is in a specified region, whether a current time is within a specified time during which a script may be executed, and a rate of movement of the mobile device.
14 . The one or more non-transitory computer-readable media of claim 10 , wherein the plurality of user preferences comprises preferences for power consumption availability for executing a script, number of software tests for which scripts can be executed during a time interval, and minimum amount of resources associated with a software test.
15 . The one or more non-transitory computer-readable media of claim 10 , wherein the mobile device is configured to execute a first version of a network feature and the software tests comprise a software test for assessing performance of a second version of the network feature, and wherein the method further comprises: retrieving one or more baseline metrics indicating network performance of the mobile device during execution of the first version of the network feature; determining a difference between the one or more baseline metrics and the one or more metrics indicating network performance of the mobile device during execution of the second version of the network feature; and responsive to determining that the difference exceeds a threshold, generating a recommendation for an action to improve network performance.
16 . A system for assessing network features by selectively executing software tests to enhance a customer's experience with a telecommunications service network, the system comprising: one or more processors; and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause operations comprising: accessing, from a database, a suite of test profiles corresponding to software tests for assessing a network feature, wherein each test profile comprises (1) at least a portion of a script to be executed on a mobile device, (2) a set of conditions for triggering execution on the mobile device, (3) power consumption availability threshold required for execution, and (4) one or more metrics to be measured during execution; receiving, from the mobile device, data indicative of power consumption availability of the mobile device and a set of values corresponding to one or more conditions for triggering execution for the software tests; identifying a plurality of software tests that the mobile device is capable of executing based on the power consumption availability, the set of values corresponding to one or more set of conditions of one or more associated test profiles, and a plurality of user preferences; transmitting, to the mobile device, a set of test script indicators corresponding to one or more of the plurality of software tests; and receiving, from the mobile device via a telecommunications service network, one or more metrics indicating network performance of the mobile device during execution of at least one of the one or more of the plurality of software tests at the mobile device.
17 . The system of claim 16 , wherein the one or more non-transitory computer-readable media further cause operations comprising: transmitting, for display on an interface of the mobile device, options for selecting one or more software tests from the plurality of software tests; and receiving, from the mobile device, a user selection for one or more software tests.
18 . The system of claim 16 , wherein receiving the one or more metrics indicating network coverage strength comprises receiving a set of records associated with the mobile device, wherein the set of records includes: latency, signal speed, and/or jitter.
19 . The system of claim 16 , wherein the set of conditions may include whether or not the mobile device is in a specified region, whether a current time is within a specified time during which a script may be executed, and a rate of movement of the mobile device.
20 . The system of claim 16 , wherein the plurality of user preferences are determined based on user input via the mobile device and/or are obtained from a user profile associated with the mobile device stored at the database.

Description

BACKGROUND A telecommunications network is established via a complex arrangement and configuration of many cell sites that are deployed across a geographical area. For example, there can be different types of cell sites (e.g., macro cells, microcells, and so on) positioned in a specific geographical location, such as a city, neighborhood, and so on). These cell sites strive to provide adequate, reliable coverage for mobile devices (e.g., smart phones, tablets, and so on) via different frequency bands and radio networks such as a Global System for Mobile (GSM) mobile communications network, a code/time division multiple access (CDMA/TDMA) mobile communications network, a 3rd or 4th generation (3G/4G) mobile communications network (e.g., General Packet Radio Service (GPRS/EGPRS)), Enhanced Data rates for GSM Evolution (EDGE), Universal Mobile Telecommunications System (UMTS), or Long Term Evolution (LTE) network), 5G mobile communications network, IEEE 802.11 (WiFi), or other communications networks. The devices can seek access to the telecommunications network for various services provided by the network, such as services that facilitate the transmission of data over the network and/or provide content to the devices. As remote device usage continues to rise at an impressive rate, there are often many people using applications requiring network connectivity in various urban and remote locations. Software tests are often performed as part of a routine check to ensure that a network functions correctly at different locations. Before new features are introduced or when they are integrated into a larger software, they are tested via software tests in different regions to ensure proper function. For example, telecommunications networks often use drive testing to measure and assess coverage, capacity, quality of service, and other indicators of network performance for mobile networks. The process may include driving along a designated route using a motor vehicle equipped with network measurement equipment to collect data. The collected data is sent to operators who use the information to make decisions for adjustments to the network (e.g., tweaking cell tower configurations, adding new equipment, or planning for the installation of new cell sites). BRIEF DESCRIPTION OF THE DRAWINGS Detailed descriptions of implementations of the present invention will be described and explained through the use of the accompanying drawings. FIG. 1 is a block diagram that illustrates a wireless communications system that can implement aspects of the present technology. FIG. 2 is a block diagram that illustrates 5G core network functions (NFs) that can implement aspects of the present technology. FIG. 3 is a block diagram illustrating a suitable computing environment within which to assessing network features by selectively executing software tests. FIG. 4 is a block diagram illustrating the components of an telecommunications services testing system. FIG. 5A is a flow diagram illustrating a process of identifying software tests for assessing network features. FIG. 5B is a flow diagram illustrating a process of executing software tests and transmitting data to a telecommunications services testing system. FIG. 6 is a block diagram that illustrates an example of a computer system in which at least some operations described herein can be implemented. The technologies described herein will become more apparent to those skilled in the art from studying the Detailed Description in conjunction with the drawings. Embodiments or implementations describing aspects of the invention are illustrated by way of example, and the same references can indicate similar elements. While the drawings depict various implementations for the purpose of illustration, those skilled in the art will recognize that alternative implementations can be employed without departing from the principles of the present technologies. Accordingly, while specific implementations are shown in the drawings, the technology is amenable to various modifications. DETAILED DESCRIPTION Information regarding coverage, capacity, quality of service, and other indicators of network performance for mobile networks are important to telecommunication network providers/operators to help them make decisions for adjustments to a network (e.g., tweaking cell tower configurations, adding new equipment, planning for the installation of new cell sites, and the like). Conventionally, drive testing, where a tester drives along a designated route using a motor vehicle equipped with network measurement equipment, is employed to ensure network performance. However, there are many limitations and problems with drive tests. Drive tests are cost-intensive, technologically ineffective under dynamic circumstances, fail to provide data regarding in-building coverage, limited to data collected with only a few types of user equipment, and limited in geographic scope. Because drive testing involves vehicles tra