US-12625711-B2 - Vocabulary to control insight delivery

Abstract

A computer-implemented method and system provide for insights/recommendations to a user of a software product. A computer application autonomously determines a count of distinct commands executed by a user. Based on the count, the computer application autonomously determines the insight that includes a feature or new distinct command. The computer application then autonomously recommends the insight to the user.

Inventors

• Jonathan Simon Pascal Steinberg

Assignees

• AUTODESK, INC.

Dates

Publication Date

20260512

Application Date

20240129

Claims (18)

1. 1 . A computer-implemented method for providing an insight, comprising: autonomously determining, in a computer application, a count of distinct commands executed by a user, wherein the autonomously determining the count comprises: determining that a new distinct command is a synonym or variant of another distinct command that has already been included in the count; and based on the determining that the new distinct command is a synonym or variant, not including the new distinct command in the count; based on the count, autonomously determining, in the computer application, the insight, wherein the insight comprises a feature or new distinct command; and autonomously recommending, via the computer application, the insight to the user.
2. 2 . The computer-implemented method of claim 1 , wherein the autonomously determining the count comprises: isolating data representing human triggered commands from automatic commands and program issued data.
3. 3 . The computer-implemented method of claim 1 , wherein the autonomously determining the count comprises: determining a modal state of the application when each distinct command is executed; and determining a separate count for each modal state.
4. 4 . The computer-implemented method of claim 1 , wherein the autonomously determining the count comprises: maintaining a list of stop commands; recognizing that the distinct command is in the list; and based on the recognizing, not including the distinct command in the count.
5. 5 . The computer-implemented method of claim 1 , further comprising: maintaining a skill tree comprising a taxonomy of one or more classifications of a user's skills; linking a set of the distinct commands to a first classification of the one or more classifications; determining a task for the user; and selecting the set based on the task and the user's skills; and autonomously determining the insight based on the selected set.
6. 6 . The computer-implemented method of claim 5 , further comprising: identifying a first distinct command based on input from the user; determining where the first distinct command is located in the skill tree; and selecting the set based on where the first distinct command is located in the skill tree.
7. 7 . The computer-implemented method of claim 1 , wherein: the autonomously determining the insight is based on historical data over a defined time period; and the historical data comprises the distinct commands the user has executed and the distinct commands the user has not executed over the defined time period.
8. 8 . The computer-implemented method of claim 1 , wherein the insights are autonomously determined based on a hierarchical organization of the distinct commands.
9. 9 . The computer-implemented method of claim 1 , further comprising: determining a first vocabulary growth curve for a first group of users, wherein the first vocabulary growth curve represents the count of a first set of distinct commands executed by users in the first group; determining a second vocabulary growth curve for a second group of users, wherein the second vocabulary growth curve represents the count of a second set of distinct commands executed by users in the second group; comparing the first vocabulary growth curve to the second vocabulary growth curve to determine a growth rate difference; and comparing the first set of distinct commands to the second set of distinct commands to determine a total vocabulary achievement difference; and wherein the autonomously determining the insight is based on the growth rate difference and the total vocabulary achievement difference.
10. 10 . A computer-implemented system for providing an insight, comprising: (a) a computer having a memory; (b) a processor executing on the computer; (c) the memory storing a set of instructions for a computer application, wherein the set of instructions, when executed by the processor cause the processor to perform operations comprising: (i) autonomously determining a count of distinct commands executed by a user, wherein the autonomously determining the count comprises: (1) determining that a new distinct command is a synonym or variant of another distinct command that has already been included in the count; and (2) based on the determining that the new distinct command is a synonym or variant, not including the new distinct command in the count; (ii) based on the count, autonomously determining the insight, wherein the insight comprises a feature or new distinct command; and (iii) autonomously recommending the insight to the user.
11. 11 . The computer-implemented system of claim 10 , wherein the autonomously determining the count comprises: isolating data representing human triggered commands from automatic commands and program issued data.
12. 12 . The computer-implemented system of claim 10 , wherein the autonomously determining the count comprises: determining a modal state of the computer application when each distinct command is executed; and determining a separate count for each modal state.
13. 13 . The computer-implemented system of claim 10 , wherein the autonomously determining the count comprises: maintaining a list of stop commands; recognizing that the distinct command is in the list; and based on the recognizing, not including the distinct command in the count.
14. 14 . The computer-implemented system of claim 10 , the operations further comprising: maintaining a skill tree comprising a taxonomy of one or more classifications of a user's skills; linking a set of the distinct commands to a first classification of the one or more classifications; determining a task for the user; and selecting the set based on the task and the user's skills; and autonomously determining the insight based on the selected set.
15. 15 . The computer-implemented system of claim 14 , the operations further comprising: identifying a first distinct command based on input from the user; determining where the first distinct command is located in the skill tree; and selecting the set based on where the first distinct command is located in the skill tree.
16. 16 . The computer-implemented system of claim 10 , wherein: the autonomously determining the insight is based on historical data over a defined time period; and the historical data comprises the distinct commands the user has executed and the distinct commands the user has not executed over the defined time period.
17. 17 . The computer-implemented system of claim 10 , wherein the insights are autonomously determined based on a hierarchical organization of the distinct commands.
18. 18 . The computer-implemented system of claim 10 , the operations further comprising: determining a first vocabulary growth curve for a first group of users, wherein the first vocabulary growth curve represents the count of a first set of distinct commands executed by users in the first group; determining a second vocabulary growth curve for a second group of users, wherein the second vocabulary growth curve represents the count of a second set of distinct commands executed by users in the second group; comparing the first vocabulary growth curve to the second vocabulary growth curve to determine a growth rate difference; and comparing the first set of distinct commands to the second set of distinct commands to determine a total vocabulary achievement difference; and wherein the autonomously determining the insight is based on the growth rate difference and the total vocabulary achievement difference.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates generally to improving user proficiency with computer software, and in particular, to a method, apparatus, system, and article of manufacture for determining how, when, and where insights are provided to a user to improve their efficiency and/or knowledge. 2. Description of the Related Art When a user begins using a software product (e.g., a trial user), they often know nothing about the product and/or how to use the product. Further, for an intermediate user to become an advanced/experienced user, that user needs to learn additional commands/actions/processes on the software. To develop a user's skill and increase their proficiency with the product, it may be useful to provide insights and/or suggested commands/actions that the user should learn/become familiar with. In particular, it is desirable to determine what insights to display as well as the timing and cadence of delivering insights to the user/customer. Prior art systems attempted to control insight cadence using session count (one session is the time the product runs from launch until the user quits) as the trigger. In other words, session count merely looked at the lapsed/total time the user used a product. However, such an approach is inadequate as it is both insufficiently granular and extremely variable amongst the target population, and resulted in a mediocre correlation to desired outcome. SUMMARY OF THE INVENTION Embodiments of the invention provide suggestions to users within a computer application based on their behaviors. Content to suggest is curated based on a level of proficiency that a user has with a particular product. Such a curation may be based on an analysis of how advanced the user's actions are. More specifically, in one or more embodiments, a count of unique commands executed by the user is maintained as a “vocabulary”. This vocabulary may then be compared to the vocabularies of ideal, different or more advanced users to determine which commands to recommend/place into insights provided to the user. The selection of which commands and the pace of delivery of the commands may all be autonomously determined based on modeling a range of different types of users of a software product and an attempt to shift users from trial users to commercial users. BRIEF DESCRIPTION OF THE DRAWINGS Referring now to the drawings in which like reference numbers represent corresponding parts throughout: FIG. 1 illustrates the logical flow for providing an insight in accordance with one or more embodiments of the invention; FIG. 2 illustrates a screen shot of the FUSION application that includes a displayed guide with the recommendation in accordance with one or more embodiments of the invention; FIG. 3 illustrates a screen shot of a guide that includes a recommendation displayed in accordance with one or more embodiments of the invention; FIG. 4 is an exemplary hardware and software environment used to implement one or more embodiments of the invention; and FIG. 5 schematically illustrates a typical distributed/cloud-based computer system utilized in accordance with one or more embodiments of the invention. DETAILED DESCRIPTION OF THE INVENTION In the following description, reference is made to the accompanying drawings which form a part hereof, and which is shown, by way of illustration, several embodiments of the present invention. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention. Overview Embodiments of the invention address two inter-related problems/challenges: (1) measuring how “proficient” users are (and whether the embodiments of the invention are actually improving their proficiency); and (2) deciding on the timing and cadence of delivering the insight to the customer—it is undesirable to deliver insights that are too advanced before a user is ready, or have too many insights displayed at the same time. Embodiments of the invention provide “insights” to users based on their product usage. As used herein, insights are typically recommendations of features or commands that would be useful to that individual. Vocabulary Measurement Embodiments of the invention measure the user's “vocabulary”—the count of distinct (unique) commands executed by the user within a software product (e.g., first-time and/or trial users of a product such as AUTODESK FUSION360, AUTOCAD, REVIT, etc.) cumulatively from their first usage throughout a trial period (e.g., a defined period of time). Embodiments of the invention may also isolate the data that represents “human triggered” commands from automatic and program issued data. The general idea is that users—especially new users during a trial period may execute just a few commands in total or many hundreds of commands (but mostly repeats of commands they've used before). However, evidence that they are learning to use the product c