US-12627676-B2 - System and method for evaluating online data

Abstract

A method and system detects at a plurality of network locations a plurality of accuracy ratings of a plurality of media instances and detects the plurality of media instances. A particular accuracy rating of one or more particular media instances is detected at a particular network location, and the one or more particular media instances are detected. A bias of the particular accuracy rating is determined based on the particular accuracy rating, the one or more particular media instances, the plurality of accuracy ratings, and the plurality of media instances. An indication is transmitted to a user based on the bias of the particular accuracy rating.

Inventors

• Sadia Afroz
• Vibhor Sehgal

Assignees

• AVAST Software s.r.o.

Dates

Publication Date

20260512

Application Date

20230126

Claims (15)

1. 1 . A method comprising: detecting at a first plurality of network locations a plurality of accuracy ratings of a plurality of media instances; detecting at a second plurality of network locations the plurality of media instances, each of the plurality of accuracy ratings associated with at least one of the plurality of media instances; training a classifier in a form of a machine learning model based on the plurality of accuracy ratings and the plurality of media instances, the classifier comprising bidirectional encoder representations from transformers (“BERT”) and an embedding layer, the training of the classifier comprising for each of the plurality of accuracy ratings: inputting to the BERT via the embedding layer one or more words of the accuracy rating, a plurality of words of the at least one of the plurality of media instances associated with the accuracy rating, and a separator separating the one or more words of the accuracy rating from the plurality of words of the at least one of the plurality of media instances associated with the accuracy rating; and providing as an output to the BERT a predetermined bias; detecting on a computing device browsing at a particular network location by a user, the computing device comprising a user interface comprising a display; detecting at the particular network location based on the browsing at the particular network location a particular accuracy rating of at least one particular media instance in a window in the display; detecting the at least one particular media instance at least one other network location; applying the classifier to the particular accuracy rating and the at least one particular media instance to determine a bias of the particular accuracy rating, the applying the classifier comprising inputting to the BERT via the embedding layer one or more words of the particular accuracy rating, a plurality of words of the at least one particular media instance, and the separator separating the one or more words of the particular accuracy rating from the plurality of words of the at least one particular media instance to determine the bias of the particular accuracy rating responsive to the browsing at the particular network location; and displaying an indication to the user in the window concurrently with the particular accuracy rating based on the bias of the particular accuracy rating responsive to the browsing at the particular network location.
2. 2 . The method of claim 1 , further comprising determining a bias of the particular network location based on the bias of the particular accuracy rating.
3. 3 . The method of claim 1 , further comprising: detecting at the particular network location at least one other accuracy rating of at least one other media instance; detecting the at least one other media instance; determining a bias of the at least one other accuracy rating based on the at least one other accuracy rating, the at least one other media instance, the plurality of accuracy ratings, and the plurality of media instances; and displaying the indication to the user further based on the bias of the at least one other accuracy rating.
4. 4 . The method of claim 3 , further comprising: determining a bias of the particular network location based on the bias of the particular accuracy rating and the bias of the at least one other accuracy rating.
5. 5 . The method of claim 3 , further comprising: determining the bias of the at least one other accuracy rating further based on the particular accuracy rating and the at least one particular media instance; determining a bias of the particular network location based on the bias of the particular accuracy rating and the bias of the at least one other accuracy rating; and displaying the indication to the user further based on the bias of the particular network location.
6. 6 . The method of claim 3 , wherein the particular accuracy rating and the at least one other accuracy rating are originated by an entity; determining a bias of the entity based on the bias of the particular accuracy rating and the bias of the at least one other accuracy rating; and transmitting the indication to the user further based on the bias of the entity.
7. 7 . The method of claim 1 , further comprising: detecting at the particular network location multiple other accuracy ratings of multiple other media instances; detecting the multiple other media instances; determining biases of the multiple other accuracy ratings based on the multiple other accuracy ratings, the multiple other media instances, the plurality of accuracy ratings, and the plurality of media instances; and displaying the indication to the user further based on the biases of the multiple other accuracy ratings.
8. 8 . The method of claim 7 , wherein the particular accuracy rating is originated by a first entity, and the multiple other accuracy ratings are originated by a second entity; determining a bias of the first entity based on the bias of the particular accuracy rating; determining a bias of the second entity based on the biases of the multiple other accuracy ratings; determining a bias of the particular network location based on the bias of the first entity and the bias of the second entity; and displaying the indication to the user further based on the bias of the particular network location.
9. 9 . The method of claim 8 , further comprising: determining a weight of the bias of the first entity based on at least one of a rating history of the first entity or a consensus by the first entity with other entities; and determining the bias of the particular network location further based on the weight of the bias of the first entity and a weight of the bias of the second entity.
10. 10 . The method of claim 8 , further comprising: detecting at the particular network location additional accuracy ratings of additional media instances; detecting the additional media instances; determining biases of the additional accuracy ratings based on the additional accuracy ratings, the additional media instances, the plurality of accuracy ratings, and the plurality of media instances; and determining the bias of the first entity further based on the biases of the additional accuracy ratings.
11. 11 . A computing system comprising at least one processor and at least one non-transitory computer readable storage medium having encoded thereon instructions that when executed by the at least one processor cause the computing system to perform a process including: detecting at a first plurality of network locations a plurality of accuracy ratings of a plurality of media instances; detecting at a second plurality of network locations the plurality of media instances, each of the plurality of accuracy ratings associated with at least one of the plurality of media instances; training a classifier in a form of a machine learning model based on the plurality of accuracy ratings and the plurality of media instances, the classifier comprising bidirectional encoder representations from transformers (“BERT”) and an embedding layer, the training of the classifier comprising for each of the plurality of accuracy ratings: inputting to the BERT via the embedding layer one or more words of the accuracy rating, a plurality of words of the at least one of the plurality of media instances associated with the accuracy rating, and a separator separating the one or more words of the accuracy rating from the plurality of words of the at least one of the plurality of media instances associated with the accuracy rating; and providing as an output to the BERT a predetermined bias; detecting browsing at a particular network location by a user; detecting at the particular network location based on the browsing at the particular network location a particular accuracy rating of at least one particular media instance in a window in a display; detecting the at least one particular media instance at least one other network location; applying the classifier to the particular accuracy rating and the at least one particular media instance to determine a bias of the particular accuracy rating, the applying the classifier comprising inputting to the BERT via the embedding layer one or more words of the particular accuracy rating, a plurality of words of the at least one particular media instance, and the separator separating the one or more words of the particular accuracy rating from the plurality of words of the at least one particular media instance to determine the bias of the particular accuracy rating responsive to the browsing at the particular network location; and displaying an indication to the user in the window concurrently with the particular accuracy rating based on the bias of the particular accuracy rating responsive to the browsing at the particular network location.
12. 12 . A network-enabled evaluation system comprising: a first computing system comprising at least a first processor and at least a first non-transitory computer readable storage medium having encoded thereon first instructions that when executed by the at least the first processor cause the first computing system to perform a first process including: detecting at a first plurality of network locations a plurality of accuracy ratings of a plurality of media instances; detecting at a second plurality of network locations the plurality of media instances, each of the plurality of accuracy ratings associated with at least one of the plurality of media instances; training a classifier in a form of a machine learning model based on the plurality of accuracy ratings and the plurality of media instances, the classifier comprising bidirectional encoder representations from transformers (“BERT”) and an embedding layer, the training of the classifier comprising for each of the plurality of accuracy ratings: inputting to the BERT via the embedding layer one or more words of the accuracy rating, a plurality of words of the at least one of the plurality of media instances associated with the accuracy rating, and a separator separating the one or more words of the accuracy rating from the plurality of words of the at least one of the plurality of media instances associated with the accuracy rating; and providing as an output to the BERT a predetermined bias; detecting at a particular network location a particular accuracy rating of at least one particular media instance; detecting the at least one particular media instance at least one other network location; and applying the classifier to the particular accuracy rating and the at least one particular media instance to determine a bias of the particular accuracy rating, the applying the classifier comprising inputting to the BERT via the embedding layer one or more words of the particular accuracy rating, a plurality of words of the at least one particular media instance, and the separator separating the one or more words of the particular accuracy rating from the plurality of words of the at least one particular media instance to determine the bias of the particular accuracy rating at the particular network location; and a second computing system comprising a user interface comprising a display, at least a second processor, and at least a second non-transitory computer readable storage medium having encoded thereon second instructions that when executed by the at least the second processor cause the second computing system to perform a second process including: detecting browsing at the particular network location by a user; detecting at the particular network location based on the browsing at the particular network location the particular accuracy rating of the at least one particular media instance in a window in the display; receiving the bias of the particular accuracy rating from the first computing system responsive to the browsing at the particular network location; and displaying an indication to the user in the window concurrently with the particular accuracy rating based on the bias of the particular accuracy rating responsive to the browsing at the particular network location.
13. 13 . A non-transitory computer-readable storage medium storing executable instructions that, as a result of execution by one or more processors of a computing system, cause the computing system to perform operations comprising: detecting at a first plurality of network locations a plurality of accuracy ratings of a plurality of media instances; detecting at a second plurality of network locations the plurality of media instances, each of the plurality of accuracy ratings associated with at least one of the plurality of media instances; training a classifier in a form of a machine learning model based on the plurality of accuracy ratings and the plurality of media instances, the classifier comprising bidirectional encoder representations from transformers (“BERT”) and an embedding layer, the training of the classifier comprising for each of the plurality of accuracy ratings: inputting to the BERT via the embedding layer one or more words of the accuracy rating, a plurality of words of the at least one of the plurality of media instances associated with the accuracy rating, and a separator separating the one or more words of the accuracy rating from the plurality of words of the at least one of the plurality of media instances associated with the accuracy rating; and providing as an output to the BERT a predetermined bias; detecting on a computing device browsing at a particular network location by a user, the computing device comprising a user interface comprising a display; detecting at the particular network location based on the browsing at the particular network location a particular accuracy rating of at least one particular media instance in a window in the display; detecting the at least one particular media instance at least one other network location; applying the classifier to the particular accuracy rating and the at least one particular media instance to determine a bias of the particular accuracy rating, the applying the classifier comprising inputting to the BERT via the embedding layer one or more words of the particular accuracy rating, a plurality of words of the at least one particular media instance, and the separator separating the one or more words of the particular accuracy rating from the plurality of words of the at least one particular media instance to determine the bias of the particular accuracy rating responsive to the browsing at the particular network location; and displaying an indication to the user in the window concurrently with the particular accuracy rating based on the bias of the particular accuracy rating responsive to the browsing at the particular network location.
14. 14 . The method of claim 1 , further comprising detecting at the first plurality of network locations the plurality of accuracy ratings of the plurality of media instances by the computing device.
15. 15 . The method of claim 1 , the training of the classifier further comprising for each of the plurality of accuracy ratings determining the predetermined bias based on at least one of a fact checking system operating at one of the first plurality of network locations or a media source system operating at one of the second plurality of network locations.

Description

TECHNICAL FIELD The disclosure relates generally to evaluating data rendered accessible via a network. BACKGROUND Determining the authenticity of a piece of information online can be challenging. Several online fact checking entities exist that analyze and label the veracity of online information. Different fact checking entities use different criteria for labeling online information. Fact checking entities may have biases, for example they may be influenced based on their financial interests and may label their competitors as biased. Since there is no easy way for online users to verify identities and biases of fact checking entities, bad actors posing as fact checkers can spread misinformation online. SUMMARY This Summary introduces simplified concepts that are further described below in the Detailed Description of Illustrative Embodiments. This Summary is not intended to identify key features or essential features of the claimed subject matter and is not intended to be used to limit the scope of the claimed subject matter. A method is provided including detecting at a plurality of network locations a plurality of accuracy ratings of a plurality of media instances and detecting the plurality of media instances. A particular accuracy rating of one or more particular media instances is detected at a particular network location, and the one or more particular media instances are detected. A bias of the particular accuracy rating is determined based on the particular accuracy rating, the one or more particular media instances, the plurality of accuracy ratings, and the plurality of media instances. An indication is transmitted to a user via a computing device based on the bias of the particular accuracy rating. A further method is provided including detecting at a plurality of network locations a plurality of accuracy ratings of a plurality of media instances and detecting the plurality of media instances. A classifier is trained based on the plurality of accuracy ratings and the plurality of media instances. A particular accuracy rating of one or more particular media instances is detected at a particular network location. The one or more particular media instances are detected. The classifier is applied to the particular accuracy rating and the one or more particular media instances to determine a bias of the particular accuracy rating. An indication is transmitted to a user via a computing device based on the bias of the particular accuracy rating. A computing system is provided including one or more processors and one or more non-transitory computer readable storage media having encoded thereon instructions that when executed by the one or more processors cause the computing system to perform a process. The process includes detecting at a plurality of network locations a plurality of accuracy ratings of a plurality of media instances and detecting the plurality of media instances. The process also includes detecting at a particular network location a particular accuracy rating of one or more particular media instances, detecting the one or more particular media instances, and determining a bias of the particular accuracy rating based on the particular accuracy rating, the one or more particular media instances, the plurality of accuracy ratings, and the plurality of media instances. The process further includes transmitting an indication to a user based on the bias of the particular accuracy rating. A network-enabled evaluation system is provided including a first computing system and a second computing system. The first computing system includes at least a first processor and at least a first non-transitory computer readable storage medium having encoded thereon first instructions that when executed by the at least the first processor cause the first computing system to perform a first process. The first process includes detecting at a plurality of network locations a plurality of accuracy ratings of a plurality of media instances and detecting the plurality of media instances. The first process also includes detecting at a particular network location a particular accuracy rating of one or more particular media instances and detecting the one or more particular media instances. The first process further includes determining a bias of the particular accuracy rating based on the particular accuracy rating, the one or more particular media instances, the plurality of accuracy ratings, and the plurality of media instances. The second computing system includes at least a second processor and at least a second non-transitory computer readable storage medium having encoded thereon second instructions that when executed by the at least the second processor cause the second computing system to perform a second process. The second process includes detecting browsing at the particular network location by a user and receiving the bias of the particular accuracy rating from the first computing system. The second proc