CN-117010400-B - Sensitive word detection method and device, electronic equipment and storage medium

CN117010400BCN 117010400 BCN117010400 BCN 117010400BCN-117010400-B

Abstract

The application discloses a method, a device, electronic equipment and a storage medium for detecting sensitive words, belongs to the field of communication, and solves the problem that illegal words in related technologies cannot be effectively intercepted. The method comprises the steps of receiving a report message for first information sent by a first terminal corresponding to a first user, obtaining at least one piece of pre-stored reported information matched with the first key word from a preset first database according to a first key word in the sensitive information, determining that report is successful and determining the first key word as a target sensitive word if the ratio of the number of the at least one piece of reported information to the number of all pieces of information stored in the first database is greater than a first preset threshold value, storing the target sensitive word into a preset second database to obtain an updated second database, and performing sensitive word detection operation on the received message to be processed according to the updated second database.

Inventors

LIU JUAN
Xie Xinbiao
SONG RUI
Ban Zhengbo
QIN ZHICHAO
DONG YU

Assignees

中国移动通信集团贵州有限公司
中国移动通信集团有限公司

Dates

Publication Date: 20260508
Application Date: 20220428

Claims (8)

1. A method for detecting a sensitive word, comprising: Receiving a report message for first information sent by a first terminal corresponding to a first user, wherein the report message comprises the first information, and sensitive information designated by the first user is marked in the first information; Acquiring at least one piece of pre-stored reported information matched with the first keyword from a preset first database according to the first keyword in the sensitive information, wherein the first database is used for storing information after reporting is successful; if the ratio of the number of the at least one reported information to the number of all the information stored in the first database is greater than a first preset threshold, determining that the reporting is successful and determining the first keyword as a target sensitive word; storing the target sensitive words into a preset second database to obtain an updated second database; performing sensitive word detection operation on the received message to be processed according to the updated second database; Before receiving the report message for the first information sent by the first terminal corresponding to the first user, the method further includes: receiving second information sent by a second terminal corresponding to a second user; According to the pre-stored sensitive words in the second database, if the sensitive words are detected to be contained in the second information, the sensitive words in the second information are filtered to obtain the first information, and the first information is sent to the first terminal; If the second information is detected to not contain sensitive information, determining the second information as the first information and sending the first information to the first terminal; After receiving the second information sent by the second terminal corresponding to the second user, the method further comprises: Determining a second keyword in the second information; determining emotion attributes corresponding to the second keywords; determining the second keyword as a first sensitive word and the second information as first sensitive information under the condition that the emotion attribute is anger; Storing the first sensitive word to the second database and storing the first sensitive information to the first database; The determining the second keyword as the first sensitive word and the second information as the first sensitive information includes: Receiving third information replied by the first user aiming at the second information; Determining a third keyword in the third information; Determining emotion attributes corresponding to the third keywords; determining the third information as second sensitive information in case the emotional attribute is a negative emotion; And if the ratio of the number of the second sensitive information to the number of all the information stored in the first database is greater than a second preset threshold, determining the second keyword as the first sensitive word, and determining the second information as the first sensitive information.
2. The method of claim 1, further comprising, after said determining said second keyword as said first sensitive word and said second information as said first sensitive information: storing the second sensitive information to the first database.
3. The method for detecting a sensitive word according to claim 1, wherein the determining the emotion attribute corresponding to the third keyword includes: And determining the emotion attribute corresponding to the third keyword as a negative emotion under the condition that the third keyword is matched with the sensitive word stored in the second database in advance.
4. The method for detecting sensitive words according to claim 1, wherein the sensitive words in the second database are sensitive words obtained from internet, and wherein the sensitive words include malicious words and/or guide consuming words.
5. A sensitive word detection apparatus for performing the steps of the method of claim 1, comprising: The receiving module is used for receiving a report message for first information sent by a first terminal corresponding to a first user, wherein the report message comprises the first information, and sensitive information appointed by the first user is marked in the first information; The acquisition module is used for acquiring at least one piece of reported information which is stored in advance and matched with the first keyword from a preset first database according to the first keyword in the sensitive information, wherein the first database is used for storing the information after reporting successfully; The determining module is used for determining that the reporting is successful and determining the first keyword as a target sensitive word if the ratio of the number of the at least one reported information to the number of all the information stored in the first database is greater than a first preset threshold; the storage module is used for storing the target sensitive words into a preset second database to obtain an updated second database; And the detection module is used for carrying out sensitive word detection operation on the received message to be processed according to the updated second database.
6. The apparatus for detecting a sensitive word as claimed in claim 5, wherein the receiving module is further configured to: receiving second information sent by a second terminal corresponding to a second user; According to the pre-stored sensitive words in the second database, if the sensitive words are detected to be contained in the second information, the sensitive words in the second information are filtered to obtain the first information, and the first information is sent to the first terminal; and if the second information is detected to not contain sensitive information, determining the second information as the first information and sending the first information to the first terminal.
7. An electronic device comprising a processor, a memory and a program or instruction stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the method of detecting a sensitive word as claimed in any one of claims 1 to 4.
8. A readable storage medium, wherein a program or instructions is stored on the readable storage medium, which when executed by a processor, implements the steps of the method for detecting a sensitive word as claimed in any one of claims 1-4.

Description

Sensitive word detection method and device, electronic equipment and storage medium Technical Field The application belongs to the field of communication, and particularly relates to a method and a device for detecting sensitive words, electronic equipment and a storage medium. Background In the related art, when a server receives a message sent by a client, it needs to check the illegal words in the message, and if the illegal words exist in the message, the illegal words are replaced or deleted. The illegal words in the related art are all sensitive words or guide consumption words which are set in advance. However, once the preset illegal vocabulary is determined, the illegal vocabulary is difficult to change, and the vocabulary in the message is rapidly evolved along with the propagation of the network information, so that the illegal vocabulary in the related technology is easy to lose timeliness in the use process, and the illegal vocabulary in the message cannot be comprehensively intercepted. Disclosure of Invention The embodiment of the application provides a method, a device, electronic equipment and a storage medium for detecting sensitive words, which can solve the problem that illegal words in the related technology cannot be effectively intercepted. In a first aspect, an embodiment of the present application provides a method for detecting a sensitive word, where the method includes receiving a report message for first information sent by a first terminal corresponding to a first user, where the report message includes the first information, and the first information is marked with sensitive information specified by the first user, acquiring, according to a first keyword in the sensitive information, at least one pre-stored reported information matched with the first keyword from a pre-set first database, where the first database is used to store information after reporting successfully, determining that reporting succeeds and determining the first keyword as a target sensitive word if a ratio of a number of the at least one reported information to a number of all information stored in the first database is greater than a first preset threshold, storing the target sensitive word in a pre-set second database, acquiring, according to the first keyword in the sensitive information, the updated second database, and performing a sensitive word detection operation on a received message to be processed. In a second aspect, the embodiment of the application provides a sensitive word detection device, which comprises a receiving module, a storage module, an acquisition module and a processing module, wherein the receiving module is used for receiving a reporting message aiming at first information sent by a first terminal corresponding to a first user, the first information comprises the first information, sensitive information designated by the first user is marked in the first information, the acquisition module is used for acquiring at least one pre-stored reported information matched with a first keyword from a pre-set first database according to the first keyword in the sensitive information, the first database is used for storing the information after the reporting is successful, the determination module is used for determining that the reporting is successful and determining the first keyword as a target sensitive word if the ratio of the number of the at least one reported information to the number of all the information stored in the first database is larger than a first preset threshold, the storage module is used for storing the target sensitive word into the pre-set second database according to the first keyword, the second database is acquired after the second database is updated, and the detection module is used for processing the received sensitive information after the second database is updated according to the detection operation. In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a program or instruction stored on the memory and executable on the processor, the program or instruction implementing the steps of the method according to the first aspect when executed by the processor. In a fourth aspect, embodiments of the present application provide a readable storage medium having stored thereon a program or instructions which when executed by a processor perform the steps of the method according to the first aspect. In a fifth aspect, an embodiment of the present application provides a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and where the processor is configured to execute a program or instructions to implement a method according to the first aspect. In the embodiment of the application, the first information is sent by a first terminal corresponding to a first user through receiving the reporting information of th