US-12620226-B2 - Method and device for detecting violations

US12620226B2US 12620226 B2US12620226 B2US 12620226B2US-12620226-B2

Abstract

A method for detecting violations, includes: acquiring a plurality of image frames corresponding to a region to be detected; performing scene semantic segmentation on the plurality of image frames to obtain forbidden violation regions in the plurality of image frames; detecting the plurality of image frames by using a rotating target detection network, the rotating target detection network being used for detecting whether an image frame includes a target detection object; determining, in a case where at least one image frame in the plurality of image frames includes the target detection object, that a violation exists in the region to be detected according to the target detection object and a forbidden violation region corresponding to the target detection object.

Inventors

Pan ZHONG

Assignees

BOE TECHNOLOGY GROUP CO., LTD.

Dates

Publication Date: 20260505
Application Date: 20220328

Claims (17)

1 . A method for detecting violations, comprising: acquiring a plurality of image frames corresponding to a region to be detected; performing scene semantic segmentation on the plurality of image frames to obtain forbidden violation regions in the plurality of image frames; detecting the plurality of image frames by using a rotating target detection network, wherein the rotating target detection network is used for detecting whether an image frame includes a target detection object; and determining, in a case where at least one image frame in the plurality of image frames includes the target detection object, that a violation exists in the region to be detected according to the target detection object and a forbidden violation region corresponding to the target detection object; wherein determining that the violation exists in the region to be detected according to the target detection object and the forbidden violation region corresponding to the target detection object includes: determining, according to an intersection area of a region of a detection frame of the target detection object and the forbidden violation region corresponding to the target detection object, a ratio of the intersection area to an area of the detection frame of the target detection object; determining that the target detection object is in the forbidden violation region in a case where the ratio of the intersection area to the area of the detection frame of the target detection object is greater than or equal to a first preset threshold; and determining that the violation exists in the region to be detected according to a detection state count value of the target detection object; wherein the target detection object includes clothing and a pedestrian, and determining that the violation exists in the region to be detected according to the detection state count value of the target detection object includes: adding one to a detection state count value of the clothing in a case where a center point of a detection frame of the clothing is outside a region of a detection frame of at least one pedestrian; and determining that the violation exists in the region to be detected in a case where the detection state count value of the clothing is greater than or equal to a fourth preset threshold.
2 . The method according to claim 1 , wherein the target detection object further includes an advertisement.
3 . The method according to claim 2 , wherein in a case where the target detection object further includes the advertisement, determining that the violation exists in the region to be detected according to the detection state count value of the target detection object further includes: adding one to the detection state count value of the target detection object in a case where a displacement of coordinate points of the target detection object in two adjacent image frames in the at least one image frame is less than a second preset threshold; and determining that the violation exists in the region to be detected in a case where the detection state count value of the target detection object is greater than or equal to a third preset threshold.
4 . The method according to claim 1 , wherein an area of the detection frame of the at least one pedestrian is greater than or equal to a fifth preset threshold, and a center point of the detection frame of the at least one pedestrian is in the forbidden violation region corresponding to the target detection object.
5 . The method according to claim 1 , further comprising: deleting detection frames with areas less than a fifth preset threshold or center points outside the forbidden violation region in detection frames of pedestrians.
6 . The method according to claim 1 , wherein the rotating target detection network includes a Backbone network, a Neck network and a Head network, wherein an output of the Head network includes a predicted rotation angle of a detection frame.
7 . The method according to claim 6 , wherein detecting the plurality of image frames by using the rotating target detection network includes: preprocessing the plurality of image frames, and extracting feature data of the plurality of image frames through the Backbone network and the Neck network; and inputting the feature data of the plurality of image frames into the Head network to obtain a detection result of the target detection object, wherein the detection result of the target detection object includes at least one of the detection frame of the target detection object, region coordinates of the detection frame, the predicted rotation angle of the detection frame, or a category of the target detection object.
8 . The method according to claim 1 , further comprising: reporting alarm information and changing an alarm identifier in a case where the alarm identifier indicates that no alarm has occurred; and reporting the alarm information in a case where the alarm identifier indicates that the alarm has occurred and a time from a last alarm is greater than a preset time.
9 . A device for detecting violations, comprising: a processor and a transceiver, wherein the transceiver is configured to acquire a plurality of image frames corresponding to a region to be detected; and the processor is configured to: perform scene semantic segmentation on the plurality of image frames to obtain forbidden violation regions in the plurality of image frames; detect the plurality of image frames by using a rotating target detection network; and determine, in a case where at least one image frame in the plurality of image frames includes a target detection object, that a violation exists in the region to be detected according to the target detection object and a forbidden violation region corresponding to the target detection object; wherein the processor is further configured to: determine, according to an intersection area of a region of a detection frame of the target detection object and the forbidden violation region corresponding to the target detection object, a ratio of the intersection area to an area of the detection frame of the target detection object; determine that the target detection object is in the forbidden violation region in a case where the ratio of the intersection area to the area of the detection frame of the target detection object is greater than or equal to a first preset threshold; and determine that the violation exists in the region to be detected according to a detection state count value of the target detection object; wherein the target detection object includes clothing and a pedestrian, and the processor is further configured to: add one to the detection state count value of the clothing in a case where a center point of a detection frame of the clothing is outside a region of a detection frame of at least one pedestrian; and determine that the violation exists in the region to be detected in a case where the detection state count value of the clothing is greater than or equal to a fourth preset threshold.
10 . The device according to claim 9 , wherein the target detection object further includes an advertisement.
11 . The device according to claim 10 , wherein in a case where the target detection object further includes the advertisement, the processor is further configured to: add one to the detection state count value of the target detection object in a case where a displacement of coordinate points of the target detection object in two adjacent image frames in the at least one image frame is less than a second preset threshold; and determine that the violation exists in the region to be detected in a case where the detection state count value of the target detection object is greater than or equal to a third preset threshold.
12 . The device according to claim 9 , wherein an area of the detection frame of the at least one pedestrian is greater than or equal to a fifth preset threshold, and a center point of the detection frame of the at least one pedestrian is in the forbidden violation region corresponding to the target detection object.
13 . The device according to claim 9 , wherein the processor is further configured to: delete detection frames with areas less than a fifth preset threshold or center points outside the forbidden violation region in detection frames of pedestrians.
14 . The device according to claim 9 , wherein the rotating target detection network includes a Backbone network, a Neck network and a Head network; and an output of the Head network includes a predicted rotation angle of a detection frame.
15 . The device according to claim 14 , wherein the processor is further configured to: preprocess the plurality of image frames, and extract feature data of the plurality of image frames through the Backbone network and the Neck network; input the feature data of the plurality of image frames into the Head network to obtain a detection result of the target detection object, wherein the detection result of the target detection object includes at least one of the detection frame of the target detection object, region coordinates of the detection frame, the predicted rotation angle of the detection frame, or a category of the target detection object.
16 . A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions that, when executed by a detection device, implement the method for detecting violations according to claim 1 .
17 . A method for detecting violations, comprising: acquiring a plurality of image frames corresponding to a region to be detected; performing scene semantic segmentation on the plurality of image frames to obtain forbidden violation regions in the plurality of image frames; detecting the plurality of image frames by using a rotating target detection network, wherein the rotating target detection network is used for detecting whether an image frame includes a target detection object; and determining, in a case where at least one image frame in the plurality of image frames includes the target detection object, that a violation exists in the region to be detected according to the target detection object and a forbidden violation region corresponding to the target detection object; wherein determining that the violation exists in the region to be detected according to the target detection object and the forbidden violation region corresponding to the target detection object includes: determining, according to an intersection area of a region of a detection frame of the target detection object and the forbidden violation region corresponding to the target detection object, a ratio of the intersection area to an area of the detection frame of the target detection object; determining that the target detection object is in the forbidden violation region in a case where the ratio of the intersection area to the area of the detection frame of the target detection object is greater than or equal to a first preset threshold; and determining that the violation exists in the region to be detected according to a detection state count value of the target detection object; wherein the target detection object includes an advertisement, and determining that the violation exists in the region to be detected according to the detection state count value of the target detection object includes: adding one to the detection state count value of the target detection object in a case where a displacement of coordinate points of the target detection object in two adjacent image frames in the at least one image frame is less than a second preset threshold; and determining that the violation exists in the region to be detected in a case where the detection state count value of the target detection object is greater than or equal to a third preset threshold.

Description

CROSS-REFERENCE TO RELATED APPLICATION This application is a national phase entry under 35 USC 371 of International Patent Application No. PCT/CN2022/083516 filed on Mar. 28, 2022, which is incorporated herein by reference in its entirety. TECHNICAL FIELD The present disclosure relates to the field of intelligent detection technologies, and in particular, to a method for detecting violations and a device for detecting violations. BACKGROUND In order to ensure that the environment to be detected (e.g., residential area, park, office building, mall, etc.) is clean and tidy, violations in the environment to be detected may generally be detected. For example, violations such as illegal posting of advertisements and illegal drying of clothes in the environment to be detected may be detected. SUMMARY In one aspect, a method for detecting violations is provided, and the method includes: firstly, acquiring a plurality of image frames corresponding to a region to be detected; then, performing scene semantic segmentation on the plurality of image frames to obtain forbidden violation regions in the plurality of image frames; detecting the plurality of image frames by using a rotating target detection network, the rotating target detection network being used for detecting whether an image frame includes a target detection object; and finally, determining, in a case where at least one image frame in the plurality of image frames includes the target detection object, that a violation exists in the region to be detected according to the target detection object and a forbidden violation region corresponding to the target detection object. In some embodiments, the target detection object includes at least one of an advertisement, clothing, or a pedestrian. In some embodiments, determining that the violation exists in the region to be detected according to the target detection object and the forbidden violation region corresponding to the target detection object includes: determining, according to an intersection area of a region of a detection frame of the target detection object and the forbidden violation region corresponding to the target detection object, a ratio of the intersection area to an area of the detection frame of the target detection object; determining that the target detection object is in the forbidden violation region in a case where the ratio of the intersection area to the area of the detection frame of the target detection object is greater than or equal to a first preset threshold; and determining that the violation exists in the region to be detected according to a detection state count value of the target detection object. For example, in a case where the target detection object includes an advertisement, determining that the violation exists in the region to be detected according to the detection state count value of the target detection object includes: adding one to the detection state count value of the target detection object in a case where a displacement of coordinate points of the target detection object in two adjacent image frames in the at least one image frame is less than a second preset threshold; and determining that the violation exists in the region to be detected in a case where the detection state count value of the target detection object is greater than or equal to a third preset threshold. For example, in a case where a target detection object includes clothing and a pedestrian, determining that the violation exists in the region to be detected according to the detection state count value of the target detection object includes: adding one to a detection state count value of the clothing in a case where a center point of a detection frame of the clothing is outside a region of a detection frame of at least one pedestrian; and determining that the violation exists in the region to be detected in a case where the detection state count value of the clothing is greater than or equal to a fourth preset threshold. In some embodiments, an area of the detection frame of the at least one pedestrian is greater than or equal to a fifth preset threshold, and a center point of the detection frame of the at least one pedestrian is in the forbidden violation region corresponding to the target detection object. In some embodiments, the method for detecting violations further includes: deleting detection frames with areas less than the fifth preset threshold or center points outside the forbidden violation region in detection frames of pedestrians. In some embodiments, the rotating target detection network includes a Backbone network, a Neck network and a Head network, and an output of the Head network includes a predicted rotation angle of a detection frame. In some embodiments, detecting the plurality of image frames by using the rotating target detection network includes: preprocessing the plurality of image frames, and extracting feature data of the plurality of image frames through the Bac