CN-117237841-B - Lens segmentation method, device, electronic equipment and computer readable storage medium

CN117237841BCN 117237841 BCN117237841 BCN 117237841BCN-117237841-B

Abstract

The embodiment of the invention provides a shot segmentation method, a device, electronic equipment and a computer readable storage medium, which relate to the technical field of security protection and comprise the steps of improving the stability of a shot segmentation algorithm by calculating the similarity between video frames, introducing video background information by a background frame mode, fully considering the integral change of video semantics, helping to guide the segmentation of a double-threshold algorithm on a video shot, and improving the information content and semantic integrity of an output shot. In addition, the background information is combined with the depth of the video semantics, so that shots with low information content in the video are effectively removed, the value of shot information is improved, and the rapid positioning and recognition of specific targets in the video are facilitated.

Inventors

LIU JITONG
XIE FUJIN
WANG ZHIHAI
YU BO

Assignees

北京明朝万达科技股份有限公司

Dates

Publication Date: 20260505
Application Date: 20230912

Claims (8)

1. A shot cut method, the method comprising: S1, acquiring an adjacent inter-frame similarity array and a background inter-frame similarity array, wherein the adjacent inter-frame similarity array comprises a plurality of adjacent inter-frame similarities which are sequentially arranged, each adjacent inter-frame similarity is calculated based on any two adjacent video frames in a video, the background inter-frame similarity array comprises a plurality of background inter-frame similarities which are sequentially arranged, and each background inter-frame similarity is calculated based on each video frame in the video and the background frame of the video; s2, determining an adjacent inter-frame difference degree array based on the adjacent inter-frame similarity degree array, and determining a background inter-frame difference degree array based on the background inter-frame similarity degree array; S3, determining average difference degree between background frames of the video based on the difference degree group between the background frames; S4, assigning an initial index and a termination index as m, and acquiring an nth adjacent inter-frame difference degree from the adjacent inter-frame difference degree array, wherein m and n are positive integers; s5, if the difference degree between the nth adjacent frames is not smaller than a mutation threshold, the nth video frame in the video is in mutation transition, S8 is executed, and otherwise S6 is executed; S6, if the difference degree between the nth adjacent frames is in the mutation threshold value and gradual change threshold value interval, and the candidate difference degree of the nth video frame is not smaller than the mutation threshold value, the nth video frame is in a gradual change transition state, S8 is executed, and otherwise S7 is executed; S7, if the difference degree between the nth adjacent frames is smaller than the gradual change threshold, executing S9; s8, if the average value of the background frame difference degrees belonging to the index range [ m, n ] in the background frame difference degree array is larger than the average difference degree between the background frames, storing the index range into the index array, and executing m=n+1; s9, executing n=n+1, and S4 until all adjacent inter-frame difference degrees in the adjacent inter-frame difference degree array are obtained, and executing S10; s10, dividing the video into a plurality of shot files based on the index array; The determining an adjacent inter-frame difference array based on the adjacent inter-frame similarity array, and determining a background inter-frame difference array based on the background inter-frame similarity array, includes: Normalizing the adjacent inter-frame similarity group to a preset interval to obtain a normalized adjacent inter-frame similarity group, and normalizing the background inter-frame similarity group to the preset interval to obtain a normalized background inter-frame similarity group; Inverting the normalized adjacent inter-frame similarity array to obtain an adjacent inter-frame difference array, and inverting the normalized background inter-frame similarity array to obtain a background inter-frame difference array.
2. The shot cut method according to claim 1, wherein the adjacent inter-frame similarity array and the background inter-frame similarity array are generated by: Acquiring the first k video frames of the video, wherein k is a positive integer; calculating the average value of the previous k video frames to obtain a background frame of the video; Calculating the similarity between all two adjacent video frames in the video based on the sequence of the video frames to obtain an adjacent inter-frame similarity array containing a plurality of adjacent inter-frame similarities which are sequentially arranged; And calculating the similarity between each video frame in the video and the background frame based on the sequence of the video frames to obtain a background frame similarity array containing the similarity between a plurality of background frames which are sequentially arranged.
3. The shot cut method of claim 1, wherein said determining an average degree of difference between background frames of the video based on the set of degrees of difference between background frames comprises: calculating the sum of all background inter-frame differences in the background inter-frame difference array; And calculating the ratio of the sum to the length of the array of the difference degrees among the background frames to obtain the average difference degree among the background frames of the video.
4. The shot cut method according to claim 1, wherein if the difference between the nth adjacent frames is within the mutation threshold and gradual change threshold interval and the candidate difference of the nth video frame is not less than the mutation threshold, the nth video frame is in a gradual change transition state, comprising: if the difference degree between the nth adjacent frames is in the mutation threshold value and gradual change threshold value interval, calculating the candidate difference degree of the nth video frame; and if the candidate difference degree of the nth video frame is not less than the abrupt change threshold, the nth video frame is in a gradual transition state.
5. A shot cut apparatus, the apparatus comprising: The acquisition module is used for acquiring an adjacent inter-frame similarity array and a background inter-frame similarity array, wherein the adjacent inter-frame similarity array comprises a plurality of adjacent inter-frame similarities which are sequentially arranged, each adjacent inter-frame similarity is calculated based on any two adjacent video frames in a video, the background inter-frame similarity array comprises a plurality of background inter-frame similarities which are sequentially arranged, and each background inter-frame similarity is calculated based on each video frame in the video and the background frame of the video; The determining module is used for determining an adjacent inter-frame difference degree array based on the adjacent inter-frame similarity degree array and determining a background inter-frame difference degree array based on the background inter-frame similarity degree array; the determining module is further configured to determine an average difference degree between background frames of the video based on the background frame difference degree group; the assignment module is used for assigning the start and stop indexes to m; The acquisition module is further configured to acquire an nth adjacent inter-frame difference from the adjacent inter-frame difference array, where m and n are positive integers; The executing module is used for calling the storage module if the difference degree between the nth adjacent frames is not smaller than the mutation threshold value and the nth video frame in the video is in mutation transition, otherwise, calling the executing module; The execution module is further configured to, if the difference between the nth adjacent frames is within the mutation threshold and gradual change threshold interval, and if the candidate difference between the nth video frame is not less than the mutation threshold, make the nth video frame be in a gradual change transition state, call the storage module, and otherwise, call the execution module; The execution module is further configured to invoke the execution module if the difference between the nth adjacent frames is less than the gradual change threshold; The storage module is used for storing the index range into the index array and executing m=n+1 if the average value of the background inter-frame difference degrees belonging to the index range [ m, n ] in the background inter-frame difference degree array is larger than the average difference degree between the background frames; the execution module is further configured to execute n=n+1, and call the assignment module until all the adjacent inter-frame difference degrees in the adjacent inter-frame difference degree array are obtained, and call the segmentation module; The segmentation module is used for segmenting the video into a plurality of shot files based on the index array; the determining module is specifically configured to: Normalizing the adjacent inter-frame similarity group to a preset interval to obtain a normalized adjacent inter-frame similarity group, and normalizing the background inter-frame similarity group to the preset interval to obtain a normalized background inter-frame similarity group; Inverting the normalized adjacent inter-frame similarity array to obtain an adjacent inter-frame difference array, and inverting the normalized background inter-frame similarity array to obtain a background inter-frame difference array.
6. The shot cut apparatus of claim 5, wherein the set of neighboring inter-frame similarities and the set of background inter-frame similarities are generated by: Acquiring the first k video frames of the video, wherein k is a positive integer; calculating the average value of the previous k video frames to obtain a background frame of the video; Calculating the similarity between all two adjacent video frames in the video based on the sequence of the video frames to obtain an adjacent inter-frame similarity array containing a plurality of adjacent inter-frame similarities which are sequentially arranged; And calculating the similarity between each video frame in the video and the background frame based on the sequence of the video frames to obtain a background frame similarity array containing the similarity between a plurality of background frames which are sequentially arranged.
7. An electronic device comprising a processor, a memory and a computer program stored on the memory and capable of running on the processor, the computer program implementing the steps of the shot cut method according to any one of claims 1-4 when executed by the processor.
8. A computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the steps of the shot segmentation method according to any one of claims 1-4 are implemented.

Description

Lens segmentation method, device, electronic equipment and computer readable storage medium Technical Field The present invention relates to the field of security technologies, and in particular, to a shot segmentation method, a shot segmentation apparatus, an electronic device, and a computer readable storage medium. Background With the development of communication technology and the improvement of hardware equipment performance, the trend of information communication transmission to video transfer in China is increasingly remarkable. Currently, the application range of the video monitoring technology is wider and wider, so that not only is the street and the roadway visible everywhere, but also security monitoring equipment is moved into the family life of people. As a comprehensive media form integrating visual, auditory and text information, video plays an important role in various fields. The hierarchical structure of video can be divided into three levels of logical units of frames, shots, and scenes from the upper layer to the lower layer. As the minimum component unit of video semantics, a shot transition state exists between two adjacent video shots, and the transition state of the shots can be divided into abrupt transition and gradual transition according to the difference of transition and speed semantic contents. The detection and segmentation of the monitoring video shots are key upstream tasks of video key frame extraction, and have important significance for video content identification and analysis. The video shot segmentation is to detect the boundary of each shot in the video by utilizing a video shot boundary detection algorithm, and then divide the video into a plurality of independent shot units according to the detection result. At present, a common video shot boundary detection algorithm is a double-threshold method, but the method has the following problems: (1) The static target in the video cannot be considered, so that the unified lens is divided into a plurality of lenses; (2) Video semantic features are not considered, so that the semantics of the output shots are incomplete; (3) The lens with low information content cannot be removed, and the information content of the output lens is reduced. Disclosure of Invention In view of the above, embodiments of the present invention have been made to provide a shot segmentation method, a shot segmentation apparatus, an electronic device, and a computer-readable storage medium that overcome or at least partially solve the above problems. In order to solve the above problems, an embodiment of the present invention discloses a shot segmentation method, which includes: S1, acquiring an adjacent inter-frame similarity array and a background inter-frame similarity array, wherein the adjacent inter-frame similarity array comprises a plurality of adjacent inter-frame similarities which are sequentially arranged, each adjacent inter-frame similarity is calculated based on any two adjacent video frames in a video, the background inter-frame similarity array comprises a plurality of background inter-frame similarities which are sequentially arranged, and each background inter-frame similarity is calculated based on each video frame in the video and the background frame of the video; s2, determining an adjacent inter-frame difference degree array based on the adjacent inter-frame similarity degree array, and determining a background inter-frame difference degree array based on the background inter-frame similarity degree array; S3, determining average difference degree between background frames of the video based on the difference degree group between the background frames; S4, assigning an initial index and a termination index as m, and acquiring an nth adjacent inter-frame difference degree from the adjacent inter-frame difference degree array, wherein m and n are positive integers; s5, if the difference degree between the nth adjacent frames is not smaller than a mutation threshold, the nth video frame in the video is in mutation transition, S8 is executed, and otherwise S6 is executed; S6, if the difference degree between the nth adjacent frames is in the mutation threshold value and gradual change threshold value interval, and the candidate difference degree of the nth video frame is not smaller than the mutation threshold value, the nth video frame is in a gradual change transition state, S8 is executed, and otherwise S7 is executed; S7, if the difference degree between the nth adjacent frames is smaller than the gradual change threshold, executing S9; s8, if the average value of the background frame difference degrees belonging to the index range [ m, n ] in the background frame difference degree array is larger than the average difference degree between the background frames, storing the index range into the index array, and executing m=n+1; s9, executing n=n+1, and S4 until all adjacent inter-frame difference degrees in the adjacent inter