Search

CN-121998888-A - Screen image detection method, device, electronic equipment, storage medium and program

CN121998888ACN 121998888 ACN121998888 ACN 121998888ACN-121998888-A

Abstract

The invention discloses a screen image detection method, a device, electronic equipment, a storage medium and a program, which are applied to the technical field of intelligent equipment, wherein the method comprises the steps of acquiring a screen image to be processed and determining a screen area image of the screen image to be processed; the method comprises the steps of utilizing a pre-trained key detection model to extract a key area in a screen area image, determining the color distribution condition of the key area and determining the key state of the key area based on the color distribution condition, and generating a key state matrix corresponding to the screen image to be processed according to the key area and the key state, wherein elements at each position in the key state matrix correspond to the key state of one key area. The embodiment of the invention can solve the problem of low success rate of detection of the existing intelligent screen, can perform state identification based on the color distribution of the keys in the screen, can reduce the influence of an image acquisition environment on screen detection, and can realize accurate detection of the equipment screen.

Inventors

  • CHEN XUHAO

Assignees

  • 北京罗克维尔斯科技有限公司

Dates

Publication Date
20260508
Application Date
20241104

Claims (12)

  1. 1. A screen image detection method, the method comprising: Acquiring a screen image to be processed, and determining a screen area image of the screen image to be processed; extracting a key region from the screen region image by using a pre-trained key detection model; Determining the color distribution condition of the key area, and determining the key state of the key area based on the color distribution condition; And generating a key state matrix corresponding to the screen image to be processed according to the key region and the key state, wherein elements at each position in the key state matrix correspond to the key state of one key region.
  2. 2. The method of claim 1, wherein the acquiring a screen image to be processed and determining a screen area image of the screen image to be processed comprises: Receiving the screen image to be processed, and converting the screen image to be processed into a gray level image to obtain a screen gray level image; performing binarization operation on the screen gray level image according to a preset gray level threshold value to obtain a screen mask image for processing the screen image area; and dividing the screen image to be processed according to the screen mask image to obtain the screen region image.
  3. 3. The method of claim 1, wherein extracting key regions within the screen region image using a pre-trained key detection model comprises: Extracting the key detection model, wherein a training set for training the key detection model comprises screen images passing through a marked key area; Inputting the screen area image to the key detection model, and obtaining a key identification result output by the key detection model; And extracting an image area from the screen area image according to the key identification result, and taking the extracted image area as the key area.
  4. 4. The method of claim 1, wherein the determining the color distribution of the key region and determining the key state of the key region based on the color distribution comprises: Counting Gaussian distribution of pixel values in the key area as the color distribution condition aiming at the key area; determining an image binarization threshold according to the color distribution condition, and performing segmentation operation on the key region through the image binarization threshold to obtain a key outline map; and determining the key state of the key area according to the proportion of the image area occupied by the key outline map in the key area.
  5. 5. The method of claim 4, wherein the counting gaussian distribution of pixel values in the key region for the key region as the color distribution condition includes: extracting pixel values of the pixel points aiming at each pixel point in the key area, wherein the pixel values at least comprise a red channel value, a green channel value and a blue channel value; Respectively counting the mean and variance of each red channel value, each green channel value and each blue channel value, determining a first Gaussian distribution function corresponding to the red channel value based on the mean and variance of the red channel value, determining a second Gaussian distribution function corresponding to the green channel value based on the mean and variance of the green channel value, and determining a third Gaussian distribution function corresponding to the blue channel value based on the mean and variance of the blue channel value, wherein the first Gaussian distribution function, the second Gaussian distribution function and the third Gaussian distribution function are used as the color distribution condition.
  6. 6. The method of claim 4, wherein said determining an image binarization threshold from said gaussian distribution comprises: and taking the pixel value corresponding to the peak of the probability of occurrence in the Gaussian distribution function image as the image binarization threshold value, or, Determining the occurrence probability of different pixel values in the Gaussian distribution, comparing the occurrence probability of each pixel value, determining a target pixel value corresponding to the highest occurrence probability, and taking the target pixel value as the image binarization threshold.
  7. 7. The method of claim 4, further comprising, after obtaining the key profile: and extracting key edge information and key size information of the key outline drawing to serve as key attribute information of the key area.
  8. 8. The method of claim 1, wherein the generating a key state matrix corresponding to the screen image to be processed according to the key region and the key state comprises: determining the position of each key area in the area of the screen image to be processed; constructing an initial matrix corresponding to each key area, and filling key state identifiers of the key states of each key area into the initial matrix to serve as the key state matrix according to the area positions.
  9. 9. A screen image detection apparatus, the apparatus comprising: The screen extraction module is used for acquiring a screen image to be processed and determining a screen area image of the screen image to be processed; the key extraction module is used for extracting a key region from the screen region image by utilizing a pre-trained key detection model; the state determining module is used for determining the color distribution condition of the key area and determining the key state of the key area based on the color distribution condition; and the result generation module is used for generating a key state matrix corresponding to the screen image to be processed according to the key region and the key state, wherein elements at each position in the key state matrix correspond to the key state of one key region.
  10. 10. An electronic device, the electronic device comprising: At least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the screen image detection method of any one of claims 1-8.
  11. 11. A computer readable storage medium storing computer instructions for causing a processor to perform the screen image detection method of any one of claims 1-8.
  12. 12. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the screen image detection method according to any of claims 1-8.

Description

Screen image detection method, device, electronic equipment, storage medium and program Technical Field The present invention relates to the field of intelligent devices, and in particular, to a method and apparatus for detecting a screen image, an electronic device, and a storage medium. Background In the age of rapid development of current technology, the intelligent screen plays a vital role in man-machine interaction, and can bring visual information display, provide a convenient operation mode and provide personalized experience for users. In addition, the intelligent screen also becomes a docking window for cooperation between different devices. Information interaction is carried out among different devices through the intelligent screen, so that the internal information of the devices can be prevented from being acquired by other devices, and the cooperative work of the intelligent devices is realized on the premise of ensuring the information security. Therefore, the accuracy of the identification and detection of the intelligent screen directly influences the cooperative operation result of the equipment and the man-machine interaction effect. At present, aiming at the recognition and detection of an intelligent screen, the problems of low recognition rate and false recognition exist, and the main reason is that the intelligent screen is shot, the intelligent screen is restricted by the environment, the shooting conditions and the illumination environment have large differences, the difference between the intelligent screen and a data set used in the training process of a detection network is large, and the detection network generated by training cannot process the acquired image of the intelligent screen. And the recognition performance is further reduced due to factors such as screen reflection, influence, inconsistent brightness, camera angle deviation and the like. The existing scheme for improving the intelligent screen detection and recognition capability is mainly realized by means of increasing training sample size, optimizing network structure and the like, but the influence caused by the environment cannot be fundamentally solved by the means. Disclosure of Invention The invention provides a screen detection method, a device, electronic equipment, a storage medium and a program, which are used for solving the problem of low success rate of detection of the existing intelligent screen, carrying out state identification based on color distribution of keys in the screen, reducing the influence of an image acquisition environment on screen detection and realizing accurate detection of the equipment screen. According to an aspect of the present invention, there is provided a screen image detection method, wherein the method includes: Acquiring a screen image to be processed, and determining a screen area image of the screen image to be processed; extracting a key region from the screen region image according to a pre-trained key detection model; Determining the color distribution condition of the key area, and determining the key state of the key area based on the color distribution condition; And generating a key state matrix corresponding to the screen image to be processed according to the key region and the key state, wherein elements at each position in the key state matrix correspond to the key state of one key region. According to an aspect of the present invention, there is provided a screen image detecting apparatus, wherein the apparatus includes: The screen extraction module is used for acquiring a screen image to be processed and determining a screen area image of the screen image to be processed; the key extraction module is used for extracting a key region from the screen region image by utilizing a pre-trained key detection model; the state determining module is used for determining the color distribution condition of the key area and determining the key state of the key area based on the color distribution condition; and the result generation module is used for generating a key state matrix corresponding to the screen image to be processed according to the key region and the key state, wherein elements at each position in the key state matrix correspond to the key state of one key region. According to another aspect of the present invention, there is provided an electronic apparatus including: At least one processor, and A memory communicatively coupled to the at least one processor, wherein, The memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the screen image detection method according to any one of the embodiments of the present invention. According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to implement the screen image detection method according to any one of the embodiments of the present inven