CN-116310472-B - Screen detection method, screen detection model training method, device and computer equipment

CN116310472BCN 116310472 BCN116310472 BCN 116310472BCN-116310472-B

Abstract

The embodiment of the specification discloses a screen detection method, a screen detection model training method, a device and computer equipment. The screen detection method comprises the steps of obtaining a screen image of equipment, dividing the screen image to obtain a plurality of image blocks, extracting first characteristic data of the image blocks, fusing the first characteristic data of the image blocks to obtain a first characteristic data sequence, and detecting abnormality of a screen of the equipment according to the first characteristic data sequence to obtain a detection result. According to the technical scheme, the accuracy of screen detection can be improved.

Inventors

ZOU CHENG
CHENG YUAN
WANG QING
GE GUANFA
CHEN YUSHENG
YING XIAOCHENG

Assignees

支付宝(杭州)信息技术有限公司

Dates

Publication Date: 20260512
Application Date: 20221110

Claims (8)

1. A screen detection method, comprising: acquiring a screen image of the equipment; Dividing a screen image to obtain a plurality of image blocks, wherein the dividing the screen image comprises dividing a sub-image where a screen area in the screen image is located to obtain a plurality of image blocks, wherein the plurality of image blocks are used for detecting abnormality of a screen of the equipment in the screen area; Extracting first characteristic data of the image block; fusing the first characteristic data of the plurality of image blocks to obtain a first characteristic data sequence; The method comprises the steps of carrying out anomaly detection on a screen of equipment according to a first characteristic data sequence to obtain a detection result, wherein the first characteristic data sequence comprises a plurality of ordered second characteristic data, and carrying out anomaly detection on the screen of the equipment, and the method comprises the steps of inputting the second characteristic data at one preset position in the first characteristic data sequence into a classifier to obtain a detection result, or carrying out fusion on the second characteristic data at a plurality of preset positions in the first characteristic data sequence, inputting the fusion result into the classifier to obtain the detection result, or inputting the second characteristic data at a plurality of preset positions in the first characteristic data sequence into a plurality of classifiers to obtain a plurality of sub-detection results, and carrying out fusion on the plurality of sub-detection results to obtain the detection result.
2. The method of claim 1, the segmenting the screen image comprising: sliding the sliding window in the screen image according to a set step length to obtain a plurality of image blocks; the set step length is smaller than the side length of the sliding window.
3. The method of claim 1, the extracting first feature data of an image block, comprising: flattening the image block to obtain first characteristic data; the fusing the first feature data of the plurality of image blocks includes: determining the position of corresponding first characteristic data according to the position of the image block in the screen image; constructing a second characteristic data sequence according to the position of the first characteristic data; and encoding the second characteristic data sequence to obtain a first characteristic data sequence.
4. The method of claim 1, the extracting first feature data of an image block, comprising: extracting first characteristic data of the image block through a conversion layer in the screen detection model; the fusing the first feature data of the plurality of image blocks includes: Fusing the first characteristic data of the plurality of image blocks through a fusion layer in the screen detection model; the abnormality detection for the screen of the equipment comprises the following steps: And detecting the abnormality of the screen of the equipment according to the first characteristic data sequence by using a classifier in the screen detection model.
5. A screen detection model training method, comprising: The method comprises the steps of obtaining a plurality of image blocks after dividing a screen image of equipment, wherein the step of dividing the screen image of the equipment comprises the steps of dividing a sub-image where a screen area in the screen image is located, and the plurality of image blocks are used for detecting abnormality of the screen of the equipment in the screen area; Acquiring first characteristic data of an image block through a conversion layer in a screen detection model; Fusing the plurality of first characteristic data through a fusion layer in the screen detection model to obtain a first characteristic data sequence; The method comprises the steps of inputting second characteristic data at a preset position in a first characteristic data sequence into a classifier to obtain a detection result or fusing the second characteristic data at a plurality of preset positions in the first characteristic data sequence, inputting the fusion result into the classifier to obtain the detection result, inputting the second characteristic data at a plurality of preset positions in the first characteristic data sequence into a plurality of classifiers to obtain a plurality of sub-detection results under the condition that the screen detection model comprises one classifier, and fusing the plurality of sub-detection results to obtain the detection result; and according to the detection result, adjusting parameters of the screen detection model to obtain the trained screen detection model.
6. A screen detection apparatus comprising: an acquisition unit configured to acquire a screen image of the apparatus; The dividing unit is used for dividing the screen image to obtain a plurality of image blocks, wherein the dividing of the screen image comprises dividing a sub-image where a screen area in the screen image is located to obtain a plurality of image blocks, and the plurality of image blocks are used for detecting the abnormality of the screen of the equipment in the screen area; an extraction unit for extracting first feature data of the image block; The fusion unit is used for fusing the first characteristic data of the plurality of image blocks to obtain a first characteristic data sequence; the detection unit is used for carrying out abnormal detection on a screen of the equipment according to a first characteristic data sequence to obtain a detection result, wherein the first characteristic data sequence comprises a plurality of ordered second characteristic data, the abnormal detection on the screen of the equipment comprises the steps of inputting the second characteristic data at one preset position in the first characteristic data sequence into a classifier to obtain the detection result, or fusing the second characteristic data at a plurality of preset positions in the first characteristic data sequence, inputting the fusion result into the classifier to obtain the detection result, or inputting the second characteristic data at a plurality of preset positions in the first characteristic data sequence into a plurality of classifiers to obtain a plurality of sub-detection results, and fusing the plurality of sub-detection results to obtain the detection result.
7. A screen detection model training apparatus, comprising: The first acquisition unit is used for acquiring a plurality of image blocks after dividing the screen image of the equipment, wherein the dividing of the screen image of the equipment comprises dividing a sub-image where a screen area in the screen image is located, and the plurality of image blocks are used for detecting abnormality of the screen of the equipment in the screen area; The second acquisition unit is used for acquiring the first characteristic data of the image block through a conversion layer in the screen detection model; the fusion unit is used for fusing the plurality of first characteristic data through a fusion layer in the screen detection model to obtain a first characteristic data sequence; The detection unit is used for carrying out abnormal detection on a screen of the equipment according to a first characteristic data sequence by using a classifier in a screen detection model, wherein the first characteristic data sequence comprises a plurality of ordered second characteristic data, and the abnormal detection is carried out on the screen of the equipment, and comprises the steps of inputting the second characteristic data at one preset position in the first characteristic data sequence into the classifier to obtain a detection result or inputting the second characteristic data at a plurality of preset positions in the first characteristic data sequence into the classifier to obtain a detection result under the condition that the screen detection model comprises a plurality of classifiers, inputting the second characteristic data at a plurality of preset positions in the first characteristic data sequence into a plurality of classifiers to obtain a plurality of sub-detection results, and fusing the plurality of sub-detection results to obtain the detection result; And the adjusting unit is used for adjusting parameters of the screen detection model according to the detection result to obtain the trained screen detection model.
8. A computer device, comprising: At least one processor; A memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of any of claims 1-5.

Description

Screen detection method, screen detection model training method, device and computer equipment Technical Field The embodiment of the specification relates to the technical field of computers, in particular to a screen detection method, a screen detection model training device and computer equipment. Background In some application scenarios, it is necessary to perform abnormality detection on a screen of a device, for example, to detect whether the screen of the device is broken. The screen may include a cell phone screen, a tablet computer screen, a notebook computer screen, etc. In the related art, a screen image of a device may be acquired. The size of the screen image is often large, and in order to reduce the operation amount and improve the detection efficiency, the screen image may be subjected to reduction processing. The screen image after the shrinking treatment can be input into the model to obtain the detection result output by the model. However, the resolution of the screen image is reduced by performing the reduction process on the screen image, which affects the accuracy of the detection result. Disclosure of Invention The embodiment of the specification provides a screen detection method, a screen detection device and computer equipment, so as to improve the accuracy of screen detection. The technical solutions of the embodiments of the present specification are as follows. In a first aspect of embodiments of the present disclosure, there is provided a screen detection method, including: acquiring a screen image of the equipment; Dividing the screen image to obtain a plurality of image blocks; Extracting first characteristic data of the image block; fusing the first characteristic data of the plurality of image blocks to obtain a first characteristic data sequence; and according to the first characteristic data sequence, carrying out abnormality detection on a screen of the equipment to obtain a detection result. In a second aspect of embodiments of the present disclosure, a screen detection model training method is provided, including: acquiring a plurality of image blocks after dividing a screen image of the equipment; Acquiring first characteristic data of an image block through a conversion layer in a screen detection model; Fusing the plurality of first characteristic data through a fusion layer in the screen detection model to obtain a first characteristic data sequence; abnormality detection is carried out on a screen of the equipment according to the first characteristic data sequence through a classifier in the screen detection model; and according to the detection result, adjusting parameters of the screen detection model to obtain the trained screen detection model. In a third aspect of embodiments of the present specification, there is provided a screen detection apparatus including: an acquisition unit configured to acquire a screen image of the apparatus; the dividing unit is used for dividing the screen image to obtain a plurality of image blocks; an extraction unit for extracting first feature data of the image block; The fusion unit is used for fusing the first characteristic data of the plurality of image blocks to obtain a first characteristic data sequence; And the detection unit is used for carrying out abnormal detection on the screen of the equipment according to the first characteristic data sequence to obtain a detection result. In a fourth aspect of embodiments of the present disclosure, there is provided a screen detection model training apparatus, including: a first acquisition unit configured to acquire a plurality of image blocks obtained by dividing a screen image of a device; The second acquisition unit is used for acquiring the first characteristic data of the image block through a conversion layer in the screen detection model; the fusion unit is used for fusing the plurality of first characteristic data through a fusion layer in the screen detection model to obtain a first characteristic data sequence; The detection unit is used for detecting the abnormality of the screen of the equipment according to the first characteristic data sequence through the classifier in the screen detection model; And the adjusting unit is used for adjusting parameters of the screen detection model according to the detection result to obtain the trained screen detection model. In a fifth aspect of embodiments of the present specification, there is provided a computer apparatus comprising: At least one processor; A memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of the first or second aspect. According to the technical scheme provided by the embodiment of the specification, the screen image of the equipment can be acquired, the screen image can be segmented to obtain a plurality of image blocks, first character