CN-116383180-B - Image processing method and device based on continuous and discrete features

Abstract

The invention discloses an image processing method based on continuous and discrete features, which comprises the steps of A, carrying out entity management, dimension management and fact management on data assets, B, carrying out image calculation based on a bitmap mode on the data assets, extracting image data from the data assets, C, detecting and evaluating the image data, including effective range deduction and processing and complete rate detection and missing value supplementation, and D, carrying out image data output based on a multi-channel delivery mode. The image processing method based on continuous and discrete features can balance the requirements of professional images and general images, integrate key elements in the professional images and the general images, and solve the problem of the breadth of the profession and the depth of the general images.

Inventors

• TIAN LIANCHAO

Assignees

• 北京农信数智科技有限公司

Dates

Publication Date

20260512

Application Date

20230128

Claims (2)

1. 1. An image processing method based on continuous and discrete features, characterized in that the method comprises the following steps: A. Entity management, dimension management and fact management are performed on the data assets; B. Performing bitmap pattern-based representation calculation on the data asset, and extracting representation data from the data asset; C. detecting and evaluating the image data, including effective range inference and processing, and complete rate detection and missing value supplementation; D. Outputting the portrait data based on a multi-channel delivery mode; The bitmap mode-based portrait calculation of the data asset comprises the steps of fact data calculation, bitmap calculation and analysis, wherein the fact data calculation cleans the data asset and models unified data, the data asset comprises the steps of packaging data calculation logic, scheduling corresponding scripts and extracting modeled portrait data; The bitmap calculation and analysis are carried out on the modeled portrait data, and then the compressed portrait data is stored in a relational database, and decompression is carried out when the relational database is inquired; The effective range deducing and processing comprises the steps of screening the portrait data effective for the current service through different characteristic combinations, and judging the abnormal value of the portrait data in the portrait of the data effective for the current service according to the outlier size of the portrait data to eliminate or empty; Wherein the completion rate detection and missing value supplementation comprises: The complete rate detection comprises the steps of detecting each field content of the portrait data, and identifying the duty ratio of illegal values and null values in each field content of the portrait data through aggregation to obtain the complete rate of each field content of the portrait data; the content completion rate of one field of the portrait data is larger than a first preset threshold value and is used as the portrait data of the field; The missing value supplement comprises filling illegal values and null values in the contents of each field of the portrait data, and filling according to one or more methods of a field correlation method, a business rule method, a randomness method and a random proportioning method; the image data output based on the multi-channel delivery mode comprises single entity output and group delineation, wherein the group delineation comprises the steps of selecting an image data group, displaying the statistical duty ratio of each field in the image data group, converting the image data stored in a relational database into an index in distributed full-text retrieval, and performing visual output, and the group delineation further comprises the steps of further setting custom labels for different entities in the image data group and performing custom group delineation output.
2. 2. An image processing device based on continuous and discrete features is characterized in that the device comprises a management control unit, a calculation unit, a detection and evaluation unit and an output unit, wherein, The management control unit is used for performing entity management, dimension management and fact management on the data assets; the computing unit is used for carrying out portrait computation based on a bitmap mode on the data asset and extracting portrait data from the data asset; the detection and evaluation unit is used for detecting and evaluating the image data, and comprises effective range inference and processing, complete rate detection and missing value supplementation; the output unit is used for outputting the portrait data based on a multi-channel delivery mode; the computing unit comprises a fact data computing unit, a bitmap computing and analyzing unit and a query caching unit, wherein, The fact data calculation unit is used for cleaning data assets and modeling unified data and comprises the steps of packaging data calculation logic, scheduling corresponding scripts and extracting modeled portrait data; The bitmap calculation and analysis unit is used for compressing the modeled portrait data, storing the compressed portrait data into the relational database, and decompressing the relational database when inquiring the relational database; The query buffer unit is used for providing image data for the detection and evaluation unit; The detection and evaluation unit comprises an effective range judging unit and a discrete value processing unit, wherein the effective range judging unit is used for screening the image data which is effective for the current service through different characteristic combinations; the detection and evaluation unit comprises a complete rate detection unit and a missing value supplementing unit, wherein, The system comprises a complete rate detection unit, a picture data auxiliary picture data processing unit, a picture data processing unit and a picture data processing unit, wherein the complete rate detection unit is used for detecting each field content of the picture data, and identifying the ratio of illegal values to empty values in each field content of the picture data through summarization to obtain the complete rate of each field content of the picture data; The missing value supplementing unit is used for filling illegal values and null values in the contents of each field of the portrait data according to one or more methods of a field correlation method, a business rule method, a randomness method and a random proportioning method; The detection and evaluation unit comprises a complete rate detection unit and a missing value supplementing unit, wherein the output unit comprises a single entity output unit and a group setting unit, the group setting unit is used for displaying the statistical duty ratio of each field in an image data group after selecting the image data group, converting image data stored in a relational database into an index in distributed full-text retrieval and performing visual output, and the group setting unit further comprises a user-defined label which is further set for different entities in the image data group and performs user-defined group setting output.

Description

Image processing method and device based on continuous and discrete features Technical Field The invention relates to big data analysis technology, in particular to an image processing method and device based on continuous and discrete features. Background The portrait user portrait of the user or the enterprise is characterized in that various characteristics of the user or the enterprise are identified, various labels are attached to the user or the enterprise through the identification, and the user or the enterprise is divided into different groups through the labels, so that products/operation operations are respectively carried out on the different groups. The current image methods are classified into two kinds, i.e., professional image methods and general image methods. The professional portrait method generally processes data of specific sources, guides specific demands of users, and generates targeted labels through set rules. While general-purpose images generally rarely relate to the field of profession. And (3) performing independent module design on links such as data processing, label setting, portrait generation, group delineation and the like so as to meet the use of portraits in different scenes. But in reality the requirements of the representation are complex and variable. Professional images are generally specific scene processing, when the requirements change (such as different formats of data, different quality, changing of image targets, etc.), the professional image method in the prior art is difficult to meet the requirement of rapid iteration, so that the universality is blocked, and the general image in the prior art is generally of functional design and is limited to specific implementation scenes, so that the special complex requirements are difficult to meet. Disclosure of Invention In order to solve the technical problems that the professional image method in the prior art is difficult to meet the requirement of quick iteration and the universality is hindered, the general image in the prior art generally belongs to functional design and is limited by specific implementation scenes and is difficult to meet specific complex requirements, the invention provides an image processing method and an image processing device based on continuous and discrete characteristics, which can balance the requirements of the professional image and the general image, integrate key elements in the requirements, and solve the problem of the breadth of the profession and the depth of the general image. In order to achieve the object, the present invention adopts the following technical scheme. A method of image processing based on continuous and discrete features, the method comprising the steps of: A. Entity management, dimension management and fact management are performed on the data assets; B. Performing bitmap pattern-based representation calculation on the data asset, and extracting representation data from the data asset; C. detecting and evaluating the image data, including effective range inference and processing, and complete rate detection and missing value supplementation; D. And outputting the portrait data based on the multi-channel delivery mode. In addition, in the portrait processing method based on continuous and discrete features of the present invention, the portrait calculation based on the bitmap pattern for the data asset includes the fact data calculation, the bitmap calculation and the analysis, wherein, The fact data calculation cleans data assets and models unified data, and comprises the steps of packaging data calculation logic, scheduling corresponding scripts and extracting modeled portrait data; the bitmap calculation and analysis compresses the modeled portrait data and stores the compressed portrait data in a relational database, and decompresses the relational database when inquiring the relational database. In the image processing method based on continuous and discrete features of the present invention, the effective range estimation and processing includes screening image data effective for the current service and determining image data having a large outlier as an outlier from among data images effective for the current service by different feature combinations, and removing or emptying the image data. In addition, in the image processing method based on continuous and discrete features of the present invention, the complete rate detection and missing value supplementation include: The complete rate detection comprises the steps of detecting each field content of the portrait data, and identifying the duty ratio of illegal values and null values in each field content of the portrait data through aggregation to obtain the complete rate of each field content of the portrait data; the content completion rate of one field of the portrait data is larger than a first preset threshold value and is used as the portrait data of the field; the missing value supplement co