CN-121999127-A - Tongue three-dimensional form reconstruction method and system based on deep learning

CN121999127ACN 121999127 ACN121999127 ACN 121999127ACN-121999127-A

Abstract

The invention provides a tongue three-dimensional morphological reconstruction method and a tongue three-dimensional morphological reconstruction system based on deep learning, which relate to the technical field of three-dimensional reconstruction, wherein the method comprises the steps of activating an image acquisition array based on illumination parameters of a light source to perform multi-angle two-dimensional sensing on a target tongue so as to obtain an original tongue image sequence; the method comprises the steps of performing visual cutting, extracting a plurality of tongue target images, performing deep learning to construct a tongue depth map, performing three-dimensional morphological analysis to construct tongue three-dimensional point cloud coordinates, mapping the tongue three-dimensional point cloud coordinates to the plurality of tongue target images for identification, performing comparative analysis on identification results and a standard gallery to generate a comparative parameter set, correcting, performing three-dimensional reconstruction on the target tongue according to the correction results, and constructing a three-dimensional morphological model. The invention solves the technical problems that the tongue analysis method in the prior art depends on two-dimensional images or manual observation, often has strong subjectivity and serious information loss, and is difficult to obtain the real three-dimensional shape of the tongue.

Inventors

LI TONG
ZHANG JIE

Assignees

无锡智观科技有限公司

Dates

Publication Date: 20260508
Application Date: 20251225

Claims (10)

1. The tongue three-dimensional morphology reconstruction method based on deep learning is characterized by comprising the following steps of: activating an image acquisition array based on light source illumination parameters to perform multi-angle two-dimensional sensing on the target tongue so as to obtain an original tongue surface image sequence; traversing the original lingual image sequence to perform visual cutting, extracting a plurality of lingual target images, performing deep learning based on the lingual target images, and constructing a lingual depth map; Carrying out three-dimensional morphological analysis based on the tongue depth map, constructing tongue three-dimensional point cloud coordinates, mapping the tongue three-dimensional point cloud coordinates to the plurality of tongue target images for identification, carrying out comparison analysis on identification results and a standard gallery, and generating a comparison parameter set; Correcting the tongue three-dimensional point cloud coordinates based on the comparison parameter set, and carrying out three-dimensional reconstruction on the target tongue according to the correction result to construct a three-dimensional morphological model.
2. The depth learning-based tongue three-dimensional morphological reconstruction method according to claim 1, wherein the method comprises the steps of activating an image acquisition array based on light source illumination parameters to perform multi-angle two-dimensional sensing on a target tongue to obtain an original tongue image sequence: Performing illumination simulation analysis according to tongue tissue reflection characteristics, and setting a standard illumination parameter set; Traversing the image acquisition array to perform joint calibration, and constructing a camera spatial attitude relation external parameter matrix; activating an image acquisition array based on the standard illumination parameter set, and synchronously carrying out continuous multi-angle two-dimensional capturing on the target tongue according to the external parameter matrix of the spatial attitude relation of the camera to obtain a plurality of image data sets; And carrying out structural sorting on the plurality of image data sets according to the acquisition time sequence to obtain the original lingual image sequence.
3. The depth learning-based tongue three-dimensional morphological reconstruction method according to claim 1, wherein the step of traversing the original tongue image sequence to perform visual cutting, extracting a plurality of tongue target images, performing depth learning based on the plurality of tongue target images, and constructing a tongue depth map comprises the steps of: traversing the original lingual image sequence to perform color space conversion to obtain a target color space; performing background separation on a target tongue body based on the target color space, determining a background threshold range, and dividing according to the background threshold range to generate initial binarized tongue body mask parameters; Performing morphological closing operation based on the initial binarization tongue mask parameters, marking internal hole data, filling the internal hole data, and generating a first processing result; Performing morphological opening operation based on the initial binarization tongue mask parameters, identifying discrete noise point data, and removing the discrete noise point data to obtain a tongue region mask; performing external rectangle calculation according to the tongue region mask, cutting the original tongue image sequence according to a calculation result, and extracting a rectangle region image set; And carrying out pixel value normalization processing on the rectangular region image set to generate the plurality of lingual target images.
4. A depth learning based tongue three-dimensional morphology reconstruction method according to claim 3, wherein the method comprises: performing contour searching on the tongue region mask, and determining initial tongue contour parameters; Performing communication analysis based on the initial tongue profile parameters to generate a plurality of communication outside-domain profile data, and arranging the plurality of communication outside-domain profile data in descending order to generate an outside-domain profile sequence; Extracting connected outer domain contour data of a first bit sequence based on the outer contour sequence, performing geometric calculation on a target tongue body, and determining a tongue body geometric center; and carrying out circumscribed rectangle calculation on the tongue region mask by taking the geometric center of the tongue as a reference point, and generating the calculation result.
5. The depth learning-based tongue three-dimensional morphological reconstruction method according to claim 1, wherein the depth learning is performed based on the plurality of tongue target images, and a tongue depth map is constructed, the method comprising: adopting an encoder-decoder architecture to integrate a multi-view feature fusion module to construct a multi-view lingual depth map depth learning model; Synchronizing the plurality of lingual object images to the multi-view lingual depth map depth learning model for joint feature extraction of multi-view joint features; And carrying out depth regression based on the multi-view combined features to construct the lingual depth map.
6. The depth learning based tongue three-dimensional morphology reconstruction method of claim 5, wherein synchronizing the plurality of tongue target images to the multi-view tongue depth map depth learning model performs joint feature extraction of multi-view joint features, the method comprising: The encoder comprises a feature extraction backbone network, wherein the feature extraction backbone network traverses the plurality of tongue surface target images to perform feature extraction based on the feature extraction backbone network, and a multi-scale feature map is generated; Configuring a multi-view feature fusion module, and performing alignment polymerization on the multi-scale feature map through the multi-view feature fusion module to construct multi-view fusion features; And taking the multi-view fusion characteristic as an input value of a decoder to perform up-sampling convolution analysis to obtain the multi-view joint characteristic.
7. The depth learning-based tongue three-dimensional morphology reconstruction method according to claim 2, wherein three-dimensional morphology analysis is performed based on the tongue depth map, and tongue three-dimensional point cloud coordinates are constructed, the method comprising: Performing internal reference distortion calculation based on the image acquisition array to generate a plurality of distortion coefficients; performing back projection calculation on the lingual surface depth map according to the distortion coefficients to determine a plurality of three-dimensional coordinate points; transforming the plurality of three-dimensional coordinate points into a unified coordinate system based on the external reference matrix of the spatial attitude relation of the camera to obtain a plurality of unified three-dimensional coordinate points for fusion, and generating a fusion coordinate point set; And performing outlier filtering on the fused coordinate point set to generate the tongue three-dimensional point cloud coordinate.
8. The depth learning-based tongue three-dimensional morphological reconstruction method according to claim 2, wherein the tongue three-dimensional point cloud coordinates are mapped to the plurality of tongue target images to identify, and the identification result is compared with a standard gallery to generate a comparison parameter set, and the method comprises: Mapping and calculating the three-dimensional point cloud coordinates of the lingual surfaces to a plurality of lingual surface target images according to the external parameter matrix of the spatial attitude relation of the camera, and determining two-dimensional pixel coordinates; Mapping effective analysis is carried out on the plurality of tongue surface target images, and a plurality of effective mapping areas are defined; Screening the two-dimensional pixel coordinates in the effective mapping areas, determining a plurality of target three-dimensional points, and carrying out mapping analysis based on the target three-dimensional points and the two-dimensional pixel coordinates to obtain a mapping relation table; marking the plurality of tongue surface target images according to the mapping relation table to generate a marking result; introducing a standard graph library, carrying out similarity comparison analysis on the identification result and the standard graph library, calculating multi-dimensional morphological difference data, and evaluating based on the multi-dimensional morphological difference data to generate the comparison parameter set.
9. The depth learning-based tongue three-dimensional morphology reconstruction method according to claim 8, wherein the tongue three-dimensional point cloud coordinates are corrected based on the comparison parameter set, the target tongue is three-dimensionally reconstructed according to the correction result, and a three-dimensional morphology model is constructed, the method comprising: Setting an optimization target based on the multi-dimensional morphological difference data, and carrying out morphological deviation analysis on the tongue three-dimensional point cloud coordinates according to the optimization target to obtain a morphological deviation amplitude value; correcting and analyzing the tongue three-dimensional point cloud coordinates by taking the form deviation amplitude value as a form priori constraint to determine a form correction value; Correcting the tongue three-dimensional point cloud coordinates according to the morphological correction value to generate a correction result; Resampling the target tongue based on the correction result to generate resampling point cloud coordinates; Calculating a normal vector of the resampling point cloud coordinate, fitting the normal vector of the resampling point cloud coordinate with the resampling point cloud coordinate, and constructing an initial triangular grid; And carrying out iterative optimization based on the initial triangular mesh, and constructing the three-dimensional morphological model of the target lingual surface.
10. A depth learning based tongue three-dimensional morphology reconstruction system for implementing the depth learning based tongue three-dimensional morphology reconstruction method of any one of claims 1-9, the system comprising: The two-dimensional sensing module is used for activating the image acquisition array based on the illumination parameters of the light source to perform multi-angle two-dimensional sensing on the target tongue so as to obtain an original tongue surface image sequence; the deep learning module is used for traversing the original lingual image sequence to perform visual cutting, extracting a plurality of lingual target images, performing deep learning based on the lingual target images and constructing a lingual depth map; The contrast analysis module is used for carrying out three-dimensional morphological analysis based on the tongue depth map, constructing tongue three-dimensional point cloud coordinates, mapping the tongue three-dimensional point cloud coordinates to the plurality of tongue target images for identification, carrying out contrast analysis on the identification result and a standard gallery, and generating a contrast parameter set; And the three-dimensional reconstruction module is used for correcting the tongue three-dimensional point cloud coordinates based on the comparison parameter set, and carrying out three-dimensional reconstruction on the target tongue according to a correction result to construct a three-dimensional morphological model.

Description

Tongue three-dimensional form reconstruction method and system based on deep learning Technical Field The invention relates to the technical field of three-dimensional reconstruction, in particular to a tongue three-dimensional morphological reconstruction method and system based on deep learning. Background The tongue surface morphology is one of important indexes of traditional Chinese medicine diagnosis and modern medicine oral health analysis, and through analysis of the tongue surface morphology, the physiological state, pathological characteristics and tongue health condition of a human body can be reflected, and in order to realize objective and quantitative analysis of the tongue surface morphology, a two-dimensional tongue surface image is required to be converted into a three-dimensional model, so that geometrical characteristics and morphology information of the tongue surface are obtained. The traditional method mainly relies on single-view tongue surface images for diagnosis through color and texture analysis, however, the two-dimensional images cannot provide solid geometric information of the tongue surface, so that morphological measurement is inaccurate easily, and part of the method relies on manual labeling of tongue surface characteristic points or manual measurement. Disclosure of Invention The application provides a tongue three-dimensional form reconstruction method and a tongue three-dimensional form reconstruction system based on deep learning, and aims to solve the technical problems that a tongue analysis method in the prior art depends on two-dimensional images or manual observation, is high in subjectivity and serious in information loss, and is difficult to obtain a real three-dimensional form of a tongue. The application discloses a first aspect of a tongue three-dimensional morphological reconstruction method based on deep learning, which comprises the steps of activating an image acquisition array based on light source illumination parameters to perform multi-angle two-dimensional sensing on a target tongue to obtain an original tongue image sequence, traversing the original tongue image sequence to perform visual cutting, extracting a plurality of tongue target images, performing deep learning based on the plurality of tongue target images to construct a tongue depth map, performing three-dimensional morphological analysis based on the tongue depth map to construct tongue three-dimensional point cloud coordinates, mapping the tongue three-dimensional point cloud coordinates to the plurality of tongue target images to perform identification, performing contrast analysis on an identification result and a standard graph library to generate a contrast parameter set, correcting the tongue three-dimensional point cloud coordinates based on the contrast parameter set, performing three-dimensional reconstruction on the target tongue according to a correction result, and constructing a three-dimensional morphological model. The application discloses a second aspect, provides a tongue three-dimensional morphology reconstruction system based on deep learning, which is used for the tongue three-dimensional morphology reconstruction method based on deep learning, and comprises a two-dimensional sensing module, a deep learning module, a comparison analysis module and a three-dimensional reconstruction module, wherein the two-dimensional sensing module is used for activating an image acquisition array based on light source illumination parameters to conduct multi-angle two-dimensional sensing on a target tongue to obtain an original tongue image sequence, the deep learning module is used for traversing the original tongue image sequence to conduct visual cutting, extracting a plurality of tongue target images, conducting deep learning based on the plurality of tongue target images to construct a tongue depth image, the comparison analysis module is used for conducting three-dimensional morphology analysis based on the tongue depth image to construct tongue three-dimensional point cloud coordinates, mapping the tongue three-dimensional point cloud coordinates to the plurality of tongue target images to identify, conducting comparison analysis on identification results and a standard graph library to generate a comparison parameter set, and the three-dimensional reconstruction module is used for conducting correction on the tongue three-dimensional point cloud coordinates based on the comparison parameter set to conduct three-dimensional reconstruction on the target tongue according to the correction results to construct a three-dimensional morphology model. The one or more technical schemes provided by the application have at least the following beneficial effects: By activating the image acquisition array based on the illumination parameters of the light source, the tongue can be accurately acquired from multiple angles, an original tongue surface image sequence with rich visual angle inf