Search

CN-121505184-B - Three-dimensional tooth curved surface generation method based on two-dimensional tooth photo

CN121505184BCN 121505184 BCN121505184 BCN 121505184BCN-121505184-B

Abstract

The invention discloses a three-dimensional tooth curved surface generation method based on a two-dimensional tooth photo, which utilizes a multi-view two-dimensional image and a camera pose, and combines depth priori and semantic priori to fit a three-dimensional nerve radiation field so as to reconstruct a three-dimensional oral cavity environment; and then extracting a three-dimensional surface from the implicit field and performing parameterization fitting to obtain independent dentition and gum models. The method is improved by an algorithm aiming at the problems of high reflection and weak texture in an oral moist environment, realizes low-cost and high-precision three-dimensional modeling without a special scanner, and has remarkable robustness and reconstruction efficiency.

Inventors

  • Zhu Bingfan
  • Zheng youyi

Assignees

  • 浙江大学

Dates

Publication Date
20260508
Application Date
20260113

Claims (9)

  1. 1. A three-dimensional tooth curved surface generating method based on a two-dimensional tooth photo is characterized by comprising the following steps: acquiring an intraoral dental image of a multi-view patient and a corresponding camera pose; Extracting depth features and/or semantic features from the intra-oral dental image of the patient by using a pre-training network, and recording image pixel colors as a truth value supervision signal; Constructing a three-dimensional reconstruction network comprising geometric branches, appearance branches and semantic branches, carrying out integral calculation on sampling space points based on the intraoral dental image of the patient and the pose of a camera by utilizing a volume rendering technology, and outputting predicted pixel colors, predicted depth features and predicted semantic features of pixels; Extracting an implicit geometric field from the three-dimensional nerve radiation field obtained by fitting to reconstruct a three-dimensional surface, and performing parameterization fitting based on the three-dimensional surface reconstruction result to obtain parameterized dentition and gum models; The three-dimensional reconstruction network comprises a geometric branch and a depth feature network, wherein an appearance branch comprises a color network, the semantic branch comprises a semantic feature network, the geometric branch outputs a space point symbol distance function SDF value, a geometric feature and a depth feature based on space point coordinates, the appearance branch and the semantic branch respectively output a space point prediction color and a prediction semantic feature based on the geometric feature, the depth feature and a view angle direction, the three-dimensional reconstruction network converts the SDF value into density by utilizing a learnable indication parameter, and performs weighted integration on the space point prediction color, the prediction depth feature and the prediction semantic feature along a camera ray based on the density, and the prediction pixel color, the prediction depth feature and the prediction semantic feature of a pixel are output.
  2. 2. The method of claim 1, wherein the process of training the three-dimensional reconstruction network further comprises iterative optimization of a spatial point sampling strategy, wherein the spatial point sampling strategy comprises calculating density and/or cumulative weight of spatial points based on SDF values and indication parameters of geometric branch output, clipping a sampling range on a camera ray according to the density and/or the cumulative weight, discarding a region with the density or the cumulative weight smaller than a preset threshold, and re-sampling the spatial points in the clipped sampling range and inputting the spatial point sampling range into the three-dimensional reconstruction network for training.
  3. 3. The method according to claim 1, wherein the camera pose is obtained by direct recording or predicted by a motion restoration structure SfM algorithm, and the normalization operation is performed on the whole scene based on the obtained camera view angle, so that the photographed oral environment is located inside one unit sphere.
  4. 4. The method according to claim 1, wherein the pre-training network comprises a monocular depth estimation network for extracting depth features and/or a semantic segmentation network for extracting semantic features.
  5. 5. The method of claim 1, wherein the three-dimensional surface reconstruction is performed by extracting an implicit geometric field from a three-dimensional neural radiation field obtained by fitting, and specifically comprises the step of extracting an isosurface from a directional symbol field by using a moving cube algorithm to obtain a three-dimensional surface reconstruction result.
  6. 6. The method of claim 1, further comprising performing new view synthesis based on the fitted three-dimensional reconstruction network to obtain a rendered image at an unknown view angle and corresponding depth feature map and semantic feature map.
  7. 7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements a three-dimensional dental curve generating method based on two-dimensional dental photographs as claimed in any one of claims 1 to 6 when the computer program is executed by the processor.
  8. 8. A storage medium containing computer executable instructions which, when executed by a computer processor, implement a two-dimensional dental photograph-based three-dimensional dental curve generating method as claimed in any one of claims 1 to 6.
  9. 9. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of a three-dimensional dental surface generation method based on two-dimensional dental photographs as claimed in any one of claims 1 to 6.

Description

Three-dimensional tooth curved surface generation method based on two-dimensional tooth photo Technical Field The invention belongs to the field of tooth orthodontic digitization, and particularly relates to a three-dimensional tooth curved surface generating method based on a two-dimensional tooth photo, which is based on a generated three-dimensional tooth curved surface dentition and gum model and can be used for dental diagnosis, orthodontic treatment, implantation operation planning and the like. Background With the popularization of digital medical technology, computer-aided diagnosis and treatment plays an increasingly important role in the dental field, especially in orthodontic treatment. At present, obtaining a three-dimensional digital model of the mouth of a patient is a precondition for making an orthodontic scheme. Intraoral scanners are commonly used clinically to obtain dentition information to assist doctors in subsequent diagnosis and treatment planning. However, the existing intraoral scanning technology has significant limitations that firstly, the process is highly dependent on professional medical environment and expensive scanning equipment, and has high requirements on professional skills of operators, secondly, patients need to go to and back to a hospital for cooperation scanning for many times, so that the time cost and economic burden of treatment are increased, and in addition, the generated original three-dimensional model often contains redundant oral tissues, and complicated tooth segmentation treatment is still needed in the follow-up. These factors together limit the improvement of orthodontic treatment efficiency. A deep neural network is introduced, a three-dimensional model is reconstructed by utilizing a two-dimensional image, and a new path is provided for solving the problems. The method allows the patient to remotely acquire the image, can realize three-dimensional reconstruction with low cost and high efficiency while reducing the dependence on professional equipment and manpower, and has important application values for dental diagnosis, orthodontics, planting planning and the like. Despite its broad prospects, two-dimensional photograph-based intraoral reconstruction faces a great technical challenge in that, unlike an ideal laboratory environment, the oral environment is highly complex. Specifically, the Non-Lambertian photometric characteristics of the tooth surface, the light reflection and refraction interference caused by the moist environment of the oral cavity, the insufficient illumination caused by the semi-closed space, the detail shielding and other problems all put very high requirements on the reconstruction integrity and definition of the algorithm. Disclosure of Invention The invention aims to provide a three-dimensional tooth curved surface generation method based on a two-dimensional tooth photo aiming at the defects of the prior art. The aim of the invention is realized by the following technical scheme: a three-dimensional tooth curved surface generating method based on two-dimensional tooth photo comprises the following steps: And (3) acquiring data, namely acquiring tooth images in the mouth of the multi-view patient and corresponding camera pose. The truth value construction comprises the steps of extracting depth features and/or semantic features from the intraoral dental images of the patient by utilizing a pre-training network, specifically, extracting the depth features and the semantic features from the images by utilizing a pre-training monocular depth estimation network and/or a semantic segmentation network respectively to serve as depth feature truth values and semantic feature truth values, and recording colors of corresponding pixels to serve as color truth values. Three-dimensional nerve radiation field fitting, namely constructing a three-dimensional reconstruction network comprising geometric branches, appearance branches and semantic branches, carrying out integral calculation on sampling space points based on the intraoral tooth image of the patient and the pose of a camera by utilizing a volume rendering technology, and outputting predicted pixel colors, predicted depth features and predicted semantic features of pixels; The model generation and post-processing comprises the steps of extracting an implicit geometric field (SDF) from a fitted three-dimensional nerve radiation field to reconstruct a three-dimensional surface, and carrying out parameterization fitting on the reconstructed three-dimensional surface to obtain parameterized dentition and gum models containing dental position marks so as to facilitate subsequent diagnosis and treatment analysis on teeth at specific positions. Further, the camera pose is obtained through direct recording or predicted by a camera pose prediction method such as a motion restoration structure (SfM). Further, the monocular depth estimation network adopts SFMLEARNER or MonoWavelet, and th