EP-4425429-B1 - SYSTEM AND METHOD OF SOLVING THE CORRESPONDENCE PROBLEM IN 3D SCANNING SYSTEMS

Inventors

• ÖJELUND, Henrik
• MOTTELSON, Isak

Dates

Publication Date

20260513

Application Date

20240229

Claims (6)

1. A computer-implemented method for generating a three-dimensional (3D) representation of a surface of a dental object (101), the method comprising the steps of: - obtaining a set of images comprising two or more images acquired by a triangulation-based intraoral scanning device (102) comprising multiple camera units and one or more projector units configured to project a pattern on a surface of the dental object, wherein images within the set of images are acquired by different camera units, wherein each image comprises a plurality of image features, wherein each image is composed of an array of pixels, wherein each pixel comprises a pixel color defined by one or more color channels; - determining points in three-dimensional (3D) space that form a solution to a correspondence problem associated with the set of images; and - generating the three-dimensional (3D) representation based on the determined 3D points, WHEREIN the points are determined by - determining potential points in 3D space that form one or more candidate solutions to the correspondence problem based on triangulation; - assigning one or more parameters to each potential point, wherein said parameters include a color and a likelihood that the potential point is part of the solution to the correspondence problem; - generating a computed color for each camera ray corresponding to a pixel, wherein the computed color is based on the assigned colors and likelihoods; and - comparing pixel colors with the computed colors associated with camera rays corresponding to each pixel by determining a numeric value of the color and likelihood of the potential points by minimizing a cost function based on the difference between the computed colors and the pixel colors, wherein the solution to the correspondence problem is found among the potential points based on their numeric value of their likelihood.
2. The computer-implemented method according to claim 1, wherein images within a set of images are acquired simultaneously.
3. The computer-implemented method according to claims 1 or 2, wherein the method further comprises the step of determining one or more image features within the set of images, wherein each potential point is determined by triangulation based on the determined image features.
4. A scanning system comprising: - a triangulation-based intraoral scanning device (102) comprising one or more projector units configured to project a pattern on a surface of the dental object (101) and two or more camera units configured to acquire the set of images, wherein the set of images comprises at least one image from each camera unit; - one or more processors (504) configured to perform the steps of the method according to any of the claims 1-3.
5. The scanning system according to claim 4, wherein the projected pattern is static.
6. The scanning system according to any of claims 4-5, wherein the projected pattern is a polygonal pattern comprising at least 3000 pattern features.

Description

Technical field The present disclosure relates to a system and method for generating a three-dimensional (3D) representation of an object. In particular, the present disclosure relates to a system and method for determining a solution to the correspondence problem associated with imaging using a 3D scanning device. Background In scanning systems employing triangulation, a central task is to solve the so-called correspondence problem. The correspondence problem may refer to the task of determining image features in images within a set of images and associating each image feature with a projector ray, wherein said images captures the same three-dimensional (3D) scene or object. The images can be taken from a different point of view, at different times, or with objects in a scene in general motion relative to the camera(s). The correspondence problem can occur in a stereo situation when two images of the same object are acquired, or it can be generalized to an N-view correspondence problem. In the latter case, the images may come from either N different cameras photographing at the same time or from one camera which is moving relative to the object/scene. The problem is made more difficult when the objects in the scene are in motion relative to the camera(s). In order to generate a three-dimensional digital representation of a scanned object, such as a person's teeth, the correspondence problem generally need to be solved, at least when using a triangulation-based scanning device to acquire the images of the object. In general, it is desired that the 3D digital representation is generated in real-time, or at least in what is perceived as real-time to the user, e.g. the dentist. Therefore, the 3D representation is typically generated simultaneously with the acquisition of images, such that the user can immediately view the generated 3D representation while scanning a patient. Consequently, it is desired that the correspondence problem is solved in a fast, reliable, and accurate manner since it enables a fast generation of the 3D representation. Thus, it is of interest to develop improved systems and methods for generating a digital 3D representation of a 3D object. In particular, it is desired to find improved methods for solving the correspondence problem and related issues within 3D scanning systems. US20200404243A1 discloses a method for generating a 3D image includes driving structured light projector(s) to project a pattern of light on an intraoral 3D surface, and driving camera(s) to capture images, each image including at least a portion of the projected pattern, each one of the camera(s) comprising an array of pixels. A processor compares a series of images captured by each camera and determines which of the portions of the projected pattern can be tracked across the images. The processor constructs a three-dimensional model of the intraoral three-dimensional surface based at least in part on the comparison of the series of images. Other embodiments are also described. US20220280269A1 discloses an apparatus for intraoral scanning comprises an elongate handheld wand comprising a probe at a distal end, one or more light projectors, and two or more cameras. Each light projector comprises at least one light source configured to generate light and a pattern generating optical element configured to generate a pattern of light when the light is transmitted through the pattern generating optical element. Each camera comprises a camera sensor and one or more lenses and is configured to capture a plurality of images that depict at least a portion of the projected pattern of light on an intraoral surface, wherein each camera is configured to focus at an object focal plane that is located between about 1 mm and about 30 mm from a lens of the one or more lenses that is farthest from the camera sensor. Boyer, Alain & Payeur, Pierre. (2017). Enhancing structured light range imaging by adaptation of color, exposure and focus. 117-122. 10.1109/CIVEMSA.2017.7995312 presents a series of enhancements to a color-coded structured light range sensor that increases the adaptability to complex and unconstrained scenes. First, the projected pattern is made more visible on colored objects by replacing the unique colored pattern with time-multiplexed pseudo-color channels. Second, an exposure fusion algorithm is used when acquiring images to allow the detection of regions with low and high reflectance characteristics. Finally, the focus planes of the scene are automatically detected and imaged separately, enlarging the sensor's depth of field. Each improvement is detailed and integrated into a custom acquisition procedure. Experimental results demonstrate the improved robustness of the structured light range sensor and validate the proposed design. Bräuer-Burchardt, Christian & Heist, Stefan & Dietrich, Patrick & Kühmstedt, Peter & Notni, Gunther. (2015). 3D reconstruction with single image pairs and structured light projection fo