Search

CN-122023713-A - Recreation facility 3D model construction method and system based on image recognition

CN122023713ACN 122023713 ACN122023713 ACN 122023713ACN-122023713-A

Abstract

The invention relates to the technical field of computer vision and discloses a recreation facility 3D model construction method and system based on image recognition, wherein the method comprises the steps of obtaining image data of multiple visual angles of the recreation facility, extracting invariance characteristic points, performing deep learning, matching and fusion, and combining triangulation to obtain an initial three-dimensional point cloud; the method comprises the steps of completing point clouds of a shielding area to form a complete structure, dividing key parts, extracting a preliminary skeleton line path, interpolating and connecting the isolated sections to obtain a continuous skeleton line, generating local surface grids, optimizing smoothness, fusing geometric constraint correction grids, filling holes, and determining a final three-dimensional reconstruction model through texture mapping matching. The method can realize accurate extraction of continuous skeleton lines of the amusement facilities and generate a high-quality 3D model with high precision, no holes and smooth surface.

Inventors

  • MA WEIXIA
  • LIN RUHAI

Assignees

  • 深圳市龙祥康体设施发展有限公司

Dates

Publication Date
20260512
Application Date
20251225

Claims (10)

  1. 1. The method for constructing the 3D model of the amusement facility based on the image recognition is characterized by comprising the following steps of: acquiring multi-view image data of an amusement facility; extracting a feature point set with invariance from the image data, carrying out matching fusion on the feature point set, and obtaining initial three-dimensional point cloud data by combining triangulation calculation; generating a point cloud density distribution diagram according to the initial three-dimensional point cloud data, detecting a shielding region with density which does not reach a preset standard in the point cloud density distribution diagram, and carrying out data supplementation on the shielding region through multi-view fusion to form a complete three-dimensional point cloud structure; a key structural component is segmented from the complete three-dimensional point cloud structure, central line tracking is carried out on the key structural component, and a preliminary skeleton line path is obtained after topology construction and curve fitting; Screening the break paragraphs in the preliminary skeleton line path, and carrying out path continuity deviation detection and correction on the break paragraphs and the adjacent paths to obtain a continuous skeleton line sequence; Generating a local surface grid according to the continuous skeleton line sequence, and carrying out smooth transition adjustment on the local surface grid to obtain a smooth surface grid; Performing geometric feature constraint adjustment on the smooth surface grid, and then identifying and filling hole structures in the grid to obtain a hole-free surface model; And carrying out texture mapping matching on the non-hole surface model and the image data, screening an optimal texture source, optimizing joint transition, and determining a final three-dimensional digital reconstruction model.
  2. 2. The method for constructing a 3D model of a play facility based on image recognition according to claim 1, wherein the acquiring image data of multiple perspectives of the play facility comprises: Capturing multi-view original image data of an amusement facility through a multi-view image acquisition system arranged on the amusement scene, wherein the acquisition system comprises a plurality of high-definition cameras, and synchronously acquiring images from different angles respectively; preprocessing the original image data to obtain multi-view image data of the amusement facility, wherein the preprocessing comprises denoising, color correction and distortion correction.
  3. 3. The method for constructing the 3D model of the amusement ride based on the image recognition according to claim 1, wherein the steps of extracting the feature point set with invariance from the image data, carrying out matching fusion on the feature point set, and obtaining initial three-dimensional point cloud data by combining triangulation calculation comprise the following steps: Extracting a feature point set with rotation invariance and scale invariance from the image data; inputting the feature point set into a pre-trained deep learning feature matching network, and generating a feature point matching pair list with high confidence through learning semantic relations among feature points; and carrying out triangulation calculation according to the characteristic point matching pair list, solving three-dimensional coordinates of key structures of the amusement facilities, and integrating all the three-dimensional coordinates to form initial three-dimensional point cloud data.
  4. 4. The method for constructing the 3D model of the amusement facility based on image recognition according to claim 1, wherein the generating a point cloud density distribution map according to the initial three-dimensional point cloud data, detecting an occlusion region with a density which does not reach a preset standard in the point cloud density distribution map, and performing data supplementation on the occlusion region through multi-view fusion to form a complete three-dimensional point cloud structure comprises: calculating the point cloud density in each unit according to a preset space grid unit based on the initial three-dimensional point cloud data, and generating a point cloud density distribution map; Determining an area corresponding to the grid unit with the point cloud density lower than a preset density threshold value in the point cloud density distribution diagram as an occlusion area, and searching a target two-dimensional image block in the multi-view image data corresponding to the occlusion area; Parallax calculation is carried out on the target two-dimensional image block, depth information is calculated, and incremental three-dimensional point cloud data are generated; Fusing the incremental three-dimensional point cloud data with the initial three-dimensional point cloud data, and judging continuity by detecting the surface normal vector direction change of the fused point cloud; If the surface normal vector direction change is smaller than a preset angle threshold value, judging that the surface normal vector meets the continuity requirement, and further determining a complete three-dimensional point cloud structure; If the surface normal vector direction change is greater than or equal to a preset angle threshold, judging that the continuity requirement is not met, marking the area as an uncompleted shielding area, retrieving multi-view image data corresponding to the uncompleted shielding area again, generating secondary increment three-dimensional point cloud data in a supplementing mode, fusing the secondary increment three-dimensional point cloud data with the current fusion point cloud again, repeating the continuity detection step until the surface normal vector direction change is smaller than the preset angle threshold, and determining the complete three-dimensional point cloud structure.
  5. 5. The method for constructing the 3D model of the amusement ride based on image recognition according to claim 1, wherein the steps of segmenting the key structural parts from the complete three-dimensional point cloud structure, tracking the central line of the key structural parts, and obtaining a preliminary skeleton line path after topology construction and curve fitting comprise the steps of: Calculating the principal curvature value of each point in the complete three-dimensional point cloud structure, and dividing the complete three-dimensional point cloud structure according to the difference of the principal curvature values to obtain a point cloud subset corresponding to the key structural component; clustering the point cloud subsets, and iteratively updating the point cloud positions to enable the point cloud subsets to shrink towards the geometric center to generate a skeleton node set; Constructing a topological structure diagram according to the skeleton node set, traversing the topological structure diagram through depth-first search, and extracting a trunk path sequence; And performing curve fitting on the trunk path sequence to generate a smooth central line track, and taking the central line track as a preliminary skeleton line path of a key structural component.
  6. 6. The method for constructing the 3D model of the amusement ride based on the image recognition according to claim 1, wherein the screening the break paragraphs in the preliminary skeleton line path, and performing path continuity deviation checking and correction on the break paragraphs and the adjacent paths to obtain a continuous skeleton line sequence comprises: Traversing the primary skeleton line path, detecting an interruption paragraph formed by the distance mutation or connection interruption between adjacent nodes, and marking the breakpoint of the interruption paragraph; Constructing a local search range by taking each breakpoint as a center, acquiring a neighborhood path point set falling into the range, screening out a to-be-connected breakpoint pair based on spatial position relevance, calculating tangential vectors of the neighborhood path point set, and simultaneously calculating Euclidean distances between the to-be-connected breakpoint pairs, wherein the tangential vectors and the Euclidean distances jointly form a posture correlation vector; If the space included angle of the tangent vector is larger than or equal to a preset angle threshold value or the Euclidean distance is larger than or equal to a preset distance threshold value, calculating the deviation between the space included angle of the tangent vector and the preset angle threshold value and the deviation between the Euclidean distance and the preset distance threshold value, and integrating to obtain a geometric discontinuity deviation value; Generating interpolation nodes with an adaptive quantity according to the deviation value, determining the estimated connection direction between break points based on the geometric distribution trend of the neighborhood path point set and the spatial position relation of the to-be-connected break point pairs, uniformly distributing the interpolation nodes along the direction, merging the interpolation nodes with the neighborhood path point set, establishing new topological connection, replacing the original break section, and integrating to obtain a preliminary continuous skeleton line sequence; If the space included angle of the tangent vector is smaller than a preset angle threshold and the Euclidean distance is smaller than a preset distance threshold, judging that the continuity requirement is met, and filling interpolation is not needed, reserving a corresponding interruption paragraph in the preliminary skeleton line path, and filling the corresponding interruption paragraph to a corresponding position of the preliminary continuous skeleton line sequence; And integrating all the interrupt paragraphs after interpolation filling and the reserved interrupt paragraphs according to the spatial position sequence, and finally obtaining a continuous skeleton line sequence.
  7. 7. The method for constructing the 3D model of the amusement ride based on the image recognition according to claim 1, wherein the generating a local surface grid according to the continuous skeleton line sequence, performing smooth transition adjustment on the local surface grid to obtain a smooth surface grid, comprises: Taking each node of the continuous skeleton line as a center, expanding a section with a preset radius along a local normal vector direction obtained based on the geometric attribute analysis of the nodes of the continuous skeleton line, generating an initial grid ring surrounding the node, and connecting the grid rings of adjacent nodes to form a local surface grid; constructing a curvature tensor field for the local surface grid, and identifying a bending turning region with a curvature principal value exceeding a preset curvature threshold value; Calculating a curvature change gradient field of the bending turning region, and extracting a grid boundary node set of the edge of the corresponding region if the gradient amplitude exceeds a preset gradient threshold value; Calculating a local ideal curvature center according to curvature distribution of the bending turning region and neighborhood grid characteristics, calculating radial offset of each boundary node in the grid boundary node set to the local ideal curvature center, generating a radial displacement vector based on the offset, adjusting the positions of nodes in the grid boundary node set through the displacement vector, deleting deformed patches and re-triangulating, and reconstructing topological connection to obtain a smooth surface grid.
  8. 8. The method for constructing the 3D model of the amusement ride based on the image recognition according to claim 1, wherein the geometric feature constraint adjustment is performed for the smooth surface grid, and then the hole structures in the grid are recognized and filled to obtain a hole-free surface model, comprising: Traversing the smooth surface grid, calculating dihedral angles between adjacent triangular patches, and marking a common edge of which the dihedral angle is smaller than a preset angle threshold value as a geometric characteristic line to form a geometric characteristic line set; constructing a Laplace deformation constraint equation containing the geometric characteristic line set constraint, and solving the equation to obtain a deformation corrected intermediate state grid model; Traversing the topological structure of the intermediate state grid model, searching for an unclosed boundary edge and organizing the unclosed boundary edge into a hole boundary ring sequence; and expanding inwards from the initial vertex of the boundary ring sequence by adopting a wave front method, generating a new triangular patch fitting the local curvature trend, and performing topological stitching on the new triangular patch and the intermediate grid model to obtain the hole-free surface model.
  9. 9. The image recognition-based amusement ride 3D model construction method according to claim 1, wherein the texture mapping matching the hole-free surface model with the image data, screening an optimal texture source and optimizing joint transition, determining a final three-dimensional digitized reconstruction model, comprises: Calculating the included angle between the normal line of each triangular patch in the non-hole surface model and the optical axis of each camera in the image data, and screening view points with included angles smaller than a preset angle to form a candidate view point set; The candidate view point set is processed through depth buffering, the blocked triangular patches are removed, and the visual texture area and the area coverage rate corresponding to each candidate view point are determined; constructing an energy optimization function by taking the area coverage rate as a core optimization index, and screening an optimal texture source image from images corresponding to the candidate viewpoint set; Generating an initial texture map according to the optimal texture source image, identifying joint boundaries in the initial texture map, constructing a poisson equation, and carrying out smooth interpolation on the joint boundaries to obtain a fusion texture map; And registering the fusion texture map to the non-hole surface model accurately, ensuring alignment of textures and geometric structures, and determining a final three-dimensional digital reconstruction model.
  10. 10. An amusement ride 3D model construction system based on image recognition, characterized by comprising: the image acquisition module is used for acquiring multi-view image data of the amusement facility; The characteristic point cloud module is used for extracting a characteristic point set with invariance from the image data, carrying out matching fusion on the characteristic point set, and obtaining initial three-dimensional point cloud data by combining triangulation calculation; the shielding completion module is used for generating a point cloud density distribution diagram according to the initial three-dimensional point cloud data, detecting a shielding region with density which does not reach a preset standard in the point cloud density distribution diagram, and carrying out data supplementation on the shielding region through multi-view fusion to form a complete three-dimensional point cloud structure; The segmentation skeleton module is used for segmenting a key structural component from the complete three-dimensional point cloud structure, carrying out central line tracking on the key structural component, and obtaining a preliminary skeleton line path after topology construction and curve fitting; The breakpoint connection module is used for screening the interruption paragraphs in the preliminary skeleton line path, and carrying out path continuity deviation test and correction on the interruption paragraphs and the adjacent paths to obtain a continuous skeleton line sequence; The grid optimization module is used for generating a local surface grid according to the continuous skeleton line sequence, and carrying out smooth transition adjustment on the local surface grid to obtain a smooth surface grid; The correction hole filling module is used for carrying out geometric feature constraint adjustment on the smooth surface grid, and then identifying and filling hole structures in the grid to obtain a hole-free surface model; And the texture mapping module is used for performing texture mapping matching on the non-hole surface model and the image data, screening an optimal texture source, optimizing joint transition and determining a final three-dimensional digital reconstruction model.

Description

Recreation facility 3D model construction method and system based on image recognition Technical Field The invention relates to the technical field of computer vision, in particular to an amusement facility 3D model construction method and system based on image recognition. Background Along with the popularization of the augmented reality technology and the virtual reality technology, the image recognition technology is continuously advanced, the digital upgrading requirement of the theme park is continuously improved, the three-dimensional digital reconstruction of the amusement facilities becomes an industry key requirement, and the integrity, the smoothness and the accuracy of the model directly influence the safety detection efficiency, the virtual experience effect and the digital asset value. The amusement facility is composed of a large number of slender metal rods, bending tracks and supporting structures, the structure is complex, the mutual shielding is serious, and extremely high requirements are provided for the environmental adaptability and the detail capturing capability of the three-dimensional reconstruction technology. At present, the prior art in the field of three-dimensional reconstruction is mainly divided into two types, one type relies on laser scanning or multi-view structured light equipment, although the precision is higher, the operation needs professional staff, and the deployment is difficult in a large-scale outdoor recreation facility scene, so that the convenient and efficient reconstruction is difficult to realize. The other type is based on an image recognition technology, reconstruction is realized through multi-view image acquisition and feature matching, the deployment is flexible, the cost is low, but the design thought is mostly applicable to objects with conventional structures, the targeted optimization of special structures of amusement facilities is lacking, and the challenges caused by serious shielding among parts are not fully considered. In practical application, the image data is easy to be lost due to mutual shielding of an elongated rod piece and a track of an amusement facility, the existing image recognition-based method is difficult to effectively supplement shielding region data, continuous space skeleton lines cannot be accurately extracted, the skeleton lines serve as cores of topological relations of the bearing facility, and once interruption and distortion occur due to shielding or pose estimation deviation, the problems that holes, deformity or unsmooth turning positions and the like are necessarily generated in subsequent surface reconstruction are solved. The technical limitation leads to the core defect in the prior art, and the continuous and reliable space skeleton line can not be accurately extracted through the image recognition technology in the recreation facility reconstruction scene with serious shielding and complex structure, so that the 3D model has insufficient accuracy, the high quality requirement of industry on three-dimensional digital reconstruction of the recreation facility can not be met, and the problem of insufficient accuracy exists. Disclosure of Invention The invention provides a 3D model construction method and system for an amusement facility based on image recognition, which effectively solve the problem of broken skeleton lines of the amusement facility caused by complex structure and serious shielding, ensure complete topological relation of the model, remarkably improve the geometric accuracy of the 3D model and meet the high-quality requirements of safety detection, virtual experience development and the like. In order to solve the technical problems, the present invention provides a method for constructing a 3D model of an amusement ride based on image recognition, including: acquiring multi-view image data of an amusement facility; extracting a feature point set with invariance from the image data, carrying out matching fusion on the feature point set, and obtaining initial three-dimensional point cloud data by combining triangulation calculation; generating a point cloud density distribution diagram according to the initial three-dimensional point cloud data, detecting a shielding region with density which does not reach a preset standard in the point cloud density distribution diagram, and carrying out data supplementation on the shielding region through multi-view fusion to form a complete three-dimensional point cloud structure; a key structural component is segmented from the complete three-dimensional point cloud structure, central line tracking is carried out on the key structural component, and a preliminary skeleton line path is obtained after topology construction and curve fitting; Screening the break paragraphs in the preliminary skeleton line path, and carrying out path continuity deviation detection and correction on the break paragraphs and the adjacent paths to obtain a continuous skeleton line