CN-122023664-A - Method and system for modeling pig 3D based on single-view depth camera image

CN122023664ACN 122023664 ACN122023664 ACN 122023664ACN-122023664-A

Abstract

The application provides a method and a system for modeling pig 3D based on a single-view depth camera image, relates to the field of 3D modeling, and solves the technical problem that in the prior art, due to the fact that different sample 3D point cloud data have posture differences, scale changes and anatomical structure corresponding relations are fuzzy, group statistical modeling and individual morphology comparison cannot be accurately performed. The method comprises the steps of respectively carrying out semantic alignment and spatial alignment on 3D point cloud data of each sample in a 3D point cloud data set to obtain an aligned 3D point cloud data set, analyzing average shape point cloud data B and deformation feature vectors V, inputting the aligned 3D point cloud data set into a deep learning network Net model to output feature vectors C, and reconstructing according to the average shape point cloud data B, the deformation feature vectors V and the feature vectors C to obtain a complete 3D point cloud data model M of a pig. The method is used for the pig 3D modeling process.

Inventors

GUI ZHIMING
LIU CHONGCHONG
ZOU JUN

Assignees

安徽拉塞特智能科技有限公司

Dates

Publication Date: 20260512
Application Date: 20260210

Claims (10)

1. A method of modeling swine 3D based on single view depth camera images, comprising: The 3D point cloud data of each sample in the 3D point cloud data set is subjected to semantic alignment and spatial alignment respectively to obtain an aligned 3D point cloud data set; analyzing average shape point cloud data B and deformation characteristic vectors V based on the aligned 3D point cloud data sets, wherein the average shape point cloud data B represents a reference point cloud model of a typical three-dimensional form of a pig group, and the deformation characteristic vectors V represent projection coefficients of form deviations of the 3D point cloud data of each sample relative to the average shape point cloud data B in a principal component space; inputting the aligned 3D point cloud data set into a deep learning network Net model, and outputting a feature vector C; and reconstructing according to a formula M=B+ C@V to obtain a complete 3D point cloud data model M of the pig, wherein @ represents matrix multiplication.
2. The method for 3D modeling of pigs based on single view depth camera images according to claim 1, wherein the semantic alignment method of 3D point cloud data of each sample comprises: registering each group of 3D point cloud data in the 3D point cloud data with the target point cloud data by utilizing a pre-trained point cloud registration network model to obtain a point corresponding index set between each group of 3D point cloud data and the target point cloud data Wherein I is the sequence number of the element in the index set, i=0, 1,., m, m is a positive integer, and the I element in the index set I Index representing a point with the same semantic meaning as the ith point of the target point cloud data in the 3D point cloud data to be aligned; and reordering the points in each group of 3D point cloud data according to the index set, and enabling each 3D point cloud data to have the same semantics at the same index position to obtain a 3D point cloud data set with aligned semantics.
3. The method for 3D modeling of pigs based on single view depth camera images according to claim 1, wherein the method for spatial alignment of 3D point cloud data for each sample comprises: And estimating rigid transformation parameters of the semantically aligned 3D point cloud data relative to the target point cloud data through the coarse alignment network model, and aligning all the 3D point cloud data to a unified coordinate system with the target point cloud data as a center to obtain a spatially aligned 3D point cloud data set, wherein the rigid transformation parameters comprise a 3D rotation matrix and translation vectors.
4. A method of modeling swine 3D based on single view depth camera images as claimed in claim 3, wherein the constructing of the coarse alignment network model comprises: acquiring a training data set, wherein the training data set comprises a plurality of groups of marked 3D point cloud data sample pairs, and each sample pair comprises source point cloud data and target point cloud data; Preprocessing source point cloud data and target point cloud data in each sample pair; Constructing a neural network architecture, wherein the architecture comprises a shared weight point feature extraction module, a global feature aggregation module and a rigid transformation parameter regression module which are sequentially connected; Based on the training data set, performing end-to-end training on the neural network frame with the aim of minimizing an alignment loss function, wherein the alignment loss function is used for optimizing the superposition degree of source point cloud data and target point cloud data after predictive conversion and guiding a coarse alignment network to learn correct space alignment capability; and after training, obtaining a rough alignment network model for estimating the 3D point cloud data of any pig relative to the target point cloud data.
5. The method of modeling swine 3D based on single view depth camera images of claim 4, wherein the constructing of the alignment loss function comprises: based on the rigid body transformation parameters, performing space transformation on the preprocessed 3D point cloud data to obtain rotated point cloud data; calculating the cosine similarity of the included angle between each point in the rotated point cloud data and the corresponding point in the target point cloud data, and averaging the cosine similarity of the included angle to obtain an alignment loss function value : ; Wherein N is the number of sampling points, N is the serial number of the sampling points, Is the position vector of the nth point in the target point cloud data, And the position vector of the nth point in the rotated point cloud data.
6. The method for modeling 3D of pigs based on single view depth camera image according to claim 1, wherein the method for obtaining the average shape point cloud data B comprises: the method comprises the steps of constructing an aligned 3D point cloud data set into a { K, M } matrix Amr, wherein K is the number of 3D point cloud data samples, M is the coordinate dimension of each sample after flattening, calculating column average values of the matrix AMr in the K sample dimensions to obtain an average value vector with the length of M, and marking the average value vector as average shape point cloud data B.
7. The method for modeling a pig 3D based on a single view depth camera image according to claim 1, wherein the method for obtaining the deformation feature vector V comprises: Calculating the difference value between each 3D point cloud data and the average shape point cloud data B in the aligned 3D point cloud data set to obtain a decentralised deviation point cloud matrix, carrying out principal component analysis on the deviation point cloud matrix, extracting feature vectors corresponding to the first Z maximum feature values to form a principal component matrix, carrying out matrix multiplication on the deviation point cloud matrix and the principal component matrix to obtain a deformation feature matrix, wherein each behavior of the deformation feature matrix is a deformation feature vector V of one sample.
8. The method for modeling 3D of pigs based on single view depth camera image according to claim 1, wherein the method for obtaining the feature vector C comprises: Inputting the aligned 3D point cloud data set into a pre-constructed and trained deep learning network Net model, wherein the deep learning network Net comprises a convolution feature extraction module, a global context aggregation module and a fully connected regression head and is used for predicting low-dimensional semantic features related to pig body types; And forward reasoning is carried out on the 3D point cloud data through the deep learning network Net model, and a feature vector C is output and used for representing morphological or body type semantic features of the pig at the current visual angle.
9. The method of modeling swine 3D based on single view depth camera images of claim 8, wherein the construction of the deep learning network Net model comprises: acquiring a plurality of aligned 3D point cloud data and corresponding feature vectors C from historical data, integrating the aligned 3D point cloud data and the corresponding feature vectors C into a training data set and a verification data set, and performing end-to-end training on the deep learning network Net by using a gradient iteration minimization algorithm to minimize distance parameters by using the absolute value of the distance between the reconstructed 3D point cloud data and the 3D point cloud data of an original input model as an optimization target; After training, a deep learning network Net model for extracting the feature vector C from the pig deep image is obtained.
10. A system for modeling swine 3D based on a single view depth camera image, operating on a method of modeling swine 3D based on a single view depth camera image as claimed in any one of claims 1-9, comprising a processing module and a reconstruction module; the processing module is used for acquiring 3D point cloud data of a plurality of groups of pig samples to obtain a plurality of 3D point cloud data sets, and respectively carrying out semantic alignment and spatial alignment on the 3D point cloud data of each sample in the 3D point cloud data sets to obtain aligned 3D point cloud data sets; Analyzing average shape point cloud data B and deformation characteristic vectors V based on the aligned 3D point cloud data sets, wherein the average shape point cloud data B represents a datum point cloud model of a typical three-dimensional form of a pig group; inputting the aligned 3D point cloud data set into a deep learning network Net model, and outputting a feature vector C; the reconstruction module is used for reconstructing and obtaining the complete 3D point cloud data model M of the pig according to the formula M=B+ C@V.

Description

Method and system for modeling pig 3D based on single-view depth camera image Technical Field The application relates to the field of 3D modeling, in particular to a method and a system for modeling pig 3D based on a single-view depth camera image. Background In modern intelligent breeding systems, accurate acquisition of three-dimensional body type phenotypes of pigs is a key to achieving efficient breeding, health assessment and growth management. In recent years, depth cameras are becoming an important tool for animal 3D data acquisition due to their low cost and easy deployment. However, the single-view depth image can only capture the surface information of the pig body part, and is influenced by the shooting angle, pig gestures and individual scale differences, and the generated original 3D point cloud has the problems of serious shielding, inconsistent pose, disordered point sequence semantics and the like. The prior art is insufficient in coping with the defects that on one hand, the traditional point cloud registration method cannot establish an anatomical structure corresponding relation across samples, so that the same index points represent different body parts, on the other hand, the multi-view reconstruction or high-precision scanning scheme is high in hardware cost and complex in operation and difficult to apply in a large scale in an actual culture environment, and a pure data-driven deep learning model often lacks modeling on population morphology statistics rules, so that deformation characteristics with biological significance are difficult to learn under the condition of no complete 3D labeling. Therefore, a pig 3D modeling method which has consistent anatomical semantics, robust posture and strong interpretability can be realized under the conditions of single visual angle, low cost and no fine labeling, so as to support the intelligent upgrading of animal husbandry is needed. Disclosure of Invention The application provides a method and a system for modeling pig 3D based on a single-view depth camera image, which solve the technical problem that in the prior art, the corresponding relation of posture difference, scale change and anatomical structure is fuzzy due to the fact that 3D point cloud data of different samples, so that the group statistical modeling and individual morphology comparison cannot be accurately performed. In order to achieve the above purpose, the application adopts the following technical scheme: in a first aspect, a method for 3D modeling of pigs based on single view depth camera images is provided, comprising: The 3D point cloud data of each sample in the 3D point cloud data set is subjected to semantic alignment and spatial alignment respectively to obtain an aligned 3D point cloud data set; analyzing average shape point cloud data B and deformation characteristic vectors V based on the aligned 3D point cloud data sets, wherein the average shape point cloud data B represents a reference point cloud model of a typical three-dimensional form of a pig group, and the deformation characteristic vectors V represent projection coefficients of form deviations of the 3D point cloud data of each sample relative to the average shape point cloud data B in a principal component space; inputting the aligned 3D point cloud data set into a deep learning network Net model, and outputting a feature vector C; and reconstructing according to a formula M=B+ C@V to obtain a complete 3D point cloud data model M of the pig, wherein @ represents matrix multiplication. Based on the technical scheme, in the method for modeling the 3D of the pig based on the single-view depth camera image, accurate acquisition of three-dimensional body type characteristics of the pig is important for breeding, health assessment and growth monitoring in intelligent breeding. However, the original 3D point cloud data generally has the problems of inconsistent point sequence, arbitrary gesture, serious occlusion and the like, and if the original 3D point cloud data is directly used for modeling, the group average morphology is fuzzy, the individual deformation information is distorted, and reliable phenotype analysis is difficult to support. Therefore, a 3D modeling method capable of effectively separating the commonality structure from the individual differences must be established. According to the scheme, through two-stage alignment, semantic alignment is performed to unify anatomical semantics of each point cloud, for example, the ith point of all samples is guaranteed to be a tail root, spatial alignment is performed to eliminate rigid body interference caused by rotation and translation, and therefore an aligned point cloud data set with high quality and consistent structure is obtained. On the basis, the average shape point cloud data B is calculated as a reference model of a typical form of a pig group, and a deformation characteristic vector V, namely a projection coefficient of a form deviation of