CN-121999421-A - Timber intelligent tallying system based on multi-view collaboration and multi-mode fusion

CN121999421ACN 121999421 ACN121999421 ACN 121999421ACN-121999421-A

Abstract

The application belongs to the field of computer vision and intelligent logistics, in particular to a timber intelligent tallying system based on multi-view cooperation and multi-mode fusion, which comprises a multi-mode acquisition array, a space-time reference synchronization module, a depth fusion module, a geometric correction engine and a cross-view topology duplication elimination module; the projection distortion is eliminated by laser radar and camera depth integration and end surface normal vector fitting, and topology association deduplication is realized based on geographic coordinates by combining RTK-GNSS and IMU pose parameters; by adopting the technical scheme, the method and the device can realize high-precision diameter-level measurement and automatic weight removal, and remarkably improve the robustness and the operation efficiency of wood tallying.

Inventors

LI DUO
SHEN XIAOJUN
SHI YAFEI
XIA MINLEI
YIN XIAODONG
HUANG QIANG
WU YUGUO

Assignees

张家港中理外轮理货有限公司
江苏苏州港集团有限公司
张家港港务集团有限公司港埠分公司

Dates

Publication Date: 20260508
Application Date: 20260409

Claims (10)

1. An intelligent wood tallying system based on multi-view collaboration and multi-mode fusion, which is characterized in that the system is constructed on a movable tallying operation platform, and comprises: the multi-mode data acquisition array is used for acquiring multi-source perception data of the wood stack to be tidied, wherein the multi-source perception data comprise color image data, three-dimensional laser point cloud data and long-wave infrared thermal imaging data; the space-time reference synchronization module is electrically connected with the multi-mode data acquisition array and provides time alignment references and space pose parameters for the multi-source sensing data; the multi-source feature depth fusion module is used for carrying out space voxel mapping on the multi-source perception data and generating an enhanced feature map for identifying the wood end face by dynamically adjusting feature weights of different modes; The three-dimensional space geometric correction engine constructs a reverse mapping matrix of projective transformation according to the three-dimensional space gesture of the wood end surface, restores the wood outline in the two-dimensional image to be an orthocircular outline of the physical space, and calculates the physical scale parameters of the wood; The cross-view topology collaborative deduplication module is used for constructing a global dynamic space database by taking absolute geographic coordinates as references, and performing unique identification and deduplication on wood targets repeatedly observed under different view angles through space-time topology association and motion compensation logic; And the central control and data storage system coordinates the concurrent running states of the modules, generates an electronic tally bill with a geofence label and transmits the electronic tally bill to the remote management terminal.
2. The intelligent wood tallying system based on multi-view collaboration and multi-modal fusion of claim 1, wherein the multi-modal data collection array comprises: At least two groups of symmetrically distributed industrial cameras, which adopt global shutter CMOS sensors for capturing texture features of the wood end face; the multi-line laser radar is arranged above the geometric center of the industrial camera and is used for scanning the three-dimensional space point cloud of the wood stack in real time; The long-wave infrared thermal imaging sensor is used for extracting outline features of the wood end face by utilizing emissivity difference of the wood and the environmental background; the industrial camera is arranged on the cargo handling operation platform through the three-axis active anti-shake platform, and the three-axis active anti-shake platform counteracts the vibration of the cargo handling operation platform through the reverse compensation logic according to the real-time motion parameters provided by the space-time reference synchronization module; And the auxiliary infrared light supplementing lamp array is matched with the photosensitive response peak value of the industrial camera in emission wavelength, is controlled by the central control and data storage system and is automatically started when the ambient illuminance is lower than a set illuminance threshold value.
3. The intelligent wood tallying system based on multi-view collaboration and multi-mode fusion of claim 2, wherein the space-time reference synchronization module comprises: The hardware trigger controller is driven by a Field Programmable Gate Array (FPGA), is connected with the industrial camera, the multi-line laser radar and the long-wave infrared thermal imaging sensor through hard wires and is used for generating a synchronous pulse sequence with controlled frequency; the inertial measurement module records the triaxial acceleration and triaxial angular velocity of the tally operation platform at sampling instant; The differential positioning system is used for providing absolute space coordinates of the tally operation platform at sampling instant; And the space-time reference synchronization module outputs the six-degree-of-freedom real-time pose parameters of the tally work platform by fusing the data of the inertia measurement module and the differential positioning system.
4. The intelligent wood tallying system based on multi-view collaboration and multi-modal fusion of claim 3, wherein the multi-source feature depth fusion module performs the following logic during operation: The three-dimensional point cloud captured by the multi-line laser radar and the two-dimensional image captured by the industrial camera are aligned in the pixel level space through the parameter matrix obtained through off-line calibration; Dividing a space body into a plurality of voxel modules by adopting a space voxelization mapping strategy, wherein each voxel module simultaneously stores an occupied state and reflection intensity from a multi-line laser radar, RGB color components from an industrial camera and temperature components from a long-wave infrared thermal imaging sensor to form a five-dimensional characteristic data structure; When the characteristics of the wood end face are extracted, self-adaptive illumination compensation logic is introduced, and when the image contrast of the visible light mode is lower than a preset threshold value, the characteristic weights of the long-wave infrared thermal imaging sensor and the multi-line laser radar reflection intensity are automatically adjusted.
5. The intelligent wood tallying system based on multi-view collaboration and multi-modal fusion according to claim 2, wherein the three-dimensional space geometry correction engine comprises performing the following steps in calculating the wood diameter level: Obtaining local point cloud slices corresponding to each wood end face to be identified from the multi-source feature depth fusion module, and fitting a geometric plane representing the wood end face by utilizing random sampling consistency logic; calculating a normal vector of the geometric plane, and determining an actual inclined posture of the wood end face in a three-dimensional space according to a three-dimensional included angle between the normal vector and a camera optical axis direction vector; Constructing a reverse mapping matrix of projective transformation, and resampling an end face projection contour which is elliptical in a two-dimensional image into an orthocircular contour in a three-dimensional physical space; And calculating the physical diameter, perimeter and cross section area of the wood end surface by combining the focal length parameter of the industrial camera and the depth distance measured by the multi-line laser radar.
6. The intelligent wood tallying system based on multi-view collaboration and multi-modal fusion of claim 5, wherein the three-dimensional space geometry correction engine further comprises: The correction module is used for estimating the overall trend and length of a single wood by combining the sparse point cloud captured by the multi-line laser radar on the side face of the wood, and carrying out integral operation on the end face area and the estimated effective length so as to realize the measurement of the volume of the single wood; The edge screening module based on geometric curvature change is used for calculating the local curvature of sampling points on the contour line after the contour of the end face is extracted, eliminating abnormal points with curvature mutation, and fitting the residual effective characteristic points by using a robust fitting algorithm so as to eliminate measurement errors caused by wood core cracking or edge breakage.
7. A multi-view collaboration and multi-modal fusion-based timber intelligent tallying system as claimed in claim 3 wherein the cross-view topology collaboration deduplication module comprises the following logic when processing overlapping observations: establishing a global space database by taking absolute geographic coordinates provided by the differential positioning system as a reference, distributing a unique global identification number UID for each newly identified wood, and storing the three-dimensional center centroid coordinates, the plane normal vector and corrected physical characteristic parameters; When the tallying operation platform moves to a subsequent visual angle, predicting the theoretical projection position of the timber stored in the database on the current image plane according to the current pose parameters; judging whether the current target is a recorded target or not by calculating the space distance deviation and the feature vector similarity between the current frame detection target and the predicted target in the database; The feature vector comprises a wood end face annual ring distribution frequency, a core region eccentricity and a texture complexity descriptor extracted by using a deep convolution network; if the spatial distance deviation and the feature vector similarity are both in the set deviation threshold range, the same target is judged, and the pose attribute of the same target is updated.
8. The intelligent wood tallying system based on multi-view collaboration and multi-mode fusion of claim 7, wherein the cross-view topology collaboration deduplication module further comprises: The state estimation logic based on pose diagram optimization is used for generating key frames in the moving process of the tally work platform, constructing a constraint network by matching common observation targets between adjacent key frames, and adjusting the pose of the key frames by utilizing the nonlinear optimization logic so as to minimize observation residual errors in a global range; And the closed loop detection logic is used for carrying out pose drift correction through identifying the marked timber characteristics when the tallying operation platform returns to the known geographic coordinate point, and if the pose closing error exceeds a set error threshold, the pose of the full-path key frame is adjusted by utilizing the linear interpolation logic.
9. The intelligent wood tallying system based on multi-view collaboration and multi-modal fusion of claim 1, wherein the central control and data storage system comprises performing the following operations in operational management: a multithread concurrency processing mechanism is adopted, and a data acquisition thread, a feature extraction thread and a geometric correction and duplication removal thread are synchronized through mutual exclusion lock and semaphore; And providing a real-time monitoring interface based on the augmented reality AR technology, and superposing and displaying the identification result, the real-time path level data and the global identification number UID on the real-time video stream in a semitransparent layer mode.
10. The intelligent wood tallying system based on multi-view collaboration and multi-modal fusion of claim 2, further comprising: The dynamic shielding processing module detects the edges of the rear row wood at the gap by utilizing the multi-echo technology of the multi-line laser radar, deduces the shape of a shielding area by combining the three-dimensional space occupation relation, and extracts the characteristics of the target from different view angles to splice and supplement; The physical environment correction module is used for integrating a temperature and humidity sensor to acquire atmospheric environment parameters and carrying out refractive index correction on time flight data of the multi-line laser radar; And the self-diagnosis module monitors the current, the temperature and the frame rate of each sensor in the multi-mode data acquisition array in real time, and triggers an audible and visual alarm when detecting lens pollution or signal lock loss.

Description

Timber intelligent tallying system based on multi-view collaboration and multi-mode fusion Technical Field The invention belongs to the field of computer vision and intelligent logistics, and particularly relates to a timber intelligent tallying system based on multi-view collaboration and multi-mode fusion. Background The global timber trade scale expansion and forestry digital transformation propulsion, the automatic tallying of logs and products become the key for improving port storage, logistics turnover and trade settlement efficiency, and the problems of large labor intensity, subjective judgment difference, parallax fatigue, environmental shielding and the like exist in the traditional manual in-situ measurement code printing counting, so that data lag and deviation are caused; The existing automatic tallying scheme adopts a high-resolution industrial camera to capture the wood end face image, combines a deep learning model to realize detection and feature extraction, introduces a multi-view image stitching technology to construct a global digital view, has superiority when processing single wood or simple standardized piling scenes, and remarkably improves the counting automation level. The method has the advantages that the method is limited in that the vision homogenization of the wood end face in the multi-view stacking scene causes the topological misalignment of 2D feature matching, count drift and missing meter are caused, the 2D image processing scheme is influenced by random inclined projection, the real inclination angle of the end face is difficult to accurately restore, the calculation deviation of the diameter level and the timber volume is caused, the robustness of a single visible light mode under the complex working condition is low, and the core technical bottleneck is that the precise reconstruction of the multi-view topology of the large-scale stacking scene, the accurate mapping of the three-dimensional space dimension and the multi-source information fusion compensation under the oblique projection are realized. Disclosure of Invention In order to overcome the deficiencies of the prior art, at least one technical problem presented in the background art is solved. The technical scheme adopted for solving the technical problems is that the intelligent timber tallying system based on multi-view collaboration and multi-mode fusion is constructed on a movable tallying operation platform and comprises the following components: The multi-mode data acquisition array is used for acquiring multi-source perception data of the wood stack to be tidied, wherein the multi-source perception data comprise high-resolution color image data, three-dimensional laser point cloud data and long-wave infrared thermal imaging data; The space-time reference synchronization module is electrically connected with the multi-mode data acquisition array and is used for providing time alignment references and space pose parameters for the multi-source sensing data; the multi-source feature depth fusion module is used for carrying out space voxel mapping on the multi-source perception data and generating an enhanced feature map for identifying the wood end face by dynamically adjusting feature weights of different modes; The three-dimensional space geometric correction engine is used for constructing a reverse mapping matrix of projective transformation according to the three-dimensional space gesture of the wood end surface, recovering the wood outline in the two-dimensional image into an orthocircular outline of the physical space, and calculating the physical scale parameters of the wood; The cross-view topology collaborative deduplication module is used for constructing a global dynamic space database by taking absolute geographic coordinates as references, and performing unique identification and deduplication on wood targets repeatedly observed under different view angles through space-time topology association and motion compensation logic; and the central control and data storage system is used for coordinating the concurrent running states of the modules, generating an electronic tally bill with a geofence label and transmitting a tally result to the remote management terminal. The multi-mode data acquisition array is fixedly arranged on a support frame of a tallying operation platform and comprises at least two groups of high-resolution industrial cameras, a group of multi-line laser radars and a group of long-wave infrared thermal imaging sensors, wherein the high-resolution industrial cameras adopt global shutter CMOS sensors, effective pixels of the global shutter CMOS sensors are not lower than 1200 ten thousand, low-distortion industrial lenses with constant apertures are arranged for acquiring high-definition color texture information of wood end faces, the multi-line laser radars are arranged above the geometric center of the camera array, the horizontal field angle of the multi-line laser radars