Search

CN-121655538-B - Self-adaptive navigation path planning system of field robot based on convolutional neural network

CN121655538BCN 121655538 BCN121655538 BCN 121655538BCN-121655538-B

Abstract

The invention discloses a field robot self-adaptive navigation path planning system based on a convolutional neural network, and belongs to the technical field of agricultural robots. The method solves the problem that the prior art is difficult to meet the requirements of efficient and accurate operation of the field robot, identifies and marks the key entities in the field through the convolutional neural network, senses the environmental change in advance, enables the robot to react in advance, avoids failure or interruption of path planning, dynamically adjusts the path planning weight according to the task type and the real-time environment, flexibly adapts to different requirements and conditions, ensures the continuity and stability of the operation, defines a balance mechanism of automatic decision and manual intervention, combines a three-level abnormal handling strategy, ensures quick and effective response under different abnormal conditions, improves the intelligent level of the system, remarkably enhances the operation efficiency and adaptability of the robot in the complex environment, and forcefully promotes agricultural automation and intelligent development.

Inventors

  • YANG KE
  • SHI HENGLIANG
  • XIN LIJUN
  • SONG YU
  • ZHOU YUNLONG

Assignees

  • 洛阳职业技术学院

Dates

Publication Date
20260512
Application Date
20260204

Claims (6)

  1. 1. The field robot self-adaptive navigation path planning system based on the convolutional neural network is characterized by comprising the following components: the twin modeling unit is configured to construct a digital twin model reflecting dynamic changes of the field environment in real time based on the multidimensional data of the field environment; The environment sensing unit is configured to construct a convolutional neural network for environment sensing based on historical data, image data of a field environment is used as input, features in the image are automatically extracted based on the convolutional neural network, and recognition results of obstacles, crop areas and passable areas in the environment are output; the self-adaptive path planning unit is configured to carry out path planning by adopting a fuzzy logic algorithm in combination with the kinematic and dynamic constraints of the field robot and path factors based on the environmental information output by the digital twin model and the recognition result output by the convolutional neural network; The system comprises a data acquisition unit, a soil humidity sensor, a data acquisition unit and a control unit, wherein the data acquisition unit is configured to install various sensors on a field robot, and comprises a laser radar, an ultrasonic sensor, a soil humidity sensor and a soil humidity sensor, wherein the laser radar is used for accurately measuring the distance between obstacles and the height of terrain; the data processing unit is configured to fuse the image data, the laser radar data, the ultrasonic data and the soil humidity data by adopting a multi-mode fusion technology to generate multi-mode fusion data; a data processing unit comprising: The acquisition subunit is used for acquiring multi-source heterogeneous original data; The system comprises a preprocessing subunit, a non-stationary index calculation unit, a non-stationary calibration unit, a multi-scale window configuration unit, a multi-scale sampling rate calculation unit and a multi-scale sampling rate calculation unit, wherein the preprocessing subunit is used for uniformly converting various types of original data in multi-source heterogeneous original data into a matrix format of time stamp-characteristic dimension, and performing time dimension alignment on all data based on time stamps; the first fusion subunit is used for carrying out self-adaptive time-frequency decomposition on a mode with strong periodicity in the calibrated data, the strong periodicity data adopts 64-point fast Fourier transform, the weak periodicity data adopts 32-point fast Fourier transform, the amplitude and phase characteristics of the main frequency are extracted, the amplitude and phase characteristics are spliced with the time domain characteristics to form double-domain characteristics, and the double-domain characteristics are fused with the multi-scale unified characteristics to obtain a standardized multi-scale characteristic sequence; The enhancement subunit is used for processing the standardized multi-scale characteristics of each mode through the intra-mode characteristic enhancement branch formed by the improved MambaFFN module, wherein the improved MambaFFN module comprises a linear characteristic enhancement branch and a nonlinear characteristic enhancement branch, and the two branch outputs are dynamically weighted and fused according to the signal to noise ratio of the mode characteristics to obtain the intra-mode enhancement characteristics; The second fusion subunit is used for constructing a cross-mode association prior matrix, wherein matrix elements are preset association strengths among modes, and the intra-mode enhancement features of each mode and intra-mode enhancement features of other modes are weighted and summed according to the association strengths to obtain cross-mode association features; the third fusion subunit is used for splicing the intra-mode enhancement features of each mode with the cross-mode association features, inputting the intra-mode enhancement features into a mode full-connection layer, unifying feature dimensions to 1024 dimensions and outputting a final feature sequence of each mode; the screening subunit is used for calculating the credibility of the characteristic data of each mode, screening the characteristic of each mode based on the credibility and obtaining the characteristics after screening; the fourth fusion subunit is used for aligning all the screened features to a uniform length by adopting a dynamic time warping algorithm, constructing a cross-modal attention matrix, calculating the association strength of any two modal features at each time step, and weighting and fusing all the modal features according to the association strength to obtain a multi-modal fusion feature; The preprocessing subunit calculates non-stationary indexes corresponding to each mode data, wherein the non-stationary indexes are different non-stationary indexes corresponding to different data, and the non-stationary indexes comprise, but are not limited to, variance and trend item slope of laser radar data, the trend item slope is calculated through 10-second sliding window linear regression, soil humidity data is variance and periodic index, the periodic index is calculated through 60-second window FFT main frequency amplitude, ultrasonic data is variance and instantaneous fluctuation amplitude, the instantaneous fluctuation amplitude is 50ms window standard deviation, image target characteristic data is variance and target area change rate, and the target area change rate is adjacent frame relative change; Laser radar data calibration formula: Wherein, the method comprises the steps of, Sliding window mean for 10 seconds; sliding window standard deviation of 10 seconds; Is the trend slope; Is the current point in time; 、 is a learnable affine parameter; The laser radar data after calibration; Original laser radar data; soil humidity data calibration formula: Wherein, the method comprises the steps of, For the primary frequency to be the primary frequency, , Periodic amplitude and phase respectively; 、 is a learnable affine parameter; The calibrated soil humidity data; Original soil humidity data; Is the sliding window mean; Is the sliding window standard deviation; an ultrasonic data calibration formula: Wherein, the method comprises the steps of, Is the instantaneous fluctuation amplitude; A threshold value is used for judging whether instantaneous fluctuation needs to be compensated; 、 is a learnable affine parameter; The ultrasonic data after calibration; Is the original ultrasonic data; Is the sliding window mean; Is the sliding window standard deviation; Image data calibration formula: Wherein, the method comprises the steps of, Is the target area change rate; 、 is a learnable affine parameter; the image characteristic data after calibration; is the original image characteristic data; Is the sliding window mean; Is the sliding window standard deviation; for the rate of change of the area of the object, Time, the area change which is changed along with time is compensated; the ultrasonic wave and the laser radar are in a high sampling rate mode, the image acquisition is in a medium sampling rate mode, the soil humidity is in a low sampling rate mode, and all mode characteristics are unified to 64 time steps through linear interpolation/mean value pooling; The method comprises the steps of performing periodic detection on calibrated data, judging through a 30-second window autocorrelation coefficient, wherein >0.5 is strong periodicity, adopting 64-point FFT for strong periodicity data, adopting 32-point FFT for weak periodicity data, extracting the main frequency characteristic of the first 30%, and splicing with the time domain characteristic to form a double-domain characteristic; Linear characteristic enhancement branch, linear layer, sigmoid activation, intra-mode attention weight multiplication, wherein the weight ; Attention weight for modality i; Variance of features of modality i; Nonlinear characteristic enhancement branches, namely linear layer-FFN-state space model SSM-intra-mode attention weight multiplication; signal-to-noise ratio: ; is the signal to noise ratio; is the average absolute value of the signal amplitude; Is the standard deviation of noise; intra-modality enhancement features: The dimension is 256 dimensions; to enhance the modal characteristics; enhancing branch output for linear features; Enhancing branch output for nonlinear characteristics; Presetting association strength, laser radar-ultrasonic wave of 0.8, laser radar-image of 0.7, image-soil humidity of 0.3 and other of 0.2; Cross-modality association features: ; cross-modal correlation features for modality modal; intra-modal enhancement features for modality modal; The association strength is preset; Intra-modality enhancement features for other modalities j; Splicing intra-modal enhancement features and cross-modal association features, inputting a full-connection layer, and outputting 1024-dimensional final feature sequences of all modes; And (3) calculating the credibility of the characteristic data of each mode, wherein in a training stage, the real-time credibility of each mode is calculated based on error statistics, and the formula is as follows: ; The credibility of the modality modal; Is a model predictive value; A true value, an inference phase to calculate confidence based on historical prediction uncertainty, ; Preserving the mode characteristics with the credibility more than 0.3, and carrying out moving average filtering on the modes with low credibility to obtain screened characteristics; The dynamic time warping algorithm DTW aligns all modal features to 100 time steps; cross-modal attention: ; the correlation strength of the modes i and j at the time step t; 、 1024-dimensional features of modalities i and j at time step t; The adaptive path planning unit includes: The system comprises a thread creation module, a processing module and a processing module, wherein the thread creation module is configured to identify and mark key entities in a field environment by utilizing a convolutional neural network, and allocate a unique identifier for each entity; the method comprises the steps of establishing a digital thread link for each key entity in a digital twin model, recording and updating state information of each key entity in real time, including but not limited to position, form, speed and direction, synchronizing the state information into the corresponding digital thread link, acquiring historical state data of each key entity, including but not limited to position change, form change and speed change, analyzing motion rules and change trends of the key entity, constructing a prediction model by utilizing a long-short-period memory network, predicting possible states of the key entity in a future short period, including predicting swing amplitude and direction of crops by utilizing wind speed and wind direction data in combination with physical characteristics of the crops, analyzing movement tracks and speeds of suspected animal areas, predicting movement probability and direction of the suspected animal areas, outputting prediction results in a structured data form, including but not limited to position and form change range information of each key entity at a future time point, and providing environment auxiliary data for path planning; The method for constructing the prediction model by using the long-term and short-term memory network comprises the following steps: The method comprises the steps of obtaining a model training data set, carrying out feature extraction on the training data set, fusing the extracted features to obtain multi-mode fusion features, constructing a layered LSTM architecture, inputting final fusion features to capture short-term fine-grained time sequence dependence, outputting bottom features, carrying out self-adaptive time-frequency decomposition on the bottom features by a middle layer, constructing time-frequency attention weight dynamic fusion time domain features and frequency domain features, outputting middle fusion features, extracting shared features by an upper layer through a 1-layer full-connection layer, and constructing double-prediction branches, wherein the model training data set comprises laser radar data, ultrasonic data, soil humidity data and image data; The method comprises the steps of inputting sharing characteristics and physical characteristics of crops in a crop swing prediction branch, outputting swing amplitude and horizontal direction angle, inputting sharing characteristics and suspected animal area characteristics in an animal movement prediction branch, outputting movement probability, horizontal movement direction and movement speed, constructing a multidimensional loss function, performing iterative training on a model based on the multidimensional loss function until a training result meets requirements, and obtaining a prediction model; The bottom layer is a short-term time sequence feature capturing LSTM layer, the hidden layer dimension is 1024 by 2 layers of bidirectional LSTM, the dropout rate is 0.2, and the multi-mode feature is fused Capturing short-term fine-grained timing dependencies and outputting underlying features Simultaneously adding time sequence residual connection, and adding the input characteristics and the LSTM output characteristics according to elements: ; Feature vectors output for the underlying LSTM; Is a layer 2 bidirectional LSTM network; inputting final characteristics of multi-mode fusion; temporal features employ underlying output The time sequence dynamic change is reserved, the frequency domain features extract frequency domain amplitude and phase features by carrying out self-adaptive time-frequency decomposition on the bottom features, the dimensions are compressed to 1024 by global average pooling, and the time-frequency attention weight is used for balancing the time domain and frequency domain feature contributions: ; Is the time-frequency attention weight; a time domain feature vector output by the bottom LSTM; Is a frequency domain feature vector; Wherein, the method comprises the steps of, As a feature of the frequency domain, The function is activated for Sigmoid, A linear layer for outputting scalar; outputting middle layer fusion features Calculating a sequence autocorrelation coefficient and judging the time sequence dependence strength; the upper layer comprises a multi-task pre-measuring head, and outputs sharing characteristics through a 1-layer full-connection layer, reLU activation and batch normalization processing The bi-predictive branches include a crop swing predictive branch and an animal movement predictive branch, the crop swing predictive branch sharing features And the physical characteristics of the crops are spliced as input characteristics, and are output as swing amplitude and horizontal direction angle of the crops through 2 full-connection layers, and animal movement prediction branches share the characteristics And the suspicious animal region features are spliced and then used as input features, the input features are output as movement probability, horizontal movement direction and movement speed through 2 full-connection layers, a learnable interaction matrix is added between full-connection layers of crop swing prediction branches and animal movement prediction branches to restrict hidden layer features of the two branches, , wherein, Is a balance coefficient; Is a learnable interaction matrix; Hidden layer characteristics of swing branches of the crops after interaction; Hiding layer features for original crop swing branches; Hidden layer features of the predicted branch for animal movement; A multi-task weighted loss function, ; Weighting a loss function for the multitasking; Is the swing loss of crops; predicting loss for animal movement, and crop swing loss by adopting MSE loss, using MSE for swing amplitude and using angle period loss for horizontal direction angle: ; Wherein, the Is the swing loss of crops; is the predicted horizontal direction angle; is a true horizontal direction angle; is the number of time steps; is the predicted swing amplitude; The real swing amplitude; Animal movement loss, binary cross entropy for movement probability, angular period loss for movement direction, MSE for movement speed: ; Predicting total loss for animal movement; is a binary cross entropy loss; To predict a movement probability; Is the true movement probability; the angular period loss is the horizontal movement direction; To predict the horizontal movement direction angle; Is the true horizontal movement direction angle; To predict the speed of movement; Is the true moving speed; Modal consistency loss, namely calculating the pearson correlation coefficient of each modal characteristic mean value vector: Wherein, the method comprises the steps of, Is the number of modes; (. G) is a feature dimension mean; the method comprises the steps of presetting an incidence matrix; Is a loss of modal consistency; (. Quadrature.) is the pearson correlation coefficient; Is the total number of time steps; Multidimensional loss function: ; Is a multidimensional loss; weighting the loss for the multitasking; Is a loss of modal consistency.
  2. 2. The convolutional neural network-based field robot adaptive navigation path planning system of claim 1, wherein the adaptive path planning unit further comprises: the path adjustment module is configured to acquire the current task type of the field robot and evaluate the priority of the task according to the importance and the emergency degree of the task; According to the current environment information and task demands, dynamically adjusting the target weight of path planning, and meeting diversified field operation demands; in the path planning process, the change of the field environment is monitored in real time, and the environment information is updated rapidly through a digital twin model; and re-evaluating task demands and path planning target weights according to the updated environment information, and dynamically adjusting a path planning strategy.
  3. 3. The convolutional neural network-based field robot adaptive navigation path planning system of claim 2, wherein the adaptive path planning unit further comprises: the intelligent decision module is configured to define an automatic decision and manual intervention balance mechanism, and divide tasks into three priorities of high, medium and low according to the complexity, risk and importance of the tasks; When the automatic decision is made, the high-priority task is selected with priority to be manually interfered, and the low-priority task is selected with priority to be made with the automatic decision; and creating a three-level exception handling strategy, and adopting different handling modes according to the severity and complexity of the exception.
  4. 4. The adaptive navigation path planning system of a field robot based on convolutional neural network of claim 2, wherein dynamically adjusting the target weight of the path plan based on current environmental information and task requirements comprises: defining a plurality of target weights of path planning according to task demands, wherein the target weights comprise a path shortest weight, a smoothness weight and a safety weight; acquiring environment information in the digital twin model in real time, and evaluating the influence of the current environment on a path planning target; defining a fuzzy logic rule, and converting the environment information into a fuzzy set; Matching the current environment information with the rule corresponding to the task demand according to the fuzzy logic rule, and carrying out reasoning calculation to obtain a fuzzy reasoning result; And converting the fuzzy reasoning result into an adjustment value of the target weight, and dynamically adjusting the target weight.
  5. 5. The convolutional neural network-based field robot adaptive navigation path planning system of claim 2, wherein the adaptive path planning unit further comprises: A lifecycle management module configured to automatically create its digital thread links when a new critical entity is detected; For digital thread links which are not updated for a long time or have left the region of interest or the entity state is stable, the digital thread links are deleted from the archived database, and the computing resources are released.
  6. 6. The convolutional neural network-based field robot adaptive navigation path planning system of claim 1, wherein the context awareness unit comprises: And the multi-scale feature extraction module is configured to extract local features and global features in the image at the same time, and draw attention mechanisms to enable the convolutional neural network to automatically focus on a key region in the image.

Description

Self-adaptive navigation path planning system of field robot based on convolutional neural network Technical Field The invention relates to the technical field of agricultural robots, in particular to a field robot self-adaptive navigation path planning system based on a convolutional neural network. Background With the advancement of agricultural modernization, field robots are increasingly used in agricultural production. However, the field environment is complex and changeable, and many factors such as obstacles, topography fluctuation, crop growth difference and the like exist, so that the traditional navigation path planning method is difficult to meet the requirements of efficient and accurate operation of the field robot. Therefore, in order to meet the existing requirements, a field robot self-adaptive navigation path planning system based on a convolutional neural network is specially provided. Disclosure of Invention The invention aims to provide a field robot self-adaptive navigation path planning system based on a convolutional neural network, which is used for identifying and marking field key entities through the convolutional neural network, perceiving environmental changes in advance, enabling a robot to react in advance, avoiding path planning failure or operation interruption, dynamically adjusting path planning weights according to task types and real-time environments, flexibly adapting to different requirements and conditions, guaranteeing operation continuity and stability, defining a balance mechanism of automatic decision and manual intervention, combining three-level abnormal handling strategies, ensuring quick and effective response under different abnormal conditions, improving the intelligent level of the system, remarkably enhancing the operation efficiency and adaptability of the robot in complex environments, forcefully promoting agricultural automation and intelligent development, and solving the problems in the background technology. In order to achieve the above purpose, the present invention provides the following technical solutions: the field robot self-adaptive navigation path planning system based on the convolutional neural network comprises: the twin modeling unit is configured to construct a digital twin model reflecting dynamic changes of the field environment in real time based on the multidimensional data of the field environment; The environment sensing unit is configured to construct a convolutional neural network for environment sensing based on historical data, image data of a field environment is used as input, features in the image are automatically extracted based on the convolutional neural network, and recognition results of obstacles, crop areas and passable areas in the environment are output; the self-adaptive path planning unit is configured to carry out path planning by adopting a fuzzy logic algorithm in combination with the kinematic and dynamic constraints of the field robot and path factors based on the environmental information output by the digital twin model and the recognition result output by the convolutional neural network. Further, the adaptive path planning unit includes: the system comprises a thread creation module, a processing module and a processing module, wherein the thread creation module is configured to identify and mark key entities in a field environment by utilizing a convolutional neural network, and allocate a unique identifier for each entity; Creating a digital thread link for each key entity in the digital twin model, recording and updating state information of each key entity in real time, including but not limited to position, form, speed and direction, and synchronizing the state information into the corresponding digital thread link; Acquiring historical state data of each key entity, including but not limited to position change, form change and speed change, and analyzing a motion rule and a change trend of the key entity; The method comprises the steps of constructing a prediction model by utilizing a long-short-period memory network, and predicting possible states in a future short period by utilizing wind speed and wind direction data in combination with physical characteristics of crops, predicting swing amplitude and direction of the crops; The prediction results are output in the form of structured data, including but not limited to the position of each key entity at the future time point, morphological change range information, and environment auxiliary data is provided for path planning. Further, the adaptive path planning unit further includes: the path adjustment module is configured to acquire the current task type of the field robot and evaluate the priority of the task according to the importance and the emergency degree of the task; According to the current environment information and task demands, dynamically adjusting the target weight of path planning, and meeting diversified field operation deman