Search

CN-121984997-A - Land-based data automatic receiving and preprocessing management system and method

CN121984997ACN 121984997 ACN121984997 ACN 121984997ACN-121984997-A

Abstract

The invention discloses an automatic land-based data receiving and preprocessing management system and method, which relate to the technical field of marine observation data processing and comprise a communication service subsystem, a data preprocessing subsystem, an intelligent perception linkage subsystem and a data management subsystem, wherein full-flow automatic processing of underwater acoustic real-time communication submerged buoy observation data is realized through modularized integration. The method realizes the automatic receiving, safe storage and efficient pretreatment of the deep sea power and the bio-geochemical element data, provides reliable data support for ocean scientific research and environmental monitoring, and is suitable for the long-term observation requirement of a 6000-meter water depth submerged buoy system.

Inventors

  • CAO ZHIYONG
  • ZHANG ZHIXIANG
  • LI YANQING
  • ZHOU XIN
  • ZHANG XIANGGUANG
  • WANG JIANING

Assignees

  • 中国科学院海洋研究所

Dates

Publication Date
20260505
Application Date
20260116

Claims (10)

  1. 1. An automatic land-based data receiving and preprocessing management system, comprising: The communication service subsystem is used for establishing a data transmission link with the underwater acoustic real-time communication submerged buoy so as to realize automatic receiving and breakpoint continuous transmission of the observed data; The data preprocessing subsystem is used for cleaning, checking, compressing and converting the received original observed data to output standardized data; the intelligent perception linkage subsystem is used for identifying ocean phenomena based on the preprocessed observation data and generating a submerged buoy observation mode regulation and control instruction; and the data management subsystem is used for storing, retrieving and backing up the preprocessed standardized data and supporting data visualization and report generation.
  2. 2. The land-based data automatic receiving and preprocessing management system according to claim 1, wherein the communication service subsystem comprises a satellite communication receiving unit, an underwater sound link adapting unit and a breakpoint continuous transmission control unit; The satellite communication receiving unit supports dual-mode switching of a domestic Beidou satellite and a Tiantong satellite, adopts a single-module multi-card integrated circuit, receives observation data sent by a submerged buoy water surface floating body through a dynamic sub-packet transmission mechanism, and adapts an MFSK mode and a QPSK mode of the underwater acoustic communication unit in receiving rate; the underwater acoustic link adapting unit establishes data interaction with the submerged buoy underwater acoustic communication machine through an RS-232 serial port, analyzes data verification information in an underwater acoustic communication protocol and judges the integrity of data reception; The breakpoint continuous transmission control unit records the time stamp and the data block number of the received data, automatically sends a non-received data request to the submerged buoy after the link is recovered when the satellite signal is interrupted or the underwater acoustic link is abnormal, and supplements and transmits the missing data, thereby ensuring that the success rate of data reception is more than or equal to 90%.
  3. 3. The land-based data automatic receiving and preprocessing management system according to claim 2, wherein the communication service subsystem further adopts a Linux kernel operating system to avoid security holes of a Microsoft system, a dual-server hot standby mechanism is configured, and when a main server fails, a standby server switches pipe data receiving tasks within less than or equal to 5 minutes, so that receiving continuity is ensured.
  4. 4. The land-based data automatic receiving and preprocessing management system as recited in claim 3, wherein said data preprocessing subsystem comprises a data cleansing unit, a multi-dimensional verification unit, an adaptive compression unit and a format conversion unit; The data cleaning unit is used for identifying and marking invalid values and abnormal extremum values in the original data, removing an observation data layer with NaN accounting for more than 1/3, and reserving effective data segments; the multi-dimensional verification unit: logic verification, namely judging the data logic consistency based on marine dynamic parameter relevance and bio-geochemical element balance; The transmission check, namely comparing the data check sum in the underwater acoustic communication protocol with a local calculated value, screening transmission error data and triggering retransmission; The self-adaptive compression unit compresses the standardized data by adopting a compression algorithm combining arithmetic coding and dictionary coding, the compression ratio is more than or equal to 1:3, and the data precision is reserved in the compression process; the format conversion unit converts original binary data of the submerged buoy into a universal data format and is suitable for common data processing software for ocean science research.
  5. 5. The land-based data automatic receiving and preprocessing management system according to claim 4, wherein the intelligent perception linkage subsystem comprises a data feature extraction unit, a marine phenomenon recognition unit and a regulation and control instruction generation unit; The data characteristic extraction unit is used for extracting a horizontal flow velocity component from the preprocessed ADCP flow velocity data, selecting the latest 128-hour integral point data and calculating the power spectrum density S (k); The marine phenomenon identification unit is used for integrating the power spectral density S, integrating frequency bands are respectively 0.8-1.2f and 0.8 f-0.5 omega, f is a local Ke' S frequency, omega is a sampling frequency, and calculating the near inertial energy ratio through integration: setting a dynamic threshold, namely, an initial threshold of 0.42, wherein no near-inertia signal is detected for 2 months continuously, the initial threshold is adjusted down to 0.40, when the threshold is r > the near-inertia internal wave is judged to exist, and when the threshold is r < the threshold is judged to end the near-inertia internal wave; The regulation and control instruction generation unit is used for generating a submerged buoy observation mode regulation and control instruction according to the marine phenomenon recognition result, and transmitting the regulation and control instruction to the submerged buoy water surface floating body in a reverse direction through a satellite communication link to realize the self-adaptive regulation of the observation mode.
  6. 6. The land-based data automatic receiving and preprocessing management system as recited in claim 5, wherein said data management subsystem comprises a distributed storage unit, an intelligent retrieval unit, a data backup unit and a visualization unit; The distributed storage unit adopts a partition storage strategy, establishes a data storage catalog according to the type of the observation element, and supports the storage of continuous observation data for more than or equal to 6 months; the intelligent searching unit supports multi-condition combined searching, and the searching response time is less than or equal to 10 seconds; The data backup unit adopts a local backup mechanism and a remote backup mechanism, wherein the local backup period is 1 hour/time, the remote backup period is 1 day/time, and the backup data retention time is more than or equal to 3 years; the visualization unit automatically generates a time sequence diagram and a profile distribution diagram of the observed data and supports marking and deriving of data abnormal points.
  7. 7. The system of claim 6, wherein the satellite communication receiving unit further comprises a signal quality monitoring module for monitoring satellite signal strength in real time, automatically switching to a spare satellite when the signal strength is below a predetermined threshold, and recording a signal switching log.
  8. 8. The system according to claim 7, wherein the data cleansing unit further performs special processing on the ADCP data, removing non-whole point data with 10, 20, 30, 40, 50 min, retaining whole point sampling data, performing layer screening on the ADCP flow rate data, removing depth layers with poor quality of near-surface data, and retaining effective observation depth layer data.
  9. 9. A method of automatically receiving and preprocessing land-based data in accordance with any one of claims 1-8, comprising the steps of: s1, initializing a system, starting a satellite communication receiving unit and an underwater acoustic link adapting unit by a communication service subsystem, establishing a data transmission link with a submerged buoy, and initializing a data receiving record list by a breakpoint continuous transmission control unit; S2, automatically receiving data, wherein a satellite communication receiving unit receives observation data transmitted by a submerged buoy water surface floating body through a Beidou/Tiantong satellite, a hydroacoustic link adapting unit receives deep data transmitted by a hydroacoustic communication machine, and a breakpoint continuous transmission control unit records a time stamp and a data block number of the received data in real time; s3, data preprocessing, namely removing invalid values and non-integral point data by a data cleaning unit, completing logic verification and transmission verification by a multi-dimensional verification unit, compressing the verified data by a self-adaptive compression unit, and converting the data into NetCDF/CSV format by a format conversion unit; s4, performing power spectrum analysis on the preprocessed ADCP data by the intelligent sensing and linkage subsystem, calculating the near inertial energy duty ratio r, generating a submerged buoy regulation instruction according to the comparison result of the r and the dynamic threshold, and reversely transmitting the submerged buoy regulation instruction to the submerged buoy through a satellite communication link; and S5, data management, wherein the data management subsystem stores the preprocessed standardized data into a distributed storage unit, starts intelligent retrieval, double backup and visualization functions and generates a data receiving and preprocessing report.
  10. 10. The method of automatic land-based data receiving and preprocessing management system according to claim 9, wherein in step S2, an exception handling mechanism is further included, when no data is received by the communication service subsystem for 5 minutes, automatically triggering link diagnosis, if the satellite signal problem is the satellite signal problem, switching the standby satellite, if the satellite signal problem is the underwater sound link problem, recording a missing data period, and executing breakpoint continuous transmission after the link is restored, and in step S3, if the data error is found by verification, transmitting a retransmission request of the data block to the submerged buoy until correct data is received.

Description

Land-based data automatic receiving and preprocessing management system and method Technical Field The invention relates to the technical field of ocean observation data processing, in particular to a land-based data automatic receiving and preprocessing management system and method. Background With the development of deep sea ocean observation technology, underwater acoustic real-time communication submerged buoy has become a core device for acquiring ocean power (temperature, salinity, flow velocity and depth) and bioelectrochemical (chlorophyll, nitrate, dissolved oxygen and pH value) elements. The submerged buoy observation data is required to be transmitted to a land-based platform for processing and analysis, but the existing land-based data processing mode has the following key problems: The existing land-based system is not timely and poor in integrity, data is received by relying on a single satellite (such as a Beidou satellite only) or a single link (such as a hydroacoustic link only), when satellite signals are not covered enough (such as the Beidou signal success rate of a western Pacific partial area is as low as 1.63%) or the link is interrupted, the data is easy to lose, an effective breakpoint continuous transmission mechanism is lacking, missing data cannot be supplemented, and the data receiving success rate is difficult to meet scientific research requirements (the average success rate of the prior art is less than 80%). The data preprocessing efficiency is low, the accuracy is insufficient, the original data of the submerged buoy comprise invalid values (NaN), non-integral point data and transmission error data, the existing processing mode is mostly dependent on manual screening and verification, the efficiency is low, errors are prone to occurring, and particularly, the ADCP flow velocity data are not specially processed according to the sampling characteristics (high reliability of integral point data) of the ADCP flow velocity data, so that the deviation of the results of the follow-up marine phenomenon recognition (such as near inertial internal waves) is large. The existing land-based system only has the data receiving and storing functions, can not identify the ocean phenomenon (such as near-inertial internal wave) based on the data characteristics and reversely regulate and control the observation mode of the submerged buoy, so that the submerged buoy still maintains the fixed observation frequency (such as ADCP1 time/hour), and the high-frequency capturing of the sudden phenomenon and the low-power consumption requirement of the conventional period are difficult to be considered. The data security and management capability are weak, a part of land-based systems adopt Windows operation systems, security holes exist, data storage is lack of partition strategies and double backup mechanisms, data are easy to be confused or lost, the retrieval efficiency is low, observation data of specific elements and specific time periods cannot be rapidly positioned, and scientific research analysis efficiency is affected. In view of the foregoing, there is a need for a land-based data system with automatic reception, efficient preprocessing, intelligent linkage and safety management functions to adapt to the long-term, multi-element observation requirements of deep sea submerged buoy. Disclosure of Invention Aiming at the defects of the prior art, the automatic land-based data receiving and preprocessing management system and method are provided, automatic receiving, accurate preprocessing, intelligent linkage regulation and control and safety management of the submerged buoy observation data are realized through modular design, the success rate of data receiving, preprocessing efficiency and scientific research applicability are improved, and the defects of the prior art are overcome. A land-based data automatic reception and preprocessing management system comprising: The communication service subsystem is used for establishing a data transmission link with the underwater acoustic real-time communication submerged buoy so as to realize automatic receiving and breakpoint continuous transmission of the observed data; The data preprocessing subsystem is used for cleaning, checking, compressing and converting the received original observed data to output standardized data; the intelligent perception linkage subsystem is used for identifying ocean phenomena based on the preprocessed observation data and generating a submerged buoy observation mode regulation and control instruction; and the data management subsystem is used for storing, retrieving and backing up the preprocessed standardized data and supporting data visualization and report generation. The communication service subsystem comprises a satellite communication receiving unit, an underwater acoustic link adapting unit and a breakpoint continuous transmission control unit; the satellite communication receiving unit supports dual-mode s