Search

CN-121989275-A - Hydropower station amphibious robot control system based on cross-domain body large model

CN121989275ACN 121989275 ACN121989275 ACN 121989275ACN-121989275-A

Abstract

The invention belongs to the technical field of robot control and large models, and discloses a hydropower station amphibious robot control system based on a cross-domain large model. The system integrates a cross-domain sensing encoder, a hydrodynamic world model solver and an amphibious mode switching controller. The cross-domain sensing encoder realizes cross-modal alignment of water environmental data through contrast learning and outputs unified semantic features, the hydrodynamic world model solver calculates action stability cost based on a hydrodynamic parameter model of an adaptive medium and outputs a compliance action command, and the amphibious modal switching controller preferentially executes modal switching logic and outputs an action driving signal after finishing the modal switching logic. The method can effectively solve the problems of perceived dislocation, unstable action and switching conflict of cross-medium inspection, and is suitable for multi-scene operation and maintenance operation of the hydropower station.

Inventors

  • DU JIAN

Assignees

  • 华电郑州机械设计研究院有限公司

Dates

Publication Date
20260508
Application Date
20260403

Claims (10)

  1. 1. A hydropower station amphibious robot control system based on a cross-domain model with a body is characterized in that the control system is carried on an amphibious inspection robot body, a cross-domain sensing encoder, a hydrodynamic world model solver and an amphibious mode switching controller are integrated, and data circulation is achieved among the modules through CAN bus interaction; The cross-domain sensing encoder is provided with a visual flow input interface and a sonar flow input interface, and is used for butting water environmental data acquired by a visual sensor through the visual flow input interface and butting underwater environmental data acquired by front-view sonar or side-scan sonar through the sonar flow input interface, respectively carrying out feature extraction and cross-mode alignment on the water environmental data and the underwater environmental data through a contrast learning mechanism, mapping the water environmental data and the underwater environmental data to the same semantic space, then outputting unified semantic features, and transmitting the unified semantic features to a hydrodynamic world model solver and an amphibious mode switching controller; The hydrodynamic world model solver is embedded in a planning layer of a cross-domain model with a large body, is used for receiving unified semantic features output by the cross-domain perceptual encoder and current mode state parameters fed back by an amphibious mode switching controller, extracting water flow velocity distribution and barrier boundary information of a target area based on the unified semantic features, calling a hydrodynamic parameter model under a corresponding medium by combining the current mode state parameters, calculating hydrodynamic stability cost corresponding to a target action by the water flow velocity distribution, the barrier boundary information and the current body gesture parameters before the robot action is generated, and automatically iterating and adjusting action output when the hydrodynamic stability cost exceeds a preset threshold value to transmit an action instruction meeting stability constraint to the amphibious mode switching controller; The amphibious mode switching controller is used for judging whether the mode switching process is currently performed after receiving the action command, converting the action command into a driving signal corresponding to an executing mechanism according to the current mode state and outputting the driving signal to a robot power system if the mode switching process is not performed, temporarily storing the action command if the mode switching process is performed, outputting the corresponding driving signal to the robot power system after finishing mode switching and finishing link adaptation, calculating the mode switching probability by combining the unified semantic features output by the cross-domain sensing encoder, outputting the mode switching signal when the switching probability exceeds a preset threshold, synchronously finishing the cross-medium adaptation of sensing, executing and communication links, and synchronously transmitting the current mode state parameters and the action executing state back to the cross-domain sensing encoder and the hydrodynamic world model solver.
  2. 2. A hydropower station amphibious robot control system based on a cross-domain large model with body according to claim 1, wherein a cross-domain sensing encoder adopts a loss function to realize feature extraction and cross-mode alignment on the above-water environment data and the underwater environment data respectively, and the loss function is defined as: Wherein sim (·) is a cosine similarity calculation function, v i is a visual feature of the ith sample, s i is a sonar feature of the ith sample corresponding to the same target, τ is a preset temperature coefficient, and the distance between the visual representation of the same physical entity and the sonar representation in a feature space is constrained by the loss function to be smaller than the distance between the visual representation and the sonar representation of different entities.
  3. 3. A hydropower station amphibious robot control system based on a cross-domain model with body as claimed in claim 1 wherein the hydrodynamic world model solver computes the hydrodynamic stability cost as a function of: Wherein x is the current state quantity of the robot, u is the control input quantity to be executed, H is the prediction time domain, F buoy is the buoyancy force born by the robot, F drag(v) is the water resistance born by the robot at the current flow velocity v, F thrust(u) is the thrust output by the propeller corresponding to the control input u, m is the robot mass, g is the gravity acceleration, ω is the angular speed of the machine body, λ is the angular speed weight coefficient, and when J stab exceeds the preset threshold, the model automatically iterates and optimizes the control input u until the stability cost meets the constraint.
  4. 4. A hydropower station amphibious robot control system based on a cross-domain large model with body according to claim 1, wherein the function of the amphibious mode switching controller calculation mode switching probability is: Wherein σ is a Sigmoid activation function, h water is the current water level height acquired by a water level sensor, h th is a preset mode switching water level threshold, F contact is the contact force between a wheel set acquired by a contact force sensor and a wall surface, α and β are preset weight coefficients, and when Pswitch is more than 0.9, mode switching logic is triggered.
  5. 5. The hydropower station amphibious robot control system based on a cross-domain large model with body according to claim 1, wherein the amphibious modality switching controller comprises a perception link layer, an execution link layer and a communication link layer; The perception link layer is used for starting an RGB visual sensor and a GPS/UWB positioning module in an underwater mode, closing the RGB visual sensor in an underwater mode, and starting a forward looking sonar and a blue-green laser sensor; the execution link layer is used for closing the vector propeller and activating the wheel/track adsorption mechanism in a water mode, and activating the vector propeller to output a pre-thrust to offset buoyancy interference and improve the working voltage of the adsorption mechanism in an underwater mode; the communication link layer is used for enabling Wi-Fi/5G wireless communication protocol in the water mode and switching to blue light communication or underwater sound communication protocol in the underwater mode.
  6. 6. A hydropower station amphibious robot control system based on a cross-domain large model as claimed in claim 1 wherein the cross-domain perceptual encoder comprises a physical attribute inference module; The physical attribute deducing module is used for deducing the material quality, hardness and friction coefficient physical attribute of the target object by combining the color texture information of the visual features and the echo intensity information of the sonar features, wherein the material quality, the hardness and the friction coefficient physical attribute are used for generating subsequent operation actions.
  7. 7. The hydropower station amphibious robot control system based on a cross-domain large model with body according to claim 1, wherein the cross-domain large model with body supports cross-medium strategy migration, wall walking and target detection strategies obtained through training under an air medium, and the control system is migrated to gate inspection and wall resident tasks under an underwater medium through a domain self-adaptive mechanism.
  8. 8. A hydropower station amphibious robot control method based on a cross-domain large body model, characterized in that the hydropower station amphibious robot control method based on a cross-domain large body model is applied to the hydropower station amphibious robot control system based on a cross-domain large body model as claimed in any one of claims 1 to 7, the method comprising: The cross-domain sensing encoder is used for interfacing water environment data acquired by a visual sensor through the visual flow input interface and interfacing underwater environment data acquired by front-view sonar or side-scan sonar through the sonar flow input interface, respectively carrying out feature extraction and cross-modal alignment on the water environment data and the underwater environment data through a contrast learning mechanism, mapping the water environment data and the underwater environment data to the same semantic space, then outputting unified semantic features, and transmitting the unified semantic features to a hydrodynamic world model solver and an amphibious modal switching controller; The hydrodynamic world model solver receives unified semantic features output by the cross-domain sensing encoder and current mode state parameters fed back by the amphibious mode switching controller, extracts water flow velocity distribution and barrier boundary information of a target area based on the unified semantic features, invokes a hydrodynamic parameter model under a corresponding medium in combination with the current mode state parameters, calculates hydrodynamic stability costs corresponding to the target actions by the water flow velocity distribution, the barrier boundary information and the current body posture parameters before the robot actions are generated, and automatically iterates and adjusts action output when the hydrodynamic stability costs exceed a preset threshold value to transmit action instructions meeting stability constraints to the amphibious mode switching controller; After receiving the action command, the amphibious mode switching controller judges whether the current mode switching process is performed, if the current mode switching process is not performed, the action command is converted into a driving signal corresponding to the executing mechanism according to the current mode state and is output to the robot power system, if the current mode switching process is performed, the action command is temporarily stored, and after the mode switching is completed and the link adaptation is finished, the corresponding driving signal is output to the robot power system; And the amphibious mode switching controller calculates mode switching probability by combining the unified semantic features output by the cross-domain sensing encoder, outputs a mode switching signal when the switching probability exceeds a preset threshold value, synchronously completes the cross-medium adaptation of sensing, executing and communication links, and synchronously returns the current mode state parameters and action executing states to the cross-domain sensing encoder and the hydrodynamic world model solver.
  9. 9. Hydropower station amphibious robot control equipment based on cross-domain large body model is characterized in that the hydropower station amphibious robot control equipment based on the cross-domain large body model comprises a memory, a processor and a hydropower station amphibious robot control program based on the cross-domain large body model, wherein the hydropower station amphibious robot control program based on the cross-domain large body model is stored on the memory and can run on the processor, and is configured to realize the steps of the hydropower station amphibious robot control method based on the cross-domain large body model according to any one of claims 1 to 7.
  10. 10. A storage medium having stored thereon a hydropower station amphibious robot control program based on a cross-domain large body model, which when executed by a processor, implements the steps of the hydropower station amphibious robot control method based on a cross-domain large body model as claimed in any one of claims 1 to 7.

Description

Hydropower station amphibious robot control system based on cross-domain body large model Technical Field The invention relates to the technical field of robot control and large models, in particular to a hydropower station amphibious robot control system based on a cross-domain large model. Background The inspection of core facilities such as a hydropower station stilling pool, a gate, a runner and the like requires robots to perform operations of crossing water and Liu Jiezhi, three types of pain points exist in the existing amphibious inspection robots generally, namely, the water vision and underwater sonar perception characteristics belong to different semantic spaces, the cross-medium target recognition error is large, the action planning is not adapted to the hydrodynamic characteristics of different mediums, the underwater operation is easy to occur unstable, the mode switching and the action execution are free of priority constraint, the link conflict easily occurs in the switching process to cause out of control, and the reliability requirement of complex operation and maintenance scenes cannot be met. The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art. Disclosure of Invention The invention mainly aims to provide a hydropower station amphibious robot control system based on a cross-domain body large model, and aims to solve the technical problems that a target identification error is large due to cross-medium perception characteristic dislocation, underwater action is unstable due to the fact that a hydrodynamic planning is not suitable for medium characteristics, and switching is out of control easily due to no priority constraint in mode switching and action execution. In order to achieve the above purpose, the invention provides a hydropower station amphibious robot control system based on a cross-domain large model with a body, wherein the control system is carried on an amphibious inspection robot body, and a cross-domain sensing encoder, a hydrodynamic world model solver and an amphibious mode switching controller are integrated to realize data circulation through CAN bus interaction between the modules; The cross-domain sensing encoder is provided with a visual flow input interface and a sonar flow input interface, and is used for butting water environmental data acquired by a visual sensor through the visual flow input interface and butting underwater environmental data acquired by front-view sonar or side-scan sonar through the sonar flow input interface, respectively carrying out feature extraction and cross-mode alignment on the water environmental data and the underwater environmental data through a contrast learning mechanism, mapping the water environmental data and the underwater environmental data to the same semantic space, then outputting unified semantic features, and transmitting the unified semantic features to a hydrodynamic world model solver and an amphibious mode switching controller; The hydrodynamic world model solver is embedded in a planning layer of a cross-domain model with a large body, is used for receiving unified semantic features output by the cross-domain perceptual encoder and current mode state parameters fed back by an amphibious mode switching controller, extracting water flow velocity distribution and barrier boundary information of a target area based on the unified semantic features, calling a hydrodynamic parameter model under a corresponding medium by combining the current mode state parameters, calculating hydrodynamic stability cost corresponding to a target action by the water flow velocity distribution, the barrier boundary information and the current body gesture parameters before the robot action is generated, and automatically iterating and adjusting action output when the hydrodynamic stability cost exceeds a preset threshold value to transmit an action instruction meeting stability constraint to the amphibious mode switching controller; The amphibious mode switching controller is used for calculating mode switching probability by combining the environment semantic features output by the cross-domain sensing encoder, outputting mode switching signals when the switching probability exceeds a preset threshold, synchronously completing sensing, executing and cross-medium adaptation of a communication link, and simultaneously transmitting current mode state parameters back to the cross-domain sensing encoder and the hydrodynamic world model solver to guide sensing feature weight adjustment and planning logic adaptation. The amphibious mode switching controller is used for judging whether the mode switching process is currently performed after receiving the action command, converting the action command into a driving signal corresponding to an executing mechanism according to the current mode s