CN-122004922-A - Multi-mode ultrasonic scanning method, device and related system

CN122004922ACN 122004922 ACN122004922 ACN 122004922ACN-122004922-A

Abstract

The application relates to the technical field of medical intelligent diagnosis and provides a multi-mode ultrasonic scanning method, a device and a related system, wherein the method collects depth image data of a skin area of a user through a camera module, a mechanical arm is controlled to adjust an ultrasonic probe to a first target pose according to scanning intention information and the depth image data of the user, wherein the ultrasonic probe and a force feedback sensor are integrated at the tail end of the mechanical arm, an ultrasonic quality evaluation parameter is calculated according to first ultrasonic image data of the ultrasonic probe and first contact pressure data of the force feedback sensor, a first ultrasonic quality evaluation parameter is obtained, and the ultrasonic probe is adjusted to a second target pose through the mechanical arm according to the first ultrasonic quality evaluation parameter; and scanning the user according to the depth image data, the second ultrasonic image data of the ultrasonic probe and the second contact pressure data of the force feedback sensor to obtain a scanning result. Thus, stability and safety of the ultrasonic probe during scanning are improved.

Inventors

CAI JINZHONG
CHENG LIXIN
HU HAO
HUANG YUN
WU FANGJING
GENG QINGSHAN
WANG GUANGSUO
SU WEIZHU
YU HAITIAN
WANG QINGJIE
LIU DONG
LIU MING
GONG WEI

Assignees

深圳市人民医院

Dates

Publication Date: 20260512
Application Date: 20260413

Claims (10)

1. A multi-modality ultrasound scanning method for use with a host device in an ultrasound scanning system, the method comprising: collecting depth image data of a target skin area of a user through a camera module; Controlling a mechanical arm to adjust an ultrasonic probe to a first target pose according to scanning intention information of a user and the depth image data, wherein the ultrasonic probe and a force feedback sensor are integrated at the tail end of the mechanical arm; calculating an ultrasonic quality evaluation parameter according to the first ultrasonic image data of the ultrasonic probe and the first contact pressure data of the force feedback sensor to obtain a first ultrasonic quality evaluation parameter; Adjusting the ultrasonic probe to a second target pose through the mechanical arm according to the first ultrasonic quality evaluation parameters; and scanning the user according to the depth image data, the second ultrasonic image data of the ultrasonic probe and the second contact pressure data of the force feedback sensor to obtain an ultrasonic scanning result of the target skin region.
2. The method of claim 1, wherein calculating an ultrasonic quality assessment parameter from the first ultrasonic image data of the ultrasonic probe and the first contact pressure data of the force feedback sensor, results in a first ultrasonic quality assessment parameter, comprising: Calculating the first ultrasonic image data based on a preset edge detection algorithm to obtain image edge contrast parameters, wherein the image edge contrast parameters comprise gray scale contrast parameters and signal-to-noise ratio parameters; Inputting the first ultrasonic image data into a preset section structure identification model to obtain a target structure identification result; Determining the confidence score of the tangent plane structure corresponding to the target structure identification result to obtain a structure confidence score; Calculating a contact pressure stability parameter of the ultrasonic probe according to the first contact pressure data; When the signal-to-noise ratio parameter is larger than or equal to a preset signal-to-noise ratio threshold, determining an image quality score corresponding to the first ultrasonic image data according to the gray contrast parameter; And calculating based on the image quality score, the structure confidence score and the contact pressure stability parameter to obtain the first ultrasonic quality evaluation parameter.
3. The method of claim 1 or 2, wherein said adjusting the ultrasound probe to a second target pose by the robotic arm according to the first ultrasound quality assessment parameter comprises: Determining pose adjustment parameters of the mechanical arm according to the first target pose; constructing a comprehensive rewarding function of a reinforcement learning algorithm based on the first ultrasonic quality evaluation parameter; An enhanced state space constructed for a state variable based on the first target pose and the first ultrasound image data; constructing an enhanced action space based on the pose adjustment parameters; constructing a reinforcement learning strategy model based on the reinforcement action space and the comprehensive rewarding function, and inputting the reinforcement state space into the reinforcement learning strategy model for solving to obtain pose control action parameters; The pose control action parameters are sent to the mechanical arm, and after the mechanical arm is controlled to adjust the pose according to the pose control action parameters, third ultrasonic image data and third contact pressure data are acquired through the camera module and the force feedback sensor; Determining a second ultrasonic quality evaluation value according to the third ultrasonic image data and the third contact pressure data, and inputting the second ultrasonic quality evaluation into the comprehensive rewarding function to obtain a rewarding value corresponding to the pose control action parameter; and carrying out iterative optimization solving on the reinforcement learning strategy model according to the reward value, and obtaining the second target pose when the output value of the comprehensive reward function meets a preset convergence condition.
4. The method of claim 1 or 2, wherein controlling the robotic arm to adjust the ultrasound probe to the first target pose based on the user's scanning intent information and the depth image data comprises: determining three-dimensional space coordinates of the target skin region according to the depth image data; constructing a world coordinate system based on a preset hand-eye-instrument calibration matrix, the three-dimensional space coordinates and the first pose parameters of the ultrasonic probe; Carrying out structural analysis on the scanning intention information to obtain a structural intention descriptor, wherein the structural intention descriptor comprises a second pose parameter of the ultrasonic probe, a standard tangent plane constraint parameter and a contact force range parameter of the ultrasonic probe; Determining a motion trail parameter of the mechanical arm according to the standard tangent plane constraint parameter, the first pose parameter and the second pose parameter; And under the constraint of the contact force range parameter and the standard tangent plane constraint parameter, adjusting the ultrasonic probe to a first target pose through the mechanical arm according to the motion track parameter.
5. The method of claim 4, wherein said adjusting the ultrasound probe to a first target pose by the robotic arm in accordance with the motion profile parameters under the constraints of the contact force range parameters and the standard tangent plane constraint parameters comprises: Determining pose control instructions of the mechanical arm in a preset time period according to the motion trail parameters to obtain a control instruction sequence; Performing parameter deconstructing processing on the control instruction sequence based on a preset mechanical arm operation intelligent body to obtain a plurality of control parameters of the mechanical arm, wherein the control parameters at least comprise a speed parameter, an acceleration parameter and a rotary coding parameter; Determining a plurality of pressure parameters from the contact force range parameters; Generating a first motion control instruction for driving the mechanical arm according to the speed parameter, the acceleration parameter and the rotary coding parameter; generating a force control instruction of the mechanical arm according to the pressure parameters; and controlling the mechanical arm to execute a first operation according to the first motion control instruction and controlling the mechanical arm to execute a second operation according to the force control instruction so as to adjust the ultrasonic probe to the first target pose.
6. The method of claim 5, wherein generating the first motion control command to drive the robotic arm based on the speed parameter, the acceleration parameter, and the rotational encoding parameter comprises: Acquiring joint angle parameters and joint angular velocity parameters of a plurality of joints of the mechanical arm; determining the pose of the tail end of the mechanical arm in the world coordinate system based on the motion trail parameters to obtain a second target pose; Operating an intelligent body through the mechanical arm to generate joint motion parameters of the joints according to the second target pose; generating joint motion tracks corresponding to the joints of the mechanical arm according to the acceleration parameters and the joint angle parameters; Determining driving control parameters corresponding to the joints according to the rotation coding parameters, the joint angular velocity parameters and the joint movement track to obtain a plurality of driving control parameters; And determining the first motion control instruction for driving the mechanical arm according to the driving control parameters.
7. The method of claim 1 or 2, wherein after the scanning the user based on the depth image data, the second ultrasound image data of the ultrasound probe, and the second contact pressure data of the force feedback sensor, the method further comprises: extracting body surface three-dimensional point cloud data of the target skin area from the depth image data; performing non-rigid deformation matching on a preset human body digital model based on the three-dimensional point cloud data to obtain a first digital twin model of the user; Extracting ultrasonic image boundary data corresponding to a target tissue section from the second ultrasonic image data to obtain ultrasonic boundary data, wherein the target tissue section is any partial area in the target skin area; determining contact pressure data of a region corresponding to the target tissue section from the second contact pressure data to obtain target contact pressure data; Performing voxel dynamic correction on the first digital twin model based on the ultrasonic image boundary data and the target contact pressure data to obtain a target digital twin model; Performing space matching on the second ultrasonic image data and the target digital twin model to determine a section position parameter corresponding to the second ultrasonic image data; determining tissue characteristic parameters and focus characteristic parameters corresponding to the target tissue section according to the section position parameters; And inputting the tissue characteristic parameters and the focus characteristic parameters into a preset large diagnosis model to obtain a diagnosis result of the user.
8. A multi-modality ultrasound scanning apparatus for use with a host device in an ultrasound scanning system, the apparatus comprising: the acquisition unit is used for acquiring depth image data of a target skin area of a user through the camera module; The control unit is used for controlling the mechanical arm to adjust the ultrasonic probe to a first target pose according to the scanning intention information of the user and the depth image data, and the ultrasonic probe and the force feedback sensor are integrated at the tail end of the mechanical arm; the computing unit is used for computing ultrasonic quality evaluation parameters according to the first ultrasonic image data of the ultrasonic probe and the first contact pressure data of the force feedback sensor to obtain first ultrasonic quality evaluation parameters; The control unit is further configured to adjust the ultrasound probe to a second target pose through the mechanical arm according to the first ultrasound quality evaluation parameter, and scan the user according to the depth image data, the second ultrasound image data of the ultrasound probe, and the second contact pressure data of the force feedback sensor, so as to obtain an ultrasound scanning result of the target skin region.
9. An electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method of any of claims 1-7.
10. A multi-modality ultrasound scanning system, wherein the multi-modality ultrasound scanning system performs the method of any of claims 1-7.

Description

Multi-mode ultrasonic scanning method, device and related system Technical Field The application relates to the technical field of medical intelligent diagnosis, in particular to a multi-mode ultrasonic scanning method, a multi-mode ultrasonic scanning device and a related system. Background At present, ultrasonic diagnosis is widely applied to early screening and accurate diagnosis of multi-part lesions such as abdomen, cardiovascular, superficial organs and the like as one of means of clinical imaging examination. With the development of ultrasonic robots and intelligent diagnosis and treatment technologies, automatic and accurate ultrasonic diagnosis becomes an important direction for improving diagnosis and treatment efficiency. However, the prior art has remarkable limitation in coping with human physiological motion interference, in the process of ultrasonic diagnosis, the skin position of a scanning area is periodically fluctuated up and down due to spontaneous breathing of a patient, the physiological motion directly causes fluctuation of the contact state of a probe and a body surface, and on the other hand, the probe is in virtual contact due to respiratory displacement, so that ultrasonic imaging is fuzzy, the section structure is distorted, and the accuracy of a diagnosis result is reduced. Therefore, how to effectively offset the displacement deviation caused by the respiratory motion of the patient, and improve the stability and the operation safety during ultrasonic scanning is needed to be solved. Disclosure of Invention The application aims to provide a multi-mode ultrasonic scanning method, a multi-mode ultrasonic scanning device and a related system, which solve the problem that the contact stability and safety are poor when an ultrasonic probe scans a skin area in the existing ultrasonic diagnosis field. In order to achieve the purpose of the application, the application provides the following technical scheme: in a first aspect, the present application provides a multi-modal ultrasound scanning method applied to a host device in an ultrasound scanning system, the method comprising: collecting depth image data of a target skin area of a user through a camera module; Controlling a mechanical arm to adjust an ultrasonic probe to a first target pose according to scanning intention information of a user and the depth image data, wherein the ultrasonic probe and a force feedback sensor are integrated at the tail end of the mechanical arm; calculating an ultrasonic quality evaluation parameter according to the first ultrasonic image data of the ultrasonic probe and the first contact pressure data of the force feedback sensor to obtain a first ultrasonic quality evaluation parameter; Adjusting the ultrasonic probe to a second target pose through the mechanical arm according to the first ultrasonic quality evaluation parameters; and scanning the user according to the depth image data, the second ultrasonic image data of the ultrasonic probe and the second contact pressure data of the force feedback sensor to obtain an ultrasonic scanning result of the target skin region. In a second aspect, the present application provides a multi-modal ultrasound scanning apparatus for use with a host device in an ultrasound scanning system, the apparatus comprising: the acquisition unit is used for acquiring depth image data of a target skin area of a user through the camera module; The control unit is used for controlling the mechanical arm to adjust the ultrasonic probe to a first target pose according to the scanning intention information of the user and the depth image data, and the ultrasonic probe and the force feedback sensor are integrated at the tail end of the mechanical arm; the computing unit is used for computing ultrasonic quality evaluation parameters according to the first ultrasonic image data of the ultrasonic probe and the first contact pressure data of the force feedback sensor to obtain first ultrasonic quality evaluation parameters; The control unit is further configured to adjust the ultrasound probe to a second target pose through the mechanical arm according to the first ultrasound quality evaluation parameter, and scan the user according to the depth image data, the second ultrasound image data of the ultrasound probe, and the second contact pressure data of the force feedback sensor, so as to obtain an ultrasound scanning result of the target skin region. In a third aspect, the present application provides an electronic device comprising a processor, a memory, a communication interface and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps of any of the methods of the first aspect of the embodiments of the present application. In a fourth aspect, embodiments of the present application provide a multi-modality ultrasound scanning system, wherein th