US-12622569-B2 - Surgical system, processor and control method

Abstract

A surgical system includes an endoscope including an imager which captures an endoscopic image, a medical manipulator with an injection needle at a distal end section thereof, a drive device which controls the medical manipulator to control a position of the injection needle, and a processor. The processor is configured to: control to inject a target amount of a injection fluid into a syringe connected to the injection needle; acquire, from the imager, the endoscopic image in which a treatment target is captured; and perform a prescribed determination to determine success/failure of injection using the endoscopic image, and control the drive device to change the position of the injection needle based on a determination result of the prescribed determination.

Inventors

• Masahiro Fujii
• Shiori YASUDA
• Ryohei Ogawa

Assignees

• OLYMPUS CORPORATION

Dates

Publication Date

20260512

Application Date

20240130

Claims (9)

1. 1 . A surgical system comprising: an endoscope including an imager which captures an endoscopic image; a medical manipulator with an injection needle at a distal end section of the medical manipulator; a drive device which controls the medical manipulator to control a position of the injection needle; and a processor, wherein the processor is configured to; control to inject a target amount of a injection fluid into a syringe connected to the injection needle; acquire, from the imager, the endoscopic image in which a treatment target is captured; and perform a prescribed determination using the endoscopic image to determine success/failure of injection, and control the drive device to change the position of the injection needle based on a determination result of the prescribed determination.
2. 2 . The surgical system as defined in claim 1 , comprising a first trained model trained based on learning data including the endoscopic image in which the treatment target, into which the injection fluid was injected, is captured, wherein the processor: determines, based on the endoscopic image, sufficiency of shape change of the treatment target using the first trained model; and controls the drive device to pull out the injection needle by a predetermined amount when determining that the shape change of the treatment target is insufficient.
3. 3 . The surgical system as defined in claim 1 , wherein the processor: determines success/failure of the injection in the prescribed determination, further using a control value of an injection amount of the injection fluid; and controls the drive device based on the determination result of the prescribed determination to change the position of the injection needle.
4. 4 . The surgical system as defined in claim 3 , comprising a first trained model trained based on learning data including the injection amount of the injection fluid and the endoscopic image in which the treatment target, into which the injection fluid corresponding to the injection amount was injected, is captured, wherein the processor: determines, based on the endoscopic image, sufficiency of shape change of the treatment target for the control value using the first trained model; and controls the drive device to pull out the injection needle by a predetermined amount when the shape change of the treatment target is insufficient.
5. 5 . The surgical system as defined in claim 1 , comprising a force sensor that is capable of detecting resistance upon injecting the injection fluid into the treatment target, wherein the processor controls the drive device to pull out the injection needle by a predetermined amount when shape change of the treatment target is insufficient and a detection value of the force sensor is equal to or greater than a predetermined value.
6. 6 . The surgical system as defined in claim 1 , comprising a position sensor that is capable of detecting a displacement amount of a plunger, wherein the processor controls the drive device to pull out the injection needle by a predetermined amount when shape change of the treatment target is insufficient and a detection value of the position sensor is less than a predetermined value.
7. 7 . The surgical system as defined in claim 1 , wherein the processor decides, based on the endoscopic image, a position in the treatment target where the injection needle is to be inserted and the target amount of the injection fluid corresponding to the position where the injection needle is to be inserted.
8. 8 . The surgical system as defined in claim 7 , comprising a second trained model trained based on learning data including the endoscopic image in which the treatment target is captured and position information indicative of the position where the injection needle is to be inserted, wherein the processor performs a first decision process using the second trained model to decide, based on the endoscopic image, the position where the injection needle is to be inserted.
9. 9 . The surgical system as defined in claim 8 , comprising a third trained model trained based on learning data including the endoscopic image in which the treatment target is captured, the position information indicative of the position where the injection needle is to be inserted, and the target amount of the injection fluid corresponding to the position information, wherein the processor performs a second decision process using the third trained model to decide the target amount of the injection fluid at each position where the injection needle is to be inserted based on the endoscopic image and the position information decided in the first decision process.

Description

CROSS REFERENCE TO RELATED APPLICATIONS This application is based upon and claims the benefit of priority to U.S. Provisional Patent Application No. 63/527,837 filed on Jul. 20, 2023, the entire contents of which are incorporated herein by reference. BACKGROUND OF THE INVENTION A surgical system which controls a medical equipment such as an endoscope by a robot arm has been known. Japanese Unexamined Patent Application Publication No. 2021-074242 discloses a surgical system which automatically operates using machine learning. SUMMARY OF THE INVENTION In accordance with one of some aspect, there is provided a surgical system comprising: an endoscope including an imager which captures an endoscopic image;a medical manipulator with an injection needle at a distal end section thereof;a drive device which controls the medical manipulator to control a position of the injection needle; anda processor,wherein the processor is configured to;control to inject a target amount of a injection fluid into a syringe connected to the injection needle;acquire, from the imager, the endoscopic image in which a treatment target is captured; andperform a prescribed determination using the endoscopic image to determine success/failure of injection, and control the drive device to change the position of the injection needle based on a determination result of the prescribed determination. In accordance with one of some aspect, there is provided a processor that controls an endoscope including an imager which captures an endoscopic image, and a drive device which controls a medical manipulator with an injection needle at a distal end section thereof to control a position of the injection needle, wherein the processor is configured to:control to inject a target amount of a injection fluid into a syringe connected to the injection needle;acquire, from the imager, the endoscopic image in which a treatment target is captured; andperform a prescribed determination to determine success/failure of injection using the endoscopic image, and control the drive device to change the position of the injection needle based on a determination result of the prescribed determination. In accordance with one of some aspect, there is provided a control method for controlling an endoscope including an imager which captures an endoscopic image, and a drive device which controls a medical manipulator with an injection needle at a distal end section thereof to control a position of the injection needle, the method comprising:controlling to inject a target amount of a injection fluid into a syringe connected to the injection needle;acquiring, from the imager, the endoscopic image in which a treatment target is captured; andperforming a prescribed determination to determine success/failure of injection using the endoscopic image, and controlling the drive device to change the position of the injection needle based on a determination result of the prescribed determination. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram illustrating an example configuration of a surgical system. FIG. 2 is a diagram illustrating an example configuration of a drive device. FIG. 3 is a diagram illustrating an example in which three treatment tool channels are included. FIG. 4 is another diagram illustrating an example in which three treatment tool channels are included. FIG. 5 is another diagram illustrating an example in which three treatment tool channels are included. FIG. 6 is a diagram illustrating an example configuration of a second treatment tool drive device. FIG. 7 is a diagram illustrating an example configuration of a first treatment tool drive device. FIG. 8 is a diagram illustrating a specific example configuration of a surgical system. FIG. 9 is a diagram illustrating an example configuration of a console. FIG. 10 is a diagram illustrating a drive device which is controlled by a foot pedal and a handle. FIG. 11 is a diagram illustrating an example configuration of a handle. FIG. 12 is a diagram illustrating an example of setting for a control signal transmitted from a console. FIG. 13 is a diagram illustrating an example screen displayed on a display of a console. FIG. 14 is a diagram illustrating an example of a treatment flow of ESD. FIG. 15 is a diagram illustrating another example of the treatment flow of ESD. FIG. 16 is a diagram illustrating another example configuration of the drive device. FIG. 17 is a diagram illustrating another example configuration of the drive device. FIG. 18 is a flowchart illustrating example processing of a injection process. FIG. 19 is a flowchart illustrating example processing of first determination. FIG. 20 is a diagram illustrating an effect of the present embodiment. FIG. 21 is another diagram illustrating an effect of the present embodiment. FIG. 22 is another diagram illustrating an effect of the present embodiment. FIG. 23 is a flowchart illustrating another example processing of the first determination. FIG. 24