CN-115150441-B - Work support system and work support composite system

CN115150441BCN 115150441 BCN115150441 BCN 115150441BCN-115150441-B

Abstract

The invention provides a remote operation system and an operation auxiliary composite system. It is possible to confirm whether or not the working machine operated according to the operation of the operator coincides with the selected working machine. The remote operation system has a plurality of work machines and a remote operation device for remotely operating the cooperative work machines. The remote operation system includes a first auxiliary processing element that acquires a captured image captured by an actual camera of the first work machine for which communication is established via a first communication path and transmits the captured image to the remote operation device, and a second auxiliary processing element that recognizes that a specific operation, such as a specific operation that is a part of a plurality of operations for operating the second work machine for which communication is established via a second communication path and that maps a change pattern corresponding to the operation in the captured image, is performed and transmits a command signal corresponding to the specific operation to the second work machine.

Inventors

Kaya toichi
YUUKI MORITA
MATSUDA YUYA
OHTANI MAKOTO
SAEKI KOJI
YAMAZAKI YOICHIRO

Assignees

神钢建机株式会社

Dates

Publication Date: 20260508
Application Date: 20220323
Priority Date: 20210329

Claims (10)

1. A remote operation system having a plurality of working machines and a remote operation device for remotely operating a cooperative working machine which is one working machine constituting a remote operation object among the plurality of working machines, the remote operation system having a function of causing each of the plurality of working machines to communicate with the remote operation device, The remote operation system is characterized by comprising: A first auxiliary processing element that outputs a determination result of determining in the remote operation device whether or not a specified operation of the cooperative work machine as selection information of a remote operation object among the plurality of work machines is performed, and a photographed image photographed by an actual camera mounted on a first work machine that establishes communication with the remote operation device through a first communication path established in association with the selection information, to an output interface of the remote operation device, the photographed image being photographed by the actual camera mounted on the first work machine when the selection information is selected in the remote operation device And a second auxiliary processing element that recognizes that a specific operation, which is a specific operation of a plurality of operations of making a second work machine for which communication with the remote operation device is established through a second communication path associated with the selection information, and that maps a change form of an operation corresponding to the specific operation in the captured image, is executed, and that transmits a command signal corresponding to the specific operation to the second work machine.
2. The remote operation system according to claim 1, wherein, The second auxiliary processing element performs a process of, in a case where a release condition is not satisfied, reserving transmission of a command signal corresponding to another specified operation for operating the cooperative work machine to the second work machine, and in a case where the release condition is satisfied, allowing transmission of a command signal corresponding to another specified operation for operating the cooperative work machine to the second work machine, The release condition includes a case where a release operation is performed in the remote operation device.
3. The remote operation system according to claim 1, wherein, The second auxiliary processing element acquires a result of determination as to whether or not the one specified operation is performed, and when the one specified operation is performed, allows another specified operation for operating the cooperative work machine to be performed, and transmits a command signal corresponding to the other specified operation to the second work machine.
4. The remote operation system according to claim 1, wherein, The second auxiliary processing element performs processing of reserving transmission of a command signal corresponding to another specified operation for operating the cooperative work machine to the second work machine in a case where a release condition is not satisfied, allowing transmission of a command signal corresponding to another specified operation for operating the cooperative work machine to the second work machine in a case where the release condition is satisfied, In the case where the one specified operation is performed, the release condition includes a case where the second auxiliary processing element obtains a determination result of whether or not there is a change form of an action corresponding to the one specified operation in the captured image, the determination result being a change form of an action corresponding to the one specified operation.
5. The remote operation system according to claim 2, wherein, The second auxiliary processing element performs processing of, in a case where a release condition is not satisfied, reserving transmission of a command signal corresponding to another specified operation for operating the cooperative work machine to the second work machine, and in a case where the release condition is satisfied, allowing transmission of a command signal corresponding to another specified operation for operating the cooperative work machine to the second work machine, In the case where the one specified operation is performed, the release condition includes a case where the second auxiliary processing element obtains a determination result of whether or not there is a change form of an action corresponding to the one specified operation in the captured image, the determination result being a change form of an action corresponding to the one specified operation.
6. A remote operation system according to claim 3, wherein, The second auxiliary processing element performs processing of reserving transmission of a command signal corresponding to another specified operation for operating the cooperative work machine to the second work machine in a case where a release condition is not satisfied, allowing transmission of a command signal corresponding to another specified operation for operating the cooperative work machine to the second work machine in a case where the release condition is satisfied, In the case where the one specified operation is performed, the release condition includes a case where the second auxiliary processing element obtains a determination result of whether or not there is a change form of an action corresponding to the one specified operation in the captured image, the determination result being a change form of an action corresponding to the one specified operation.
7. The remote operation system according to claim 1, wherein, The first auxiliary processing element acquires position data of the plurality of work machines constituting a remote operation object, The second auxiliary processing element generates a display image of each of the plurality of work machines based on the position data of the plurality of work machines, and transmits the display image to the remote operation device.
8. The remote operation system according to claim 2, wherein, The first auxiliary processing element acquires position data of the plurality of work machines constituting a remote operation object, The second auxiliary processing element generates a display image of each of the plurality of work machines based on the position data of the plurality of work machines, and transmits the display image to the remote operation device.
9. A remote operation system according to claim 3, wherein, The first auxiliary processing element acquires position data of the plurality of work machines constituting a remote operation object, The second auxiliary processing element generates a display image of each of the plurality of work machines based on the position data of the plurality of work machines, and transmits the display image to the remote operation device.
10. The remote operation system according to claim 4, wherein, The first auxiliary processing element acquires position data of the plurality of work machines constituting a remote operation object, The second auxiliary processing element generates a display image of each of the plurality of work machines based on the position data of the plurality of work machines, and transmits the display image to the remote operation device.

Description

Work support system and work support composite system Technical Field The present invention relates to a remote operation system including a plurality of work machines and a remote operation device for remotely operating one of the plurality of work machines, which is a remote operation object, and having a function of enabling the remote operation device and the plurality of work machines to communicate with each other. Background Conventionally, an operator remotely operates a work machine without riding on the work machine. As such a working machine, for example, patent document 1 discloses a remote operation system in which a master and a slave are provided so as to be communicable with each other, and an operator operates an operation lever provided to the master, whereby the slave disposed in a driver's seat of the working machine operates the operation lever of the working machine. Prior art literature Patent literature Patent document 1 Japanese patent application laid-open No. 2019-1761401 Disclosure of Invention Problems to be solved by the invention However, when two or more working machines are remotely operated by one master, it is necessary to connect the master to the slaves provided in the respective working machines so as to be able to communicate with each other. The operator remotely operates the operation lever or the like while observing an image of the periphery of the work machine as an operation target through a display of the host computer. However, there are cases where the image displayed on the monitor of the host computer is an image of the surroundings of the work machine different from the operation target. When the operator actually gets on the work machine, the appearance of the work machine and the situation of the site where the work machine is located can be checked before getting on the work machine, but when the work machine is remotely operated, such a check cannot be performed. Therefore, it is difficult for the operator to notice that the remote operation device is connected to an unexpected work machine (different slave). Even if it is desired to confirm the images by a camera mounted on the work machine, these images are substantially similar and therefore difficult to distinguish. In such a case, if the remote operation is performed, the operator may operate the work machine that is not recognized as the operation target. The present invention has been made in view of the above-described problems, and an object thereof is to provide a remote operation system capable of confirming whether or not a work machine operated according to an operation by an operator matches a work machine selected by the operator. Means for solving the problems In order to achieve the above object, a remote operation system according to the present invention includes a plurality of working machines and a remote operation device for remotely operating a cooperative working machine, which is one working machine constituting a remote operation target, among the plurality of working machines, the remote operation system having a function of causing each of the plurality of working machines to communicate with the remote operation device, The remote operation system is characterized by comprising: A first auxiliary processing means for outputting, to an output interface of the remote operation device, a determination result of determining in the remote operation device whether or not a specified operation of the cooperative work machine as selection information of remote operation objects among the plurality of work machines is performed, and a photographed image obtained by photographing the photographed image by an actual-machine photographing device mounted on the first work machine when the selection information is selected in the remote operation device, the first work machine establishes communication with the remote operation device through a first communication path associated with the selection information, and a second auxiliary processing element that recognizes that a specified operation is performed, which is a specified operation that causes a part of a plurality of operations of a second work machine operation for which communication with the remote operation device is established through a second communication path associated with the selection information, and that maps a change form of an action corresponding to the operation in the captured image, and transmits a command signal corresponding to the specified operation to the second work machine. Brief description of the drawings Fig. 1 is a structural explanatory diagram of a work support composite system according to an embodiment of the present invention. Fig. 2 is an explanatory diagram showing an example of a data structure stored in a database of the job assistance server. Fig. 3 is an explanatory diagram of the structure of the remote operation device. Fig. 4 is an explanatory diagram showing an example of screen displ