JP-7857110-B2 - Camera terminal, instruction terminal, communication system, work instruction method and program

JP7857110B2JP 7857110 B2JP7857110 B2JP 7857110B2JP-7857110-B2

Inventors

森孝弘

Assignees

シャープ株式会社

Dates

Publication Date: 20260512
Application Date: 20220210

Claims (11)

The shooting unit captures multiple frames of video footage in chronological order, A recording unit records motion information indicating the movement of the shooting terminal at the first time, associated with a first timestamp information indicating the first time when each frame image among the plurality of frame images was acquired. An instruction information receiving unit receives instruction information which is associated with instruction content information indicating the instruction content for a target frame video among the plurality of frame videos, and second timestamp information indicating a second time point in time when the target frame video was acquired. A generation unit generates display content to be displayed on the display unit based on the instruction content information and motion information of a movement determination associated with the first timestamp information of the movement determination indicating a time after the second time point, Equipped with, The instruction content information includes instruction coordinate information indicating the coordinates of the instruction position within the target frame video, The generation unit includes in the display content a movement request that, if the shooting terminal moves after the second time point and the indicated position is not within the display frame image displayed on the display unit , the shooting terminal moves backward from the movement indicated by the motion information of the movement determination.
A movement determination unit determines whether or not the shooting terminal has moved from the second time point onward, based on the motion information of the movement determination; The shooting terminal according to claim 1, wherein the generation unit determines whether or not the indicated position exists within the display frame image based on the indicated coordinate information and the motion information for the movement determination, and generates the display content according to whether or not the shooting terminal has moved from the second time point onward and whether or not the indicated position exists within the display frame image.
The shooting terminal according to claim 2, wherein the generation unit includes the instruction content in the display content if the instruction position is located within a new display frame image displayed on the display unit after the movement request has been included in the display content.
The system further includes a command position calculation unit that corrects the coordinates of the command position based on the motion information of the movement determination and calculates the corrected coordinates. The shooting terminal according to claim 2 or 3, wherein the generation unit, if the shooting terminal moves after the second time point and the indicated position is within the display frame image, includes the indicated content in the display content and superimposes the indicated content on the corrected coordinates within the display frame image.
The generating unit, if the shooting terminal does not move after the second time point, includes the instruction content in the display content and superimposes the instruction content on the coordinates in the display frame image, according to any one of claims 2 to 4.
The recording unit further records each of the frame images, The shooting terminal according to claim 2 or 3, wherein, if the shooting terminal moves after the second time point, the generation unit includes the target frame video read from the recording unit and played back in the display content, and superimposes the instruction content on the coordinates in the target frame video.
A communication system comprising: a shooting unit that takes pictures in a time series and acquires multiple frame images; a recording unit that records motion information indicating the movement of the shooting terminal at a first time, associated with a first timestamp information indicating a first time when each of the multiple frame images was acquired; an instruction information receiving unit that receives instruction information associated with instruction content information indicating instruction content for a target frame image among the multiple frame images and a second timestamp information indicating a second time when the target frame image was acquired; and a generation unit that generates display content to be displayed on a display unit based on the instruction content information and motion information for movement determination associated with a first timestamp information for movement determination indicating a time after the second time, and an instruction terminal that can communicate with the shooting terminal, wherein The aforementioned shooting terminal is The system further includes a video transmission unit that associates each frame video with the first timestamp information and transmits it to the instruction terminal, The aforementioned instruction terminal is The system includes an instruction information transmission unit that transmits the instruction information to the camera terminal, The instruction content information includes instruction coordinate information indicating the coordinates of the instruction position within the target frame video, The generation unit is a communication system that includes a stop request in the display content requesting the shooting terminal to stop if the shooting terminal moves after the second time point and the indicated position is not within the display frame image displayed on the display unit .
A communication system comprising: a shooting unit that takes pictures in a time series and acquires multiple frame images; a recording unit that records motion information indicating the movement of the shooting terminal at a first time, associated with a first timestamp information indicating a first time when each of the multiple frame images was acquired; an instruction information receiving unit that receives instruction information associated with instruction content information indicating instruction content for a target frame image among the multiple frame images and a second timestamp information indicating a second time when the target frame image was acquired; and a generation unit that generates display content to be displayed on a display unit based on the instruction content information and motion information for movement determination associated with a first timestamp information for movement determination indicating a time after the second time, and an instruction terminal that can communicate with the shooting terminal, wherein The instruction content information includes instruction coordinate information indicating the coordinates of the instruction position within the target frame video, The aforementioned shooting terminal is A video transmission unit that associates each frame video with the first timestamp information and transmits it to the instruction terminal, If the shooting terminal moves after the second time point and the indicated position is not within the display frame image displayed on the display unit , the instruction response transmission unit transmits instruction response information indicating that instruction is not possible to the instruction terminal. Furthermore, The aforementioned instruction terminal is An instruction information transmission unit that transmits the instruction information to the camera terminal, The instruction result output unit outputs a message indicating that the instruction is not possible when it receives the instruction response information from the aforementioned shooting terminal, A communication system equipped with [the following features].
If the shooting terminal moves after the second time point and the indicated position is not within the displayed frame image, the generation unit includes a still request in the display content requesting the shooting terminal to stop moving. The communication system according to claim 8, wherein the instruction response transmission unit transmits a re-instruction request to the instruction terminal requesting new instruction information when the shooting terminal comes to a standstill after the standstill request has been displayed on the display unit.
The process involves taking photos in chronological order to acquire multiple frames of video, A step of recording motion information indicating the movement of the shooting terminal at the first time, associated with a first timestamp information indicating the first time when each frame image among the plurality of frame images was acquired, A step of receiving instruction information which is associated with instruction content information indicating the content of an instruction for a target frame video among the plurality of frame videos, and a second timestamp information indicating a second time point in time when the target frame video was acquired. A step of generating display content to be displayed on the display unit based on the instruction content information and motion information of the movement determination associated with the first timestamp information of the movement determination indicating a time after the second time point, Includes, The instruction content information includes instruction coordinate information indicating the coordinates of the instruction position within the target frame video, A work instruction method that, in the process of generating the display content, if the shooting terminal moves after the second time point and the indicated position is not within the display frame image displayed on the display unit , includes in the display content a movement request that requests the shooting terminal to move backward from the movement indicated by the motion information of the movement determination.
On the computer, A function that captures multiple frames of video by shooting in chronological order, A function to record motion information indicating the movement of the shooting terminal at the first time, associated with a first timestamp information indicating the first time when each frame image among the plurality of frame images was acquired, A function for receiving instruction information that associates instruction content information indicating the content of an instruction for a target frame video among the multiple frame videos, and a second timestamp information indicating a second time point in time when the target frame video was acquired. A function to generate display content to be displayed on the display unit based on the instruction content information and motion information of the movement determination associated with the first timestamp information of the movement determination indicating a time after the second time point, Make it run, The instruction content information includes instruction coordinate information indicating the coordinates of the instruction position within the target frame video, A program that generates the display content, wherein if the shooting terminal moves after the second time point and the indicated position is not within the display frame image displayed on the display unit , the program includes in the display content a movement request that requests the shooting terminal to move backward from the movement indicated by the motion information of the movement determination.

Description

This disclosure relates to a camera terminal, an instruction terminal, a communication system, a work instruction method, and a program. Patent Document 1 discloses a technology for estimating the position and orientation of a worker from video footage captured by a work-side terminal, and displaying the estimated position and orientation of the worker on a control-side terminal. In the technology disclosed in Patent Document 1, the control-side terminal instructs the work-side terminal on a virtual screen to determine the next work direction, and the work-side terminal displays an image indicating the next work direction instructed by the control-side terminal. International release WO2017/158718 This is a diagram showing an example of a communication system configuration.This is a block diagram showing an example of the configuration of a camera terminal.This is a block diagram showing an example of the configuration of the control unit of the shooting terminal according to the first embodiment.This figure shows an example of information where motion information and first timestamp information are associated and stored in the motion information storage area.This is a block diagram showing an example of the configuration of a control terminal.This is a block diagram showing an example of a control unit for an instruction terminal.This flowchart shows an example of the operation of the shooting terminal according to the first embodiment.Figure 7 shows a flowchart illustrating an example of the operation of the camera terminal.This flowchart shows an example of the operation of the instruction terminal according to the first embodiment.This figure shows an example of the displayed content when the camera terminal according to the first embodiment is moved.This figure shows an example of the displayed content when the camera terminal does not move according to the first embodiment.This is a block diagram showing an example of the configuration of the control unit of the shooting terminal according to the second embodiment.This flowchart shows an example of the operation of the shooting terminal according to the second embodiment.Figure 13 shows a flowchart illustrating an example of the operation of the camera terminal.This figure shows an example of the display content of a camera terminal according to the second embodiment.This flowchart shows an example of the operation of the shooting terminal according to the third embodiment.This flowchart shows an example of the operation of the instruction terminal according to the third embodiment.This figure shows an example of the display content of a shooting terminal according to the third embodiment.This figure shows an example of the display content of the camera terminal, following Figure 18. (First Embodiment) Figure 1 shows an example of the configuration of the communication system 100. The communication system 100 includes a camera terminal 101 and an instruction terminal 102. The camera terminal 101 and the instruction terminal 102 are connected via a network 103. The network 103 consists of a mobile communication network such as LTE (Long Term Evolution) or 5G, or a LAN (Local Area Network) such as Wi-Fi (registered trademark). The recording terminal 101 is an information processing device used by the worker. The worker uses the recording terminal 101 to record video of the work site in chronological order. The recording terminal 101 transmits the recorded frames to the instruction terminal 102 located remotely. The recording terminal 101 is, for example, a smartphone or tablet device. The instruction terminal 102 is an information processing device used by the instructor. The instructor is the person who gives instructions to the worker. The instruction terminal 102 is, for example, a PC (Personal Computer). The instruction terminal 102 plays back the frame video transmitted from the shooting terminal 101 in chronological order. For example, the instruction terminal 102 streams a video composed of frame video transmitted from the shooting terminal 101. The instruction terminal 102 then acquires the instruction content related to the target frame video being played back, based on the instructor's operation. The instruction content includes the instruction position within the target frame video, various instructions, and the designation of the object to be photographed. Various instructions include instructions indicating approaching the designated object, and instructions indicating the worker's position and direction of movement. Figure 2 is a block diagram showing an example of the configuration of the camera terminal 101. It comprises a communication unit 201, an audio output unit 202, an audio input unit 203, a touch panel unit 204, an imaging unit 205, a motion sensor 206, a storage unit 207, a memory 208, and a control unit 209. The communication unit 201 is an interface for connecting to the network 103 for voice calls and packet communication. Th