EP-4318396-B1 - INFERENCE DEVICE, LEARNING DEVICE, INFERENCE METHOD, LEARNING METHOD, INFERENCE PROGRAM, AND LEARNING PROGRAM

EP4318396B1EP 4318396 B1EP4318396 B1EP 4318396B1EP-4318396-B1

Inventors

ARAI, Akito
KUSANO, KATSUHIRO
SHIMIZU, SHOGO
OKUMURA, SEIJI

Dates

Publication Date: 20260513
Application Date: 20210413

Claims (15)

An inference apparatus (200) comprising: a shape similarity computation unit (2021) to compute, as a shape similarity, a similarity in shape between a learning movement path curve which is a movement path curve obtained through learning and an observation movement path curve which is a movement path curve obtained through observation; a position similarity computation unit (2022) to compute, as a position similarity, a similarity in position between the learning movement path curve and the observation movement path curve when the learning movement path curve and the observation movement path curve are placed in a same coordinate space; and a conformity computation unit (2031) to compute a conformity between the learning movement path curve and the observation movement path curve, using the shape similarity and the position similarity, wherein the learning movement path curve is divided into a plurality of curve components constituting the learning movement path curve, the curve component being a plot point corresponding to each time within the learning movement path curve, the observation movement path curve is divided into a plurality of curve components corresponding to the plurality of curve components of the learning movement path curve, the curve component being a plot point corresponding to each time within the observation movement path curve, the shape similarity computation unit computes the shape similarity by comparing corresponding curve components between the learning movement path curve and the observation movement path curve, the position similarity computation unit computes the position similarity by comparing corresponding curve components between the learning movement path curve and the observation movement path curve, and the conformity computation unit sets a significance level for each curve component of the learning movement path curve based on an attribute of each curve component, and computes the conformity, using the shape similarity, the position similarity, and the significance level of each curve component, wherein the learning movement path curve is a movement path curve in which a movement path of a moving object is represented, and the conformity computation unit sets the significance level for each curve component based on a movement speed of the object for each curve component, the moving speed being the attribute of each curve component of the learning movement path curve, wherein the learning movement path curve is a movement path curve generated by aggregating a plurality of basic movement path curves, each basic movement path curve being a movement path curve generated from image data (400) capturing an action of the moving object, and the conformity computation unit sets the significance level for each curve component based on a distribution of the plurality of basic movement path curves for each curve component, the distribution representing a probability distribution (p) for each point of the learning movement path curve when the learning movement path curve and the plurality of basic movement path curves are superimposed in the same coordinate space, the distribution being the attribute of each curve component of the learning movement path curve.
The inference apparatus according to claim 1, wherein the conformity computation unit sets a higher significance level for a curve component with a lower movement speed of the object.
The inference apparatus according to claim 1 or 2, wherein the conformity computation unit sets a higher significance level for a curve component at which a likelihood that is calculated from the distribution of the plurality of basic movement path curves is higher.
The inference apparatus according to claim 1, wherein the conformity computation unit corrects at least either of the shape similarity and the position similarity for each curve component, using the significance level of the same curve component, and after making correction using the significance level for each curve component, computes the conformity between the learning movement path curve and the observation movement path curve, using the shape similarity and the position similarity for each curve component.
The inference apparatus according to claim 1, wherein the shape similarity computation unit computes the shape similarity between each of a plurality of learning movement path curves and the observation movement path curve, the position similarity computation unit computes the position similarity between each of the plurality of learning movement path curves and the observation movement path curve, and the conformity computation unit computes the conformity between each of the plurality of learning movement path curves and the observation movement path curve, using the shape similarity and the position similarity computed between each of the plurality of learning movement path curves and the observation movement path curve, and selects a learning movement path curve from among the plurality of learning movement path curves based on a result of computation of the conformity.
The inference apparatus according to any one of the preceding claims, wherein the conformity computation unit sets the significance level for a curve component of the learning movement path curve based on a multimodal distribution of the plurality of basic movement path curves at the curve component.
The inference apparatus according to any one of the preceding claims, wherein the conformity computation unit sets the significance level for a curve component of the learning movement path curve based on a distribution of the plurality of basic movement path curves for the curve component and on a heading direction of at least any basic movement path curve of the plurality of basic movement path curves for the curve component.
The inference apparatus according to claim 5, wherein the shape similarity computation unit computes the shape similarity between the plurality of learning movement path curves, and the position similarity computation unit computes the position similarity between each of the plurality of learning movement path curves and the observation movement path curve if the shape similarity between the plurality of learning movement path curves is greater than or equal to a threshold value.
A learning apparatus (100) comprising: a learning movement path curve generation unit (101) to generate a learning movement path curve through learning, the learning movement path curve being a movement path curve to be divided into a plurality of curve components; and a coefficient value setting unit (102) to set a coefficient value for each curve component of the learning movement path curve based on an attribute of each curve component, each curve component being obtained by dividing the movement path curve into a plurality of curve components, the curve component being a plot point corresponding to each time within the learning movement path curve, wherein the learning movement path curve generation unit generates the learning movement path curve by aggregating a plurality of basic movement path curves, each basic movement path curve being a movement path curve generated from image data (400) capturing an action of the moving object, and the coefficient value setting unit sets the coefficient value for each curve component based on a distribution of the plurality of basic movement path curves for each curve component, the distribution representing a probability distribution (p) for each point of the learning movement path curve when the learning movement path curve and the plurality of basic movement path curves are superimposed in a same coordinate space, the distribution being the attribute of each curve component of the learning movement path curve, wherein the learning movement path curve generation unit generates a movement path curve in which a movement path of a moving object is represented as the learning movement path curve, and the coefficient value setting unit sets the coefficient value for each curve component based on a movement speed of the object for each curve component, the moving speed being the attribute of each curve component of the learning movement path curve.
The learning apparatus according to claim 9, wherein the coefficient value setting unit sets a higher coefficient value for a curve component at which a likelihood that is calculated from the distribution of the plurality of basic movement path curves is higher.
The learning apparatus according to claim 10, wherein the coefficient value setting unit sets the coefficient value for a curve component of the learning movement path curve based on a multimodal distribution of the plurality of basic movement path curves at the curve component.
An inference method comprising: computing, by a computer, as a shape similarity, a similarity in shape between a learning movement path curve which is a movement path curve obtained through learning and an observation movement path curve which is a movement path curve obtained through observation; computing, by the computer, as a position similarity, a similarity in position between the learning movement path curve and the observation movement path curve when the learning movement path curve and the observation movement path curve are placed in a same coordinate space; and computing, by the computer, a conformity between the learning movement path curve and the observation movement path curve, using the shape similarity and the position similarity, wherein the learning movement path curve is divided into a plurality of curve components constituting the learning movement path curve, the curve component being a plot point corresponding to each time within the learning movement path curve, the observation movement path curve is divided into a plurality of curve components corresponding to the plurality of curve components of the learning movement path curve, the curve component being a plot point corresponding to each time within the observation movement path curve, in the computing of the shape similarity, the computer computes the shape similarity by comparing corresponding curve components between the learning movement path curve and the observation movement path curve, in the computing of the position similarity, the computer computes the position similarity by comparing corresponding curve components between the learning movement path curve and the observation movement path curve, and in the computing of the conformity, the computer sets a significance level for each curve component of the learning movement path curve based on an attribute of each curve component, and computes the conformity, using the shape similarity, the position similarity, and the significance level of each curve component, wherein the learning movement path curve is a movement path curve in which a movement path of a moving object is represented, and the conformity computation unit sets the significance level for each curve component based on a movement speed of the object for each curve component, the moving speed being the attribute of each curve component of the learning movement path curve, wherein the learning movement path curve is a movement path curve generated by aggregating a plurality of basic movement path curves, each basic movement path curve being a movement path curve generated from image data (400) capturing an action of the moving object, and the computer sets the significance level for each curve component based on a distribution of the plurality of basic movement path curves for each curve component, the distribution representing a probability distribution (p) for each point of the learning movement path curve when the learning movement path curve and the plurality of basic movement path curves are superimposed in the same coordinate space, the distribution being the attribute of each curve component of the learning movement path curve.
A learning method comprising: generating, by a computer, a learning movement path curve through learning, the learning movement path curve being a movement path curve to be divided into a plurality of curve components, the curve component being a plot point corresponding to each time within the learning movement path curve; and setting, by the computer, a coefficient value for each curve component of the learning movement path curve based on an attribute of each curve component, wherein in the generating of the learning movement path curve, the computer generates the learning movement path curve by aggregating a plurality of basic movement path curves, each basic movement path curve being a movement path curve generated from image data (400) capturing an action of the moving object, and in the setting of the coefficient value, the computer sets the coefficient value for each curve component based on a distribution of the plurality of basic movement path curves for each curve component that would arise when the learning movement path curve and the plurality of basic movement path curves are superimposed in a same coordinate space, the distribution being the attribute of each curve component of the learning movement path curve, wherein the learning movement path curve is a movement path curve in which a movement path of a moving object is represented, the computer sets the coefficient value for each curve component based on a movement speed of the object for each curve component, the moving speed being the attribute of each curve component of the learning movement path curve.
An inference program that causes a computer to execute the method according to claim 12.
A learning program that causes a computer to execute the method according to claim 13.

Description

Technical Field The present disclosure relates to analysis of a movement path curve. The movement path curve is a curve representing a movement path of an object. Background Art There is an image analysis technique that captures an image of movement of an object with an image capturing device and recognizes the type of the movement of the object through analysis of the captured image. Such an image analysis technique analyzes captured images and extracts a movement path curve representing the movement path of the object. Then, the extracted movement path curve is compared to each of multiple learned movement path curves which are prepared in advance. Each of the learned movement path curves has a label set for it. The label indicates the type of movement. As a result of comparison, a learned movement path curve that is most similar to the extracted movement path curve is selected. Then, according to the label of the selected learned movement path curve, the type of movement of the object is recognized. Patent Literature 1 discloses a technique as an application of such an image analysis technique, particularly a technique to recognize an action of a factory operator from a movement path curve at each body part of the operator. Porikli Fatih: "Trajectory Distance Metric Using Hidden Markov Model Based Representation", 30 November 2004 (2004-11 -30), pages 1-10, XP093145871 discloses a set of novel distance metrics that use model based representations for trajectories. The similarity of trajectories using the conformity of the corresponding HMM models is determined. These metrics enable the comparison of trajectories without any limitations of the conventional measures. Kong W. W. ET AL: "Sign Language Phoneme Transcription with Rule-based Hand Trajectory Segmentation", Journal of Signal Processing Systems, vol. 59, no. 2, 16 October 2008 (2008-10-16), pages 211 -222 discloses a method for extracting phonemes from American Sign Language (ASL) sentences. It introduces a rule-based segmentation algorithm to divide signed sentences into hand motion trajectories. Feature descriptors based on principal component analysis are then extracted to efficiently represent these segments. The segments are clustered using k-means to derive phonemes. Citation List Patent Literature Patent Literature 1: Japanese application 2020-528365 Summary of Invention Technical Problem Patent Literature 1 discloses a movement type recognition technique that is robust against difference in position by comparing shapes of movement curves, in order to solve the problem that incorrect recognition arises when coordinate values of movement curves are different even if the types of movement are the same and the shapes of curves are also the same. However, due to comparison of the shapes of movement path curves, it has a problem of being unable to correctly recognize the movement type when there are two or more learned movement path curves that are similar to each other in shape. A primary object of the present disclosure is to solve such a problem. More specifically, the present disclosure is primarily aimed at obtaining a configuration that is capable of correctly recognizing movement type even when there are two or more learned movement path curves that are similar to each other in shape, while having performance that is robust against difference in position. Solution to Problem The invention is defined in the appended set of claims. An inference apparatus according to the present disclosure, includes: a shape similarity computation unit to compute, as a shape similarity, a similarity in shape between a learning movement path curve which is a movement path curve obtained through learning and an observation movement path curve which is a movement path curve obtained through observation;a position similarity computation unit to compute, as a position similarity, a similarity in position between the learning movement path curve and the observation movement path curve when the learning movement path curve and the observation movement path curve are placed in a same coordinate space; anda conformity computation unit to compute a conformity between the learning movement path curve and the observation movement path curve, using the shape similarity and the position similarity. Advantageous Effects of Invention According to the present disclosure, movement type can be correctly recognized even when there are two or more learned movement path curves that are similar to each other in shape. Brief Description of Drawings Fig. 1 shows a configuration example of an analysis system according to Embodiment 1.Fig. 2 shows a functional configuration example of a learning apparatus according to Embodiment 1.Fig. 3 shows a hardware configuration example of the learning apparatus according to Embodiment 1.Fig. 4 shows a functional configuration example of an inference apparatus according to Embodiment 1.Fig. 5 shows a hardware configuration example of the infere