KR-102963799-B1 - The Method And System For Personalized Object Tracking Using Personalized Dictionary Database

KR102963799B1KR 102963799 B1KR102963799 B1KR 102963799B1KR-102963799-B1

Abstract

The present invention relates to a personalized object tracking method performed in a computing system, wherein the method recognizes user objects and object objects using a deep learning-based object recognition model and assigns IDs to images or videos captured from cameras installed in each space, generates ownership relationships based on the time when user objects and object objects exist together in the same space and stores them in a log DB, extracts location text from images containing object objects using a Vision-Language Model (VLM) and stores it in a log DB, and, upon a request for location tracking of an object object by a user object, queries the location text of the object object matched with the user object from the log DB to generate a response.

Inventors

이다연
배상우
정찬성
김희원
이건희
이혜영
이예진
윤동식
김종은
오중균
최가윤
곽민지
황준영
이선빈

Assignees

에이치디씨랩스 주식회사

Dates

Publication Date: 20260511
Application Date: 20251126

Claims (7)

A personalized object tracking method performed in a computing system comprising one or more processors and one or more memories, wherein An object recognition step of recognizing user objects and object objects using a deep learning-based object recognition model for images or videos captured from cameras installed in each space, and assigning a user object ID and an object object ID to each user object and object object; A log DB storage step for each recognized object, which stores relationship information regarding ownership relationships in a log DB by matching a user object ID determined to be the owner to a corresponding object object ID based on a time interval in which the object object and the user object exist identically in the same space; A location text storage step for each recognized object, wherein an image in which the object exists is input into an internal or external VLM (Vision-Language Model) to extract location text regarding the location information of the object, and the location text for the object is matched with the object ID and stored in a log DB; and Regarding a request for location tracking of an object object input from a user object, a first response statement generation step is included, which derives a user object ID related to the request, derives a related object object ID matched to the user object ID, and generates a first response statement to be provided to the user object based on location text for the object object ID. The above object recognition step is, The recognition cycle for recognizing user objects and object objects varies depending on the presence or absence of user objects within a specific space, and Assign User Object IDs and Object Object IDs only to one or more User Objects and one or more Object Objects included in the pre-configured Initial Target Object List as Object Recognition Targets, and The above recognition period is set longer when there is no user object in a specific space than when there is a user object in a specific space, and An object tracking method in which the creation cycle for generating the ownership relationship between user objects and object objects is set identically regardless of whether there is a user object within a specific space.
delete
A personalized object tracking method performed in a computing system comprising one or more processors and one or more memories, wherein An object recognition step of recognizing user objects and object objects using a deep learning-based object recognition model for images or videos captured from cameras installed in each space, and assigning a user object ID and an object object ID to each user object and object object; A log DB storage step for each recognized object, which stores relationship information regarding ownership relationships in a log DB by matching a user object ID determined to be the owner to a corresponding object object ID based on a time interval in which the object object and the user object exist identically in the same space; A location text storage step for each recognized object, wherein an image in which the object exists is input into an internal or external VLM (Vision-Language Model) to extract location text regarding the location information of the object, and the location text for the object is matched with the object ID and stored in a log DB; and Regarding a request for location tracking of an object object input from a user object, a first response statement generation step is included, which derives a user object ID related to the request, derives a related object object ID matched to the user object ID, and generates a first response statement to be provided to the user object based on location text for the object object ID. The above log DB storage step is, It further includes a user object DB storage step of storing ownership relationship frequencies, including the frequency of occurrence of ownership relationships between user objects and object objects, together with object ID, time information, and spatial information, in the user object DB; The above log DB storage step is, When multiple user object IDs are matched as owners to the same object ID, an owner matching step of matching the user object ID having the highest ownership relationship frequency with said object ID as the owner of said object; and An object tracking method comprising: a final matching step in which, when the frequency of ownership relationship between a user object ID and an object ID exceeds a preset threshold frequency, the user object ID is permanently matched as the final owner of the object ID.
A personalized object tracking method performed in a computing system comprising one or more processors and one or more memories, wherein An object recognition step of recognizing user objects and object objects using a deep learning-based object recognition model for images or videos captured from cameras installed in each space, and assigning a user object ID and an object object ID to each user object and object object; A log DB storage step for each recognized object, which stores relationship information regarding ownership relationships in a log DB by matching a user object ID determined to be the owner to a corresponding object object ID based on a time interval in which the object object and the user object exist identically in the same space; A location text storage step for each recognized object, wherein an image in which the object exists is input into an internal or external VLM (Vision-Language Model) to extract location text regarding the location information of the object, and the location text for the object is matched with the object ID and stored in a log DB; and Regarding a request for location tracking of an object object input from a user object, a first response statement generation step is included, which derives a user object ID related to the request, derives a related object object ID matched to the user object ID, and generates a first response statement to be provided to the user object based on location text for the object object ID. The above log DB storage step is, If there is a time when the above user object and the above object object are together in the same space, the above user object ID is matched to the owner of the above object object ID after a preset time has elapsed from the start point of that time, and An object tracking method that matches the user object ID to the owner of the object object ID when the difference between the time when recognition of the user object ends and the time when recognition of the object object begins in the same space thereafter is within a preset difference range standard.
A personalized object tracking method performed in a computing system comprising one or more processors and one or more memories, wherein An object recognition step of recognizing user objects and object objects using a deep learning-based object recognition model for images or videos captured from cameras installed in each space, and assigning a user object ID and an object object ID to each user object and object object; A log DB storage step for each recognized object, which stores relationship information regarding ownership relationships in a log DB by matching a user object ID determined to be the owner to a corresponding object object ID based on a time interval in which the object object and the user object exist identically in the same space; A location text storage step for each recognized object, wherein an image in which the object exists is input into an internal or external VLM (Vision-Language Model) to extract location text regarding the location information of the object, and the location text for the object is matched with the object ID and stored in a log DB; and Regarding a request for location tracking of an object object input from a user object, a first response statement generation step is included, which derives a user object ID related to the request, derives a related object object ID matched to the user object ID, and generates a first response statement to be provided to the user object based on location text for the object object ID. The above location text storage step is, A location text extraction step of inputting a plurality of images captured by a plurality of installed cameras into a deep learning-based VLM to extract corresponding location text; and An object tracking method comprising: a second response generation step in which, when a user object ID related to an object ID cannot be derived from the log DB, a location tracking request for the object, an image of the object, and an image of each space are input into the VLM to generate a second response.
A personalized object tracking method performed in a computing system comprising one or more processors and one or more memories, wherein An object recognition step of recognizing user objects and object objects using a deep learning-based object recognition model for images or videos captured from cameras installed in each space, and assigning a user object ID and an object object ID to each user object and object object; A log DB storage step for each recognized object, which stores relationship information regarding ownership relationships in a log DB by matching a user object ID determined to be the owner to a corresponding object object ID based on a time interval in which the object object and the user object exist identically in the same space; A location text storage step for each recognized object, wherein an image in which the object exists is input into an internal or external VLM (Vision-Language Model) to extract location text regarding the location information of the object, and the location text for the object is matched with the object ID and stored in a log DB; and Regarding a request for location tracking of an object object input from a user object, a first response statement generation step is included, which derives a user object ID related to the request, derives a related object object ID matched to the user object ID, and generates a first response statement to be provided to the user object based on location text for the object object ID. The above object tracking method is, A one-time request determination step for determining a user's request that is entered at a rate below a preset ratio standard among multiple requests entered from a user object as a one-time request; and An object tracking method comprising: a schema removal step for removing the schema for the object in the user object DB when the time at which the above one-time request was last entered has elapsed from the current time.
A computing system comprising one or more processors and one or more memories, and performing a personalized object tracking method, An object recognition unit that recognizes user objects and object objects using a deep learning-based object recognition model for images or videos captured from cameras installed in each space, and assigns a user object ID and an object object ID to each user object and object object; A log DB storage unit that, for each recognized object, stores relationship information regarding ownership relationships in a log DB by matching a user object ID determined to be the owner to a corresponding object object ID based on a time interval in which the object object and the user object exist identically in the same space; A location text storage unit that, for each recognized object, inputs an image in which the object exists into an internal or external VLM (Vision-Language Model) to extract location text regarding the location information of the object, and stores the location text for the object in a log DB by matching it with the object ID; and A first response generation unit for a location tracking request of an object object input from a user object, deriving a user object ID related to the request, deriving a related object object ID matched to the user object ID, and generating a first response to be provided to the user object based on location text for the object object ID; The above object recognition unit is, The recognition cycle for recognizing user objects and object objects varies depending on the presence or absence of user objects within a specific space, and Assign User Object IDs and Object Object IDs only to one or more User Objects and one or more Object Objects included in the pre-configured Initial Target Object List as Object Recognition Targets, and The above recognition period is set longer when there is no user object in a specific space than when there is a user object in a specific space, and A computing system that sets the creation cycle for generating the ownership relationship between user objects and object objects identically, regardless of whether user objects exist within a specific space.

Description

The Method and System for Personalized Object Tracking Using Personalized Dictionary Database The present invention relates to a personalized object tracking method performed in a computing system, wherein the method recognizes user objects and object objects using a deep learning-based object recognition model and assigns IDs to images or videos captured from cameras installed in each space, generates ownership relationships based on the time when user objects and object objects exist together in the same space and stores them in a log DB, extracts location text from images containing object objects using a Vision-Language Model (VLM) and stores it in a log DB, and, upon a request for location tracking of an object object by a user object, queries the location text of the object object matched with the user object from the log DB to generate a response. With the recent advancement of Internet of Things (IoT) technology and the widespread adoption of smart home cameras in residential environments, intelligent services that enhance user convenience within the home are gaining attention. These services perform various functions, such as recognizing user voice commands to control home appliances or providing services corresponding to those commands. In particular, there is a growing market demand for object tracking technology that can quickly locate frequently used items, such as mobile phones, wallets, and car keys, when users lose their location within the home. Meanwhile, regarding conventional object tracking technologies, there are devices that track objects using tags on specific objects, such as those described in Korean Registered Patent No. 10-0179670. Such conventional technology primarily relies on a method where a user attaches a tag to a specific object and locates its position by detecting the tag's signal. However, this method has the drawbacks of requiring the user to attach the tag to the object in advance, and it is impossible to track objects that are not tagged. Additionally, while object detection technology using installed cameras continuously detects and recognizes objects, it has the problem of being unable to distinguish which user the object belongs to. Therefore, there is a need for technology capable of tracking personalized objects by matching ownership relationships through the temporal occupation of a specific space between users and objects. FIG. 1 schematically illustrates the structure of a computing system that performs a personalized object tracking method using a personalized dictionary DB according to one embodiment of the present invention. FIG. 2 schematically illustrates the steps for performing a personalized object tracking method using a personalized dictionary DB according to an embodiment of the present invention. FIG. 3 schematically illustrates an object recognition step according to one embodiment of the present invention. FIG. 4 schematically illustrates a recognition cycle for recognizing a user object and an object object according to an embodiment of the present invention, and a creation cycle for creating an ownership relationship between the user object and the object object. FIG. 5 illustrates a user object DB storage step according to an embodiment of the present invention. FIG. 6 schematically illustrates the process of matching a user object ID to an object ID as the owner based on the frequency of ownership relationships according to one embodiment of the present invention. FIG. 7 schematically illustrates the process of extracting ownership relationship frequencies for time intervals existing within the same space of a user object and an object according to one embodiment of the present invention. FIG. 8 schematically illustrates the process of storing the owner of an object ID along with an object image in a log DB according to one embodiment of the present invention. FIG. 9 schematically illustrates the process of extracting location text of an object and storing it in a log DB according to one embodiment of the present invention. FIG. 10 schematically illustrates a second answer generation step according to an embodiment of the present invention. FIG. 11 schematically illustrates the process of removing a schema for an object from a user object DB according to an embodiment of the present invention. FIG. 12 illustrates, in an exemplary manner, the internal configuration of a computing device according to one embodiment of the present invention. Hereinafter, various embodiments and/or aspects are disclosed with reference to the drawings. For illustrative purposes, numerous specific details are disclosed in the following description to aid in a general understanding of one or more aspects. However, it will also be recognized by those skilled in the art that these aspects may be practiced without such specific details. The following description and the accompanying drawings describe specific exemplary aspects of one or more aspects in detail. Howeve