US-20260126515-A1 - MULTI-CHANNEL SPATIAL POSITIONING SYSTEM

US20260126515A1US 20260126515 A1US20260126515 A1US 20260126515A1US-20260126515-A1

Abstract

A method includes acquiring a set of images from a plurality of cameras in a monitored environment, detecting a first entity based on the set of images, and determining a first set of locations of the first entity based on locations of the first entity in the set of images. The method also includes acquiring sensor measurements from a plurality of sensors and determining a second set of locations of the first entity based on the sensor measurements. The method also includes determining whether the first set of locations should be associated with the second set of locations based on a confidence factor, and, in response to determining that the first and second set of locations should be associated, determining a sequence of locations of the first entity through the monitored environment.

Inventors

James Francis Hallett
Kirk Arnold Moir

Assignees

INPIXON

Dates

Publication Date: 20260507
Application Date: 20250924

Claims (20)

1 . A non-transitory, machine-readable medium storing instructions that, when executed by one or more processors, effectuate operations comprising: acquiring, using a computer system, a set of images from a plurality of cameras, wherein: each of the plurality of cameras have a different respective field of view, and at least part of the fields of view are of a monitored environment; detecting and localizing, using the computer system, in at least some of the set of images, a first entity moving through the monitored environment; determining, using the computer system, a first set of locations within the monitored environment of the first entity based on locations of the first entity in the set of images, wherein each of the first set of locations is associated with an image acquisition time; acquiring, using the computer system, a set of sensor measurements of the monitored environment from a plurality of sensors, the plurality of sensors being different from the plurality of cameras; determining, using the computer system, a second set of locations within the monitored environment of the first entity based on the set of sensor measurements, wherein each of the second set of locations is associated with a sensor measurement time; determining, using the computer system, whether the first set of locations should be associated with the second set of locations based on a set of confidence factors calculated based on the first set of locations and the second set of locations, the set of confidence factors being indicative of the second set of locations being locations of the first entity and not another entity; in response to determining that the first set of locations should be associated with the second set of locations, determining, using the computer system, a sequence of locations of the first entity through the monitored environment; and storing, using the computer system, the sequence of locations in a computer-readable media in communication with the computer system.
2 . The medium of claim 1 , wherein a first subset of the set of images is acquired by a first camera and a second subset of the set of images is acquired by a second camera, and wherein determining the first entity comprises: detecting, using the computer system, the first entity based on the first subset of the set of images using a convolutional neural network; determining, using the computer system, a first set of attributes associated with the first entity based on the first subset of the set of images; detecting, using the computer system, a second entity based on the second subset of the set of images using the convolutional neural network; determining, using the computer system, a second set of attributes associated with the second entity based on the second subset of the set of images; determining, using the computer system, a matching entity confidence factor, wherein the matching entity confidence factor is based on the first set of attributes and the second set of attributes; and associating the first entity with the second entity based on the matching entity confidence factor.
3 . The medium of claim 2 , wherein: the first set of attributes comprises a first gait attribute associated with the first entity, wherein determining the first set of attributes comprises determining the first gait attribute based on the first subset of the set of images, and wherein the first gait attribute comprises at least one of a movement speed, postural sway, stride frequency, gait symmetry, gait dynamic range, or gait characteristic curve; and the second set of attributes comprises a second gait attribute associated with the second entity, wherein determining the second set of attributes comprises determining the second gait attribute based on the second subset of the set of images, wherein the second gait attribute comprises a same attribute type as the first gait attribute.
4 . The medium of claim 2 , wherein determining the matching entity confidence factor comprises using a first image comprising a portion of the first entity and a second image comprising a portion of the second entity as inputs for a capsule neural network.
5 . The medium of claim 2 , wherein determining the matching entity confidence factor comprises using a siamese neural network to determine the matching entity confidence factor, wherein the siamese neural network is trained using a first image comprising a portion of the first entity, and wherein using the siamese neural network comprises using a second image comprising a portion of the second entity as inputs.
6 . The medium of claim 2 , wherein the operations further comprises: determining a boundary of the first entity with respect to a first location, wherein the first entity is detected to be at the first location at a first measurement time, and wherein the boundary is determined based on the first location and a time difference between the first measurement time and a second measurement time; and wherein determining the matching entity confidence factor comprises determining the matching entity confidence factor based on a distance between the boundary and a second location, wherein the second entity is determined to be at the second location at a second measurement time, wherein the second measurement time is after the first measurement time.
7 . The medium of claim 2 , wherein the operations further comprises: determining a boundary of the first entity with respect to a first location, wherein the first entity is detected to be at the first location at a first measurement time, and wherein the boundary is determined based on the first location and a time difference between the first measurement time and a second measurement time; and determining that a field of view of the second camera is within the boundary.
8 . The medium of claim 1 , wherein detecting and localizing the first entity comprises determining an attention weight, wherein the attention weight is based on the detection of an attribute associated with the first entity, and wherein the attention weight is used by a neural network to determine the set of confidence factors.
9 . The medium of claim 1 , wherein detecting and localizing the first entity comprises: segmenting a first image of the set of images into a set of grid cells; for each respective grid cell of the set of grid cells, determining a bounding box associated with the respective grid cell using a convolution operation; and detecting the entity based on the bounding box.
10 . The medium of claim 1 , wherein detecting the first entity comprises: determining a set visual features in a first image of the set of images and a set of visual feature positions associated with the set of visual features; generating a set of bounding boxes for each of the set of features, wherein each of the set of bounding boxes encloses one of the set of feature positions using a convolutional operation; for each respective bounding box in the set of bounding boxes, determine a respective class score based on a portion of the first image bounded by the respective bounding box using a convolution operation, wherein the respective class score is associated with a first object type in a set of object types, and wherein the respective class score is indicative of a likelihood that the respective bounding box is bounding an object of the object type; and detecting the first entity based on the set of bounding boxes and a set of class scores comprising the respective class scores.
11 . The medium of claim 1 , wherein determining the sequence of locations of the first entity through the monitored environment the operations further comprises applying a Kalman filter to determine the sequence of locations based on the second set of locations.
12 . The medium of claim 1 , wherein the plurality of sensors is a first plurality of sensors, and wherein the operations further comprising: acquiring, using the computer system, a second set of sensor measurements of the monitored environment from a second plurality of sensors, wherein the second plurality of sensors are different from the plurality of cameras, and wherein the second plurality of sensors are different from the first plurality of sensors; determining, using the computer system, a third set of locations within the monitored environment of the first entity based on the second set of sensor measurements, wherein each of the third set of locations is associated with the sensor measurement time, wherein determining the sequence of locations of the first entity comprises determining the sequence of locations of the first entity based on the third set of locations.
13 . The medium of claim 1 , wherein the plurality of sensors comprises electronic emission sensors, and wherein determining the sequence of locations comprises determining a location based on a time of arrival of a signal from a mobile computing device or an angle of arrival of the signal from the mobile computing device.
14 . The medium of claim 1 , wherein the plurality of sensors comprises a temperature sensor, and wherein the operations further comprise: determining an entity temperature of the first entity using the temperature sensor, wherein the entity temperature is measured at a first time of measurement; and determining an entity location associated with the entity temperature at the first time of measurement, wherein the second set of locations comprises the entity location.
15 . The medium of claim 1 , wherein the plurality of sensors comprises a chemical sensor, and wherein the operations further comprise: determining a volatile chemical signature of the first entity using the chemical sensor, wherein the volatile chemical signature is measured at a first time of measurement; and determining an entity location associated with the volatile chemical signature at the first time of measurement, wherein the second set of locations comprises the entity location.
16 . The medium of claim 1 , wherein the plurality of sensors comprises an ultrasonic sound sensor, and wherein the operations further comprise: determining a sound of the first entity using the ultrasonic sound sensor, wherein the sound is measured at a first time of measurement; and determining an entity location associated with the sound at the first time of measurement, wherein the second set of locations comprises the entity location.
17 . The medium of claim 1 , wherein a sensor of the plurality of sensors is attached to a camera of the plurality of cameras.
18 . The medium of claim 1 , further comprising: determining, using the computer system, whether a location in the sequence of locations is in a restricted area of the monitored environment; and in response to a determination that the location is in the restricted area of the monitored environment, display a warning to a graphical display device.
19 . The medium of claim 1 , further comprising: determining, using the computer system, whether a location in the sequence of locations is outside a permitted area of the monitored environment; and in response to a determination that the location is outside the permitted area of the monitored environment, display a warning to a graphical display device.
20 . A method comprising: acquiring, using a computer system, a set of images from a plurality of cameras, wherein: each of the plurality of cameras have a different respective field of view, and at least part of the fields of view are of a monitored environment; detecting and localizing, using the computer system, in at least some of the set of images, a first entity moving through the monitored environment; determining, using the computer system, a first set of locations within the monitored environment of the first entity based on locations of the first entity in the set of images, wherein each of the first set of locations is associated with an image acquisition time; acquiring, using the computer system, a set of sensor measurements of the monitored environment from a plurality of sensors, the plurality of sensors being different from the plurality of cameras; determining, using the computer system, a second set of locations within the monitored environment of the first entity based on the set of sensor measurements, wherein each of the second set of locations is associated with a sensor measurement time; determining, using the computer system, whether the first set of locations should be associated with the second set of locations based on a set of confidence factors calculated based on the first set of locations and the second set of locations, the set of confidence factors being indicative of the second set of locations being locations of the first entity and not another entity; in response to determining that the first set of locations should be associated with the second set of locations, determining, using the computer system, a sequence of locations of the first entity through the monitored environment; and storing, using the computer system, the sequence of locations in a computer-readable media in communication with the computer system.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This patent application is a continuation of U.S. Non-Provisional patent application Ser. No. 17/493,450, filed Oct. 4, 2021, titled “MULTI-CHANNEL SPATIAL POSITIONING SYSTEM”, which is a continuation of U.S. Non-Provisional Patent Application 16/668,180, filed 30 Oct. 2019, titled “MULTI-CHANNEL SPATIAL POSITIONING SYSTEM”, now U.S. Pat. No. 11,164,329. U.S. Non-Provisional patent application Ser. No. 16/668,180 claims the benefit of U.S. Provisional Patent Application 62/754,446, filed 1 Nov. 2018, titled “A MULTI-SENSOR SYSTEM FOR MONITORING CROWDS OF INDIVIDUALS.” The entire content of each afore-mentioned patent filing is hereby incorporated by reference. BACKGROUND 1. Field This disclosure relates generally to positioning systems and, more particularly, to multi-sensor positioning systems. 2. Background Understanding visitor behavior is useful for the design, operation and optimization of public or semi-public spaces. Visitor behavior information is valuable to various stakeholders of the space, such as an owner of the space, an operator of the space, security personnel and merchant staff operating in the space, or the like. Existing computer systems to determine indoor positions of visitors suffer from a number of challenges. Some systems require the visitor to wear a tracking device, but these arrangements are often untenable in public places, where people are unwilling to wear such devices. Some systems merely afford low-resolution, sparse indications of position, like card-based electronic door access controls. SUMMARY The following is a non-exhaustive listing of some aspects of the present techniques. These and other aspects are described in the following disclosure. Some aspects include a process that includes acquiring, using a computer system, a set of images from a plurality of cameras, wherein: each of the plurality of cameras have a different respective field of view, and at least part of the fields of view are of a monitored environment; detecting and localizing, using the computer system, in at least some of the set of images, a first entity moving through the monitored environment; determining, using the computer system, a first set of locations within the monitored environment of the first entity based on locations of the first entity in the set of images, wherein each of the first set of locations is associated with an image acquisition time; acquiring, using the computer system, a set of sensor measurements of the monitored environment from a plurality of sensors, the plurality of sensors being different from the plurality of cameras; determining, using the computer system, a second set of locations within the monitored environment of the first entity based on the set of sensor measurements, wherein each of the second set of locations is associated with a sensor measurement time; determining, using the computer system, whether the first set of locations should be associated with the second set of locations based on a set of confidence factors calculated based on the first set of locations and the second set of locations, the set of confidence factors being indicative of the second set of locations being locations of the first entity and not another entity; in response to determining that the first set of locations should be associated with the second set of locations, determining, using the computer system, a sequence of locations of the first entity through the monitored environment; and storing, using the computer system, the sequence of locations in a computer-readable media in communication with the computer system. Some aspects include a tangible, non-transitory, machine-readable medium storing instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations including the above-mentioned process. Some aspects include a system, including: one or more processors; and memory storing instructions that when executed by the processors cause the processors to effectuate operations of the above-mentioned process. BRIEF DESCRIPTION OF THE DRAWINGS The above-mentioned aspects and other aspects of the present techniques will be better understood when the present application is read in view of the following figures in which like numbers indicate similar or identical elements: FIG. 1 is a diagram of a monitored environment in which various entities may be tracked using the present techniques, in accordance with some embodiments; FIG. 2 is a flowchart of operations to determine a sequence of locations based on a set of images and sensor measurements, in accordance with some embodiments; FIG. 3 is a flowchart of operations to determine associations between devices and entities, in accordance with some embodiments; and FIG. 4 shows an example of a computing device by which the present techniques may be implemented, in accordance with some embodiments. While the present techniques are susceptible to va