US-12626372-B2 - System and method for estimating the direction of a movement in video images

Abstract

The present description concerns a system and method of determining at least one classifier of a general movement along a first direction in video images of a scene, comprising determining a differential image based on two video images, selecting pixels of the differential image corresponding to edges of objects, determining, for each selected pixel, at least one classifier of a local movement along the first direction at least at a first or second value, determining a first indicator of the local movement along the first direction, which depends on the sum of the local movement classifiers at the first value and a second indicator of the local movement along the first direction, which depends on the sum of the local movement classifiers at the second value, and determining the general movement classifier based on the comparison of the first and second local movement indicators.

Inventors

• Marina Nicolas

Assignees

• STMICROELECTRONICS INTERNATIONAL N.V.

Dates

Publication Date

20260512

Application Date

20231211

Priority Date

20221213

Claims (20)

1. 1 . A method of determining at least one classifier of a general movement along a first direction in a scene, the method comprising steps: a) determining a differential image based on first and second video images of the scene; b) selecting pixels of the differential image corresponding to edges of objects of the scene; c) determining, for each pixel selected at step b), at least one classifier of a local movement along the first direction at least at a first value and at a second value; d) determining a first indicator of the local movement along the first direction which depends on a sum of the at least one local movement classifier along the first direction at the first value and a second indicator of the local movement along the first direction which depends on a sum of the at least one local movement classifier along the first direction at the second value; and e) determining the at least one classifier of the general movement along the first direction based on a comparison of the first and second indicators of the local movement along the first direction.
2. 2 . The method according to claim 1 , wherein, at step a), each pixel of the differential image is an affine function of a difference between the pixels at the same position in the first and second video images.
3. 3 . The method according to claim 1 , wherein step a) further comprises a step of segmentation of the differential image to delimit at least two portions of the differential image, steps b), c), d), and e) being implemented separately for each portion of the differential image.
4. 4 . The method according to claim 1 , wherein step b) comprises a first selection of pixels of the differential image for which an absolute value of a difference between each pixel and a constant is greater than a first pixel value threshold.
5. 5 . The method according to claim 4 , wherein step b) comprises a second selection of pixels of the differential image among the pixels of the first selection, comprising selecting, for each group of adjacent pixels along the first direction, only of a pixel at a beginning of the group and of a pixel at an end of the group.
6. 6 . The method according to claim 4 , wherein step b) comprises removing, from the first selection, of each group of adjacent pixels along the first direction for which a number of pixels is smaller than a pixel number threshold.
7. 7 . The method according to claim 1 , wherein step c) comprises, for each pixel of the differential image selected at step b), the determination of the classifier of the local movement along the first direction of the selected pixel at least at the first value or at the second value based on a value of the selected pixel and on a result of the comparison between a first pixel equal to the pixel of the first or second video image located along the first direction just before the selected pixel or a first noise level equal to a noise level of pixels of the differential image located along the first direction just before the selected pixel and a second pixel equal to the pixel of the first or second video image located along the first direction just after the selected pixel or a second noise level equal to a noise level of pixels of the differential image located along the first direction just after the selected pixel.
8. 8 . The method according to claim 7 , wherein, for each pixel of the differential image selected at step b), step c) comprises: in response to the selected pixel being lower than a second pixel value threshold and in response to the first pixel being lower than the second pixel or the first noise level being lower than the second noise level, the classifier of the local movement along the first direction of the selected pixel is set to the first value; in response to the selected pixel being higher than the second pixel value threshold and in response to the first pixel being lower than the second pixel or the first noise level being lower than the second noise level, the classifier of the local movement along the first direction of the selected pixel is set to the second value; in response to the selected pixel being lower than the second pixel value threshold and in response to the first pixel being higher than the second pixel or the first noise level being higher than the second noise level, the classifier of the local movement along the first direction of the selected pixel is set to the second value; and in response to the selected pixel being higher than the second pixel value threshold and in response to the first pixel being higher than the second pixel or the first noise level being higher than the second noise level, the classifier of the local movement along the first direction of the selected pixel is set to the first value.
9. 9 . The method according to claim 7 , wherein the first noise level is equal to a standard deviation of values of a number K of adjacent pixels of the differential image along the first direction located just before the selected pixel, and the second noise level is equal to a standard deviation of values of the number K of adjacent pixels of the differential image along the first direction located just after the selected pixel.
10. 10 . The method according to claim 9 , wherein the first noise level and/or the second noise level are replaced with a maximum noise level value in response to the respective K pixels being higher than a third pixel value threshold.
11. 11 . The method according to claim 1 , further comprising: at step c), determining, for each pixel of the differential image selected at step b), at least one classifier of the local movement along a second direction, different from the first direction, at least at a third value or at a fourth value; at step d), determining a first indicator of the local movement along the second direction which depends on the sum of the local movement classifiers along the second direction at the third value and a second indicator of the local movement along the second direction which depends on the sum of the local movement classifiers along the second direction at the fourth value; and at step e), determining a classifier of the general movement along the second direction based on the comparison of the first and second indicators of the local movement along the second direction.
12. 12 . The method according to claim 11 , wherein, at step d), for each pixel of each group of pixels selected at step b) and aligned along a third direction inclined to within 10° along a bisector with respect to the first and second directions, when the classifier of the local movement along the first direction of the pixel in the group is at the first value and the classifier of the local movement along the second direction of the pixel in the group is at the fourth value, the classifier of the local movement along the first direction of the pixel in the group is weighted by a factor smaller than one in the determination of the first indicator of the local movement along the first direction and the classifier of the local movement along the second direction of the pixel in the group is weighted by the factor in the determination of the first indicator of the local movement along the second direction, and, when the classifier of the local movement along the first direction of the pixel in the group is at the second value and the classifier of the local movement along the second direction of the pixel in the group is at the fourth value, the classifier of the local movement along the first direction of the pixel is weighted by the factor in the determination of the second indicator of the local movement along the first direction and the classifier of the local movement along the second direction of the pixel in the group is weighted by the factor in the determination of the second indicator of the local movement along the second direction.
13. 13 . The method according to claim 11 , wherein the pixels of the video images and of the differential image are arranged in rows and in columns, the first direction corresponding to a row direction and the second direction corresponding to a column direction.
14. 14 . The method according to claim 1 , further comprising: illuminating the scene with a radiation; and capturing the radiation to acquire the video images of the scene.
15. 15 . A system for determining at least one classifier of a general movement along a first direction in a scene, the system comprising: a video image acquisition image sensor; and a processing module configured to: a) determine a differential image based on first and second video images of the scene received from the video image acquisition image sensor; b) select pixels of the differential image corresponding to edges of objects of the scene; c) determine, for each pixel selected at step b), at least one classifier of a local movement along the first direction at least at a first value and at a second value; d) determine a first indicator of the local movement along the first direction which depends on a sum of the at least one local movement classifier along the first direction at the first value and a second indicator of the local movement along the first direction which depends on a sum of the at least one local movement classifier along the first direction at the second value; and e) determine the at least one classifier of the general movement along the first direction based on a comparison of the first and second indicators of the local movement along the first direction.
16. 16 . The system according to claim 15 , wherein, at step a), each pixel of the differential image is an affine function of a difference between the pixels at the same position in the first and second video images.
17. 17 . The system according to claim 15 , wherein step a) further comprises a step of segmentation of the differential image to delimit at least two portions of the differential image, steps b), c), d), and e) being implemented separately for each portion of the differential image.
18. 18 . The system according to claim 15 , wherein step b) comprises a first selection of pixels of the differential image for which an absolute value of a difference between each pixel and a constant is greater than a first pixel value threshold.
19. 19 . The system according to claim 18 , wherein step b) comprises a second selection of pixels of the differential image among the pixels of the first selection, comprising a selection, for each group of adjacent pixels along the first direction, only of a pixel at a beginning of the group and of a pixel at an end of the group.
20. 20 . The system according to claim 18 , wherein step b) comprises a removal, from the first selection, of each group of adjacent pixels along the first direction for which a number of pixels is smaller than a pixel number threshold.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of French Patent Application No. 2213216, filed on Dec. 13, 2022, which application is hereby incorporated herein by reference. TECHNICAL FIELD The present disclosure generally concerns a system and method of estimation of the direction of a movement in video images. BACKGROUND For certain applications using an image sensor supplying video images of a scene, it is desirable to have an estimate of the general direction of a movement in the images supplied by the image sensor. It may be the estimate of the direction of movements of mobile objects of the scene or the estimate of the direction of the movement of the image sensor when the latter is mobile with respect to the scene. An example of application concerns the image sensor of a virtual reality helmet. There exist methods of estimation of the general direction of a movement in video images based on the analysis of the images supplied by the image sensor. However, such methods generally require the processing of a plurality of images to operate correctly, which implies the storage of information of previous images, while, for certain applications, it is desirable for the estimate of the general direction to be available with a minimum memory cost, rapidly after the acquisition of a new video image, without waiting for a latency period directly induced by the number of images/second of the system. SUMMARY There exists a need to overcome all or part of the disadvantages of known methods of estimation of the general direction of a movement in video images. An embodiment provides a method of determination of at least one classifier of a general movement along a first direction in video images of a scene, comprising the following steps: a) determining a differential image based on two video images;b) selecting pixels of the differential image corresponding to edges of objects of the scene;c) determining, for each pixel selected at step b), at least one classifier of a local movement along the first direction at least at a first value or at a second value;d) determining a first indicator of the local movement along the first direction which depends on the sum of the classifiers of the local movement along the first direction at the first value and a second indicator of the local movement along the first direction which depends on the sum of the classifiers of the local movement along the first direction at the second value; ande) determining the classifier of the general movement along the first direction based on the comparison of the first and second indicators of the local movement along the first direction. An embodiment also provides a system of determination of at least one classifier of a general movement along a first direction in video images of a scene, comprising an image sensor of acquisition of the video images and a processing module configured to: a) determine a differential image based on two video images;b) select pixels of the differential image corresponding to edges of objects of the scene;c) determine, for each pixel selected at step b), at least one classifier of the local movement along the first direction at least at a first value or at a second value;d) determine a first indicator of the local movement along the first direction which depends on the sum of the classifiers of the local movement along the first direction at the first value and a second indicator of the local movement along the first direction which depends on the sum of the classifiers of the local movement along the first direction at the second value; ande) determine the classifier of the general movement along the first direction based on the comparison of the first and second indicators of the local movement along the first direction. According to an embodiment, at step a), each pixel of the differential image is an affine function of the difference between the pixels at the same position in the two video images. According to an embodiment, step a) further comprises a step of segmentation of the differential image to delimit at least two portions of the differential image, steps b), c), d), and e) being implemented separately for each portion of the differential image. According to an embodiment, step b) comprises a first selection of pixels of the differential image for which the absolute value of the difference between the pixel and a constant is greater than a first pixel value threshold. According to an embodiment, step b) comprises a second selection of pixels of the differential image among the pixels of the first selection comprising the selection, for each group of adjacent pixels along the first direction, only of the pixel at the beginning of the group and of the pixel at the end of the group. According to an embodiment, step b) comprises the removal from the first selection of each group of adjacent pixels along the first direction for which the number of pixels is smaller than a pi