CN-121999259-A - Ground contact behavior identification method, device, equipment, medium and product

Abstract

The application discloses a method, a device, equipment, a medium and a product for identifying a touchdown behavior, and relates to the technical field of computers. The method comprises the steps of obtaining a depth image acquired at a first moment through a first image acquisition module and a view image acquired at the first moment through a second image acquisition module, identifying first distance information corresponding to a first object in a scene object and second distance information corresponding to a ground object in the scene object based on distance information represented by the depth image and texture information represented by the view image, and determining a touchdown detection result of the first object at the first moment according to a distance difference value between the first distance information and the second distance information. The method enables the identification equipment for performing geodetic detection to be configured at any position capable of shooting the first object in the scene, so that the situation that a non-contact type touchdown detection scene exists can be met.

Inventors

• Request for anonymity
• Request for anonymity

Assignees

• 极壳科技(上海)有限公司

Dates

Publication Date

20260508

Application Date

20241104

Claims (20)

1. 1. A method for identifying touchdown behavior, the method comprising: Acquiring a depth map acquired at a first time through a first image acquisition module and a view image acquired at the first time through a second image acquisition module, wherein the depth map comprises distance information between a scene object and the first image acquisition module, and the view image comprises texture information of a scene; Identifying first distance information corresponding to a first object in the scene objects and second distance information corresponding to a ground object in the scene objects based on the distance information represented by the depth map and the texture information represented by the view image, wherein a ground contact relationship exists between the first object and the ground object, the first distance information is used for expressing a first distance between the identified first object and the first image acquisition module, and the second distance information is used for expressing a second distance between the identified ground object and the first image acquisition module; And determining a touchdown detection result of the first object at the first moment according to the distance difference value between the first distance information and the second distance information.
2. 2. The method of claim 1, wherein the identifying the first distance information corresponding to a first object of the scene objects and the second distance information corresponding to a ground object of the scene objects based on the distance information characterized by the depth map and based on texture information characterized by the field of view image comprises: Identifying a first object region corresponding to the first object in the visual field image and a first ground region corresponding to the ground object based on the distance information represented by the depth map and the texture information represented by the visual field image, or identifying the first object region corresponding to the first object in the visual field image and the first ground region corresponding to the ground object based on the texture information represented by the visual field image, wherein the first object region is an image region in which the first object in the visual field image is located, and the first ground region is an image region in which the ground object in the visual field image is located; And determining first distance information corresponding to the first object area and second distance information corresponding to the first ground area based on the depth map and the view image.
3. 3. The method of claim 2, wherein the identifying a first object region corresponding to the first object in the field of view image and a first ground region corresponding to the ground object based on the distance information characterized by the depth map and based on texture information characterized by the field of view image comprises: And inputting the depth map and the view image into a region identification model, identifying the first object and the ground object through the region identification model, and determining the first object region and the first ground region in the view image, wherein the region identification model is obtained through pre-training and is used for analyzing the spatial position information expressed by the depth map and the texture information expressed by the view image so as to identify the models of the first object and the ground object from the scene object.
4. 4. A method according to claim 3, wherein said inputting the depth map and the field of view image into a pre-trained region identification model, identifying the first object of the field of view range and the ground object by the region identification model, determining the first object region and the first ground region in the field of view image comprises: Establishing a pixel mapping relation between a first pixel in the view image and a second pixel in the depth map; and inputting the depth map, the visual field image and the pixel mapping relation into the region identification model, identifying the first object and the ground object through the region identification model, and determining the first object region and the first ground region in the visual field image.
5. 5. A method according to claim 3, wherein the region recognition model comprises a first recognition model for recognizing a region in the view image in which the first object is located and a second recognition model for recognizing a region in which the ground object is located; The inputting the depth map and the view image into a region identification model, identifying the first object and the ground object by the region identification model, determining the first object region and the first ground region in the view image, comprising: Inputting the depth map and the view image into the first recognition model, recognizing the first object through the first recognition model, and determining the first object area in the view image; And inputting the depth map and the view image into the second recognition model, recognizing the ground object through the second recognition model, and determining the first ground area in the view image.
6. 6. The method according to any one of claims 2 to 5, wherein determining first distance information corresponding to the first object region and second distance information corresponding to the first ground region based on the depth map and the field of view image includes: Establishing a pixel mapping relation between a first pixel in the view image and a second pixel in the depth map; Determining a second object region corresponding to the first object region in the depth map based on the pixel mapping relation; Obtaining the first distance information according to the depth value indicated by the second pixel in the second object region in the depth map; determining a second ground area corresponding to the first ground area in the depth map based on the pixel mapping relation; And obtaining the second distance information according to the depth value indicated by the second pixel in the second ground area in the depth map.
7. 7. The method of claim 6, wherein the obtaining the second distance information from the depth value indicated by the second pixel in the second ground area in the depth map comprises: Acquiring region depth data corresponding to the second ground region in the depth map, wherein the region depth data comprises depth values corresponding to second pixels in the second ground region respectively; And carrying out plane fitting on the regional depth data through a random sampling consistency algorithm, determining an average depth value corresponding to the second ground region, and taking the average depth value as the second distance information.
8. 8. The method of claim 7, wherein the obtaining the region depth data corresponding to the second ground region in the depth map comprises: acquiring shooting angle information of the first image acquisition module at the first moment; Correcting a pixel horizontal plane of a second pixel in the depth map based on the shooting angle information to obtain a corrected depth map; and acquiring the region depth data corresponding to the second ground region in the corrected depth map.
9. 9. The method according to any one of claims 1 to 5, further comprising: acquiring attitude information of the first image acquisition module and the second image acquisition module through an inertial measurement unit; And performing visual angle compensation on the depth map and the visual field image through the gesture information to obtain a compensated depth map and a compensated visual field image.
10. 10. The method according to any one of claims 1 to 5, further comprising: Acquiring motion information of the first image acquisition module and the second image acquisition module through an inertial measurement unit; And adjusting a first visual angle of the first image acquisition module and/or a second visual angle of the second image acquisition module based on the motion information.
11. 11. The method according to any one of claims 1 to 5, wherein the touchdown detection result includes a touchdown state and a touchdown state; The determining, according to the distance difference between the first distance information and the second distance information, a touchdown detection result of the first object at the first moment includes: acquiring a preset difference threshold; Determining that the first object is in the touchdown state when the distance difference between the first distance information and the second distance information is smaller than the preset difference threshold; and determining that the first object is in the ground-leaving state under the condition that the distance difference value between the first distance information and the second distance information is greater than or equal to the preset difference value threshold value.
12. 12. The method of claim 11, wherein the obtaining a preset difference threshold comprises: Detecting the terrain type of the ground object to obtain terrain information of the ground object, wherein the terrain information is used for indicating the terrain type of the ground object contacted by the first object; And acquiring the preset difference threshold matched with the terrain information, wherein the magnitude of the preset difference threshold and the hardness of the terrain material of the terrain type are in negative correlation.
13. 13. The method of claim 12, wherein the detecting the terrain type of the ground object to obtain the terrain information of the ground object comprises: determining the first ground area in the visual field image, wherein the first ground area is an image area where the ground object is located in the visual field image; Inputting the field image with the marking information of the first ground area to a terrain recognition model, recognizing the terrain type of the ground object through the terrain recognition model to obtain the terrain information, wherein the terrain recognition model is used for classifying the terrain type of the ground object according to the texture of the first ground area.
14. 14. The method of claim 13, wherein the determining that the first object is in the touchdown state if a distance difference between the first distance information and the second distance information is less than the preset difference threshold comprises: acquiring object speed information of the first object at the first moment, and acquiring a preset speed threshold; And determining that the first object is in the ground contact state under the condition that the distance difference value between the first distance information and the second distance information is smaller than the preset difference value threshold value and the object speed information is smaller than the preset speed threshold value.
15. 15. The method of claim 14, wherein the obtaining object velocity information of the first object at the first time comprises: Acquiring a plurality of continuous depth maps continuously acquired by the first image acquisition module in a first period, wherein the first period comprises the first moment; determining at least one key point corresponding to the first object in the plurality of depth maps respectively; Determining inter-frame pixel displacement of the keypoint between the plurality of depth maps; Converting the inter-frame pixel displacement into a spatial displacement; the object velocity information of the first object at the first time instant is determined based on the spatial displacement and an inter-frame time difference between the plurality of depth maps.
16. 16. The method according to any one of claims 1 to 5, further comprising: Obtaining a grounding detection result respectively corresponding to each detection moment of the first object in a second period; And based on the ground contact detection result in the second period, performing gait analysis on the first object to obtain a gait analysis result, wherein the gait analysis result is used for indicating whether the gait of the first object in the second period is abnormal or not.
17. 17. A touchdown behavior recognition device, the device comprising: The device comprises an acquisition module, a first image acquisition module and a second image acquisition module, wherein the acquisition module is used for acquiring a depth image acquired at a first time through the first image acquisition module and a view image acquired at the first time through the second image acquisition module, the depth image comprises distance information between a scene object and the first image acquisition module, and the view image comprises texture information of a scene; The recognition module is used for recognizing first distance information corresponding to a first object in the scene objects and second distance information corresponding to a ground object in the scene objects based on the distance information represented by the depth map and texture information represented by the visual field image, wherein a ground contact relation exists between the first object and the ground object, the first distance information is used for expressing a first distance between the recognized first object and the first image acquisition module, and the second distance information is used for expressing a second distance between the recognized ground object and the first image acquisition module; and the judging module is used for determining a touchdown detection result of the first object at the first moment according to the distance difference value between the first distance information and the second distance information.
18. 18. A computer device comprising a processor and a memory, wherein the memory has stored therein at least one program that is loaded and executed by the processor to implement the touchdown behavior recognition method of any of claims 1-16.
19. 19. A computer readable storage medium having stored therein at least one program code loaded and executed by a processor to implement a touchdown behavior recognition method according to any of claims 1 to 16.
20. 20. A computer program product comprising a computer program or instructions which, when executed by a processor, implements a method of ground engaging behavior recognition according to any one of claims 1 to 16.

Description

Ground contact behavior identification method, device, equipment, medium and product Technical Field The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, a medium, and a product for identifying a touchdown behavior. Background The method has important significance for the real-time prediction and identification of foot bottoming behaviors in the fields of gait analysis, motion monitoring, human body behavior identification, robot control and the like. The conventional ground contact detection method mainly relies on physical sensors such as a pressure sensor and an accelerometer, namely, pressure data and speed data are detected in real time through sensors arranged on the foot, so that whether the foot is in a ground contact state or a ground leaving state is judged according to the pressure data and the speed data. However, the sensor in the above method needs to be in direct contact with the human body or the sole, which has the problems of inconvenient equipment wearing, use of field Jing Shouxian, and the like, and is difficult to meet the requirements of non-contact and long-distance detection. Disclosure of Invention The embodiment of the application provides a method, a device, equipment, a medium and a product for identifying a ground contact behavior. The technical scheme is as follows: In one aspect, a method for identifying a touchdown behavior is provided, the method comprising: Acquiring a depth map acquired at a first time through a first image acquisition module and a view image acquired at the first time through a second image acquisition module, wherein the depth map comprises distance information between a scene object and the first image acquisition module, and the view image comprises texture information of a scene; Identifying first distance information corresponding to a first object in the scene objects and second distance information corresponding to a ground object in the scene objects based on the distance information represented by the depth map and the texture information represented by the view image, wherein a ground contact relationship exists between the first object and the ground object, the first distance information is used for expressing a first distance between the identified first object and the first image acquisition module, and the second distance information is used for expressing a second distance between the identified ground object and the first image acquisition module; And determining a touchdown detection result of the first object at the first moment according to the distance difference value between the first distance information and the second distance information. In another aspect, there is provided a touchdown behavior recognition device, the device comprising: The device comprises an acquisition module, a first image acquisition module and a second image acquisition module, wherein the acquisition module is used for acquiring a depth image acquired at a first time through the first image acquisition module and a view image acquired at the first time through the second image acquisition module, the depth image comprises distance information between a scene object and the first image acquisition module, and the view image comprises texture information of a scene; The recognition module is used for recognizing first distance information corresponding to a first object in the scene objects and second distance information corresponding to a ground object in the scene objects based on the distance information represented by the depth map and texture information represented by the visual field image, wherein a ground contact relation exists between the first object and the ground object, the first distance information is used for expressing a first distance between the recognized first object and the first image acquisition module, and the second distance information is used for expressing a second distance between the recognized ground object and the first image acquisition module; and the judging module is used for determining a touchdown detection result of the first object at the first moment according to the distance difference value between the first distance information and the second distance information. In another aspect, a computer device is provided, where the computer device includes a processor and a memory, where the memory stores at least one instruction, at least one program, a code set, or an instruction set, where the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement a method for identifying a touchdown behavior according to any of the embodiments of the present application. In another aspect, a computer readable storage medium is provided, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored, where the at least one instructi