CN-122018682-A - Hand binding method, device, electronic equipment, storage medium and product

CN122018682ACN 122018682 ACN122018682 ACN 122018682ACN-122018682-A

Abstract

The application provides a hand binding method, a device, electronic equipment, a storage medium and a product, wherein the method detects a hand frame, a first human frame, skeleton points and external human frames corresponding to the skeleton points from a cabin image; the method comprises the steps of matching a first human frame in a cabin image with a preset seat area, matching an external human frame corresponding to a skeleton point with the first human frame, determining seat information corresponding to the skeleton point, matching the skeleton point with a hand frame, and determining the seat information corresponding to the skeleton point matched with the hand frame as the seat information corresponding to the hand frame. According to the scheme, the seat of the human-shaped frame is transferred to the skeleton point and then to the hand through the matching of the human-shaped frame and the seat area, the matching of the skeleton point and the human-shaped frame and the matching of the skeleton point and the hand frame, so that the corresponding seat of the hand can be determined, the binding between the hand and the seat is realized, the seat of a user executing the hand action is effectively positioned, and the recognition accuracy of the gesture control instruction is improved.

Inventors

WANG XU
SHAO BO
ZHANG YUSHUN
Gong Suming

Assignees

科大讯飞股份有限公司

Dates

Publication Date: 20260512
Application Date: 20260116

Claims (14)

1.A method of hand binding, comprising: Detecting a hand frame, a first human-shaped frame, skeleton points and external human-shaped frames corresponding to the skeleton points from a cabin image; matching a first human frame in the cabin image with a preset seat area, matching an external human frame corresponding to the skeleton point with the first human frame, and determining seat information corresponding to the skeleton point; and matching the skeleton points with the hand frame, and determining seat information corresponding to the skeleton points matched with the hand frame as seat information corresponding to the hand frame.
2. The hand binding method of claim 1, wherein matching a first human frame in the cabin image with a preset seating area, and matching an circumscribing human frame corresponding to the skeletal point with the first human frame, determining seating information corresponding to the skeletal point, comprises: Matching a first human frame in the cabin image with a preset seat area, and determining seat information corresponding to the first human frame; And matching the external human-shaped frame corresponding to the skeleton point with the first human-shaped frame, and determining the seat information corresponding to the first human-shaped frame matched with the external human-shaped frame corresponding to the skeleton point as the seat information corresponding to the skeleton point.
3. The hand binding method of claim 1, wherein the skeletal points comprise wrist skeletal points and hand skeletal points; Matching the skeleton points with the hand frame, and determining seat information corresponding to the skeleton points matched with the hand frame as seat information corresponding to the hand frame, wherein the method comprises the following steps: Based on the wrist skeleton points and the hand skeleton points, determining an external hand frame corresponding to the skeleton points, and determining seat information corresponding to the skeleton points as seat information of the external hand frame corresponding to the skeleton points; And matching the external hand frame corresponding to the skeleton point with the hand frame, and determining the seat information of the external hand frame matched with the hand frame as the seat information corresponding to the hand frame.
4. The hand binding method of claim 1, wherein matching a first human frame in the cabin image with a preset seating area, and matching an circumscribing human frame corresponding to the skeletal point with the first human frame, before determining the seating information corresponding to the skeletal point, further comprises: Detecting a cockpit image and determining a head frame of a person in the cockpit image; Matching a head frame in the cabin image with a first human-shaped frame in the cabin image, and judging whether the first human-shaped frame matched with the head frame is abnormal or not; If the first human frame matched with the human head frame is abnormal, adjusting the first human frame based on the human head frame to obtain a target human frame; Correspondingly, matching a first human-shaped frame in the cabin image with a preset seat area, and matching an external human-shaped frame corresponding to the skeleton point with the first human-shaped frame, and determining seat information corresponding to the skeleton point, including: and matching the target humanoid frame with a preset seat area, matching the external humanoid frame corresponding to the skeleton point with the target humanoid frame, and determining the seat information corresponding to the skeleton point.
5. The hand binding method of claim 4, wherein determining whether a first human frame that matches the human head frame is abnormal comprises: Calculating the distance between the center point of the first human head frame matched with the human head frame and the center point of the human head frame on the abscissa axis; If the distance is within the preset distance range, determining that a first human-shaped frame matched with the human head frame is normal; Or alternatively And if the distance is not within the preset distance range, determining that the first human frame matched with the human head frame is abnormal.
6. The hand binding method of claim 4, wherein adjusting the first humanoid frame based on the humanoid frame results in a target humanoid frame, comprising: Expanding the head frame according to preset proportional parameters between the head frame and the human-shaped frame to obtain a second human-shaped frame corresponding to the head frame; and fusing the first human-shaped frame matched with the human head frame and the second human-shaped frame corresponding to the human head frame according to a preset human-shaped frame adjustment rule to obtain a target human-shaped frame.
7. The hand binding method of claim 1, wherein before matching the skeletal points with the hand frame and determining the seat information corresponding to the skeletal points matched with the hand frame as the seat information corresponding to the hand frame, further comprising: selecting a target hand frame from the hand frames based on preset player rules; Correspondingly, the skeletal points are matched with the hand frame, and seat information corresponding to the skeletal points matched with the hand frame is determined as seat information corresponding to the hand frame, and the method comprises the following steps: And matching the skeleton points with the target hand frame, and determining seat information corresponding to the skeleton points matched with the target hand frame as seat information corresponding to the target hand frame.
8. The hand binding method of claim 7, wherein selecting a target hand frame from the hand frames based on a pre-set player rule comprises: setting player anchor points in a preset gesture area; calculating the distance between the player anchor point and the center point of the hand frame and the intersection ratio between the gesture area and the hand frame; And selecting a target hand frame corresponding to the player anchor point according to a preset player rule, the distance and the cross ratio.
9. The hand binding method of claim 1, wherein detecting a hand frame, a first human frame, a skeletal point, and an circumscribing human frame corresponding to the skeletal point from a cabin image comprises: extracting features of the cabin image to obtain hand features and human-shaped features in the cabin image; Based on the hand characteristics, predicting a hand frame in the cabin image, based on the humanoid characteristics, predicting a first humanoid frame in the cabin image, and based on the hand characteristics and the splicing characteristics of the humanoid characteristics, predicting skeleton points in the cabin image and external humanoid frames corresponding to the skeleton points.
10. The hand binding method of claim 9, wherein feature extraction is performed on a cockpit image to obtain hand features and humanoid features in the cockpit image, comprising: inputting the cockpit image into a pre-constructed feature extraction network to obtain hand features and human-shaped features in the cockpit image, wherein the feature extraction network comprises a main network and a PANet fusion network.
11. A hand binding apparatus, comprising: The image detection module is used for detecting a hand frame, a first human frame, skeleton points and external human frames corresponding to the skeleton points from the cabin image; the skeleton point seat determining module is used for matching a first human frame in the cabin image with a preset seat area, matching an external human frame corresponding to the skeleton point with the first human frame and determining seat information corresponding to the skeleton point; and the hand seat determining module is used for matching the skeleton points with the hand frames and determining seat information corresponding to the skeleton points matched with the hand frames as seat information corresponding to the hand frames.
12. An electronic device is characterized by comprising a memory and a processor; the memory is connected with the processor and used for storing programs; The processor is configured to implement the hand binding method according to any one of claims 1 to 10 by running a program in the memory.
13. A storage medium having stored thereon a computer program which, when executed by a processor, implements the hand binding method of any one of claims 1 to 10.
14. A computer program product comprising computer program instructions which, when executed by a processor, cause the processor to implement the hand binding method of any one of claims 1 to 10.

Description

Hand binding method, device, electronic equipment, storage medium and product Technical Field The application relates to the technical field of artificial intelligence, in particular to a hand binding method, a hand binding device, electronic equipment, a storage medium and a hand binding product. Background With the rapid development of artificial intelligence technology, in the field of intelligent cabins, acquiring instructions through an artificial intelligence gesture recognition technology has become a main upper input of the current intelligent vehicle-mounted user interaction function, so that the driving experience of a real vehicle is greatly improved. Users at different positions in the cabin have different gesture control authorities, and under a multi-person and multi-hand scene, the traditional hand detection method cannot effectively locate the position of the user executing the hand action in the cabin, so that the recognition accuracy of gesture control instructions is affected. Disclosure of Invention Based on the above requirements, the application provides a hand binding method, a device, an electronic device, a storage medium and a product, which can bind the positions of all cabins with the positions of the hands, thereby effectively positioning the positions of users executing hand actions in cabins and further improving the recognition accuracy of gesture control instructions. In order to achieve the above purpose, the present application proposes the following technical scheme: according to a first aspect of an embodiment of the present application, there is provided a hand binding method, including: Detecting a hand frame, a first human-shaped frame, skeleton points and external human-shaped frames corresponding to the skeleton points from a cabin image; matching a first human frame in the cabin image with a preset seat area, matching an external human frame corresponding to the skeleton point with the first human frame, and determining seat information corresponding to the skeleton point; and matching the skeleton points with the hand frame, and determining seat information corresponding to the skeleton points matched with the hand frame as seat information corresponding to the hand frame. Optionally, matching the first human frame in the cabin image with a preset seat area, and matching the external human frame corresponding to the skeleton point with the first human frame, to determine seat information corresponding to the skeleton point, including: Matching a first human frame in the cabin image with a preset seat area, and determining seat information corresponding to the first human frame; And matching the external human-shaped frame corresponding to the skeleton point with the first human-shaped frame, and determining the seat information corresponding to the first human-shaped frame matched with the external human-shaped frame corresponding to the skeleton point as the seat information corresponding to the skeleton point. Optionally, the skeletal points include wrist skeletal points and hand skeletal points; Matching the skeleton points with the hand frame, and determining seat information corresponding to the skeleton points matched with the hand frame as seat information corresponding to the hand frame, wherein the method comprises the following steps: Based on the wrist skeleton points and the hand skeleton points, determining an external hand frame corresponding to the skeleton points, and determining seat information corresponding to the skeleton points as seat information of the external hand frame corresponding to the skeleton points; And matching the external hand frame corresponding to the skeleton point with the hand frame, and determining the seat information of the external hand frame matched with the hand frame as the seat information corresponding to the hand frame. Optionally, the matching of the first human frame in the cabin image with a preset seat area, and the matching of the external human frame corresponding to the skeleton point with the first human frame, before determining the seat information corresponding to the skeleton point, further includes: Detecting a cockpit image and determining a head frame of a person in the cockpit image; Matching a head frame in the cabin image with a first human-shaped frame in the cabin image, and judging whether the first human-shaped frame matched with the head frame is abnormal or not; If the first human frame matched with the human head frame is abnormal, adjusting the first human frame based on the human head frame to obtain a target human frame; Correspondingly, matching a first human-shaped frame in the cabin image with a preset seat area, and matching an external human-shaped frame corresponding to the skeleton point with the first human-shaped frame, and determining seat information corresponding to the skeleton point, including: and matching the target humanoid frame with a preset seat area, matching the ex