CN-116052213-B - Human body contour point positioning method and system, equipment and storage medium

CN116052213BCN 116052213 BCN116052213 BCN 116052213BCN-116052213-B

Abstract

The invention discloses a human body contour point positioning method, a system, equipment and a storage medium, which comprise the steps of constructing a human body contour point positioning neural network model; the method comprises the steps of obtaining a training data set, training a neural network model by using the training data set, inputting an image with a human body into the neural network model to obtain human skeleton point coordinates and human contour point coordinates on the image, and obtaining contour points after human body block connection based on the human skeleton point coordinates and the human contour point coordinates on the image. The neural network model predicts the contour points with independent blocks, then the blocks are connected through rules, the definition of the contour points with independent blocks is clear, when the human body posture is continuously changed, the change of the contour points is also continuous, and the advantages of the contour points with independent blocks are beneficial to the training of the neural network so as to obtain a good prediction result.

Inventors

LI DONGPING
YING LEBIN
Yang Kaihang

Assignees

杭州相芯科技有限公司

Dates

Publication Date: 20260508
Application Date: 20230130

Claims (8)

1. A human body contour point positioning method, characterized by comprising: constructing a human body contour point positioning neural network model; acquiring a training data set and training the neural network model by using the training data set; Inputting an image with a human body into the neural network model to obtain human body skeleton point coordinates and human body contour point coordinates on the image, wherein the method comprises the following steps: Acquiring a human body frame in the image with the human body, modifying the human body frame to be equal to the aspect ratio of the neural network model and scaling the image to be a fixed size; Inputting the images with fixed sizes into the neural network model to obtain a heat map of N human skeleton points and a heat map of M human contour points; The method comprises the steps of respectively obtaining a human skeleton point coordinate, a human skeleton point confidence coefficient, a human contour point coordinate and a human contour point confidence coefficient on an image with a fixed size according to a heat degree map of a human skeleton point and a heat degree map of a human contour point, respectively obtaining a pixel point position with a maximum response value on each single-channel heat degree map based on the heat degree map of the human skeleton point and the heat degree map of the human contour point, dividing a rectangular area by taking the pixel point position with the maximum response value as a center, taking a weighted average value of coordinate point positions of the rectangular area as a coordinate corresponding to the human skeleton point or the human contour point on the single-channel heat degree map, and taking the maximum response value as the corresponding confidence coefficient, and calculating to obtain the human skeleton point coordinate and the human contour point coordinate on the image with the fixed size based on the human skeleton point or the human contour point on the single-channel heat degree map, wherein a calculation formula is as follows: x_netin = x_hm / W hm * W; y_netin = y_hm / H hm * H; wherein x_netin and y_ netin are the coordinates of a human skeleton point or a human contour point respectively, W hm 、H hm is the width and the height of a heat map of the human skeleton point or a heat map of the human contour point respectively, and W, H is the width and the height of an image with fixed size respectively; Acquiring human skeleton point coordinates and human contour point coordinates on the image based on the human skeleton point coordinates and the human contour point coordinates on the image with fixed size; And obtaining contour points after the human body blocks are connected based on the human body skeleton point coordinates and the human body contour point coordinates on the image.
2. The human body contour point positioning method as set forth in claim 1, wherein the training the neural network model step based on the image with human body comprises: The neural network model comprises a back bone network and two headnet networks, wherein the two headnet networks are a human contour point headnet network and a human skeleton point headnet network respectively; Freezing the human body contour point headnet network, training the back bone network and the human body skeletal point headnet network using the training dataset; After training is completed, freezing the backup network and the human skeleton point headnet network, and training the human contour point headnet network by using the training data set; after training is completed, the backhaul network and the two headnet networks are trained using the training data set.
3. The human body contour point positioning method according to claim 2, wherein the training data set includes a positive sample data set and a negative sample data set, the positive sample data set is a picture set with human body frames, human body bone points and human body contour point labels, and the negative sample data set is a picture set without human body.
4. The human body contour point positioning method as defined in claim 1, wherein obtaining contour points after human body block connection based on human body skeleton point coordinates and human body contour point coordinates on the image comprises: The human body block comprises a shoulder neck block, a chest, waist and crotch block, a right big arm block, a right forearm block, a right palm block, a left big arm block, a left forearm block, a left palm block, a right thigh block, a right calf block, a right sole block, a left thigh block, a left calf block and a left sole block; connecting the right big arm block, the right small arm block and the right palm block based on the human skeleton point coordinates and the human contour point coordinates on the image to obtain an independent right arm contour; similarly, connecting the left thigh block, the left forearm block and the left palm block to obtain independent left arm outlines, connecting the right thigh block, the right calf block and the right sole block to obtain independent right leg outlines, and connecting the left thigh block, the left calf block and the left sole block to obtain independent left leg outlines; based on the human skeleton point coordinates and the human contour point coordinates of the shoulder-neck block and the chest-waist-crotch block, respectively judging the front and back sides of the shoulder-neck block and the chest-waist-crotch block; If the shoulder and neck block is the front surface, the independent right arm contour is connected with the left contour of the shoulder and neck block and the left contour of the chest, waist and crotch block at the same time, and the independent left arm contour is connected with the other side contour of the shoulder and neck block and the other side contour of the chest, waist and crotch block at the same time; If the shoulder and neck block is the back, the independent left arm contour is connected with the left contour of the shoulder and neck block and the left contour of the chest, waist and crotch block at the same time, and the independent right arm contour is connected with the other side contour of the shoulder and neck block and the other side contour of the chest, waist and crotch block at the same time; If the chest-waist crotch block is a front surface, the left outline of the independent right leg outline is connected with the left outline of the chest-waist crotch block, the right outline of the independent right leg outline is connected with the crotch point of the chest-waist crotch block, the right outline of the independent left leg outline is connected with the right outline of the chest-waist crotch block, and the left outline of the independent left leg outline is connected with the crotch point of the chest-waist crotch block; If the chest-waist crotch block is a back surface, the left outline of the independent left leg outline is connected with the left outline of the chest-waist crotch block, the right outline of the independent left leg outline is connected with the crotch point of the chest-waist crotch block, the right outline of the independent right leg outline is connected with the right outline of the chest-waist crotch block, and the left outline of the independent right leg outline is connected with the crotch point of the chest-waist crotch block; And connecting the human skeleton point coordinates and the human contour point coordinates on the image to obtain contour points after connecting human blocks.
5. The body contour point positioning method as defined in claim 4, wherein the judging of the front and back sides of the shoulder-neck block, chest-waist-crotch block based on the body skeleton point coordinates and body contour point coordinates of the shoulder-neck block, chest-waist-crotch block, respectively, comprises: When the confidence degrees of the contour points and the skeleton points of the shoulder-neck block and the chest-waist-crotch block are larger than a preset threshold value, judging the front-back tendency of the shoulder-neck block and the chest-waist-crotch block by using the distance between the left and right contour points and the left and right skeleton points; When the confidence of the bone points of only the shoulder-neck region and the chest-waist-crotch region is greater than a preset threshold, the bone points of the shoulder-neck region and the chest-waist-crotch region are used to judge the front-back inclination of the shoulder-neck region and the chest-waist-crotch region.
6. A system for locating points of a human body contour, for implementing the method of any of claims 1-5, comprising: The building module is used for building a human body contour point positioning neural network model; The training module is used for acquiring a training data set and training the neural network model by using the training data set; the acquisition module is used for inputting the image with the human body into the neural network model to obtain the human body skeleton point coordinates and the human body contour point coordinates on the image; and the calculation module is used for obtaining the contour points after the human body blocks are connected based on the human skeleton point coordinates and the human contour point coordinates on the image.
7. An apparatus comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program which, when executed by the processing unit, causes the processing unit to perform the method of any of claims 1-5.
8. A storage medium storing a computer program executable by a device, the program when run on the device causing the device to perform the method of any one of claims 1-5.

Description

Human body contour point positioning method and system, equipment and storage medium Technical Field The invention belongs to the technical field of artificial intelligence, and particularly relates to a human body contour point positioning method, a system, equipment and a storage medium. Background With the development of artificial intelligence technology, artificial intelligence technology is widely used. For example, when an image of a user is acquired by the image acquisition device, key points of human bones of the user can be given according to the image of the user, but a detection result obtained based on the detection method is often not accurate enough, the detection result only comprises coordinates of a plurality of key points associated with the bones of the user, and coordinates of human contour points of the user, which are more and more detailed distributed on each part of the human body of the user, cannot be given. In a certain application scenario, the human body image editing operation such as body shaping and slimming of the human body image is needed, so that the coordinates of a plurality of human body skeleton points and a plurality of human body contour points of the current user are required to be accurately known, the number of key points in the detection result obtained by the existing detection method is too small, and the obtained key points are the skeleton key points of the user, so that the detection result only shows a two-dimensional skeleton of the user, and the user experience is low. Disclosure of Invention In view of the above problems, the present invention provides a method, a system, a device, and a storage medium for positioning human body contour points, where the method includes: constructing a human body contour point positioning neural network model; acquiring a training data set and training the neural network model by using the training data set; Inputting an image with a human body into the neural network model to obtain human skeleton point coordinates and human contour point coordinates on the image; And obtaining contour points after the human body blocks are connected based on the human body skeleton point coordinates and the human body contour point coordinates on the image. Preferably, the training the neural network model step includes, based on the image with the human body: The neural network model comprises a back bone network and two headnet networks, wherein the two headnet networks are a human contour point headnet network and a human skeleton point headnet network respectively; Freezing the human body contour point headnet network, training the back bone network and the human body skeletal point headnet network using the training dataset; After training is completed, freezing the backup network and the human skeleton point headnet network, and training the human contour point headnet network by using the training data set; after training is completed, the backhaul network and the two headnet networks are trained using the training data set. Preferably, the training data set includes a positive sample data set and a negative sample data set, the positive sample data set is a picture set with human body frame, human body skeleton points and human body contour point marks, and the negative sample data set is a picture set without human body. Preferably, inputting the image with the human body into the neural network model, and obtaining the human body skeleton point coordinates and the human body contour point coordinates on the image includes: Acquiring a human body frame in the image with the human body, modifying the human body frame to be equal to the aspect ratio of the neural network model and scaling the image to be a fixed size; Inputting the images with fixed sizes into the neural network model to obtain a heat map of N human skeleton points and a heat map of M human contour points; According to the heat map of the human skeleton points and the heat map of the human contour points, respectively obtaining human skeleton point coordinates, human skeleton point confidence, human contour point coordinates and human contour point confidence on the image with fixed size; and obtaining the human skeleton point coordinates and the human contour point coordinates on the image based on the human skeleton point coordinates and the human contour point coordinates on the image with fixed size. Preferably, the obtaining the human skeleton point coordinates, the human skeleton point confidence, the human contour point coordinates and the human contour point confidence on the image with the fixed size according to the heat map of the human skeleton point and the heat map of the human contour point respectively includes: Based on the heat map of the human skeleton points and the heat map of the human contour points, respectively acquiring the pixel point position with the maximum response value on each single-channel heat map; Dividing a rectangular are