CN-117058234-B - Method and device for planting message

CN117058234BCN 117058234 BCN117058234 BCN 117058234BCN-117058234-B

Abstract

The application relates to a planting information method, a planting information method and a planting device. The method for generating the hair planting information comprises the steps of obtaining a head video of an object to be planted, carrying out head modeling on the head video to obtain a head model, carrying out hair follicle reconstruction on the head model to obtain an original hair follicle point cloud, carrying out hairline estimation on the basis of the head video to determine a target hairline position, and determining the hair planting information on the basis of the original hair follicle point cloud and the target hairline position. The hair planting method comprises the steps of obtaining hair planting information generated based on the hair planting information generation method, and controlling the mechanical arm to execute hair planting operation or hair taking operation based on the hair planting information. By adopting the method, the accuracy of the plant information can be improved.

Inventors

Request for anonymity
Request for anonymity
Request for anonymity
Request for anonymity
Request for anonymity

Assignees

深圳微美机器人有限公司

Dates

Publication Date: 20260508
Application Date: 20230808

Claims (13)

1. A method of generating plant information, the method comprising: acquiring a head video of an object to be planted, and performing head modeling on the head video to obtain a head model; Performing posture correction on the head model to obtain a third head model; reconstructing the hair follicle of the third head model to obtain an original hair follicle point cloud; performing face gesture detection on the head video to determine a target face image, wherein the face gesture detection is performed on each frame in a video stream of the head video, selecting a frame with the smallest difference compared with a reference as an optimal face image, and taking the optimal face image as the target face image; Estimating a hairline according to the target face image to obtain a target hairline position, wherein the target hairline position is an optimal hairline position estimated according to a to-be-planted hair object; Determining hair planting information based on the original hair follicle point cloud and the target hairline position; the performing posture correction on the head model to obtain a third head model includes: Performing attitude correction on the head model based on the initial symmetry plane to obtain a first head model; performing attitude correction on the first head model based on the target symmetry plane to obtain a second head model; Performing gesture correction on the second head model based on the face information and the rotation information corresponding to the second head model to obtain a third head model, so that the forward direction of the Y axis of the reference coordinate axis corresponding to the third head model passes through the rotation of the head of the third head model, and the forward direction of the Z axis passes through the face of the third head model; The initial symmetry plane is obtained based on the head model and a face mirror image model of the head model, wherein a Y axis of a reference coordinate axis on the corresponding side of the head model passes through the top end of the head model, an X axis passes through one side of eyes of the head model, and a Z axis passes through one side of ears of the head model, and the target symmetry plane is obtained based on a nose point of the first head model and a nose point of the face mirror image model of the first head model.
2. The method of claim 1, wherein prior to said pose correction of said head model based on initial symmetry, further comprising: Performing principal component analysis on the head model to determine a reference coordinate axis; determining a face position of the head model; Acquiring a first mirror image model of the head model about a first coordinate plane, wherein the first coordinate plane is determined by a reference coordinate axis passing through a human face and a Y axis; registering the head model and the first mirror model; An initial symmetry plane is determined based on the registered head model and the first mirror model.
3. The method of claim 1, wherein prior to said pose correction of said first head model based on object symmetry, further comprising: Carrying out posture adjustment on the initial symmetrical plane so that the initial symmetrical plane coincides with a second coordinate plane to obtain a posture adjustment relation, wherein the second coordinate plane is determined by a Y axis and a Z axis; Performing posture adjustment on the head model according to the posture adjustment relation to obtain a first head model; acquiring a second mirror model of the first head model with respect to a second coordinate plane; acquiring nose points of the first head model and the second mirror image model respectively; moving the first and second mirror image models such that nose points of the first and second mirror image models coincide; And determining a target symmetrical plane based on the first head model and the second mirror image model after nose tip points are overlapped.
4. The method of any of claims 1-3, wherein the determining hair planting information based on the original hair follicle point cloud and the target hairline position comprises: Performing posture correction on the third head model according to the target face image; Based on the original hair follicle point cloud, carrying out hair follicle point cloud complementation on the third head model with the corrected posture to obtain a current hair follicle point cloud; rendering the third head model after the hair follicle point cloud is completed to a front view direction to obtain a target rendering diagram; Migrating a target hairline corresponding to the target hairline position to the target rendering graph, and judging whether the difference between the target hairline and the current hairline in the target rendering graph meets the difference requirement or not; when the difference between the target hairline and the current hairline in the target rendering graph meets the difference requirement, determining a hair implantation area according to the current hair follicle point cloud and the original hair follicle point cloud; when the difference between the target hairline and the current hairline in the target rendering diagram does not meet the difference requirement, taking the current hair follicle point cloud as an original hair follicle point cloud, continuing to execute the hair follicle point cloud-based on the original hair follicle point cloud, and completing the hair follicle point cloud of the third head model after the posture correction to obtain the current hair follicle point cloud until the difference between the target hairline and the current hairline in the target rendering diagram meets the difference requirement, and determining a hair implantation area according to the current hair follicle point cloud and the original hair follicle point cloud obtained by hair follicle reconstruction.
5. The method of claim 4, wherein the determining hair planting information based on the original hair follicle point cloud and the target hairline position further comprises: acquiring the density of the hair implantation and the density of the hair taking area after hair taking; extracting a first target hair follicle point cloud from the original hair follicle point cloud; obtaining target hair follicles with neighborhood density meeting density requirements in the first target hair follicle point cloud; And determining a hair taking area by taking the first target hair follicle as a center according to the hair planting area, the hair planting density and the density of the hair taking area after hair taking.
6. The method of claim 5, wherein after determining the hair planting information based on the original hair follicle point cloud and the target hairline position, comprising: Processing the original hair follicle point cloud based on the hair taking area and the hair planting area to obtain a second target hair follicle point cloud, combining the second target hair follicle point cloud with the third head model after posture correction, and obtaining a two-dimensional rendering diagram of multiple view angles of the combined third head model; Acquiring a target hairstyle; Generating a three-dimensional effect graph after hair implantation based on the two-dimensional rendering graph and the target hairstyle; Receiving an adjustment instruction aiming at the plant-sent information; And adjusting the plant information according to the adjustment instruction, updating the second target hair follicle point cloud based on the plant information, and continuously combining the second target hair follicle point cloud with the third head model after posture correction until the plant information is adjusted.
7. A method according to any one of claims 1 to 3, wherein after head modeling the head video to obtain a head model, the method further comprises: Acquiring a face part of the head model; Projecting the face part to a target symmetrical plane to obtain a two-dimensional face image; Detecting characteristic points of the two-dimensional face image, and back-projecting the detected characteristic points into the head model; calculating a first distance between feature points in the head model, and acquiring a second distance between feature points in the target face image; And obtaining a scale factor according to the first distance and the second distance, and adjusting the size of the head model based on the scale factor.
8. A hair planting method, characterized in that the hair planting method comprises: acquiring the plant information generated based on the plant information generation method according to any one of claims 1 to 7; controlling the mechanical arm to execute hair planting operation or hair taking operation based on the hair planting information; Acquiring an initial real-time image of an object to be planted, and reconstructing hair follicles based on the initial real-time image to obtain an initial real-time hair follicle point cloud; registering the initial real-time hair follicle point cloud with a preoperative target hair follicle point cloud to obtain a first registration relationship, wherein the preoperative target hair follicle point cloud is obtained according to a head model and implant information; And displaying the hair follicle change after the hair implantation operation or the hair extraction operation in the head model in real time based on the first registration relation.
9. The method of hair implantation as set forth in claim 8, wherein before registering the initial real-time hair follicle point cloud with the preoperative target hair follicle point cloud, further comprising: adjusting the pose of the head model based on the pose of the binocular camera; A preoperative target hair follicle point cloud is determined based on the head model after the pose adjustment and the implant information.
10. The hair transplantation method according to claim 9, wherein said adjusting the pose of said head model based on the pose of said binocular camera comprises: Acquiring the gesture of a binocular camera, and generating the gesture of a virtual camera based on the gesture of the binocular camera; rendering the head model based on the gesture of the virtual camera to obtain a reference rendering diagram; acquiring a real-time image acquired by the binocular camera; Continuing to adjust the pose of the head model when the similarity between the reference rendering map and the real-time image is greater than or equal to a threshold; when the similarity between the reference rendering map and the real-time image is smaller than the threshold value, the posture adjustment of the head model is completed.
11. The method of claim 10, wherein the determining the preoperative target hair follicle point cloud based on the adjusted head model and the implant information comprises: determining a segmentation plane based on a target symmetry plane of the head model; and moving the segmentation plane along the direction perpendicular to the segmentation plane until the reference rendering graph is changed, acquiring the position of the segmentation plane, segmenting the head model through the acquired segmentation plane, acquiring a target part of the head model after segmentation, and determining preoperative target hair follicle point cloud based on the target part and implantation information.
12. The method according to claim 9, wherein the plant message includes a plant message area and a take-out area, and wherein the control mechanism arm performs the plant message operation or the take-out operation based on the plant message information, and further comprising: dividing the hair planting area and the hair taking area into areas to obtain subareas; And the first point of the current subarea is determined based on the last point of the last subarea, and the first point of the first subarea is randomly determined or is determined based on the point of the leftmost upper corner of the first subarea.
13. A plant information generating apparatus, the apparatus comprising: The head model generation module is used for acquiring a head video of the object to be planted, and performing head modeling on the head video to obtain a head model; The first posture correction module is used for correcting the posture of the head model to obtain a third head model; The original hair follicle point cloud reconstruction module is used for reconstructing hair follicles of the third head model to obtain an original hair follicle point cloud; The hairline estimation module is used for carrying out face gesture detection on the head video to determine a target face image, and comprises the steps of carrying out face gesture detection on each frame in a video stream of the head video, selecting a frame with the smallest difference compared with a reference as an optimal face image, and taking the optimal face image as a target face image; The plant information determining module is used for determining plant information based on the original hair follicle point cloud and the target hairline position; The first posture correction module is specifically configured to perform posture correction on the head model based on an initial symmetry plane to obtain a first head model, perform posture correction on the first head model based on a target symmetry plane to obtain a second head model, perform posture correction on the second head model based on face information and rotation information corresponding to the second head model to obtain a third head model, so that a forward direction of a Y axis of a corresponding side reference coordinate axis of the third head model passes through rotation of a head of the third head model, and a forward direction of a Z axis passes through a face of the third head model, wherein the initial symmetry plane is obtained based on the head model and a face mirror model of the head model, a Y axis of the corresponding side reference coordinate axis of the head model passes through a top end of the head model, an X axis passes through a side where eyes of the head model are located, and a Z axis passes through a side where ears of the head model are located, and the target symmetry plane is obtained based on a nose point of the first head model and a nose point of the face mirror model of the first head model.

Description

Method and device for planting message Technical Field The application relates to the technical field of intelligent medical treatment, in particular to a planting and messaging method, a planting and messaging method and a planting and messaging device. Background Since hair loss changes the appearance of a person, many hair loss patients treat hair loss through hair-planting surgery. In the prior art, most hair implantation operations are designed manually by doctors, namely, preoperative doctors detect hair follicles of patients so as to know the conditions and density distribution of the hair follicles of the heads of the patients, and design an hairline according to the face shape of the patients, so that hair taking areas, hair implantation areas, the number of hair follicles to be transplanted and the like are planned, and in the operation, the doctors extract and separate the hair follicles by using a hair follicle transplanting device through naked eye observation and self experience. When the number of hair follicles reaches the number required to be transplanted, the extracted hair follicles are planted in the hair planting area. However, doctors can only judge the overall density and health condition of hair follicles in the area to plan, and can not plan each hair follicle, so that the obtained hair implantation information is inaccurate. Disclosure of Invention In view of the foregoing, it is desirable to provide a method, a device, a computer device, and a storage medium for generating a message, which can improve the accuracy of the message. In a first aspect, the present application provides a method of generating a plant information, the method comprising: acquiring a head video of an object to be planted, and performing head modeling on the head video to obtain a head model; Reconstructing hair follicles of the head model to obtain an original hair follicle point cloud; estimating a hairline based on the head video to determine a target hairline position; And determining the hair planting information based on the original hair follicle point cloud and the target hairline position. In one embodiment, before the reconstructing the hair follicle of the head model to obtain the original hair follicle point cloud, the method further includes: -pose correction of the head model by at least one of: Performing posture correction on the head model based on initial symmetry, or performing posture correction on the head model based on target symmetry, or performing posture correction on the head model based on face information and spin information corresponding to the head model; The initial symmetry plane is obtained based on the head model and a face mirror model of the head model, the target symmetry plane is obtained by a first head model and a face mirror model of the first head model, and the first head model is the head model subjected to posture correction according to the initial symmetry plane. In one embodiment, before the posture correction of the head model based on the initial symmetry plane, the method further includes: performing principal component analysis on the head model to determine a reference coordinate axis, wherein a Y axis of the reference coordinate axis passes through the top end of the head model; determining a face position of the head model; Acquiring a first mirror image model of the head model about a first coordinate plane, wherein the first coordinate plane is determined by a reference coordinate axis passing through a human face and a Y axis; registering the head model and the first mirror model; An initial symmetry plane is determined based on the registered head model and the first mirror model. In one embodiment, before the posture correction is performed on the head model based on the object symmetry, the method further includes: Carrying out posture adjustment on the initial symmetrical plane so that the initial symmetrical plane coincides with a second coordinate plane to obtain a posture adjustment relation, wherein the second coordinate plane is determined by a Y axis and a Z axis; performing posture adjustment on the head model according to the posture adjustment relation to obtain a second head model; acquiring a second mirror image model of the second head model with respect to a second coordinate plane; Respectively acquiring nose points of the second head model and the second mirror image model; Moving the second head model and the second mirror model such that nose points of the second head model and the second mirror model coincide; and determining a target symmetrical plane based on the second head model and the second mirror image model after nose tip points are overlapped. In one embodiment, the determining the target hairline position based on the hairline estimation performed by the head video includes: Detecting the facial gestures of the head video to determine a target facial image; and estimating the hairline according to the target face i