Search

CN-116168417-B - Nail recognition and positioning method and system

CN116168417BCN 116168417 BCN116168417 BCN 116168417BCN-116168417-B

Abstract

The invention discloses a nail identifying and positioning method and system, comprising the steps of S1, shooting by an identifying device to obtain first image information of a working area, sending the first image information to a target detection network to generate corresponding second image information, dividing the first image information, comparing the second image information with the target detection network to obtain nail mask images, S2, calculating the original 3D point cloud and the nail mask images to obtain real coordinates of the nail point cloud, S3, converting the nail point cloud into a calibration coordinate system, determining the calibration coordinate system by setting the position of a printing device below the identifying device, S4, traversing each nail to obtain nail angle information according to the point cloud information of the nail, S5, projecting the point cloud coordinates of the nail onto a two-dimensional plane according to camera external parameters to obtain the deflection angle of each nail, and calculating the nail mask images and the original 3D point cloud by the device to obtain the real point cloud coordinates of each nail, wherein the shape of the nail can be embodied by a plurality of point cloud coordinates.

Inventors

  • XIANG YONG
  • YAN JIN

Assignees

  • 上海魅奈儿科技有限公司

Dates

Publication Date
20260512
Application Date
20221229

Claims (9)

  1. 1. A method of nail identification positioning, comprising: S1, a recognition device photographs to obtain first image information of a working area, locates the nail position in the first image information through a target detection network, generates corresponding second image information, transmits the second image information to a segmentation network, segments the second image information into a plurality of areas, and compares the first image information with the segmented second image information after a probability value is preset to obtain a nail mask image of the first image information; S2, calculating to obtain real coordinates of the nail point cloud through the original 3D point cloud and the nail mask image, and carrying out statistical filtering on the nail point cloud to eliminate noise points of the point cloud; s3, converting the nail point cloud into a calibration coordinate system, wherein the calibration coordinate system is determined by setting the position of the printing device below the identification device; s4, traversing each nail, and obtaining the highest point, the left-right inclination angle and the front-back inclination angle information of the nail according to the point cloud information of the nail; s5, projecting the point cloud coordinates of the nails onto a two-dimensional plane according to camera external parameters, performing Gaussian filtering to eliminate possible noise points, and acquiring deflection angles of each nail according to a two-dimensional image; In the step S4, further includes: S41, traversing the nails according to the point clouds of each nail, and combining the point cloud data to obtain the basic data of the nails; S42, after traversing the point cloud data, performing data arrangement on the point cloud of the nails to obtain the highest point, the left-right inclination angle and the front-back inclination angle of the point cloud of the nails.
  2. 2. The method for nail recognition positioning according to claim 1, further comprising, in said step S1: S11, the recognition device photographs the finger position in the working area, and the obtained first image information is sent to the target detection network to position the fingernail part in the first image information to generate corresponding second image information, wherein the second image information mainly comprises the image information of the fingernail part; And S12, sending the second image information to a segmentation network, presetting a judgment probability value, acquiring the probability value of each pixel belonging to the nail region in the original image of the hand, marking the pixel points with the probability value larger than the preset probability value as the nail region, and marking the pixel points with the probability value smaller than the preset probability value as the non-nail region, so as to obtain the nail mask image of the first image information.
  3. 3. The method for nail recognition positioning according to claim 1, further comprising, in said step S2: S21, calculating the nail mask image and original 3D point cloud data to obtain real point cloud coordinates of nails, wherein the original 3D point cloud data is set in the real world; s22, filtering each nail mask image, eliminating noise points of point clouds, and feeding back the accuracy of image information through the point clouds.
  4. 4. The method for nail recognition positioning according to claim 1, further comprising, in said step S3: s31, setting the initial position of the printing device below the identification device as a calibration coordinate system according to the identification device; and S32, transmitting the real point cloud of the nail to the calibration coordinate system, wherein the calibration coordinate system is obtained by combining the real point cloud.
  5. 5. The method for nail recognition positioning according to claim 1, further comprising, in said step S5 S51, projecting the point cloud coordinates of the nails onto a two-dimensional plane according to external parameters of a camera, performing Gaussian filtering to eliminate possible noise points, and acquiring deflection angles of each nail according to a two-dimensional image; S52, extracting nail mask images of all nails in advance, extracting image contours of all nail masks through an image processing function library, fitting by using a minimum rectangle, and returning to a rotation angle of the minimum rectangle with an upper left vertex as an origin after fitting is completed; s53, obtaining a minimum rectangle of the nail outline through an image processing function, obtaining left and right and upper and lower endpoints of the nail according to the intersection point of the rectangle and the nail outline, calculating left and right inclination angles of the nail according to point cloud coordinates (x, y, z) of the left and right endpoints, and calculating the angle of the upward and downward inclination according to the point cloud coordinates of the upper and lower endpoints.
  6. 6. A method for nail recognition positioning according to claim 1, wherein said segmentation network comprises Unet, pspnet, deeplab series, and wherein said second image information is segmented and compared.
  7. 7. A method of nail recognition positioning according to claim 1, wherein the recognition device is one of a 3D camera, RGB binocular, lidar or tof camera.
  8. 8. A system for nail identification and location, said system being adapted for use in a method as claimed in any one of claims 1 to 7, comprising The image processing module (1) is used for receiving the first image information sent by the identification device and generating second image information according to the nail part corresponding to the first image information; The communication module (2) can send the second image information to a segmentation network through the communication module, segment the nail part image in the second image information and transmit the segmented nail mask image back to the image processing module; The control module (3) is used for presetting a probability value, comparing the segmented second image information with the first image information, judging the probability value that the corresponding pixel belongs to the nail region, marking the pixel point with the probability value larger than the preset probability value as the nail region, marking the pixel point with the probability value smaller than the preset probability value as the non-nail region, and obtaining a nail mask image of the first image information; the 3D point cloud operation module (4) combines the nail mask image with the original 3D point cloud to obtain real point cloud coordinates of the nail, and converts the nail point cloud into a calibration coordinate system to enable the printing device to print according to the position of the calibration coordinate system; And the information processing module (5) is used for collecting and processing the calculated or converted 3D point cloud coordinates and acquiring the deflection angle according to the 3D point cloud coordinates.
  9. 9. A nail recognition positioning system according to claim 8, wherein said calibration coordinate system is an xy plane, parallel to the plane of the printing device.

Description

Nail recognition and positioning method and system Technical Field The invention relates to the technical field of intelligent nail art, in particular to a nail recognition and positioning method and system. Background At present, most intelligent nail art machines only print to single nail face, and nail art machine need scribble nail polish glue in advance at the nail at first before preparing the nail art, and nail polish glue colour is different with hand support surrounding environment and finger colour, then discerns the profile of nail, has seriously restricted time and recognition accuracy of nail art. And since printing is performed only for a single nail, there is no need to recognize the rotation angle and inclination of the nail face. At present, a scheme is adopted in which four fingers (four fingers, namely an index finger, a middle finger, a ring finger and a little finger) and thumbs are arranged on different planes, namely according to the structure of a human hand, a left thumb is arranged on the right side of a palm and is vertical to the plane where the four fingers are located, and a right thumb is arranged on the left side of the palm and is vertical to the plane where the four fingers are located. In addition, in the prior art, nail polish is coated on the nail surface in advance, the color of the nail polish is different from the surrounding environment of the hand support and the color of the finger, and is usually white, and then the nail polish is identified, so that the steps are complicated, and the nail polish time is increased. It should be noted that, in the patent with publication number CN113469093a, the applicant applied for a nail recognition method and system based on deep learning, by acquiring the original image of the hand in advance, acquiring the probability value of each pixel belonging to the nail region in the original image of the hand, and acquiring the nail mask image of the original image of the hand, extracting the outline coordinates of the nail in the nail mask image, and acquiring the deflection angle and inclination information of each nail, the nail polish and the anti-overflow glue do not need to be coated on the nail in advance before nail recognition, and the nail recognition method and system can simultaneously recognize a plurality of nails, and the recognition precision is higher, and the whole required time is less, so that the problems in the technology can be solved, but the acquired nail mask image is a two-dimensional image, and the position and the region of the nail are reflected through the two-dimensional mask image. The problem of the patent is that the nail is an arc-shaped surface, each finger is not right-side-up, a certain inclination angle exists, rotation information is contained in depth (Z axis), a two-dimensional image cannot be processed, meanwhile, the height of each finger nail is completely different, the two-dimensional information cannot judge the height without Z axis data, serious errors can be generated in the spraying process of the lack of Z axis data, and the nail beautifying effect is affected. The slope of the nail in the X and Y axes is referred to as the rotation angle, and the slope of the nail in the Z axis is referred to as the tilt angle. Disclosure of Invention The invention aims to provide a nail recognition positioning method and system aiming at the problem that errors occur due to the fact that an inclination angle and a height difference exist between the existing nail recognition products in the nail recognition spraying process. In order to achieve the above purpose, the invention adopts the following technical scheme that the nail identifying and positioning method comprises the following steps: S1, a recognition device photographs to obtain first image information of a working area, locates the nail position in the first image information through a target detection network, generates corresponding second image information, transmits the second image information to a segmentation network, segments the second image information into a plurality of areas, and compares the first image information with the segmented second image information after a probability value is preset to obtain a nail mask image of the first image information; S2, calculating to obtain real coordinates of the nail point cloud through the original 3D point cloud and the nail mask image, and carrying out statistical filtering on the nail point cloud to eliminate noise points of the point cloud; s3, converting the nail point cloud into a calibration coordinate system, wherein the calibration coordinate system is determined by setting the position of the printing device below the identification device; s4, traversing each nail, and obtaining the highest point, the left-right inclination angle, the front-back inclination angle and other information of the nail according to the point cloud information of the nail; And S5, projecting the