Search

US-12619106-B2 - Method and apparatus for determining a fit of a visual equipment

US12619106B2US 12619106 B2US12619106 B2US 12619106B2US-12619106-B2

Abstract

The present disclosure relates to a method and device for evaluating a fit of visual equipment. In an embodiment, the method ( 105 ) includes receiving ( 110 ) an image(s) of a face of a user, identifying ( 115 ) a subset of eyeglasses from a database that satisfy a preliminary fit threshold, determining ( 120 ) an eyeglass rim plane of a candidate eyeglass, determining ( 125 ) an eyeglass pads plane of the candidate eyeglass, generating ( 130 ) at least one 2D profile of a nose of the user, generating ( 140 ) at least one 2D profile of a nasal section of a frame of the candidate eyeglass, virtually docking ( 150 ) the candidate eyeglass on the nose of the face of the user, calculating ( 170 ), when the virtually docked candidate eyeglass satisfies a secondary fit threshold, a final fit of the virtually docked candidate eyeglass, the final fit being defined by at least one of a comfort criterion and/or an aesthetic criterion.

Inventors

  • Julien ANDOCHE
  • Estelle NETTER
  • Fabien Muradore
  • Xin Pei ZHOU

Assignees

  • ESSILOR INTERNATIONAL

Dates

Publication Date
20260505
Application Date
20211229
Priority Date
20210128

Claims (15)

  1. 1 . A method for determining a fit of a visual equipment, comprising: receiving, by processing circuitry, at least one image of a face of a user; identifying, by the processing circuitry, a subset of eyeglasses from a database of visual equipment that satisfy a preliminary fit threshold, the preliminary fit threshold being a deviation limit of a parameter of the eyeglasses from a corresponding anatomical parameter of the face of the user, wherein whether an eyeglass of the database does or does not satisfy the preliminary fit threshold is determined by comparing the parameter of the eyeglasses with the corresponding anatomical parameter of the face of the user and by eliminating the eyeglass when the preliminary fit threshold is exceeded, the comparing being related to comfort, which is based on an adaptation of the eyeglasses to a facial width of the user, and/or being related to aesthetics, which is based on an adaptation of the eyeglasses to a user facial shape and/or skin color and/or eye color; determining, by the processing circuitry and for a candidate eyeglass of the subset of eyeglasses, an eyeglass rim plane defined by a pantoscopic angle; determining, by the processing circuitry and based on the eyeglass rim plane, an eyeglass pads plane that intersects eyeglass pads of the candidate eyeglass, the eyeglass pads plane being situated at a position offset from the eyeglass rim plane by a predetermined distance; generating, by the processing circuitry, at least one two-dimensional (2D) profile of a nose of the user based on one or more facial landmarks detected in the at least one image of the face of the user and a three-dimensional (3D) nose contour estimated from the at least one image of the face of the user, the at least one 2D profile of the nose of the user including at least one of: a first one of the at least one 2D profile of the nose of the user being defined as an intersection of a plane parallel to the eyeglass pads plane and the estimated 3D nose contour, the first one of the at least one 2D profile being situated at a predetermined distance from a facial reference landmark of the one or more facial landmarks, and a second one of the at least one 2D profile of the nose of the user being defined as an intersection of a plane parallel to the eyeglass rim plane and the estimated 3D nose contour, the second one of the at least one 2D profile being situated relative to the first one of the at least one 2D profile based on the predetermined distance of the offset of the eyeglass pads plane from the eyeglass rim plane; generating, by the processing circuitry and for the candidate eyeglass of the subset of eyeglasses, at least one 2D profile of a nasal section of a frame of the candidate eyeglass, a first one of the at least one 2D profile of the nasal section of the frame being associated with eyeglass pads of the candidate eyeglass and a second one of the at least one 2D profile of the nasal section of the frame being associated with an eyeglass rim of the candidate eyeglass; virtually docking, by the processing circuitry and based on the at least one 2D profile of the nose of the user and the at least one 2D profile of the nasal section of the frame of the candidate eyeglass, the candidate eyeglass on the nose of the face of the user; comparing, by the processing circuitry, locations of a feature of an eyeglass rim of the virtually docked candidate eyeglass to a subset of the one or more facial landmarks detected in the at least one image of the face of the user; and calculating, by the processing circuitry and when the virtually docked candidate eyeglass satisfies a secondary fit threshold representing a distance between global reference positions of the eyeglass and the face of the user, a final fit of the virtually docked candidate eyeglass on the nose of the user, the final fit of the virtually docked candidate eyeglass being a score, which is a result of a scoring function obtained by: defining comfort and/or aesthetic criteria, defining an evaluation function measuring the defined comfort and/or aesthetic criteria, wherein inputs include one or more morphological features of the face of the user, or of the eyeglass, defining a target value representing an ideal result of the evaluation function, and defining the scoring function that evaluates the distance between a result of the evaluation function and the ideal result.
  2. 2 . The method of claim 1 , further comprising combining, by the processing circuitry and in order to calculate the final fit of the virtually docked candidate eyeglass on the nose of the user, calculated values for each of a plurality of comfort criteria and/or calculated values for each of a plurality of aesthetic criteria.
  3. 3 . The method of claim 2 , wherein the combining includes calculating, by the processing circuitry, a weighted sum of the calculated values for each of the plurality of comfort criteria, and/or calculating, by the processing circuitry, a weighted sum of the calculated values for each of the plurality of aesthetic criteria.
  4. 4 . The method of claim 1 , further comprising iteratively performing, by the processing circuitry and for each remaining eyeglass of the subset of eyeglasses, the determining the eyeglass rim plane, the determining the eyeglass pads plane, the generating the at least one 2D profile of the nasal section of the frame of the candidate eyeglass, the virtually docking, the comparing, and the calculating.
  5. 5 . The method of claim 4 , further comprising ranking, by the processing circuitry, each virtually docked candidate eyeglass that satisfies the secondary fit threshold based on a comparison of a final fit of each virtually docked candidate eyeglass.
  6. 6 . The method of claim 5 , further comprising generating, by the processing circuitry and based on the ranking, a visual equipment recommendation for the user.
  7. 7 . The method of claim 4 , wherein the at least one 2D profile of the nasal section of the frame of the candidate eyeglass includes a plurality of 2D profiles associated with positions of mobile eyeglass pads of the candidate eyeglass, each of the plurality of 2D profiles being a configuration of the candidate eyeglass and representing a possible position of the mobile eyeglass pads of the candidate eyeglass.
  8. 8 . The method of claim 7 , wherein the determining the eyeglass pads plane, the virtually docking, the comparing, and the calculating are performed for each configuration of the candidate eyeglass, each configuration of the candidate eyeglass being an additional candidate eyeglass.
  9. 9 . The method of claim 1 , wherein a position of the virtually docked candidate eyeglass is calculated by optimizing a relationship between the first one of the at least one 2D profile of the nose of the user and the first one of the at least one 2D profile of the nasal section of the frame associated with the eyeglass pads of the candidate eyeglass in view of a second one of the at least one 2D profile of the nose of the user and the second one of the at least one 2D profile of the nasal section of the frame associated with the eyeglass rim of the candidate eyeglass, the second one of the at least one 2D profile of the nose of the user being associated with an intersection of the eyeglass rim plane and the estimated 3D nose contour.
  10. 10 . The method of claim 1 , wherein the generating the first one of the at least one 2D profile of the nose of the user includes generating, by the processing circuitry, a subsequent one of the at least one 2D profile of the nose of the user corresponding to a posterior nose plane of the user, the posterior nose plane being associated with an axis extending through a most posterior aspect of the nose of the user, and performing, by the processing circuitry, a transformation on the subsequent one of the at least one 2D profile of the nose of the user, a result of the transformation being the first one of the at least one 2D profile of the nose of the user, the transformation including a nose splay angle.
  11. 11 . The method of claim 1 , wherein an anatomical characteristic of the user is determined as a distance between temples of the user or as a distance between ears of the user.
  12. 12 . The method of claim 1 , wherein the eyeglass rim plane is further defined by a vertex distance.
  13. 13 . An apparatus for determining a fit of a visual equipment, comprising: processing circuitry configured to receive at least one image of a face of a user, identify a subset of eyeglasses from a database of visual equipment that satisfy a preliminary fit threshold, the preliminary fit threshold being a deviation limit of a parameter of the eyeglasses from a corresponding anatomical parameter of the face of the user, wherein whether an eyeglass of the database does or does not satisfy the preliminary fit threshold is determined by comparing the parameter of the eyeglasses with the corresponding anatomical parameter of the face of the user and by eliminating the eyeglass when the preliminary fit threshold is exceeded, the comparing being related to comfort, which is based on an adaptation of the eyeglasses to a facial width of the user, and/or being related to aesthetics, which is based on an adaptation of the eyeglasses to a user facial shape and/or skin color and/or eye color, determine, for a candidate eyeglass of the subset of eyeglasses, an eyeglass rim plane defined by a pantoscopic angle, determine, based on the eyeglass rim plane, an eyeglass pads plane that intersects eyeglass pads of the candidate eyeglass, the eyeglass pads plane being situated at a position offset from the eyeglass rim plane by a predetermined distance, generate at least one two-dimensional (2D) profile of a nose of the user based on one or more facial landmarks detected in the at least one image of the face of the user and a three-dimensional (3D) nose contour estimated from the at least one image of the face of the user, the at least one 2D profile of the nose of the user including at least one of: a first one of the at least one 2D profile of the nose of the user being defined as an intersection of a plane parallel to the eyeglass pads plane and the estimated 3D nose contour, the first one of the at least one 2D profile being situated at a predetermined distance from a facial reference landmark of the one or more facial landmarks, and a second one of the at least one 2D profile of the nose of the user being defined as an intersection of a plane parallel to the eyeglass rim plane and the estimated 3D nose contour, the second one of the at least one 2D profile being situated relative to the first one of the at least one 2D profile based on the predetermined distance of the offset of the eyeglass pads plane from the eyeglass rim plane, generate, for the candidate eyeglass of the subset of eyeglasses, at least one 2D profile of a nasal section of a frame of the candidate eyeglass, a first one of the at least one 2D profile of the nasal section of the frame being associated with eyeglass pads of the candidate eyeglass and a second one of the at least one 2D profile of the nasal section of the frame being associated with an eyeglass rim of the candidate eyeglass, virtually dock, based on the at least one 2D profile of the nose of the user and the at least one 2D profile of the nasal section of the frame of the candidate eyeglass, the candidate eyeglass on the nose of the face of the user, compare locations of a feature of an eyeglass rim of the virtually docked candidate eyeglass to a subset of the one or more facial landmarks detected in the at least one image of the face of the user, and calculate, when the virtually docked candidate eyeglass satisfies a secondary fit threshold representing a distance between global reference positions of the eyeglass and the face of the user, a final fit of the virtually docked candidate eyeglass on the nose of the user, the final fit of the virtually docked candidate eyeglass being a score, which is a result of a scoring function obtained by: defining comfort and/or aesthetic criteria, defining an evaluation function measuring the defined comfort and/or aesthetic criteria, wherein inputs include one or more morphological features of the face of the user, or of the eyeglass, defining a target value representing an ideal result of the evaluation function, and defining the scoring function that evaluates the distance between a result of the evaluation function and the ideal result.
  14. 14 . The apparatus of claim 13 , wherein the processing circuitry is further configured to rank each virtually docked candidate eyeglass that satisfies the secondary fit threshold based on a comparison of a final fit of each virtually docked candidate eyeglass, and generate, based on the ranking, a visual equipment recommendation for the user.
  15. 15 . A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by processing circuitry of a computer, cause the computer to perform a method for determining a fit of a visual equipment, the method comprising: receiving at least one image of a face of a user; identifying a subset of eyeglasses from a database of visual equipment that satisfy a preliminary fit threshold, the preliminary fit threshold being a deviation limit of a parameter of the eyeglasses from a corresponding anatomical parameter of the face of the user, wherein whether an eyeglass of the database does or does not satisfy the preliminary fit threshold is determined by comparing the parameter of the eyeglasses with the corresponding anatomical parameter of the face of the user and by eliminating the eyeglass when the preliminary fit threshold is exceeded, the comparing being related to comfort, which is based on an adaptation of the eyeglasses to a facial width of the user, and/or being related to aesthetics, which is based on an adaptation of the eyeglasses to a user facial shape and/or skin color and/or eye color; determining, for a candidate eyeglass of the subset of eyeglasses, an eyeglass rim plane defined by a pantoscopic angle; determining, based on the eyeglass rim plane, an eyeglass pads plane that intersects eyeglass pads of the candidate eyeglass, the eyeglass pads plane being situated at a position offset from the eyeglass rim plane by a predetermined distance; generating at least one two-dimensional (2D) profile of a nose of the user based on one or more facial landmarks detected in the at least one image of the face of the user and a three-dimensional (3D) nose contour estimated from the at least one image of the face of the user, the at least one 2D profile of the nose of the user including at least one of: a first one of the at least one 2D profile of the nose of the user being defined as an intersection of a plane parallel to the eyeglass pads plane and the estimated 3D nose contour, the first one of the at least one 2D profile being situated at a predetermined distance from a facial reference landmark of the one or more facial landmarks, and a second one of the at least one 2D profile of the nose of the user being defined as an intersection of a plane parallel to the eyeglass rim plane and the estimated 3D nose contour, the second one of the at least one 2D profile being situated relative to the first one of the at least one 2D profile based on the predetermined distance of the offset of the eyeglass pads plane from the eyeglass rim plane; generating, for the candidate eyeglass of the subset of eyeglasses, at least one 2D profile of a nasal section of a frame of the candidate eyeglass, a first one of the at least one 2D profile of the nasal section of the frame being associated with eyeglass pads of the candidate eyeglass and a second one of the at least one 2D profile of the nasal section of the frame being associated with an eyeglass rim of the candidate eyeglass; virtually docking, based on the at least one 2D profile of the nose of the user and the at least one 2D profile of the nasal section of the frame of the candidate eyeglass, the candidate eyeglass on the nose of the face of the user; comparing locations of a feature of an eyeglass rim of the virtually docked candidate eyeglass to a subset of the one or more facial landmarks detected in the at least one image of the face of the user; and calculating, when the virtually docked candidate eyeglass satisfies a secondary fit threshold representing a distance between global reference positions of the eyeglass and the face of the user, a final fit of the virtually docked candidate eyeglass on the nose of the user, the final fit of the virtually docked candidate eyeglass being a score, which is a result of a scoring function obtained by: defining comfort and/or aesthetic criteria, defining an evaluation function measuring the defined comfort and/or aesthetic criteria, wherein inputs include one or more morphological features of the face of the user, or of the eyeglass, defining a target value representing an ideal result of the evaluation function, and defining the scoring function that evaluates the distance between a result of the evaluation function and the ideal result.

Description

BACKGROUND Field of the Disclosure The present disclosure relates to a method for determining a virtual fit of an eyewear on a face of a user based on comfort and aesthetics. A properly fitting eyewear is an essential step to ensuring desired optical performance. Description of the Related Art Selecting new eyewear can be daunting, as brick and mortar retailers, online retailers, and combinations thereof offer myriad options for your eye care needs. Brick and mortar retailers fail to provide sufficient inventory and convenience for the modern consumer. Online businesses strive to provide consumer-specific recommendations, but are often unable to determine a best fit of a given eyewear product for a consumer due to a lack of physical interaction with the physical proportions of the consumer. Similarly, consumer uncertainty regarding fit often leads to in-person try-ons and fittings. While providing increased consumer confidence for their eyewear purchase when a good fit can be found, limiting a consumer to in-store inventory and the pressures associated with a public try-on can complicate and dampen the shopping experience. Document FR 3 065 821 A1 discloses a device for obtaining measurements and 3D images for determining morphological parameters of the face in order to make made-to-measure eyeglasses. Document WO 2015/027196 A1 discloses a method and system to create custom products from scratch. In order to exploit the convenience and inventory of online eye care services, a new approach to determining fit of eyeglasses, or visual equipment, is needed. If a fit quality of a virtual try-on of an eyeglass frame can be determined, then it may be possible to recommend an eyeglass with confidence. Thus, in order to provide an online eye care experience that generates personalized product recommendations and guarantees a good fit, a new approach is required. The foregoing “Background” description is for the purpose of generally presenting the context of the disclosure. Work of the inventors, to the extent it is described in this background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly or impliedly admitted as prior art against the present disclosure. SUMMARY The present disclosure relates to an apparatus and method for determining a fit of a visual equipment. According to an embodiment, the present disclosure further relates to a method for determining a fit of a visual equipment according to claim 1. According to an embodiment, the present disclosure further relates to an apparatus for determining a fit of a visual equipment according to claim 13. The foregoing paragraphs have been provided by way of general introduction, and are not intended to limit the scope of the following claims. The described embodiments, together with further advantages, will be best understood by reference to the following detailed description taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein: FIG. 1 is a flow diagram of a method of determining a fit of a visual equipment, according to an exemplary embodiment of the present disclosure; FIG. 2A is an illustration of a step of a method of determining a fit of a visual equipment, according to an exemplary embodiment of the present disclosure; FIG. 2B is an illustration of a step of a method of determining a fit of a visual equipment, according to an exemplary embodiment of the present disclosure; FIG. 2C is an illustration of a step of a method of determining a fit of a visual equipment, according to an exemplary embodiment of the present disclosure; FIG. 3A is a flow diagram of a sub process of a method of determining a fit of a visual equipment, according to an exemplary embodiment of the present disclosure; FIG. 3B is an illustration of a step of a sub process of a method of determining a fit of a visual equipment, according to an exemplary embodiment of the present disclosure; FIG. 3C is an illustration of a step of a sub process of a method of determining a fit of a visual equipment, according to an exemplary embodiment of the present disclosure; FIG. 3D is an illustration of a step of a sub process of a method of determining a fit of a visual equipment, according to an exemplary embodiment of the present disclosure; FIG. 3E is an illustration of a step of a sub process of a method of determining a fit of a visual equipment, according to an exemplary embodiment of the present disclosure; FIG. 4A is a flow diagram of a sub process of a method of determining a fit of a visual equipment, according to an exemplary embodiment of the present disclosure; FIG. 4B is an illustration of a step of