Search

CN-117079343-B - Control method of intelligent bathroom mirror

CN117079343BCN 117079343 BCN117079343 BCN 117079343BCN-117079343-B

Abstract

The invention discloses a control method of an intelligent bathroom mirror, which is used for controlling the intelligent bathroom mirror and comprises the following steps that step 1, an administrator collects user data and stores the data into a value user database, wherein the data comprises height, facial five sense organs and facial contour data; the invention integrates the controller, the power supply module, the three-dimensional camera, the touch screen, the lighting module, the defogging module and the communication module into the bathroom mirror to realize the automation and the intellectualization of the bathroom mirror detection, applies the human key points, the identity recognition algorithm, the high-precision Anchor-Free target detection algorithm, the behavior analysis and other artificial intelligent technologies to the bathroom mirror, can realize the operation detection of people in front of the bathroom mirror, the detection of the behavior mode, and the operation prompt according to the detection result, and the lighting module regulates the light according to the behavior mode, thereby greatly improving the intellectualization of the bathroom mirror.

Inventors

  • CHEN ZHEBO
  • Tao Shuxian
  • YU JIANAN
  • CHEN YIXIN
  • CHEN ZHENYUAN
  • WANG WANSHEN
  • CHEN LONGWEI
  • PAN LINGFENG
  • Ye Xuewang
  • WU DIWEI

Assignees

  • 浙江大学台州研究院

Dates

Publication Date
20260505
Application Date
20230717

Claims (8)

  1. 1. The control method of the intelligent bathroom mirror is characterized by comprising the following steps of: Step 1, an administrator collects user data and stores the data into a user database, wherein the data comprises height, facial features and facial contour data; step 2, real-time monitoring of the three-dimensional camera, and controlling the three-dimensional camera to acquire color images in front of the bathroom mirror in real time by the controller And depth image ; Step 3, a person enters a bathroom, whether the person moves in front of the bathroom mirror is judged, if yes, step 4 is executed, and if not, step 2 is executed; Step4, identifying the identity of the entrant; Step 5, obtaining the current time, and the lighting module is used for obtaining the time period of the current time Adjust the brightness to The color temperature is adjusted to ; Step 6, judging whether the mirror surface of the bathroom mirror and the lens of the three-dimensional camera need defogging, if so, executing the step 7 after executing the automatic defogging flow, and if not, executing the step 7; The automatic defogging process in the step 6 comprises the following steps: Step 6.1, detecting whether the mirror surface humidity of the bathroom mirror is larger than a mirror surface humidity threshold value by a humidity sensor, if so, judging that the mirror surface of the bathroom mirror needs to be defogged, executing step 6.4, and if not, executing step 6.2; step 6.2, the controller controls the defogging module to start, and the heating resistance wire defogs the mirror surface of the bathroom mirror; Step 6.3, detecting the mirror surface humidity of the bathroom mirror in real time by a humidity sensor, and when the mirror surface humidity is smaller than a mirror surface humidity threshold value, controlling the defogging module to stop by a controller, and executing the step 6.4; Step 6.4, calculating the definition index of the current three-dimensional camera lens Contrast sharpness index And sharpness threshold If (if) > Judging that the lens is clear, ending the step, if not, judging that the lens is fuzzy, and executing the step 6.5; in the above-mentioned (4), Is a color image Is defined by a coordinate system of the horizontal and vertical coordinates, To be at a coordinate point A corresponding gray value; Step 6.5, the controller controls the defogging module to start, and the heating resistance wire defogs the lens; step 6.6, calculating defogging definition index of lens Contrast defogging sharpness index And sharpness threshold If (if) > Judging that the lens is clear, step 6.8, ending the step, if not, comparing And (3) with If (if) < , If the difference is the definition difference threshold, judging that the poor definition of the lens is generated by dirt which cannot pass through the defogging module, judging that the lens is dirty, executing the step 6.7, and if not, continuing to execute the step 6.5; step 6.7, prompting a user to dirty and treat the lens by using the touch screen, and executing the step 6.6; Step 6.8, ending the step; step 7, processing the color image Acquiring face region ; Step 8, processing the color image Acquiring left hand region And right hand region ; Step 9, analyzing the behavior, judging whether the behavior is a cleaning behavior, if yes, executing the step 2 after the cleaning behavior is finished, and if not, executing the step 10; the specific step of judging whether the behavior is the cleaning behavior in the step 9 is as follows: step 9.1 calculating left hand region And right hand region Distance from lip region respectively And ; Is the left hand region of human body The actual distance of the center point from the center point of the lip region, Is the right hand region of human body The actual distance between the center point and the center point of the lip region is as follows On the image, the center point of the lip area is The coordinates are set as Left hand region Center point , The coordinates are set as Right hand region Center point , The coordinates are set as , 、 And The coordinates of (a) are given as a known parameter, 、 And At the position of The mapping points on are 、 And , Is the coordinates of (a) , Is the coordinates of (a) , Is the coordinates of (a) Obtainable according to the above formulae (1) and (2) 、 And And (3) with 、 And Mapping relation of coordinates; ; ; ; According to the above formulae (5), (6) and (7) 、 And Can obtain depth image In (a) 、 And Depth value from three-dimensional camera 、 、 ; Computing depth images With respect to the magnification of the actual photographic subject, The lateral image magnification is set to , The longitudinal image magnification is set to The physical field of view range of the actual shooting object is set as , In the form of a transverse width, Longitudinal height, can obtain ; ; In the above-mentioned (9), Is the lateral field angle of the three-dimensional camera, Is the longitudinal field angle of the three-dimensional camera, Is a three-dimensional camera and Is defined as the distance average of (a), Is a known parameter, according to the above formulas (8) and (9), can be obtained ; ; ; A world coordinate system is established with the actual photographing object, 、 And The point corresponding to the world coordinate system is set as 、 And According to the above formulas (11), (12) and (13), the reaction mixture is 、 And Coordinate conversion of (c) 、 And The coordinates of the world's coordinate system, Is the coordinates of (a) , Is the coordinates of (a) , Is the coordinates of (a) Can get ; ; Step 9.2 calculating left hand region And right hand region Overlap area and intersection ratio with lip region respectively; the area of the lip region is set as Left hand region Is set as the area of Left hand region The overlapping area with the lip area is set as Left hand region Is set to the cross-over ratio of Right hand region Is set as the area of Right hand region The overlapping area with the lip area is set as Right hand region Is set to the cross-over ratio of Can be obtained ; ; In the above formulas (15) and (16), 、 、 、 And Are all known parameters according to The image may be obtained in a form that, 、 、 、 Are all An area in the image; Step 9.3, analyzing the behaviors: If it is 、 Are all larger than , , Judging that the user is in a cleaning action; If it is Or/and (or) Greater than And is not greater than , And Greater than 0, determining to continue analysis; If it is Or/and (or) Less than , And Greater than Judging that the user is in a toilet behavior; For a maximum distance threshold of the hand to mouth region, at which distance the hand is considered to be non-interacting with the mouth, Is a set value; for a minimum distance threshold for the hand to mouth region, at which distance the hand is considered to have interaction with the mouth, Is a set value; is a threshold of the cross-over ratio; step 10, judging whether the hand holds the article, if so, executing step 12, entering an article identification process, and if not, executing step 11; step 11, judging whether an area needs to be amplified, if yes, executing an amplification flow, and if not, executing step 13; Step 12, the article identification model identifies the hand to hold the article, if the article identification model outputs 1, the article is judged to be a toothbrush and enters a tooth brushing behavior mode, if the article identification model outputs 2, the article is judged to be lipstick and enters a lipstick behavior mode, if the article identification model outputs 3, the article is judged to be a shaver and enters a shaving behavior mode; And step 13, judging whether a person is in the bathroom, if so, executing step 2, and if not, closing the lighting module, and ending the step.
  2. 2. The method for controlling the intelligent bathroom mirror according to claim 1, wherein the user-defined lighting module parameters in the step 1 are the brightness and the color temperature of the lamplight in different time periods and different behavior modes, and the controller controls the lighting module to be turned on and off, and turns on the brightness and the color temperature in the corresponding time period according to the current time, and then converts the brightness and the color temperature into the brightness and the color temperature set in the corresponding behavior mode according to the current behavior mode.
  3. 3. The method for controlling an intelligent bathroom mirror according to claim 1, wherein the step3 of determining whether the occupant is moving in front of the bathroom mirror comprises the following specific steps: step 3.1, calculating the distance between the entrant and the bathroom mirror ; Step 3.2, judging And a threshold distance between the entrant and the bathroom mirror If (if) Not greater than Step 3.3 is executed if Greater than Judging that the entrant does not move in front of the bathroom mirror; step 3.3, judging that the entering person is meeting Not greater than Activity time under the condition of (2) With an activity threshold time If (if) Not less than Judging that the entrant moves in front of the bathroom mirror, executing step 4, if Less than And (2) judging that the entrant does not move in front of the bathroom mirror, and executing the step (2).
  4. 4. The method for controlling an intelligent bathroom mirror according to claim 1, wherein the step 4 of identifying the identity of the user is specifically identifying a color image And (3) comparing the facial information with a user database to identify the user identity, if the user identity is identified, executing the step (5), if the user identity is not identified, judging that the entering person is a stranger, prompting whether the manager needs to add data by a touch screen, if so, acquiring the data of the stranger by the manager, and if not, executing the step (5).
  5. 5. The method for controlling an intelligent bathroom mirror according to claim 1, wherein the step 7 is performed with color image processing The middle face area specifically includes the following steps: Step 7.1 extracting color image Upper face key points 14-17, calculating minimum circumscribed rectangle, expanding each length and width by n pixel points to generate face region, and cutting face region to obtain face image : Step 7.2 for facial image Running a face_ landmark _localization neural network model, identifying 70 key points of the face, detecting the key points by adopting two cascaded CNNs, and identifying the outline, the eyebrow area, the nose area, the eye area and the lip area of the face to be in an oral area; and 7.3, obtaining a skin color area and a lip area according to the area identified in the step 7.2.
  6. 6. The method for controlling an intelligent bathroom mirror according to claim 1, wherein the color image is processed in the step 8 The middle hand region is specifically used for extracting color image The upper hand key points 4 and 7 calculate the minimum circumscribed rectangle, expand n pixel points each in length and width, generate left hand region And right hand region 。
  7. 7. The method for controlling an intelligent bathroom mirror according to claim 3, wherein the distance in the step 3.1 is as follows The calculation formula of (2) is as follows: color image The key points are marked as Depth image The key points are marked as , , And (3) with One-to-one correspondence; color image Is set to be The pixel height is set to Depth image Is set to be The pixel height is set to , 、 、 、 Is a known parameter, get ; Get the key point of P1 , Is the coordinates of (a) , Key point mapping Key point Is the coordinates of (a) , To obtain the known parameters which can be obtained according to (1) ; Obtaining the coordinates of the key points from the formula (2) According to Obtaining depth image Distance value of corresponding target in the middle , For the depth value of the key point distance three-dimensional camera of the actual shooting object, key points 0-7 and 14-17 are taken, 。
  8. 8. The method for controlling an intelligent bathroom mirror according to claim 1, wherein the amplifying process in the step 11 comprises the following specific steps: step 11.1, the hand of the user makes a circle at the position to be amplified in the face area; step 11.2, extracting a track of hand circle drawing; Acquiring real-time color images of hand circle from beginning to end Calculating the minimum circumscribed rectangle, expanding each length and width by n pixel points to generate a face area, and cutting the face area to obtain a face image Extracting color image The upper hand key points 4 and 7 calculate the minimum circumscribed rectangle, expand n pixel points each in length and width, generate left hand region And right hand region Detection of left hand region using a convolutionally gesture machine (CPM) And right hand region Extracting key points of fingers, and obtaining facial images Points overlapped with key points of the fingers, wherein a plurality of points form a hand circle-drawing track; step 11.3, fitting the extracted hand circle track with the horizontal minimum outer rectangle to form a track image, and separating the track image from the face image Cutting, performing secondary interpolation processing on the data of the track image, increasing resolution, and generating an amplified image; and 11.4, displaying the enlarged image on the touch screen.

Description

Control method of intelligent bathroom mirror Technical Field The invention belongs to the field of bathroom accessories, and relates to a control method of an intelligent bathroom mirror. Background The bathroom mirror is a mirror which is placed in a bathroom for people to comb and wash and is an integral part of the bathroom space. Along with popularization and application of intelligent household technology, bathroom mirrors also have intelligent application function, and intelligent bathroom mirrors on the market at present have LED lamp and touch switch function, adjust illumination colour temperature through touch switch, and whole function is single, and intelligent application is comparatively basic. Disclosure of Invention The invention provides a control method of an intelligent bathroom mirror in order to overcome the defects of the prior art. In order to achieve the purpose, the invention adopts the following technical scheme that the control method of the intelligent bathroom mirror is used for controlling the intelligent bathroom mirror and comprises the following steps of: step 1, an administrator collects user data and stores the data into a value user database, wherein the data comprises height, facial features and facial contour data; Step2, monitoring a three-dimensional camera in real time, and controlling the three-dimensional camera to acquire a color image Pm and a depth image Dm in front of a bathroom mirror in real time by a controller; Step 3, a person enters a bathroom, whether the person moves in front of the bathroom mirror is judged, if yes, step 4 is executed, and if not, step 2 is executed; Step4, identifying the identity of the entrant; Step 5, obtaining the current time, and adjusting the brightness to Lh and the color temperature to Kh by the lighting module according to the time period Th of the current time; step 6, judging whether the mirror surface of the bathroom mirror and the lens of the three-dimensional camera need defogging, if so, executing the step 7 after executing the automatic defogging flow, and if not, executing the step 7; step 7, processing the color image Pm to obtain a face area F1; step 8, processing the color image Pm to obtain a left hand region H1 and a right hand region H2; Step 9, analyzing the behavior, judging whether the behavior is a cleaning behavior, if yes, executing the step 2 after the cleaning behavior is finished, and if not, executing the step 10; step 10, judging whether the hand holds the article, if so, executing step 12, entering an article identification process, and if not, executing step 11; step 11, judging whether an area needs to be amplified, if yes, executing an amplification flow, and if not, executing step 13; Step 12, the article identification model identifies the hand to hold the article, if the article identification model outputs 1, the article is judged to be a toothbrush and enters a tooth brushing behavior mode, if the article identification model outputs 2, the article is judged to be lipstick and enters a lipstick behavior mode, if the article identification model outputs 3, the article is judged to be a shaver and enters a shaving behavior mode; And step 13, judging whether a person is in the bathroom, if so, executing step 2, and if not, closing the lighting module, and ending the step. Further, in the step 1, the parameters of the user-defined lighting module are the brightness and the color temperature of the lamplight in different time periods and different behavior modes, the time period is divided according to the user-defined setting, the brightness and the color temperature in different time periods are set according to the user-defined setting, the controller controls the lighting module to be turned on and off, the brightness and the color temperature in the corresponding time period are turned on according to the current time, and the brightness and the color temperature are converted into the brightness and the color temperature set according to the corresponding behavior mode according to the current behavior mode. Further, the specific step of determining whether the user moves in front of the bathroom mirror in the step3 is as follows: Step 3.1, calculating the distance d between the user and the bathroom mirror; step 3.2, judging d and the threshold distance S between the entrant and the bathroom mirror, if d is not more than S, Executing the step 3.3, if d is greater than S, judging that the entering person does not move in front of the bathroom mirror; And 3.3, judging the activity time delta t and the activity threshold time t of the entering person under the condition that d is not more than S, if delta t is not less than t, judging that the entering person is active in front of the bathroom mirror, executing step 4, and if delta t is less than t, judging that the entering person is not active in front of the bathroom mirror, and executing step 2. Further, the step 4 of identifying the ide