Search

CN-122004749-A - Fatigue detection method and device for head-mounted equipment

CN122004749ACN 122004749 ACN122004749 ACN 122004749ACN-122004749-A

Abstract

The embodiment of the disclosure discloses a fatigue detection method and device for head-mounted equipment, the method comprises the steps of obtaining facial data of a wearer of the head-mounted equipment, wherein the facial data comprise at least one of eye openness, eye blink frequency and mouth openness, obtaining a fatigue detection result of the wearer according to a preset fatigue detection model and the facial data, and outputting prompt information of the wearer in a fatigue state when the fatigue detection result is in the fatigue state, wherein the preset fatigue detection model comprises a first fatigue detection model, a second fatigue detection model and a third fatigue detection model, the first fatigue detection model reflects a mapping relation between the eye openness and the fatigue detection result, the second fatigue detection model reflects a mapping relation between the eye blink frequency and the fatigue detection result, and the third fatigue detection model reflects a mapping relation between the mouth openness and the fatigue detection result.

Inventors

  • GU MIN
  • WEN SHANSHAN

Assignees

  • 歌尔股份有限公司

Dates

Publication Date
20260512
Application Date
20241111

Claims (10)

  1. 1. A method of fatigue detection for a head-mounted device, the method comprising: Acquiring face data of a wearer of the head-mounted device, wherein the face data comprises at least one of eye openness, eye blink frequency and mouth openness; Obtaining a fatigue detection result of the wearer according to a preset fatigue detection model and the facial data; Outputting prompt information of the wearer in a fatigue state when the fatigue detection result is that the wearer is in the fatigue state; The preset fatigue detection model comprises a first fatigue detection model, a second fatigue detection model and a third fatigue detection model, wherein the first fatigue detection model reflects the mapping relation between the opening degree and the closing degree of eyes and the fatigue detection result, the second fatigue detection model reflects the mapping relation between the blink frequency of eyes and the fatigue detection result, and the third fatigue detection model reflects the mapping relation between the opening degree of mouth and the fatigue detection result.
  2. 2. The method of claim 1, wherein the headset includes a camera device, The acquiring facial data of a wearer of the headset device includes: acquiring a facial image of the wearer acquired by the image pickup device; And identifying the facial image and obtaining facial data of the wearer.
  3. 3. The method of claim 2, wherein the camera device comprises at least a first camera, a second camera, a third camera, and a fourth camera, the first camera is located at a first position of the headset, the second camera is located at a second position of the headset, the third camera is located at a third position of the headset, the fourth camera is located at a fourth position of the headset, and the first position, the second position, the third position, and the fourth position are different positions of the headset.
  4. 4. The method of claim 3, wherein the eye opening and closing degree comprises a left eye opening and closing degree and the eye blink frequency comprises a left eye blink frequency and a right eye blink frequency, wherein the mouth opening and closing degree comprises a left mouth angle opening and closing degree and a right mouth angle opening and closing degree, The identifying the facial image, obtaining facial data of the wearer, includes: Identifying a first facial image shot by the first camera to obtain the opening and closing degree of the left eye and the blink frequency of the left eye; identifying a second face image shot by the second camera to obtain the opening and closing degree of the right eye and the blink frequency of the right eye; Identifying a third face image shot by the third camera to obtain the opening and closing degree of the left mouth angle, and And identifying a fourth facial image shot by the fourth camera to obtain the right mouth angle opening and closing degree.
  5. 5. The method of claim 1, wherein the facial data comprises eye openness, The step of obtaining the fatigue detection result of the wearer according to a preset fatigue detection model and the face data comprises the following steps: Determining a vector value of a first feature vector for reflecting the fatigue detection result according to the eye opening and closing degree, wherein the first feature vector comprises at least one feature of current eye opening and closing degree, eye opening and closing degree change rate, accumulated eye opening and closing degree change, average eye opening and closing degree, eye opening and closing degree standard deviation, maximum eye opening and closing degree, minimum eye opening and closing degree and eye opening and closing degree fluctuation; And obtaining a fatigue detection result of the wearer according to the vector value and the first fatigue detection model.
  6. 6. The method of claim 1, wherein the facial data comprises eye blink frequency, The step of obtaining the fatigue detection result of the wearer according to a preset fatigue detection model and the face data comprises the following steps: Determining a vector value of a second feature vector for reflecting the fatigue detection result according to the eye blink frequency, wherein the second feature vector comprises at least one of a current eye blink frequency and a mean blink interval time; and obtaining a fatigue detection result of the wearer according to the vector value and the second fatigue detection model.
  7. 7. The method of claim 1, wherein the facial data comprises mouth opening and closing, The step of obtaining the fatigue detection result of the wearer according to a preset fatigue detection model and the face data comprises the following steps: Determining a vector value of a third feature vector for reflecting the fatigue detection result according to the mouth opening and closing degree, wherein the third feature vector comprises at least one feature of the current mouth opening and closing degree, the mouth opening and closing degree change rate, the accumulated mouth opening and closing degree change, the average mouth opening and closing degree, the mouth opening and closing degree standard deviation, the maximum mouth opening and closing degree, the minimum mouth opening and closing degree and the maximum mouth opening and closing degree; and obtaining a fatigue detection result of the wearer according to the vector value and the third fatigue detection model.
  8. 8. The method of claim 1, wherein the facial data comprises eye openness, eye blink frequency, and mouth openness, The step of obtaining the fatigue detection result of the wearer according to a preset fatigue detection model and the face data comprises the following steps: obtaining a first fatigue detection result of the wearer according to the first fatigue detection model and the eye opening and closing degree; Obtaining a second fatigue detection result of the wearer according to the second fatigue detection model and the eye blink frequency; Obtaining a third fatigue detection result of the wearer according to the third fatigue detection model and the mouth opening degree; And obtaining the fatigue detection result according to the first fatigue detection result, the second fatigue detection result and the third fatigue detection result.
  9. 9. The method according to claim 1, wherein the method further comprises: Acquiring a first training sample set, a second training sample set and a third training sample set, wherein each first training sample in the first training sample set comprises an eye opening and closing degree sample and a fatigue detection label of the eye opening and closing degree sample, each second training sample in the second training sample set comprises an eye blinking frequency sample and a fatigue detection label of the eye blinking frequency sample, and each third training sample in the third training sample set comprises a mouth opening and closing degree sample and a fatigue detection label of the mouth opening and closing degree sample; Training a first fatigue detection model to be trained according to the first training sample set to obtain a trained first fatigue detection model; training a second fatigue detection model to be trained according to the second training sample set to obtain a trained second fatigue detection model, and And training a third fatigue detection model to be trained according to the third training sample set to obtain the trained third fatigue detection model.
  10. 10. A fatigue detection device for a head-mounted apparatus, the device comprising: The device comprises an acquisition module, a control module and a control module, wherein the acquisition module is used for acquiring facial data of a wearer of the head-mounted device, wherein the facial data comprise eye opening and closing degree, eye blinking frequency and mouth opening and closing degree; the fatigue detection module is used for obtaining a fatigue detection result of the wearer according to a preset fatigue detection model and the face data; the prompting module is used for outputting prompting information of the fatigue state of the wearer when the fatigue detection result is that the wearer is in the fatigue state; The preset fatigue detection model comprises a first fatigue detection model, a second fatigue detection model and a third fatigue detection model, wherein the first fatigue detection model reflects the mapping relation between the opening degree and the closing degree of eyes and the fatigue detection result, the second fatigue detection model reflects the mapping relation between the blink frequency of eyes and the fatigue detection result, and the third fatigue detection model reflects the mapping relation between the opening degree of mouth and the fatigue detection result.

Description

Fatigue detection method and device for head-mounted equipment Technical Field The embodiment of the disclosure relates to the technical field of head-mounted equipment, and more particularly relates to a fatigue detection method of head-mounted equipment, a fatigue detection device of head-mounted equipment and head-mounted equipment. Background Currently, fatigue detection may be performed on a user by a headset device, such as a Virtual Reality (VR) device and an augmented Reality device (Augmented Reality, AR), and typically, the headset device may detect eye data of the user first and determine whether the wearer is in a tired state in combination with a fatigue determination threshold, however, this manner may not accurately detect whether the wearer is in a tired state, which may result in a failure to discover and alert the user in a tired state in time. Disclosure of Invention An object of an embodiment of the present disclosure is to provide a new technical solution for fatigue detection of a head-mounted device. According to a first aspect of embodiments of the present disclosure, there is provided a fatigue detection method for a head-mounted device, the method comprising: Acquiring face data of a wearer of the head-mounted device, wherein the face data comprises at least one of eye openness, eye blink frequency and mouth openness; Obtaining a fatigue detection result of the wearer according to a preset fatigue detection model and the facial data; Outputting prompt information of the wearer in a fatigue state when the fatigue detection result is that the wearer is in the fatigue state; The preset fatigue detection model comprises a first fatigue detection model, a second fatigue detection model and a third fatigue detection model, wherein the first fatigue detection model reflects the mapping relation between the opening degree and the closing degree of eyes and the fatigue detection result, the second fatigue detection model reflects the mapping relation between the blink frequency of eyes and the fatigue detection result, and the third fatigue detection model reflects the mapping relation between the opening degree of mouth and the fatigue detection result. Optionally, the headset comprises camera means, The acquiring facial data of a wearer of the headset device includes: acquiring a facial image of the wearer acquired by the image pickup device; And identifying the facial image and obtaining facial data of the wearer. Optionally, the camera device at least includes a first camera, a second camera, a third camera and a fourth camera, the first camera is located at a first position of the headset, the second camera is located at a second position of the headset, the third camera is located at a third position of the headset, the fourth camera is located at a fourth position of the headset, and the first position, the second position, the third position and the fourth position are different positions of the headset. Optionally, the eye opening and closing degree comprises a left eye opening and closing degree and a right eye opening and closing degree, the eye blinking frequency comprises a left eye blinking frequency and a right eye blinking frequency, the mouth opening and closing degree comprises a left mouth angle opening and closing degree and a right mouth angle opening and closing degree, The identifying the facial image, obtaining facial data of the wearer, includes: Identifying a first facial image shot by the first camera to obtain the opening and closing degree of the left eye and the blink frequency of the left eye; identifying a second face image shot by the second camera to obtain the opening and closing degree of the right eye and the blink frequency of the right eye; Identifying a third face image shot by the third camera to obtain the opening and closing degree of the left mouth angle, and And identifying a fourth facial image shot by the fourth camera to obtain the right mouth angle opening and closing degree. Optionally, the facial data includes eye openness, The step of obtaining the fatigue detection result of the wearer according to a preset fatigue detection model and the face data comprises the following steps: Determining a vector value of a first feature vector for reflecting the fatigue detection result according to the eye opening and closing degree, wherein the first feature vector comprises at least one feature of current eye opening and closing degree, eye opening and closing degree change rate, accumulated eye opening and closing degree change, average eye opening and closing degree, eye opening and closing degree standard deviation, maximum eye opening and closing degree, minimum eye opening and closing degree and eye opening and closing degree fluctuation; And obtaining a fatigue detection result of the wearer according to the vector value and the first fatigue detection model. Optionally, the facial data includes eye blink frequency, The step of obtaining the