Search

EP-4105819-B1 - DEEP LEARNING-BASED HORSE SPEED CALCULATION SYSTEM AND METHOD THEREOF

EP4105819B1EP 4105819 B1EP4105819 B1EP 4105819B1EP-4105819-B1

Inventors

  • KWAN, Kwok Leung

Dates

Publication Date
20260513
Application Date
20210113

Claims (7)

  1. A method for a horse speed calculation system based on deep learning, comprising the following steps: S1. before a start of a competition, horses participating in the competition walk around a competition field, a camera shoots a video of a target horse walking around the competition field and sends the video to an optical flow field calculation module; during a video shooting, the camera rotates to maintain the target horse within a shooting range; S2. performing an image extraction from the video obtained by the camera and calculating an optical flow field between consecutive images: the optical flow field calculation module decodes the video, input from the camera, into single frames, and extracts an image every set number of frames N, that is, two consecutive images are separated by N frames, and calculates the optical flow field between two consecutive images; S3. performing an object artificial intelligence detection on the video: an object artificial intelligence detection module detects objects that appear in the video, and saves a position and a size of each of the detected objects, comprising a position and a size of the target horse; a position of an object is a position of its center point, and a size comprises a length and a width; S4. an optical flow field filtering module filters out all moving objects from the optical flow field obtained in step S2 according to the objects detected in step S3, such that a maximum component of the optical flow field is the camera speed in pixels every N frames; S5. a camera speed calculation module calculates a direction and a speed of the camera according to the filtered optical flow field in step S4; S6. calculating a speed of the target horse between two consecutive images: according to the position of the target horse detected by the object artificial intelligence detection module, a displacement of the target horse between two consecutive images is calculated by subtracting a position of a previous image from a position of the target horse in a current image, a unit of the displacement is pixel/N frames; accordingly, a target horse speed calculation module calculates a final camera speed in pixels/N frames, that is, a speed H p of the target horse is: H p = ω − d ˙ − d ; ω is an angular velocity of the camera, unit is pixel/N frames, ḋ is a position of a center of the target horse in the previous image, and d is a position of a center of the target horse in the current image; S7. an output module averages all speeds of the target horse between two consecutive images obtained by calculation to obtain an average speed of the target horse.
  2. The method according to claim 1, characterized in that , in step S2, the number of frames is set to 5 frames.
  3. The method according to claim 1, characterized in that , in step S2, a method using Farneback is used to calculate the optical flow field.
  4. The method according to claim 1, characterized in that , in step S3, the object artificial intelligence detection module uses a YOLOv3 network to implement an artificial intelligence detection of the objects.
  5. The method of claim 1, characterized in that , in step S6, a unit conversion is performed on H p to obtain the target horse speed in unit of horse length/second; first, calculating the target horse speed H p in unit of pixel per second through H p : H ˙ p = H p * fps * N ; fps is a number of frames per second of the video; then, converting pixels to a horse length, and calculating the target horse speed V t in the unit of horse length/second; V t = H ˙ p / Pixels ; Ḣ p is the speed of the target horse in unit of pixels per second; Pixels is the number of pixels of the horse length, unit is pixel/horse length.
  6. The method of claim 5, characterized in that , for a length of the target horse detected by the object artificial intelligence detection module, using an abnormal state detection method RANSAC to review the detected horse length; assuming that the target horse walks normally within a short period of time after the start of the video, as new data points appear, a new regression model is updated based on this set of sampled data; checking the new data points against the new regression model, and against high-quality data points of an existing regression model, any abnormal data values will be removed and replaced with a known effective length.
  7. A horse speed calculation system based on deep learning, comprising: a camera: the camera is provided in a competition field, and is used to shoot a video of a target horse walking around the competition field before a competition; during a shooting process, the camera rotates to keep the target horse in a shooting range; an optical flow field calculation module: used to extract images from the video obtained by the camera and calculate an optical flow field between consecutive images; an object artificial intelligence detection module: used to use an artificial intelligence to detect and save a position and a size of an object in the video; an optical flow field filter module: used to filter the optical flow field calculated by the optical flow field calculation module according to a detection result of the object artificial intelligence detection module to filter out all moving objects, such that a maximum component of the optical flow field is the camera speed in pixels every N frames; a camera speed calculation module: used to calculate a direction and a speed of the camera using the optical flow field filtered by the optical flow field filter module; a target horse speed calculation module: used to calculate, according to the position of the target horse detected by the object artificial intelligence detection module, a displacement of the target horse between two consecutive images by subtracting a position of a previous image from a position of the target horse in a current image, and use the displacement of the horse to adjust the speed of the camera between two consecutive images calculated by the camera speed calculation module to obtain a speed H p of the target horse, an adjustment formula is H p = ω - ( ḋ - d ), ω is an angular velocity of the camera, unit is pixel/N frames, ḋ is a position of a center of the target horse in the previous image, and d is a position of a center of the target horse in the current image; an output module is used to average all speeds of the target horse between two consecutive images to obtain and output an average speed of the target horse.

Description

Technical Field The invention relates to the technical field of deep learning, in particular to a horse speed calculation system and method based on deep learning. Technical Background Equestrian is a sport that integrates exercise, fitness and leisure, and is loved by increasingly more people in recent years. Speed horse racing based on equestrian has developed into a competition event, which uses speed and riding skills to win and challenges the overall level of a rider. At present, in predicting the competition ranking of horses, it is generally based on naked eyes to observe the state of the horses (including fatigue level, sweating amount and movement) to make predictions. There is also a lack of more accurate scientific and intelligent means. LIU KAIZHEN et al "A Real-Time Method to Estimate Speed of Object Based on Object Detection and Optical Flow Calculation" (ISSN:1742-6588) introduces a method to real-time estimate speed of object by combining two CNN: YOLOv2 and FlowNet. In every frame, YOLOv2 provides object size; object location and object type while FlowNet providing the optical flow of whole image. On one hand, object size and object location help to select out the object part of optical flow image thus calculating out the average optical flow of every object. On the other hand, object type and object size help to figure out the relationship between optical flow and true speed by means of optics theory and priori knowledge. Therefore, with these two key information, speed of object can be estimated. This method manages to estimate multiple objects at real-time speed by only using a normal camera even in moving status, whose error is acceptable in most application fields like manless driving or robot vision. Summary of the Invention In view of the shortcomings of the prior art, the present invention aims to provide a horse speed calculation system and method based on deep learning, which realizes the use of artificial intelligence technology to observe horses and calculate horse speeds in a more scientific way. In order to achieve the above objectives, the present invention adopts the following technical solutions: A method for a horse speed calculation system based on deep learning, comprising the following steps: S1. before a start of a competition, horses participating in the competition walk around a competition field, a camera shoots a video of a target horse walking around the competition field and sends the video to an optical flow field calculation module; during a video shooting, the camera rotates to maintain the target horse within a shooting range;S2. performing an image extraction from the video obtained by the camera and calculating an optical flow field between consecutive images: the optical flow field calculation module decodes the video, input from the camera, into single frames, and extracts an image every set number of frames N, that is, two consecutive images are separated by N frames, and calculates the optical flow field between two consecutive images;S3. performing an object artificial intelligence detection on the video: an object artificial intelligence detection module detects objects that appear in the video, and saves a position and a size of each of the detected objects, comprising a position and a size of the target horse; a position of an object is a position of its center point, and a size comprises a length and a width;S4. an optical flow field filtering module filters out all moving objects from the optical flow field obtained in step S2 according to the objects detected in step S3, such that a maximum component of the optical flow field is the camera speed in pixels every N frames;S5. a camera speed calculation module calculates a direction and a speed of the camera according to the filtered optical flow field in step S4;S6. calculating a speed of the target horse between two consecutive images: according to the position of the target horse detected by the object artificial intelligence detection module, a displacement of the target horse between two consecutive images is calculated by subtracting a position of a previous image from a position of the target horse in a current image, a unit of the displacement is pixel/N frames; accordingly, a target horse speed calculation module calculates a final camera speed in pixels/N frames, that is, a speed Hp of the target horse is: Hp=ω−d˙−d;ω is an angular velocity of the camera, unit is pixel/N frames, ḋ is a position of a center of the target horse in the previous image, and d is a position of a center of the target horse in the current image;S7. an output module averages all speeds of the target horse between two consecutive images obtained by calculation to obtain an average speed of the target horse. Further, in step S2, the number of frames is set to 5 frames. Further, in step S2, a method using Farneback is used to calculate the optical flow field. Further, in step S3, the object artificial intelligence de