Search

KR-102963945-B1 - VANISHING POINT EXTRACTION DEVICE AND METHOD OF EXTRACTING VANISHING POINT

KR102963945B1KR 102963945 B1KR102963945 B1KR 102963945B1KR-102963945-B1

Abstract

A vanishing point extraction method according to the technical concept of the present disclosure may include the steps of: obtaining a straight line including a vanishing point of a first image from a first image; obtaining a plurality of sample points that intersect the straight line and an object included in the first image from the first image; obtaining at least one matching point that matches the plurality of sample points from a second image which is the next image of the first image; and obtaining a vanishing point of a second image based on at least one matching point from the second image.

Inventors

  • 최진혁

Assignees

  • 삼성전자주식회사

Dates

Publication Date
20260511
Application Date
20200214

Claims (20)

  1. A step of obtaining a straight line including the vanishing point of the first image from the first image; A step of obtaining a plurality of sample points from the first image that intersect the straight line and the object included in the first image; A step of obtaining at least one matching point that matches the plurality of sample points from a second image, which is the next image of the first image; and A method for extracting a vanishing point, comprising the step of obtaining a vanishing point of the second image based on at least one matching point from the second image.
  2. In paragraph 1, The step of obtaining at least one matching point mentioned above is, A step of obtaining a plurality of first templates corresponding to the plurality of sample points from the first image; A step of obtaining at least one second template similar to at least one of the plurality of first templates from the second image; and A vanishing point extraction method comprising the step of obtaining at least one matching point from at least one second template from the second image above.
  3. In paragraph 2, The step of obtaining at least one second template above is, A step of obtaining a plurality of search regions corresponding to each of the plurality of sample points from the second image above; and A vanishing point extraction method comprising: a step of, for each sample point, obtaining a second template for an area similar to a first template corresponding to each sample point within a search area corresponding to each sample point.
  4. In paragraph 3, The step of acquiring the above plurality of search regions is, A step of identifying points having the same coordinates as each of the plurality of sample points of the first image from the second image; and A vanishing point extraction method comprising the step of obtaining regions of a predetermined size, each containing the identified points, as the plurality of search regions from the second image.
  5. In paragraph 2, For each sample point, the step of obtaining a second template for an area similar to the first template corresponding to each sample point within a search area corresponding to each sample point is: A step of determining multiple candidate regions in a search region corresponding to each sample point; A step of calculating a correlation value between each of the above-determined candidate regions and a first template corresponding to each of the above-determined sample points; and A vanishing point extraction method comprising the step of obtaining the candidate region with the highest correlation calculated above as the second template.
  6. In paragraph 2, The straight line including the vanishing point of the first image above is, A method for extracting a vanishing point that includes the vanishing point of the first image and is a horizontal line horizontal to the horizontal axis of the first image.
  7. In paragraph 1, The object included in the first image above is a vanishing point extraction method that is a vehicle or a preset object related to a vehicle.
  8. In paragraph 1, The method further includes the step of removing an outlier of at least one matching point. The step of obtaining the vanishing point of the second image above is, A vanishing point extraction method characterized by further including the step of obtaining a vanishing point of the second image based on at least one matching point from which the outliers have been removed.
  9. In paragraph 8, The step of removing outliers of at least one matching point is, A vanishing point extraction method characterized by removing outliers using at least one of a RANSAC (RANdom Sample Consensus) model, a PROSAC (PROgressive Sample Consensus) model, or a StaRSaC (Stable Random Sample Consensus) model.
  10. In paragraph 1, The step of obtaining the vanishing point of the second image above is, A vanishing point extraction method characterized by obtaining a vanishing point of a second image by correcting the vanishing point of the first image based on at least one matching point.
  11. In Paragraph 10, The step of obtaining the vanishing point of the second image above is, A vanishing point extraction method characterized by obtaining the vanishing point of the second image by correcting the y-coordinate of the vanishing point of the first image using the y-coordinates of at least one matching point.
  12. In a vanishing point extraction device, Image sensor; and A memory storing a first image, information about a vanishing point of the first image, and information about at least one object included in the first image; and A vanishing point extraction device comprising: a processor that, upon receiving a second image from the image sensor, identifies a horizontal line including the vanishing point of the first image based on information regarding the vanishing point of the first image, acquires a plurality of sample points intersecting the at least one object and the horizontal line based on information regarding at least one object included in the first image, identifies at least one match point in the second image that matches the plurality of sample points, and acquires the vanishing point of the second image by correcting the vanishing point of the first image based on the at least one match point.
  13. In Paragraph 12, The above processor is, A vanishing point extraction device characterized by obtaining a plurality of first templates corresponding to the plurality of sample points from the first image and obtaining at least one second template similar to at least one of the plurality of first templates from the second image.
  14. In Paragraph 13, The above processor is, A plurality of search regions corresponding to each of a plurality of sample points are obtained from the second image above, and A vanishing point extraction device characterized by, for each sample point, obtaining a second template that is similar to a first template corresponding to each sample point within a search area corresponding to each sample point.
  15. In Paragraph 14, The above processor is, A vanishing point extraction device characterized by identifying points having coordinates identical to the coordinates of each of the plurality of sample points of the first image from the second image, and acquiring areas of a preset size including each of the identified points from the second image as the plurality of search areas.
  16. In Paragraph 14, The above processor is, A vanishing point extraction device characterized by determining multiple candidate regions in a search region corresponding to each sample point, calculating a correlation value between each of the determined candidate regions and a first template corresponding to each sample point, and acquiring the candidate region with the highest calculated correlation value as the second template.
  17. In Paragraph 12, The above processor is, A vanishing point extraction device characterized by removing an outlier of at least one matching point and obtaining a vanishing point of the second image based on the at least one matching point from which the outlier was removed.
  18. In Paragraph 17, The above processor is, A vanishing point extraction device characterized by removing outliers using at least one of a RANSAC (RANdom Sample Consensus) model, a PROSAC (PROgressive Sample Consensus) model, or a StaRSaC (Stable Random Sample Consensus) model.
  19. In Paragraph 12, The above processor is, A vanishing point extraction device characterized by obtaining the vanishing point of the second image by correcting the y-coordinate of the vanishing point of the first image using at least one matching point's y-coordinates.
  20. In an autonomous driving device, Image sensor; A memory storing a first image, information about a vanishing point of the first image, and information about at least one object included in the first image; and A first processor that, upon receiving a second image from the image sensor, identifies a horizontal line including the vanishing point of the first image based on information regarding the vanishing point of the first image, acquires a plurality of sample points intersecting the at least one object and the horizontal line based on information regarding at least one object included in the first image, identifies at least one match point in the second image that matches the plurality of sample points, and acquires the vanishing point of the second image by correcting the vanishing point of the first image based on the at least one match point; and An autonomous driving device comprising: a second processor that controls the operation of a host vehicle device based on information regarding the vanishing point of a second image obtained through the first processor.

Description

Vanishing Point Extraction Device and Vanishing Point Extraction Method The technical concept of the present disclosure relates to a vanishing point extraction device and a vanishing point extraction method, and specifically, to a vanishing point extraction device and a vanishing point extraction method that extracts the vanishing point of a current image based on information about the vanishing point of a previous image and objects included in the previous image. An image capturing device using an image sensor can be included in various types of electronic devices, such as smartphones, PCs, surveillance cameras, and automobiles, or used as a standalone electronic device. In the case of autonomous vehicles, safe driving can be achieved by acquiring images of the vehicle's surroundings through image sensors, determining the surrounding situation using the acquired images, and controlling the vehicle based on the determination results. For example, an autonomous vehicle can extract a vanishing point from an image using a vanishing point extraction device and use the extracted vanishing point to estimate the distance between vehicles or detect vehicle vibration. In the case of a vanishing point extraction device based on a conventional method, lanes within an image are detected, and the intersection points formed by extending the detected lanes into straight lines are identified as vanishing points. However, a problem with the conventional vanishing point extraction device was that it could not extract a vanishing point if there were no lanes nearby, or if lane detection failed in the image even if lanes existed. FIG. 1 is a block diagram showing a vanishing point extraction device according to one embodiment of the present disclosure. FIG. 2 is a drawing showing in detail a vanishing point extraction device according to one embodiment of the present disclosure. FIG. 3 is a drawing showing a host vehicle including a vanishing point extraction device according to one embodiment of the present disclosure. FIG. 4 is a drawing showing a first image according to one embodiment of the present disclosure. FIG. 5 is a drawing showing a straight line including the vanishing point of a first image according to one embodiment of the present disclosure. FIG. 6 is a drawing showing a plurality of sample points of a first image according to one embodiment of the present disclosure. FIG. 7 is a drawing showing a first template corresponding to a sample point according to one embodiment of the present disclosure. FIG. 8 is a drawing showing a second image, which is the next frame of a first image according to one embodiment of the present disclosure. FIG. 9 is a drawing showing a search area of a second image according to one embodiment of the present disclosure. FIG. 10 is a diagram illustrating a patch matching operation for a second image according to one embodiment of the present disclosure. FIG. 11 is a drawing showing a second template corresponding to a sample point according to one embodiment of the present disclosure. FIG. 12 is a drawing showing a matching point of a second template according to one embodiment of the present disclosure. FIG. 13 is a drawing showing a matching point of a second image according to one embodiment of the present disclosure. FIG. 14 is a drawing showing a plurality of matching points of a second image according to one embodiment of the present disclosure. FIG. 15 is a drawing showing the vanishing point of a second image according to one embodiment of the present disclosure. FIG. 16 is a drawing showing a plurality of matching points of a second image according to one embodiment of the present disclosure. FIG. 17 is a drawing showing the vanishing point of a second image according to one embodiment of the present disclosure. FIG. 18 is a drawing showing a plurality of sample points of a first image according to one embodiment of the present disclosure. FIG. 19 is a flowchart illustrating a vanishing point extraction method according to one embodiment of the present disclosure. FIG. 20 is a block diagram showing an autonomous driving device according to one embodiment of the present disclosure. FIG. 1 is a block diagram showing a vanishing point extraction device according to one embodiment of the present disclosure. Referring to FIG. 1, the vanishing point extraction device (10) may include an image sensor (100), a memory (200), and a processor (300). Additionally, the processor (300) may include a vanishing point extractor (310). The vanishing point extraction device (10) is a device that captures an image, analyzes the captured image, and extracts the vanishing point of the image based on the analysis results. The vanishing point extraction device (10) can be implemented as a PC (personal computer), an IoT (Internet of Things) device, or a portable electronic device. The portable electronic device can be equipped in various devices such as a laptop computer, a mobile phone, a smartphon