US-12620125-B2 - Electronic device for identifying position of external object associated with image by using information of camera and method thereof

Abstract

An electronic device according to an embodiment may include a camera and a processor. The processor may be configured to obtain, from an image obtained using the camera, two-dimensional coordinate values representing vertices of a portion related to an external object. The processor may be configured to identify a first line extending along a reference direction on a reference plane included in the image and overlapping a first vertex of the vertices, and a second line connecting a reference point within the image and a second vertex of the vertices. The processor may be configured to, based on an intersection of the first line and the second line, obtain three-dimensional coordinate values representing each of vertices of a three-dimensional external space corresponding to the external object.

Inventors

• Sukpil KO

Assignees

• THINKWARE CORPORATION

Dates

Publication Date

20260505

Application Date

20230727

Priority Date

20220729

Claims (20)

1. 1 . An electronic device, comprising: a camera configured to obtain an image; and a processor, wherein the processor is configured to: compensate for distortions of the image caused by a lens of the camera, using an intrinsic parameter for the camera; obtain, from the image obtained using the camera, two-dimensional coordinate values representing vertices of a portion related to an external object; identify a first line extending along a reference direction on a reference plane included in the image and overlapping a first vertex of the vertices, and a second line connecting a reference point within the image and a second vertex of the vertices; and based on an intersection of the first line and the second line, obtain three-dimensional coordinate values representing each of vertices of a three-dimensional external space corresponding to the external object, and wherein the vertices of the external space include a three-dimensional point corresponding to an intersection of a third line connecting a third vertex of the vertices of the portion and the intersection of the first and second lines, and a fourth line connecting the first and second vertices.
2. 2 . The electronic device of claim 1 , wherein the processor is configured to: determine the three-dimensional point by performing unprojection of the intersection of the third line and the fourth line.
3. 3 . The electronic device of claim 2 , wherein the processor is configured to: determine a three-dimensional point corresponding to an intersection of a fifth line extending along the reference direction from the three-dimensional point and a sixth line connecting the first vertex and a fourth vertex of vertices of the portion, as another vertex of the vertices of the external space.
4. 4 . The electronic device of claim 2 , wherein the processor is configured to: determine a three-dimensional point corresponding to an intersection of a seventh line extending from the three-dimensional point along a different direction perpendicular to the reference direction and an eighth line connecting the second vertex and the third vertex, as the other vertex of the vertices of the external space.
5. 5 . The electronic device of claim 1 , wherein the processor is configured to obtain the two-dimensional coordinate values, using a neural network trained for object recognition.
6. 6 . The electronic device of claim 1 , wherein the processor is configured to determine a distance to the external object based on the three-dimensional coordinate values.
7. 7 . The electronic device of claim 1 , wherein the processor is configured to obtain the two-dimensional coordinate values for the portion within the image for which the distortion has been compensated.
8. 8 . A method of an electronic device, comprising: compensating for distortions of an image obtained by a camera of the electronic device and caused by a lens of the camera, using an intrinsic parameter for the camera; obtaining, from the image, two-dimensional coordinate values representing vertices of a portion related to an external object; identifying a first line extending along a reference direction on a reference plane included in the image and overlapping a first vertex of the vertices, and a second line connecting a reference point within the image and a second vertex of the vertices; and based on an intersection of the first line and the second line, obtaining three-dimensional coordinate values representing each of vertices of a three-dimensional external space corresponding to the external object, and wherein the vertices of the external space include a three-dimensional point corresponding to an intersection of a third line connecting a third vertex of the vertices of the portion and the intersection of the first and second lines, and a fourth line connecting the first and second vertices.
9. 9 . The method of claim 8 , wherein the obtaining the three-dimensional coordinate values comprises: determining the three-dimensional point by performing unprojection of the intersection of the third line and the fourth line.
10. 10 . The method of claim 9 , wherein the obtaining the three-dimensional coordinate values comprises: determining a three-dimensional point corresponding to an intersection of a fifth line extending along the reference direction from the three-dimensional point and a sixth line connecting the first vertex and a fourth vertex of vertices of the portion, as another vertex of the vertices of the external space.
11. 11 . The method of claim 9 , wherein the obtaining the three-dimensional coordinate values comprises: determining a three-dimensional point corresponding to an intersection of a seventh line extending from the three-dimensional point along a different direction perpendicular to the reference direction and an eighth line connecting the second vertex and the third vertex, as the other vertex of the vertices of the external space.
12. 12 . The method of claim 8 , wherein the obtaining the two-dimensional coordinate values comprises: obtaining the two-dimensional coordinate values, using a neural network trained for object recognition.
13. 13 . The method of claim 8 , further comprising: determining a distance to the external object based on the three-dimensional coordinate values.
14. 14 . The method of claim 8 , wherein the obtaining the two-dimensional coordinate values comprises: obtaining the two-dimensional coordinate values for the portion within the image for which the distortion has been compensated.
15. 15 . A non-transitory computer-readable storage medium comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to: compensate for distortions of an image obtained by a camera of the electronic device and caused by a lens of the camera, using an intrinsic parameter for the camera; obtain, from the image, two-dimensional coordinate values representing vertices of a portion related to an external object; identify a first line extending along a reference direction on a reference plane included in the image and overlapping a first vertex of the vertices, and a second line connecting a reference point within the image and a second vertex of the vertices; and based on an intersection of the first line and the second line, obtain three-dimensional coordinate values representing each of vertices of a three-dimensional external space corresponding to the external object, and wherein the vertices of the external space include a three-dimensional point corresponding to an intersection of a third line connecting a third vertex of the vertices of the portion and the intersection of the first and second lines, and a fourth line connecting the first and second vertices.
16. 16 . The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed by the processor of the electronic device, cause the electronic device to: determine the three-dimensional point by performing unprojection of the intersection of the third line and the fourth line.
17. 17 . The non-transitory computer-readable storage medium of claim 16 , wherein the instructions, when executed by the processor of the electronic device, cause the electronic device to: determine a three-dimensional point corresponding to an intersection of a fifth line extending along the reference direction from the three-dimensional point and a sixth line connecting the first vertex and a fourth vertex of vertices of the portion, as another vertex of the vertices of the external space.
18. 18 . The non-transitory computer-readable storage medium of claim 16 , wherein the instructions, when executed by the processor of the electronic device, cause the electronic device to: determine a three-dimensional point corresponding to an intersection of a seventh line extending from the three-dimensional point along a different direction perpendicular to the reference direction and an eighth line connecting the second vertex and the third vertex, as the other vertex of the vertices of the external space.
19. 19 . The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed by the processor of the electronic device, cause the electronic device to obtain the two-dimensional coordinate values, using a neural network trained for object recognition.
20. 20 . The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed by the processor of the electronic device, cause the electronic device to: determine a distance to the external object based on the three-dimensional coordinate values.

Description

CROSS-REFERENCE TO RELATED APPLICATION(S) This application is based on and claims priority under 35 U.S.C. § 119 to Korean Patent Application No. 10-2022-0094678, filed on Jul. 29, 2022, in the Korean Intellectual Property Office, and Korean Patent Application No. 10-2023-0090943, filed on Jul. 13, 2023, in the Korean Intellectual Property Office, the disclosures of which are incorporated by reference herein in their entirety. BACKGROUND Technical Field The present disclosure relate to an electronic device for identifying a position of an external object associated with an image by using information of a camera and a method thereof. Description of Related Art Electronic devices and/or services based on object recognition techniques are being developed, including algorithms for inferring external objects captured in an image. For example, based on such object recognition techniques, one or more external objects (e.g., pedestrians, vehicles, etc.) may be recognized from images. Information including a result of recognizing one or more recognized external objects may be used to automate and/or replace actions of a user recognizing an external object, depending on the level of automation of a vehicle, such as for autonomous driving/autonomous driving. SUMMARY A scheme for identifying a position of an external object using only images obtained from a camera may be required. According to an embodiment, an electronic device may comprise a camera and a processor. The processor may be configured to obtain, from an image obtained using the camera, two-dimensional coordinate values representing vertices of a portion related to an external object. The processor may be configured to identify a first line extending along a reference direction on a reference plane included in the image and overlapping a first vertex of the vertices, and a second line connecting a reference point within the image and a second vertex of the vertices. The processor may be configured to, based on an intersection of the first line and the second line, obtain three-dimensional coordinate values representing each of vertices of a three-dimensional external space corresponding to the external object. According to an embodiment, a method of an electronic device may comprise obtaining, from an image obtained using a camera of the electronic device, two-dimensional coordinate values representing vertices of a portion related to an external object. The method may comprise identifying a first line extending along a reference direction on a reference plane included in the image and overlapping a first vertex of the vertices, and a second line connecting a reference point within the image and a second vertex of the vertices. The method may comprise, based on an intersection of the first line and the second line, obtaining three-dimensional coordinate values representing each of vertices of a three-dimensional external space corresponding to the external object. According to an embodiment, a computer-readable storage medium may comprise instructions. The instructions may cause, when executed by a processor of an electronic device, the electronic device to obtain, from an image obtained using the camera, two-dimensional coordinate values representing vertices of a portion related to an external object. The instructions may cause, when executed by a processor of an electronic device, the electronic device to identify a first line extending along a reference direction on a reference plane included in the image and overlapping a first vertex of the vertices, and a second line connecting a reference point within the image and a second vertex of the vertices. The instructions may cause, when executed by a processor of an electronic device, the electronic device to, based on an intersection of the first line and the second line, obtain three-dimensional coordinate values representing each of vertices of a three-dimensional external space corresponding to the external object. According to various embodiments, the electronic device may identify a position of the external object based solely on the image obtained from the camera. BRIEF DESCRIPTION OF THE DRAWINGS The above and other aspects, features, and advantages of certain embodiments of the disclosure will be more apparent from the following description, taken in conjunction with the accompanying, in which: FIG. 1 illustrates an example of an electronic device for identifying a position of an external object using an image, according to an embodiment; FIG. 2 illustrates an example of a block diagram of an electronic device, according to an embodiment; FIG. 3 illustrates an example flowchart to illustrate an operation of an electronic device, according to an embodiment; FIG. 4 illustrates an operation for obtaining intrinsic parameters of a camera of an electronic device, according to an embodiment; FIG. 5 illustrates an exemplary operation of an electronic device to compensate for distortion of an image cause