Search

KR-102961715-B1 - USER END, SYSTEM AND METHOD FOR CONTROLLING A DRONE

KR102961715B1KR 102961715 B1KR102961715 B1KR 102961715B1KR-102961715-B1

Abstract

A user terminal for controlling a drone is disclosed. The user terminal according to the present invention analyzes an original image and controls the drone to capture a reproduced image that gives the same or similar feeling as the original image. The electronic device of the present invention may be linked with an artificial intelligence module, a robot, an augmented reality (AR) device, a virtual reality (VR) device, a device related to 5G services, etc.

Inventors

  • 황미란
  • 임조영
  • 문광식

Assignees

  • 엘지전자 주식회사

Dates

Publication Date
20260508
Application Date
20191218

Claims (20)

  1. In a user terminal for controlling a drone, First wireless communication unit; and It includes a processor that controls the drone by communicating data with the drone through the first wireless communication unit, The above processor is, When data regarding the original first image is input to the user terminal, the hardware specifications of the second camera used to capture the first image and the composition of the first subject included in the first image are analyzed, and Analyze the hardware specifications of the first camera equipped in the above-mentioned drone, and The hardware specifications of the first camera and the hardware specifications of the second camera are compared, and based on the comparison result, In order to enable the first camera to photograph the second subject with the same or similar magnification ratio, angle of view, and composition as the magnification ratio, angle of view, and composition used when the second camera captured the first image, After generating a first camera control signal for controlling the first camera and a first drone control signal for moving the drone to a position where the drone can photograph the second subject by the first camera control signal, the first drone control signal and the first camera control signal are transmitted to the drone through the first wireless communication unit, The first camera captures a second image, which is a reproduction image that is identical or similar to the original image, and The above processor is, The first image is divided into a plurality of first divided images in frames per second, and the first area ratio occupied by the first subject in each first divided image is calculated. In order to minimize the difference between the first image and the second image caused by the difference between the hardware specifications of the first camera and the hardware specifications of the second camera, The above first drone control signal is, It includes a position control signal that causes the drone to be offset by a certain distance from the second subject so as to adjust the distance between the second subject and the first camera, and The first camera control signal above is, A user terminal comprising a shooting control signal that controls the magnification ratio, angle of view, depth, and illumination of the first camera.
  2. delete
  3. delete
  4. In Article 1, The above processor is, If the amount of change in the first area ratio in a specific section of the first image is greater than or equal to the first criterion, it is determined that zoom-in shooting has been performed on the first subject, and A user terminal that determines that zoom-out shooting has been performed on the first subject if the amount of change in the first area ratio in a specific section of the first image is less than or equal to the second criterion.
  5. ◈Claim 5 was waived upon payment of the establishment registration fee.◈ In Paragraph 4, The above processor is, A user terminal that generates a shooting control signal to zoom in or zoom out the first camera depending on whether the above zoom-in or zoom-out shooting is performed.
  6. In Article 1, The above processor is configured to perform AI (Artificial Intelligence) learning, and A user terminal that generates the first drone control signal and the first camera control signal by AI learning the magnification ratio, angle of view, focal length, depth of field, and illumination of the second camera used to capture the first video, the type and composition of the first subject included in the first video, and the overall hardware specifications of the drone and the first camera.
  7. In Article 1, It further includes a first memory that stores information about the drone, data about the first image and the second image, and Information regarding the above drone is, A user terminal containing information on the overall hardware specifications of the drone, including the model name, size, diagonal axis distance, number and capacity of batteries, weight, maximum takeoff weight, maximum payload, hovering accuracy range, maximum angular velocity, maximum pitch angle, maximum ascent speed, maximum descent speed, maximum speed, maximum practical ascent limit altitude, maximum wind speed withstandable, maximum flight time per payload, motor model name, propeller model name, presence or absence of landing gear, operating temperature, type and model name of mounted sensors, wireless signal reception range, pixels, magnification, and field of view of the mounted camera.
  8. In Article 1, It further includes a first position data processing unit capable of processing data regarding the location of the user terminal, the location of the second subject, and the location of the drone. The above processor is, A user terminal that generates a second drone control signal to control the drone to move in correspondence with the direction and speed of movement of the second subject when the second subject moves.
  9. In Article 1, The above-mentioned first wireless communication unit is, Communicate with at least two access points located around the drone and the user terminal using the IEEE 802.11mc protocol, and A user terminal that generates location data of the user terminal and the drone by performing triangulation using the RTT (Round Trip Time) method on at least two access points.
  10. In Article 1, The data for the first image above A user terminal stored in at least one of an external server, a web-based server, a cloud server, and the storage of the user terminal.
  11. In a system for controlling a drone, It includes a first user terminal capable of data communication with a drone equipped with a first camera and controlling the movement of said drone, The above-mentioned first user terminal is, When data regarding the original video, the first video, is input to the first user terminal, the hardware specifications of the second camera used to produce the first video and the composition of the first subject included in the first video are analyzed, and Analyze the hardware specifications of the first camera equipped in the above-mentioned drone, and The hardware specifications of the first camera and the hardware specifications of the second camera are compared, and based on the comparison result, A first camera control signal is generated to cause the first camera to photograph a second subject with the magnification ratio, angle of view, and composition of the second camera and to generate a second image, and The first image is divided into a plurality of first divided images in frames per second, and the first area ratio occupied by the first subject in each first divided image is calculated. In order to minimize the difference between the first image and the second image caused by the difference between the hardware specifications of the first camera and the hardware specifications of the second camera, A first drone control signal is generated to control the movement of the drone so that the drone is offset by a certain distance from the second subject, thereby allowing the distance between the second subject and the first camera to be adjusted. The first camera control signal above is, A system comprising a shooting control signal that controls the magnification ratio, angle of view, depth, and illumination of the first camera.
  12. In Paragraph 11, The above-mentioned first user terminal is, A system for dividing the second image into a plurality of second divided images and calculating the second area ratio occupied by the second subject for each second divided image.
  13. ◈Claim 13 was waived upon payment of the establishment registration fee.◈ In Paragraph 11, The above-mentioned first user terminal is, A system that transmits the first drone control signal and the first camera control signal to the drone to control the movement of the drone and the shooting of the first camera.
  14. ◈Claim 14 was waived upon payment of the establishment registration fee.◈ In Paragraph 11, The above-mentioned first user terminal is, A system including an AI module capable of performing AI learning.
  15. ◈Claim 15 was waived upon payment of the establishment registration fee.◈ In Paragraph 11, The above drone is a system that films the above second subject while tracking it.
  16. ◈Claim 16 was waived upon payment of the establishment registration fee.◈ In Paragraph 12, The above-mentioned first user terminal is, If the amount of change in the first area ratio in a specific section of the first image is greater than or equal to the first criterion, it is determined that zoom-in shooting has been performed on the first subject, and A system that determines that zoom-out shooting has been performed on the first subject if the amount of change in the first area ratio in a specific section of the first image is less than or equal to the second criterion.
  17. ◈Claim 17 was waived upon payment of the establishment registration fee.◈ In Article 16, The above-mentioned first user terminal is, A system that generates a shooting control signal to cause the first camera to zoom in or zoom out depending on whether the zoom-in or zoom-out shooting is performed.
  18. ◈Claim 18 was waived upon payment of the establishment registration fee.◈ In Article 11, It includes at least one access point capable of wirelessly connecting to the drone and the first user terminal, respectively, and A system in which the drone, the first user terminal, and the access point communicate with each other using the IEEE 802.11mc protocol.
  19. Regarding drone control methods, A step in which the original video, the first video, is downloaded to the user terminal; Through the above user terminal, A step of analyzing the hardware specifications of the second camera used to capture the first image and the composition of the first subject included in the first image; A step of analyzing the hardware specifications of a first camera equipped in the above-mentioned drone; A step of comparing the hardware specifications of the first camera and the hardware specifications of the second camera, and based on the comparison result, generating a first camera control signal to control the first camera to photograph a second subject with a magnification ratio, angle of view, depth of field, and illumination that are identical or similar to the magnification ratio, angle of view, depth of field, and illumination used when the second camera captured the first image; A step of generating a first drone control signal for moving the drone to a position where the second subject can be photographed by the first camera control signal; and The method includes the step of transmitting the first drone control signal and the first camera control signal to the drone so that the first camera captures a second image, which is a reproduction image that is identical or similar to the original image. The step of analyzing the hardware specifications of the second camera used to capture the first image and the composition of the first subject included in the first image is: A step of dividing the first image into a plurality of first divided images at a frame-per-second rate; and The method further includes the step of calculating a first area ratio occupied by the first subject in each of the first segmented images, and In order to minimize the difference between the first image and the second image caused by the difference between the hardware specifications of the first camera and the hardware specifications of the second camera, The above first drone control signal is, It includes a position control signal that causes the drone to be offset by a certain distance from the second subject so as to adjust the distance between the second subject and the first camera, and The first camera control signal above is, A drone control method comprising a shooting control signal that controls the magnification ratio, angle of view, depth, and illumination of the first camera.
  20. delete

Description

User terminal, system and method for controlling a drone The present invention relates to a user terminal, a system, and a control method for controlling a drone to capture images. Unmanned aerial vehicles (UAVs) are a collective term for airplanes and helicopter-shaped unmanned aerial vehicles capable of flying and being controlled by radio guidance without a pilot. Recently, in addition to military uses such as reconnaissance and attack, the utilization of UAVs is increasing in various civilian and commercial fields, including video recording, unmanned delivery services, and disaster observation. Meanwhile, private and commercial unmanned aerial vehicles (UAVs) are inevitably operated on a limited basis due to the insufficient establishment of infrastructure, such as various regulations, certifications, and legal frameworks; furthermore, it is difficult for users to recognize the potential dangers or risks they may pose to the public. In particular, there is a growing trend of incidents, including collisions, flights in security zones, and privacy violations, resulting from the indiscriminate use of UAVs. Many countries are striving to improve new regulations, standards, policies, and procedures regarding the operation of unmanned aerial vehicles. Recently, as the use of unmanned aerial vehicles (UAVs) for video recording has become more common, there is a growing demand to provide users with various video recording modes when using UAVs. In particular, as many videos mimicking or recreating memorable scenes from movies are being created and shared on the web, users recently have been demanding more drones that can capture reenactment videos that are identical or similar to the original footage. FIG. 1 shows a perspective view of an unmanned aerial vehicle according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating the control relationships between the main components of the unmanned aerial vehicle of FIG. 1. FIG. 3 is a block diagram illustrating the control relationships between the main components of an aircraft control system according to one embodiment of the present invention. FIG. 4 illustrates a block diagram of a wireless communication system to which the methods proposed in this specification can be applied. Figure 5 is a figure showing an example of a signal transmission/reception method in a wireless communication system. Figure 6 shows an example of the basic operation of a robot and a 5G network in a 5G communication system. Figure 7 illustrates an example of a basic operation between robots using 5G communication. Figure 8 is a diagram showing an example of a 3GPP system concept diagram including a UAS. Figure 9 shows an example of a C2 communication model for a UAV. FIG. 10 is a flowchart illustrating an example of a measurement method to which the present invention can be applied. FIG. 11 is a conceptual diagram showing one embodiment of an AI device. FIG. 12 is a block diagram showing a drone control system according to one embodiment of the present invention. FIG. 13 is a conceptual diagram showing a user terminal acquiring a first image according to an embodiment of the present invention. FIG. 14 is a conceptual diagram showing a drone according to one embodiment of the present invention photographing a second subject under the control of a user terminal. FIGS. 15 and 16 are drawings showing a user terminal controlling a drone according to an embodiment of the present invention when the composition of the subject being expressed and the composition of the subject being expressed in the reproduced image are different. Figure 17 is a diagram comparing how the representation of a captured image changes depending on the size of the image sensor. FIG. 18 is a diagram showing a user terminal according to an embodiment of the present invention recognizing the face of a subject using a drone. Referring to FIG. 19, a drone control system (60) according to another embodiment of the present invention includes a drone (100), an external server (200), and a plurality of user terminals (300, 310). FIG. 19 is a block diagram showing a drone control system according to another embodiment of the present invention. FIGS. 20 and 21 are conceptual diagrams showing a drone control system according to another embodiment of the present invention capturing a reproduction image. FIG. 22 is a conceptual diagram showing a drone control system according to another embodiment of the present invention controlling the distance between a moving subject and a drone. FIG. 23 is a conceptual diagram showing a drone control system according to another embodiment of the present invention measuring the positions of a subject and a drone using surrounding access points. FIG. 24 is a conceptual diagram of the WiFi 802.11mc communication standard used by a drone control system according to another embodiment of the present invention. FIG. 25 is a conceptual diagram showing a drone control syste