KR-20260062167-A - DEEP LEARNING BASED METHOD AND DEVICE FOR FIELD ENVIROMENT VISUAL RECOGNITION USING A SINGLE CAMERA

KR20260062167AKR 20260062167 AKR20260062167 AKR 20260062167AKR-20260062167-A

Abstract

The present invention relates to a fruit harvesting robot device and a method of operation, and more specifically, includes a shooting unit that captures RGB images with a single camera, a detection unit that detects the fruit by analyzing the images, a moving unit that moves the robot to the vicinity of the detected fruit, and a harvesting unit that harvests the fruit. The position of the fruit can be recognized in three dimensions using only a single camera image, and the income of the farm can be significantly improved by automatically harvesting only high-marketability fruit without the intervention of a worker.

Inventors

이상준

Assignees

전북대학교산학협력단

Dates

Publication Date: 20260507
Application Date: 20241025

Claims (10)

In a robotic device that performs communication with an external device and harvests fruit trees, A shooting unit that captures color images with a single camera; A detection unit that detects the fruit tree by analyzing the above image; A moving unit that moves the robot to the vicinity of the detected fruit tree; and A harvesting unit for harvesting the above fruit trees; A fruit harvesting robot device including
In paragraph 1, The above robot further includes a communication unit that performs wireless communication with an external device, and The above external device performs management or control of one or more of the above robots, and A fruit harvesting robot device characterized by including an input unit that receives the type of fruit in text or voice to one or more of the above external device or the above robot.
In paragraph 2, The above detection unit is located at either the external device or the robot, and A conversion unit that converts input data of the above-mentioned input unit into text; and It includes an analysis unit that analyzes the above text and the above image, and The above analysis unit An object detection unit that detects the location and size of the fruit tree by analyzing the text and the image based on a visual language model; A depth estimation unit that analyzes the above image and calculates the distance to the fruit tree; A judgment unit that determines the marketability of the fruit based on the color of the fruit; and A setting unit that sets the harvest target by comprehensively considering the location, size, distance, and marketability of the above-mentioned fruit tree; A fruit harvesting robot device characterized by further including
In paragraph 1, The above-mentioned moving unit performs movement using any one of tires, crawler tracks, or rails for the movement of the robot, and A fruit harvesting robot device characterized by the moving part and the harvesting part being configured as independent modules that can be exchanged according to the user's needs.
In paragraph 1, The above harvesting unit A fixing part for fixing the above fruit tree; A cutting section separating the fruit tree from the tree; and A robot arm that brings the fixed part to the fruit tree using one or more joints; A fruit harvesting robot device characterized by including
In the operation method of a fruit harvesting robot, A step of obtaining information on the type of fruit tree to be harvested through an input unit; A step of capturing a color image with a single camera in the imaging unit; A step of detecting the fruit by analyzing the image in the detection unit; A step of moving the robot in the moving part; and A step of harvesting the fruit tree at the harvesting section; A method of operation of a fruit harvesting robot including
In paragraph 6, The step of detecting the fruit tree above is, A step of converting the type information of the fruit tree into text in a conversion unit; and The analysis unit includes a step of analyzing the text and the image to determine the location of the fruit tree in the image, and The step of determining the location of the fruit tree is, A step of detecting the location of the fruit tree by analyzing the text and the image based on a visual language model in an object detection unit; A step of analyzing the image in the depth estimation unit and calculating the distance to the fruit tree; A step of determining the marketability of the fruit, including the degree of ripeness and size of the fruit, based on the color of the fruit in a judgment unit; and A step of setting the harvest target by comprehensively considering the location, distance, and marketability of the fruit tree in the setting unit; A method of operation for a fruit harvesting robot characterized by further including
In Paragraph 7, The step of receiving the type of fruit tree is performed in an input unit that receives the type of fruit tree as text or voice from one or more of the external device or the robot, and The above external device and the robot perform wireless communication through a communication unit, and A method of operation for a fruit harvesting robot, characterized in that the above external device is connected to one or more of the robots to manage or control them.
In paragraph 6, The above-mentioned moving unit performs movement using any one of tires, crawler tracks, or rails for the movement of the robot, and A method of operation for a fruit harvesting robot characterized in that the moving part and the harvesting part are configured in the form of independent modules that can be exchanged according to the user's needs.
In paragraph 6, The above harvesting step is, A step of approaching the end effector of the robot to the said number using one or more joints of the robot arm; The method further includes the step of harvesting the fruit using the end effector, The step of harvesting the fruit tree mentioned above is, A step of fixing the fruit tree using a fixing part; and A step of separating the fruit tree from the tree using a cutting section; A method of operation for a fruit harvesting robot characterized by further including

Description

Deep Learning Based Method and Device for Field Environment Visual Recognition Using a Single Camera The present invention relates to a fruit tree harvesting robot device and a method of operating the robot, and specifically to a robot device and a method of operating the robot for harvesting fruit trees in an open field environment using a single camera. Korea, along with Japan and the United States, is reaching a super-aged society where the elderly population aged 65 or older accounts for more than 21% of the total population. At the same time, as a phenomenon of avoiding physical labor known as "3D" (dirty, dangerous, and difficult) occurs, leading to a contraction in investment, the agricultural environment in rural areas is facing various difficulties. Smart farms, which incorporate artificial intelligence technology into agriculture to overcome these difficulties, are emerging and receiving significant attention. They are particularly noteworthy because they can achieve automation and improved efficiency in the crop harvesting process in environments such as orchards that require a lot of human intervention. Although crop harvesting automation technology combined with deep learning algorithms is being researched in various ways to achieve fruit cultivation automation using manipulators, which are robots composed of links and joints, conventional technology only provides transport robots that acquire images of fruit trees to distinguish them or move harvested fruit trees to preset locations, so there is an increasing demand for technology that actually harvests fruit trees. FIG. 1 is a diagram showing the configuration of a fruit harvesting robot device according to one embodiment of the present invention. FIG. 2 is a diagram illustrating the operation of a fruit harvesting robot device using a deep learning-based visual language model according to an embodiment of the present invention. FIG. 3 is a diagram illustrating the structure of a visual language model of a fruit harvesting robot device according to one embodiment of the present invention. FIG. 4 is a diagram showing a fruit classification method of a fruit harvesting robot device according to one embodiment of the present invention. FIG. 5 is a flowchart showing the operation of a fruit harvesting robot device according to one embodiment of the present invention. FIG. 6 is a flowchart illustrating a method for selecting a harvesting target for a fruit harvesting robot device according to one embodiment of the present invention. The present invention is capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the invention to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application. Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate an overall understanding of the present invention, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted. FIG. 1 is a diagram showing the configuration of a fruit harvesting robot device according to one embodiment of the present invention. A fruit harvesting robot (100) (hereinafter, harvesting robot) that automatically harvests fruit according to one embodiment of the present invention may include a shooting unit (110), a detection unit (120), a moving unit (130), a harvesting unit (140), and a communication unit (150), and may include an external device (200) that performs wireless communication with the communication unit (150). Additionally, although not shown in the drawing, one or more of the