US-20260128711-A1 - PHOTOVOLTAIC MODULE INSTALLATION METHOD BASED ON SURFACE LINEAR FEATURES AND MODULE INSTALLATION ROBOT

US20260128711A1US 20260128711 A1US20260128711 A1US 20260128711A1US-20260128711-A1

Abstract

A photovoltaic module installation method based on surface linear features and a module installation robot are disclosed. The method comprises: grasping a to-be-installed module, moving it to a fine-tuning position and hovering it; obtaining a first image at the fine-tuning position, calculating the average angle between the horizontal lines of the to-be-installed module and a reference module based on the first image, and fine-tuning the to-be-installed module based on the average angle; obtaining a second image, calculating the actual distance between the corresponding vertical lines of the two modules and the actual distance between the edges of the two modules based on the second image; fine-tuning the to-be-installed module horizontally based on the former, and fine-tuning the to-be-installed module vertically based on the latter; and pressing the to-be-installed module downward to the installation location. The present application reduces installation errors and improves the installation accuracy of photovoltaic modules.

Inventors

Chenxin Zou
Shitao Wang
Ye Zhang

Assignees

Huzhou Leapting Technology Co., Ltd.

Dates

Publication Date: 20260507
Application Date: 20250912
Priority Date: 20241104

Claims (11)

1 . A photovoltaic module installation method based on surface linear features, which is used for a module installation robot, the module installation robot comprises a robotic arm and a camera located at an end of the robotic arm, characterized by comprising: grasping a module to be installed via the robotic arm, moving the module to a fine-tuning position, and hovering it at the fine-tuning position, wherein the fine-tuning position is located above an installation location of the module to be installed and spaced from the installation location by a first preset distance; obtaining a first image including a reference module and the module to be installed via the camera, wherein the reference module is a previously installed photovoltaic module immediately close to the module to be installed; calculating an average angle between horizontal lines of the reference module and horizontal lines of the module to be installed in the first image, wherein the horizontal lines of the reference module and the module to be installed are formed by connection lines between multiple cells, and performing a first fine-tuning adjustment on the module to be installed based on the average angle so that the horizontal lines of the reference module and the horizontal lines of the module to be installed are parallel; obtaining a second image including the reference module and the first fine-tuned module to be installed via the camera; calculating a first actual distance between vertical lines of the reference module and vertical lines of the corresponding module to be installed, wherein the vertical lines of the reference module and the module to be installed are formed by connection lines between multiple cells, and a second actual distance between a lower edge horizontal line of the reference module and an upper edge horizontal line of the module to be installed based on the second image; translating the module to be installed along a horizontal direction based on the first actual distance so as to align the vertical lines of the reference module with the vertical lines of the corresponding module to be installed; translating the module to be installed along a vertical direction based on the second actual distance so as to maintain a second preset distance between the reference module and the module to be installed; pressing the module to be installed downward to the installation location; said calculating an average angle between horizontal lines of the reference module and horizontal lines of the module to be installed in the first image comprises: performing a morphological opening operation on the first image, and then extracting edge horizontal lines of the two modules in the first image using a Hough detection algorithm; obtaining intersection points between the edge horizontal lines and a first vertical detection line, wherein the first vertical detection line is a straight line located at a center of the first image and parallel to a Y-axis; clustering the edge horizontal lines based on vertical coordinates of the intersection points to obtain a plurality of clusters, and identifying a position of the corresponding edge horizontal line by using a midpoint value of each cluster; calculating an intermediate value between a topmost cluster and a bottommost cluster, wherein the edge horizontal lines of the clusters located above the intermediate value are determined as the edge horizontal lines of the reference module, and the edge horizontal lines of the clusters located below the intermediate value are determined as the edge horizontal lines of the module to be installed, thereby identifying the edges of the reference module and the module to be installed in the first image; distinguishing the reference module and the module to be installed in the first image based on the edges; calculating an average inclination angle of the horizontal lines of the reference module in the first image; calculating an average inclination angle of the horizontal lines of the module to be installed in the first image; obtaining the average angle between the horizontal lines of the reference module and the horizontal lines of the module to be installed based on the average inclination angle of the horizontal lines of the reference module and the average inclination angle of the horizontal lines of the module to be installed; said calculating an average inclination angle of the horizontal lines of the reference module in the first image comprises: extracting all horizontal lines of the reference module using a Hough detection algorithm, and obtaining an inclination angle for each horizontal line of the reference module; obtaining intersection points between the horizontal lines and the first vertical detection line; clustering the horizontal lines based on vertical coordinates of the intersection points to obtain a plurality of clusters; performing a median filtering on inclination angles of the horizontal lines within each cluster, and constructing a first point set using the inclination angles of remaining horizontal lines after filtering and vertical coordinates of their intersection points between the first vertical detection line and the remaining horizontal lines after filtering; wherein for each point in the first point set, its horizontal coordinate is a vertical coordinate of the intersection point between the remaining horizontal lines after filtering and the first vertical detection line, and its vertical coordinate is the inclination angle of remaining horizontal lines after filtering; fitting the first point set to obtain a first linear equation, wherein the average inclination angle of all horizontal lines of the reference module is equal to an intercept b1 of the first linear equation.
2 - 4 . (canceled)
5 . The photovoltaic module installation method based on surface linear features according to claim 1 , characterized in that said calculating an average inclination angle of the horizontal lines of the module to be installed in the first image comprises: extracting the horizontal lines of the module to be installed in the first image using a Hough detection algorithm, and obtaining an inclination angle for each horizontal line; obtaining intersection points between the horizontal lines of the module to be installed and the first vertical detection line; clustering the horizontal lines of the module to be installed based on vertical coordinates of the intersection points to obtain a plurality of clusters; performing a median filtering on inclination angles of the horizontal lines within each cluster, and constructing a second point set using the inclination angles of remaining horizontal lines after filtering and vertical coordinates of their intersection points between the first vertical detection line and the remaining horizontal lines after filtering, wherein for each point in the second point set, its horizontal coordinate is a vertical coordinate of the intersection point between the remaining horizontal lines after filtering and the first vertical detection line, and its vertical coordinate is the inclination angle of remaining horizontal lines after filtering; fitting the second point set with a second linear equation, wherein a slope of the second linear equation is equal to a slope of the first linear equation, which comprises: substituting each point in the second point set into the second linear equation to obtain a corresponding intercept value of the second linear equation, and averaging all intercept values to obtain an intercept of the second linear equation; wherein the average inclination angle of all horizontal lines of the module to be installed is equal to the intercept b2 of the second linear equation.
6 . The photovoltaic module installation method based on surface linear features according to claim 1 , characterized in that said calculating a first actual distance between vertical lines of the reference module and vertical lines of the corresponding module to be installed based on the second image comprises: determining edges of the reference module and the module to be installed in the second image, and identifying the lower edge horizontal line of the reference module and the upper edge horizontal line of the module to be installed; distinguishing the reference module and the module to be installed in the second image based on the edges, and determining a range of the reference module and a range of the module to be installed; extracting a vertical line of the reference module and a vertical line of the corresponding module to be installed from the second image as an upper module reference line and a lower module reference line, respectively; extracting vertical lines of the reference module and the module to be installed other than the upper module reference line and the lower module reference line from the second image; using horizontal coordinate values of first-type intersection points between a first horizontal detection line and the vertical lines of the reference module to identify positions of corresponding vertical lines, and using horizontal coordinate values of second-type intersection points between a second horizontal detection line and the vertical lines of the module to be installed to identify positions of corresponding vertical lines; wherein the first horizontal detection line is a straight line located in the range of the reference module and parallel to an X-axis, and the second horizontal detection line is a straight line located in the range of the module to be installed and parallel to the X-axis; calculating relative positions of the vertical lines of the reference module with respect to the upper module reference line, calculating relative positions of the vertical lines of the module to be installed with respect to the lower module reference line; and matching the vertical lines of the reference module and the module to be installed having same relative positions to form vertical line pairs; calculating a first pixel distance between the vertical line of the reference module and the vertical line of the corresponding module to be installed based on positions of any two vertical lines among all vertical line pairs; obtaining a second scaling ratio r2 between a pixel distance and an actual distance within the range of the module to be installed in the second image; multiplying the first pixel distance by the second scaling ratio r2 to obtain the first actual distance.
7 . The photovoltaic module installation method based on surface linear features according to claim 6 , characterized in that said extracting a vertical line of the reference module and a vertical line of the corresponding module to be installed from the second image as an upper module reference line and a lower module reference line, respectively, comprises: the second image comprises a first-type vertical line and a plurality of second-type vertical lines, wherein the first-type vertical line is wider than the second-type vertical line; performing a morphological opening operation on the second image, and then extracting the first-type vertical line using a Hough detection algorithm, wherein the first-type vertical line located in the range of the reference module is used as the upper module reference line, and the first-type vertical line located in the range of the module to be installed is used as the lower module reference line.
8 . The photovoltaic module installation method based on surface linear features according to claim 7 , characterized in that said using horizontal coordinate values of first-type intersection points between a first horizontal detection line and the vertical lines of the reference module to identify positions of corresponding vertical lines comprises: selecting the first horizontal detection line located at a center of the range of the reference module, and obtaining the first-type intersection points between the upper module reference line and the first horizontal detection line; clustering horizontal coordinates of the first-type intersection points to obtain a plurality of clusters; obtaining a midpoint value of each cluster, averaging the midpoint values of all clusters, and using an average value as a position identifier of the upper module reference line; said using horizontal coordinate values of second-type intersection points between a second horizontal detection line and the vertical lines of the module to be installed to identify positions of corresponding vertical lines comprises: selecting the second horizontal detection line located at a center of the range of the module to be installed, and obtaining the second-type intersection points between the lower module reference line and the second horizontal detection line; clustering horizontal coordinates of the second-type intersection points to obtain a plurality of clusters; obtaining a midpoint value of each cluster, averaging the midpoint values of all clusters, and using an average value as a position identifier of the lower module reference line.
9 . The photovoltaic module installation method based on surface linear features according to claim 8 , characterized in that said extracting vertical lines of the reference module and the module to be installed other than the upper module reference line and the lower module reference line from the second image comprises: extracting the second-type vertical lines of the reference module and the second-type vertical lines of the module to be installed in the second image using a Hough detection algorithm; said using horizontal coordinate values of first-type intersection points between a first horizontal detection line and the vertical lines of the reference module to identify positions of corresponding vertical lines comprises: obtaining the first-type intersection points between the second-type vertical lines of the reference module and the first horizontal detection line; clustering the second-type vertical lines of the reference module based on horizontal coordinates of the first-type intersection points to obtain a plurality of clusters, and using a midpoint value of each cluster as a position identifier of the second-type vertical line of the corresponding reference module; said using horizontal coordinate values of second-type intersection points between a second horizontal detection line and the vertical lines of the module to be installed to identify positions of corresponding vertical lines comprises: obtaining the second-type intersection points between the second-type vertical lines of the module to be installed and the second horizontal detection line; clustering the second-type vertical lines of the module to be installed based on horizontal coordinates of the second-type intersection points to obtain a plurality of clusters, and using a midpoint value of each cluster as a position identifier of the second-type vertical line of the corresponding module to be installed.
10 . The photovoltaic module installation method based on surface linear features according to claim 6 , characterized in that said calculating a first pixel distance between the vertical line of the reference module and the vertical line of the corresponding module to be installed based on positions of any two vertical lines among all vertical line pairs comprises: constructing a third point set, wherein points of the third point set correspond to the vertical line pairs, and the horizontal coordinate value of each point in the third point set is the horizontal coordinate value of the first-type intersection point, and the vertical coordinate value is the horizontal coordinate value of the second-type intersection point; fitting the third point set to obtain a third linear equation, wherein an independent variable of the third linear equation is a position of the vertical line of the reference module in the vertical line pairs, and a dependent variable is a position of the vertical line of the corresponding module to be installed; obtaining a corresponding dependent variable based on the third linear equation when the independent variable is a horizontal coordinate value of a center of the second image, and subtracting the independent variable value from the dependent variable value to obtain the first pixel distance.
11 . The photovoltaic module installation method based on surface linear features according to claim 7 , characterized in that said calculating a second actual distance between a lower edge horizontal line of the reference module and an upper edge horizontal line of the module to be installed based on the second image comprises: calculating a pixel distance y1 from a center of the second image to the lower edge horizontal line of the reference module, and a pixel distance y2 from the center to the upper edge horizontal line of the module to be installed; obtaining a first proportional relationship r1 between a pixel distance and an actual distance at the range of the reference module, and the second proportional relationship r2 between the pixel distance and the actual distance at the range of the module to be installed; multiplying the pixel distance y1 by the first proportional relationship r1 to obtain the corresponding actual distance I; multiplying the pixel distance y2 by the second proportional relationship r2 to obtain the corresponding actual distance II; obtaining the second actual distance based on the actual distance I and the actual distance II.
12 . The photovoltaic module installation method based on surface linear features according to claim 11 , characterized in that said obtaining a first proportional relationship r1 between a pixel distance and an actual distance at the range of the reference module comprises: calculating distances between adjacent second-type vertical lines at the range of the reference module based on positions of the second-type vertical lines of the reference module; obtaining a pixel length of a cell at the range of the reference module in the second image based on the distances between all adjacent second-type vertical lines at the range of the reference module; obtaining the first proportional relationship r1 between the pixel distance and the actual distance of the range of the reference module based on the pixel length and an actual length of the cell in the range of the reference module; said obtaining a second proportional relationship r2 between the pixel distance and an actual distance of the range of the module to be installed comprises: calculating distances between adjacent second-type vertical lines within the range of the module to be installed based on positions of the second-type vertical lines of the module to be installed in the second image; obtaining a pixel length of a cell within the range of the module to be installed in the second image based on the distances between all adjacent second-type vertical lines within the range of the module to be installed; obtaining a second proportional relationship r2 between the pixel distance and the actual distance of the range of the module to be installed based on the pixel length and an actual length of the cell in the range of the module to be installed.
13 . A module installation robot, characterized by comprising: a robotic arm, which is configured to grasp, move, and adjust a module to be installed; a camera, which is located at an end of the robotic arm, configured for obtaining an image containing a reference module and the module to be installed; a memory, which is configured to store computer programs; a processor, which is configured to implement the photovoltaic module installation method based on surface linear features as described in any one of claim 1 to 12 when running the computer programs.

Description

FIELD OF THE DISCLOSURE The present application relates to the field of photovoltaic module installation, in particular to a photovoltaic module installation method based on surface linear features and a module installation robot. BACKGROUND Photovoltaic modules can convert solar energy into electricity and are essential for using clean energy. However, setting up a photovoltaic power station requires huge labor and is extremely dangerous. Now, autonomous installation of photovoltaic modules is possible with module installation robots. These robots use visual recognition to determine the installation location of the photovoltaic modules and then install these photovoltaic modules. However, factors such as the long-distance movement of the robotic arm can lead to large installation errors, currently the rough installation is possible, and the installation errors is difficult to control. SUMMARY OF THE INVENTION One of the purposes of the present application is to address the problems existing in the prior art and provide a photovoltaic module installation method based on surface linear features and a module installation robot. The technical solutions provided by the present application are as follows: A photovoltaic module installation method based on surface linear features is used for a module installation robot. The module installation robot comprises a robotic arm and a camera located at an end of the robotic arm. The installation method comprises: grasping a module to be installed via a robotic arm, moving the module to a fine-tuning position, and hovering it at the fine-tuning position, wherein the fine-tuning position is located above an installation location and spaced from the installation location by a first preset distance;obtaining a first image including a reference module and the module to be installed via the camera, wherein the reference module is a previously installed photovoltaic module immediately close to the module to be installed;calculating an average angle between horizontal lines of the reference module and horizontal lines of the module to be installed in the first image, and performing a first fine-tuning adjustment on the module to be installed based on the average angle so that the horizontal lines of the reference module and the horizontal lines of the module to be installed are parallel;obtaining a second image including the reference module and the first fine-tuned module to be installed via the camera;calculating a first actual distance between vertical lines of the reference module and vertical lines of the corresponding module to be installed, and a second actual distance between a lower edge horizontal line of the reference module and an upper edge horizontal line of the module to be installed based on the second image;translating the module to be installed along a horizontal direction based on the first actual distance so as to align the vertical lines of the reference module with the vertical lines of the corresponding module to be installed;translating the module to be installed along a vertical direction based on the second actual distance so as to maintain a second preset distance between the reference module and the module to be installed;pressing the module to be installed downward to the installation location. In some embodiments, said calculating an average angle between horizontal lines of the reference module and horizontal lines of the module to be installed in the first image comprises: determining edges of the reference module and the module to be installed in the first image;distinguishing the reference module and the module to be installed in the first image based on the edges;calculating an average inclination angle of the horizontal lines of the reference module in the first image;calculating an average inclination angle of the horizontal lines of the module to be installed in the first image;obtaining the average angle between the horizontal lines of the reference module and the horizontal lines of the module to be installed based on the average inclination angle of the horizontal lines of the reference module and the average inclination angle of the horizontal lines of the module to be installed. In some embodiments, said determining edges of the reference module and the module to be installed in the first image comprises: performing a morphological opening operation on the first image, and then extracting edge horizontal lines of the two modules in the first image using a Hough detection algorithm;obtaining intersection points between the edge horizontal lines and a first vertical detection line, wherein the first vertical detection line is a straight line located at a center of the first image and parallel to a Y-axis;clustering the edge horizontal lines based on vertical coordinates of the intersection points to obtain a plurality of clusters, and identifying a position of the corresponding edge horizontal line by using a midpoint value of each clus