CN-122002009-A - Unmanned aerial vehicle low-delay high-definition image transmission method

CN122002009ACN 122002009 ACN122002009 ACN 122002009ACN-122002009-A

Abstract

The invention provides a method for unmanned aerial vehicle low-delay high-definition image transmission, which comprises the steps of collecting and processing video frames by an image collecting module, carrying out H.265 hardware coding by an encoding module by adopting an intra-frame refreshing mechanism, storing the coded frames into an annular buffer area, monitoring the number of frames N of the buffer area in real time by a code rate control module, progressively adjusting an output code rate through a state machine, emptying the buffer area and triggering an emergency signal if N exceeds a high water level threshold value, reading data by a transmission module, dynamically adjusting FEC redundancy according to a packet loss rate fed back by a receiving end, sending after encoding, receiving and attempting FEC decoding by the receiving end, if successful, decoding a display image and feeding back the packet loss rate, if unsuccessful, judging whether to initiate ARQ retransmission request by combining with a time stamp, if effective, requesting retransmission, adjusting FEC by a transmitting end according to feedback, responding the emergency signal, and continuously monitoring the buffer area. The invention solves the problems of coding delay, transmission reliability and congestion control lag, and realizes low-delay high-definition image transmission in a long-distance complex environment.

Inventors

LIU JUN
DENG CHENGLONG
ZHUANG LIHUA
LIANG JIUZHEN
XUE CAIXIA
SHEN ZIQIAN

Assignees

常州大学

Dates

Publication Date: 20260508
Application Date: 20260318

Claims (9)

1. The method for transmitting the low-delay high-definition image of the unmanned aerial vehicle is characterized by comprising the following steps of: step S01, starting a transmitting end and a receiving end, collecting and processing video frames by an image collecting module, carrying out H.265 hardware coding by an encoding module by adopting an intra-frame refreshing mechanism, and storing the coded video frames into an annular buffer area; Step S02, a code rate control module monitors the number of frames of a buffer area in real time, dynamically adjusts the coding code rate through a state machine model, and executes emergency processing when the number of frames exceeds a high water level threshold value; step S03, receiving data by a receiving end and attempting FEC decoding, if successful, carrying out H.265 decoding display and feeding back packet loss rate, if failed, triggering ARQ retransmission mechanism, and judging retransmission effectiveness by combining with a time stamp; And step S04, the sender dynamically adjusts the FEC redundancy according to the packet loss rate fed back by the receiver, responds to the emergency signal and returns to step S02 to continue execution.
2. The method for low-latency high-definition image transmission of unmanned aerial vehicle according to claim 1, wherein the image acquisition module in step S01 acquires video frame data and processes the video frame data through VI, ISP, VPSS.
3. The method of claim 1, wherein the state machine model in step S02 includes a hold state, a code up state, and a code down state, and the state transition is performed according to a comparison result between the buffer frame number N and the target frame number threshold t_target and the high water level threshold t_high.
4. The method for low-latency high-definition graphics of unmanned aerial vehicle according to claim 3, wherein the state transition specifically comprises: when the device is in a holding state and N is smaller than T_target, starting timing, and switching to a code lifting state if N is continuously smaller than T_target in a preset holding period; When the bit is in a holding state and N=T_target, immediately switching to a code dropping state, and recording the current code rate as a peak code rate; when in the code lifting state and N < T_target, increasing the code rate by one step every time a preset code lifting interval period until reaching a preset upper limit code rate; when in the code-ascending state and n=t_target, immediately switching to the code-descending state; When in a code-dropping state and N=T_target, reducing the code rate by one step every time a preset code-dropping interval period is passed until a preset lower limit code rate is reached; When in the code-down state and N < T_target, switching to the hold state; when N is more than or equal to T_high, the code rate control module triggers emergency treatment, wherein the emergency treatment comprises the steps of emptying the annular buffer zone, recording emergency times and sending an emergency event signal to the transmission module through a message queue.
5. The method of claim 4, wherein the transmission module performs at least one of increasing FEC redundancy, forcing a switching of operating frequency points, and requesting the encoding module to transmit a key frame after receiving the emergency signal.
6. The method for low-delay high-definition image transmission of the unmanned aerial vehicle according to claim 1, wherein the specific steps of the step S03 are as follows: Step S031, the receiving end receives data through the wireless transceiver unit, performs FEC decoding attempt, if decoding is successful, the step S032 is entered, and if decoding is failed, the step S033 is entered; step S032, H.265 decodes and displays the video picture, and simultaneously counts the packet loss rate and feeds back to the sending end; Step S033, triggering ARQ retransmission request, combining with video frame decoding time stamp to judge retransmission validity, if yes, sending retransmission request to the sending end, retransmitting corresponding data packet by the sending end and returning to step S031, if not, discarding retransmission.
7. The method for determining the validity of retransmission in step S033, wherein the receiving end calculates the remaining available time of the data packet to be retransmitted according to the decoding timestamp of the video frame, and if the expected arrival time of the retransmission packet exceeds the decoding time, the receiving end gives up the retransmission request; When triggering selective retransmission, the transmission module combines the decoding time stamp of the video frame to judge whether the data packet to be retransmitted is still effective, and if the predicted arrival time exceeds the decoding time, the retransmission request is abandoned.
8. The method for unmanned aerial vehicle low-delay high-definition image transmission according to claim 1, further comprising the step of frequency hopping anti-interference, wherein the step of monitoring RSSI and packet loss rate of each candidate frequency point in real time is carried out, when the channel quality of the current working frequency point is lower than a preset threshold value, the optimal frequency point is dynamically switched, and synchronous switching is realized between the receiving party and the transmitting party.
9. The utility model provides a device that unmanned aerial vehicle low delay high definition image passed which characterized in that, this device includes: The image acquisition module is configured at the transmitting end and comprises an image sensor, a VI capturing unit of an MIPI interface, an ISP image processing unit and a VPSS video processing unit, and is used for acquiring and processing video frame data; The coding module is connected with the image acquisition module and is used for carrying out H.265 hardware coding on video frame data, the coding module adopts an intra-frame refreshing mechanism to replace a traditional I frame, key frame refreshing loads are uniformly dispersed to a plurality of P frames, code rate peaks are eliminated, and the depth of a coding buffer zone is compressed to 1 frame; The annular buffer area is connected with the coding module and used for buffering coded video frame data, and the capacity of the annular buffer area is preset to be 10 frames; The code rate control module is connected with the annular buffer area and used for monitoring the number N of frames buffered in the annular buffer area in real time and progressively adjusting the output code rate of the coding module through the state machine model according to the comparison result of the number N of frames and a preset threshold value; the transmission module is configured at the sending end and the receiving end and is used for transmitting video data through a wireless channel, the transmission module adopts an FEC and ARQ mixed transmission protocol, dynamically adjusts the redundancy of forward error correction according to the real-time packet loss rate fed back by the receiving end, triggers selective retransmission when FEC error correction fails, and meanwhile combines a video frame decoding time stamp to judge the effectiveness of retransmission so as to avoid invalid retransmission and delay increase; the wireless receiving and transmitting unit is configured at the transmitting end and the receiving end and is responsible for wireless transmission and reception of video data; The receiving end processing module is configured at the receiving end and comprises an FEC decoding unit, an H.265 decoding unit, a display unit and a packet loss rate statistical feedback unit, and is used for receiving video data, performing FEC decoding attempt, H.265 decoding display and real-time statistical packet loss rate feedback to the transmitting end; and the debugging and feedback interface is configured between the sending end and the receiving end and is used for transmitting a packet loss rate feedback signal and an ARQ retransmission request.

Description

Unmanned aerial vehicle low-delay high-definition image transmission method Technical Field The invention relates to the technical field of wireless video transmission, in particular to a method for transmitting a low-delay high-definition image of an unmanned aerial vehicle. Background The unmanned aerial vehicle high-definition digital image transmission system is a core component for realizing remote control and real-time video transmission of the unmanned aerial vehicle. Along with the continuous expansion of unmanned aerial vehicle application scenes, the industry has put forward increasingly stringent requirements on the transmission distance, image quality and end-to-end delay of an image transmission system. However, when the existing unmanned aerial vehicle image transmission technology meets the above requirements, various defects still exist, and the defects mainly appear in the following three aspects: 1. the currently widely adopted H.264/H.265 coding standard relies on periodic I frames as a decoding reference. Since the amount of I-frame data is large, typically up to 5 to 10 times that of P-frames, its generation and transmission can cause the encoder buffer to instantaneously expand, thereby introducing an additional delay of 2-3 frames. Meanwhile, the code rate peak generated by the method is extremely easy to cause network congestion, and further worsens transmission performance. 2. Existing schemes have limitations in reliable transport mechanisms. If the ARQ mechanism is simply relied on, a complete RTT needs to be waited for retransmission recovery after packet loss occurs, which significantly increases end-to-end delay. If only the FEC mechanism is adopted, a fixed redundancy bandwidth needs to be preset, which leads to waste of bandwidth resources when the channel condition is good, and can not effectively correct errors due to insufficient redundancy when the channel is bad. 3. The traditional code rate control strategy is mostly based on the delay indexes such as packet loss rate or RTT fed back by the receiving end, and cannot accurately sense the backlog state of the buffer zone of the sending end in real time. This feedback lag makes the system slow to react when network conditions fluctuate suddenly, easily resulting in buffer overflow, video frame loss, and a sharp rise in delay. In summary, the prior art has relatively isolated response to problems at various levels, and is difficult to cooperate. Therefore, there is a need for an unmanned aerial vehicle image transmission system capable of performing multi-dimensional depth collaborative optimization from encoding and transmission to congestion control, so as to finally realize long-distance, low-delay and high-image-quality stable video transmission. Disclosure of Invention In order to solve the problems, the invention solves the problems of coding delay, transmission reliability and congestion control lag through the collaborative design of intra-frame refreshing coding, dynamic code rate control based on a ring buffer water level and a state machine and FEC and ARQ hybrid transmission protocol, and realizes low-delay high-definition image transmission in a long-distance complex environment. According to the embodiment of the invention, a method for low-delay high-definition image transmission of an unmanned aerial vehicle is provided. In a first aspect of the invention, a method for low-latency high-definition graphics transmission of a unmanned aerial vehicle is provided. The method comprises the following steps: step S01, starting a transmitting end and a receiving end, collecting and processing video frames by an image collecting module, carrying out H.265 hardware coding by an encoding module by adopting an intra-frame refreshing mechanism, and storing the coded video frames into an annular buffer area; Step S02, a code rate control module monitors the number of frames of a buffer area in real time, dynamically adjusts the coding code rate through a state machine model, and executes emergency processing when the number of frames exceeds a high water level threshold value; step S03, receiving data by a receiving end and attempting FEC decoding, if successful, carrying out H.265 decoding display and feeding back packet loss rate, if failed, triggering ARQ retransmission mechanism, and judging retransmission effectiveness by combining with a time stamp; And step S04, the sender dynamically adjusts the FEC redundancy according to the packet loss rate fed back by the receiver, responds to the emergency signal and returns to step S02 to continue execution. Further, the image acquisition module in step S01 acquires video frame data and processes the video frame data through VI, ISP, VPSS. Further, the state machine model in step S02 includes a hold state, an up-code state, and a down-code state, and performs state transition according to the comparison result of the buffer frame number N with the target frame number threshol