CN-121711377-B - UDP-network-oriented large-model cross-domain remote calling method and device

CN121711377BCN 121711377 BCN121711377 BCN 121711377BCN-121711377-B

Abstract

A method for remotely calling large model cross domain includes combining large model dialogue request and history dialogue data of local service end to form model call data, packing said model call data into UDP data packet according to UDP data packet format, sending said UDP data packet to far end, combining all data packets of same complete message after decrypting and checking said data packet by far end proxy service to reconstruct dialogue request data, submitting said dialogue request data to large model to obtain stream response data returned by large model, returning said data to local through UDP protocol after packing, returning said data packet to client through SSE stream output data format after processing said data packet by local service end. The invention can realize the correct transmission of the large model dialogue data in the UDP network environment, can effectively solve the problems of network delay and data packet loss, ensures the reliability and the safety of the data, and satisfies the long-distance use of the large model in the special network.

Inventors

CHEN TAO
BI SHENG
RUAN YING
ZHOU DAN
WU CUILING
LI JIE

Assignees

之江实验室

Dates

Publication Date: 20260508
Application Date: 20260212

Claims (9)

1. A large model cross-domain remote calling method facing UDP network is characterized by comprising the following steps: (1) Analyzing the request to obtain the current request parameter, and combining with the history dialogue data to form the transmission data of remote model call; (2) Encapsulating the transmission data according to a preset UDP data packet format to obtain at least one UDP data packet, transmitting the UDP data packet to a remote proxy service through a UDP protocol, and locally caching the transmitted data packet; (3) When all data packets belonging to the same complete message are successfully received, combining all the data packets to reconstruct complete dialogue request data; (4) The remote proxy service submits the complete dialogue request data to the large model, acquires the streaming response data returned by the large model, combines the streaming response data and encapsulates the streaming response data into at least one UDP data packet, and sends the UDP data packet back to the local server; (5) The local server receives the UDP data packet from the remote proxy service, distributes the data packet to the corresponding message buffer for processing according to the dialogue ID, sequentially checks and overtime processes the messages in the message buffer through an independent message processing thread, and returns the correct message to the client through an SSE streaming output mode; (6) In the data transmission process, the receiving end checks the UDP data packet and feeds back the confirmation information, and the transmitting end judges whether to retransmit the data packet according to the received confirmation information or the fact that the confirmation information is not received after overtime.
2. The large-model cross-domain remote call method for UDP network according to claim 1, wherein the request parameters comprise question information, request terminal IP, user information, dialogue mode; the history dialogue data is obtained from a relational database, and the data format of the history dialogue data is JSON format.
3. The large-model cross-domain remote call method for UDP network of claim 1, wherein said UDP packet format comprises a header of a fixed length and a message body of a limited length, wherein said header comprises a packet type, a session ID, a complete message ID, a packet sequence number, a total packet number, a data checksum, and a data length field, and wherein said message body is limited in length to within a predetermined byte to adapt to network transmission requirements.
4. The large-model cross-domain remote call method for UDP network according to claim 3, wherein in the step (2), when the transmission data is packaged according to a preset UDP data packet format, the transmission data called by the remote model is decomposed according to the limit length of the message body, the message body is subjected to data encryption and then a packet header is added to be packaged into a UDP data packet, wherein the data encryption adopts an AES-CTR stream encryption algorithm to ensure that the lengths of the data packets are consistent, and the UDP data packet comprises the packet header and the encrypted message body.
5. The UDP network oriented large model cross-domain remote call method of claim 1, wherein in step (2), the locally buffering the transmitted data packet specifically comprises: the method comprises the steps of taking a data packet ID in a UDP data packet as a key, taking a corresponding UDP data packet as a value, putting a key value pair into a data packet buffer for buffering so as to resend when the data packet is abnormal, and cleaning the overtime data packet in the data packet buffer at regular time, wherein the data packet buffer is constructed based on a key value pair set.
6. The method of claim 1, wherein in the step (3), the receiving end uses the complete message ID as a key to temporarily store the single UDP packet successfully checked in the buffer structure, and when detecting that the number of UDP packets associated with the same complete message ID has reached the total number of packets indicated in the packet header, it triggers the data inclusion and operation.
7. The method according to claim 1, wherein in the step (5), a separate message buffer is built for each session, the message buffer is built based on an ordered key-value pair set, and the received message is stored with the complete message ID as a key for masking duplicate data and identifying data missing.
8. The method of claim 7, wherein the message processing thread maintains a current output sequence number, circularly checks a complete message ID of header data in the message buffer, takes the complete message ID of the header data out and encapsulates the complete message ID of the header data into an SSE stream output data format if the complete message ID of the header data is equal to the current output sequence number, returns the complete message ID of the header data to the client through the SSE stream output mode, increases the current output sequence number while discarding the header data if the complete message ID of the header data is smaller than the current output sequence number, waits until timeout, and initiates a data retransmission request or terminates a session according to a timeout strategy.
9. An apparatus for implementing the UDP network oriented large model cross-domain remote invocation method of any one of claims 1-8, comprising: The local service module is used for receiving a large model dialogue request of the client, combining and generating transmission data of remote model call, receiving and processing large model streaming response data from remote proxy service, and returning the large model streaming response data to the client in an SSE streaming output mode; the data encapsulation and transmission module is used for encapsulating transmission data called by the remote model or streaming response data of the large model into UDP data packets with predefined formats, transmitting the UDP data packets between the local service module and the remote proxy service module through a UDP protocol, and taking charge of caching, confirmation and retransmission management of the data packets; The remote proxy service module is used for receiving, checking and merging UDP data packets from the local service module to reconstruct dialogue request data, submitting the dialogue request data to the large model, acquiring streaming response data returned by the large model, and packaging the streaming response data through the data packaging and transmission module and then transmitting the streaming response data back to the local service module.

Description

UDP-network-oriented large-model cross-domain remote calling method and device Technical Field The invention relates to the technical field of information technology artificial intelligence application, in particular to a large-model cross-domain remote calling method and device for a UDP (user datagram protocol) network. Background Along with the rapid development of AI technology, a large language model (Large Language Model, LLM, simply referred to as "large model") is playing an increasingly important role in the production and life of human beings, and the large language model obtains deep language understanding and generating capability through massive data training, can perform smooth dialogue communication like human beings, solves various problems, and completes various tasks such as text creation, code writing, language translation, information induction and the like, so that the large language model becomes an intelligent assistant for improving work efficiency and exciting creativity. Under the general environment, the OpenAI-style API interface has become a fact standard in the field of large models, and is an HTTP interface which abstracts large model capability into calls of request and response modes, takes a message list as input and takes large model completion as output. In the field of the front of scientific research, because of the limited environment, such as certain space engineering environments, high-performance computing power cannot be built locally to run model reasoning, meanwhile, the network environment is special, the problems of instability, high time delay and the like exist, external communication is only applicable to UDP (user datagram protocol ) protocols, and a OpenAI mode cannot be used for conveniently calling a remote model like a general environment. Aiming at the application requirement of the large model of the special environment, the invention provides a UDP-network-oriented large-model cross-domain remote calling method, which enables the application of the large model in the special network environment to be possible and provides an effective solution for network problems such as packet loss, time delay and the like. Disclosure of Invention The invention aims to provide a large-model cross-domain remote calling method and device for a UDP network, aiming at the defects of the prior art. The invention can solve the problem of cross-domain remote call of large model interaction in UDP special network environment. The invention aims at realizing the following technical scheme that the first aspect of the embodiment of the invention provides a large-model cross-domain remote calling method facing UDP network, which comprises the following steps: (1) Analyzing the request to obtain the current request parameter, and combining with the history dialogue data to form the transmission data of remote model call; (2) Encapsulating the transmission data according to a preset UDP data packet format to obtain at least one UDP data packet, transmitting the UDP data packet to a remote proxy service through a UDP protocol, and locally caching the transmitted data packet; (3) When all data packets belonging to the same complete message are successfully received, combining all the data packets to reconstruct complete dialogue request data; (4) The remote proxy service submits the complete dialogue request data to the large model, acquires the streaming response data returned by the large model, combines the streaming response data and encapsulates the streaming response data into at least one UDP data packet, and sends the UDP data packet back to the local server; (5) The local server receives the UDP data packet from the remote proxy service, distributes the data packet to the corresponding message buffer for processing according to the dialogue ID, sequentially checks and overtime processes the messages in the message buffer through an independent message processing thread, and returns the correct message to the client through an SSE streaming output mode; (6) In the data transmission process, the receiving end checks the UDP data packet and feeds back the confirmation information, and the transmitting end judges whether to retransmit the data packet according to the received confirmation information or the fact that the confirmation information is not received after overtime. Further, the request parameters comprise question information, request terminal IP, user information and dialogue mode; the history dialogue data is obtained from a relational database, and the data format of the history dialogue data is JSON format. Further, the UDP data packet format comprises a packet header with a fixed length and a message body with a limited length, wherein the packet header comprises a data packet type, a dialogue ID, a complete message ID, a data packet ID, a packet serial number, a total packet number, a data checksum and a data length field, and the length of the message body