CN-122019694-A - Terminal device and security query method for external LLM service

Abstract

The present invention provides a terminal device and a security QUERY (QUERY) method for external LLM service. The security query method includes receiving an AI query. The security query method also includes processing private data stored in the memory of the terminal device according to the AI query to obtain the private hint. The security query method also includes transmitting the AI query to a second language generation model. The security query method also includes receiving an initial answer generated by the second language generation model from the AI query. The security query method also includes inputting the initial answer and the private prompt to the first language generation model of the terminal device to obtain a final answer generated by the first language generation model of the terminal device.

Inventors

• HUANG YUMING
• LI YONGJUN
• LIN YANBO

Assignees

• 旺宏电子股份有限公司

Dates

Publication Date

20260512

Application Date

20241209

Priority Date

20241108

Claims (16)

1. 1. A terminal device includes A user interface configured to receive an AI query; A processor coupled to the user interface; a memory coupled to the processor and configured to store private data; a neural engine coupled to the processor and configured to execute a first language generation model, and A communication module coupled to the processor, Wherein the processor is configured to: When an AI query is entered from the user interface, the private data is processed according to the AI query to obtain a private hint, The communication module transmits the AI query to a second language generation model of an external server, An initial answer generated by the second language generative model is received by the communication module, And inputting the initial answer and the private prompt into the first language generation model to obtain a final answer, Wherein the number of model parameters of the second language generation model is greater than the number of model parameters of the first language generation model.
2. 2. The terminal device of claim 1, wherein the processor executes a decision model configured to determine whether the terminal device meets a solution requirement of the AI query based on capabilities of the first language generation model executed by the neural engine of the terminal device.
3. 3. The terminal device of claim 1, wherein the first language generation model modifies the initial answer based on the private prompt to produce the final answer.
4. 4. The terminal device of claim 1, wherein the processor is configured to execute a hint engineering application to the private data stored in the memory to obtain the private hint based on the AI query.
5. 5. The terminal device of claim 1, wherein the processor is configured to perform a model fine-tuning of the private data stored in the memory to obtain the private hint based on the AI query.
6. 6. The terminal device of claim 1, wherein the processor is configured to perform a search enhancement generation to obtain an embedded vector for searching the private data stored in the memory to obtain the private hint, Wherein the embedded vector is generated from a vector database that embeds the AI query and the private data.
7. 7. A security query method for an external LLM service, comprising: Receiving an AI query by a user interface of a terminal device; Processing, by a processor of the terminal device, private data stored in a memory of the terminal device according to the AI query to obtain a private hint; Transmitting the AI query to a second language generation model of an external server by a communication module of the terminal device; receiving, by the communication module, an initial answer generated by the second language generation model based on the AI query, and Inputting the initial answer and the private prompt into a first language generation model executed by a neural engine of the terminal device to obtain a final answer generated by the first language generation model of the terminal device, Wherein the number of model parameters of the second language generation model is greater than the number of model parameters of the first language generation model.
8. 8. The security query method of claim 7 wherein deriving the final answer generated by the first language generation model of the terminal device includes modifying the initial answer based on the private prompt to generate the final answer.
9. 9. The security query method of claim 7, wherein processing the private data stored in the memory of the terminal device to obtain the private hint comprises the processor being configured to execute a hint engineering application to the private data stored in the memory to obtain the private hint based on the AI query.
10. 10. The security query method of claim 7, wherein processing the private data stored in the memory of the terminal device to obtain the private hint comprises the processor being configured to execute a model trim application to the private data stored in the memory to obtain the private hint based on the AI query.
11. 11. The security query method of claim 7, wherein processing the private data stored in the memory of the terminal device to obtain the private hint comprises the processor being configured to perform a search enhancement generation to obtain an embedded vector for searching the private data stored in the memory to obtain the private hint, Wherein the embedded vector is generated from a vector database that embeds the AI query and the private data.
12. 12. A security query method for an external LLM service, comprising: Receiving an AI query by a user interface of a terminal device; Processing, by a processor of the terminal device, private data stored in a memory of the terminal device according to the AI query to obtain a private hint; Judging whether the terminal device meets a solution requirement of the AI query or not according to the capability of a first language generation model executed by a neural engine of the terminal device by the processor of the terminal device; When the terminal device is judged to meet the answer requirement of the AI query, directly inputting the AI query and the private prompt into the first language generation model to obtain a final answer generated by the first language generation model of the terminal device; When the terminal device is judged not to meet the answer requirement of the AI query, a communication module of the terminal device transmits the AI query to a second language generation model of the external LLM service; Receiving, by the communication module, an initial answer generated by the second language generation model of the external LLM service based on the AI query, and Inputting the initial answer and the private prompt to the first language generation model of the terminal device to obtain the final answer generated by the first language generation model of the terminal device, Wherein the number of model parameters of the second language generation model is greater than the number of model parameters of the first language generation model.
13. 13. The security query method of claim 12 wherein the final answer derived from the first language generation model includes revising the initial answer based on the private prompt to produce the final answer.
14. 14. The security query method of claim 12, wherein processing the private data stored in the memory of the terminal device to obtain the private hint comprises the processor being configured to execute a hint engineering application to the private data stored in the memory to obtain the private hint based on the AI query.
15. 15. The security query method of claim 12, wherein processing the private data stored in the memory of the terminal device to obtain the private hint comprises the processor being configured to execute a model trim application to the private data stored in the memory to obtain the private hint based on the AI query.
16. 16. The security query method of claim 12, wherein processing the private data stored in the memory of the terminal device to obtain the private hint comprises the processor being configured to perform a search enhancement generation to obtain an embedded vector for searching the private data stored in the memory to obtain the private hint, Wherein the embedded vector is generated from a vector database that embeds the AI query and the private data.

Description

Terminal device and security query method for external LLM service Technical Field The present invention is directed to a security QUERY (QUERY) method for an external LLM service, and a terminal apparatus for performing the security QUERY method for the external LLM service. Background The application and demand for generating artificial intelligence (GENERATIVE ARTIFICIAL INTELLIGENT, GAI) has grown rapidly in the near future, and the application demand for GAI in terminal devices, such as smartphones, edge devices, or portable devices has also increased. Considering that GAI shows great potential for its widespread use, for example, chatGPT of OpenAI company generates native answers to various questions based on a large language model (Large language model, LLM), the computation and memory overhead of such LLM is quite high, whereas known LLM services can only be used on cloud computing, which brings security concerns and the internet demands of necessity. Furthermore, LLM is known to have a illusion that it may result from virtually incorrect or meaningless answers due to limitations in training data, bias in models, or complexity in language content. Additional data may be used as a hint (prompt) to reduce the illusion of LLM, but this also raises security concerns. Because the additional data may be confidential (proprietary) files, graphics or images that must be uploaded to the cloud. Thus, techniques for obtaining AI answers from LLM services without uploading private data to the cloud while improving the accuracy of AI answers are desirable. Disclosure of Invention The present invention describes a technique for secure querying of a terminal device with SLM and private data in cooperation with LLM services provided by a cloud server or an external computing system. A first aspect of the present invention features a terminal device (EDGE DEVICE). The terminal device comprises a user interface arranged to receive AI-queries. The terminal device also includes a processor coupled to the user interface. The terminal device also includes a memory coupled to the processor and configured to store private data (PRIVATE DATA). The terminal device also includes a neural engine coupled to the processor and configured to execute the first language generation model (language generation model). The terminal device also includes a communication module coupled to the processor. The processor is configured to process the private data according to the AI query to obtain a private prompt (private prompt) when the AI query is entered from the user interface. The processor is also configured to transmit an AI query by the communication module to a second language generation model of the external server. The processor is also configured to receive, by the communication module, an initial answer generated by the second language generative model. The processor is also configured to input the initial answer and the private prompt into the first language generation model to obtain a final answer. The number of model parameters of the second language generation model is greater than the number of model parameters of the first language generation model. A second aspect of the present invention features a security query method for an external LLM service. The security query method includes receiving, by a user interface of the terminal device, an AI query. The security query method also includes processing, by the processor of the terminal device, the private data stored in the memory of the terminal device according to the AI query to obtain the private hint. The security query method also includes transmitting, by the communication module of the terminal device, the AI query to a second language generation model of the external LLM service. The security query method also includes receiving, by the communication module, an initial answer generated by the second language generation model from the AI query. The security query method also includes inputting the initial answer and the private prompt to a first language generation model executed by the neural engine of the terminal device to obtain a final answer generated by the first language generation model of the terminal device. The number of model parameters of the second language generation model is greater than the number of model parameters of the first language generation model. A third aspect of the present invention is characterized by a security query method for an external LLM service. The security query method includes receiving, by a user interface of the terminal device, an AI query. The security query method also includes processing, by the processor of the terminal device, the private data stored in the memory of the terminal device according to the AI query to obtain the private hint. The security query method also includes determining, by the processor of the terminal device, whether the terminal device meets the answer requirements of the AI query based on t