CN-122022126-A - Journey planning and intelligent navigation system based on multi-mode large model

CN122022126ACN 122022126 ACN122022126 ACN 122022126ACN-122022126-A

Abstract

The invention relates to the technical field of data processing and discloses a journey planning and intelligent navigation system based on a multi-mode large model, which comprises a heterogeneous data unified access module, a heterogeneous data processing module and a data processing module, wherein the heterogeneous data unified access module is used for receiving heterogeneous original data streams from a user side and Internet services; the multi-modal semantic alignment and enhancement module is used for carrying out feature extraction and cross-modal semantic fusion on multi-modal heterogeneous signals submitted by users and further comprises a structural intention and context awareness module, a dynamic space-time environment map construction module, an accommodation-traffic-activity joint optimization engine module and a context self-adaptive closed-loop navigation module. According to the scheme, the problem of lodging, traffic and activity planning and splitting is relieved through the end-to-end collaborative optimization framework, fuzzy intention of a user is converted into computable structural constraint, multidimensional dynamic environment factors are fused in a planning stage, a situation awareness and hierarchical re-planning mechanism is introduced in an execution stage, a closed-loop decision process is formed, and the practical adaptability, individuation level and execution continuity of a travel scheme are improved.

Inventors

WANG QINTING
LIU HAO
JIANG LIHUI
WANG ZAORONG

Assignees

浙江深大智能科技有限公司

Dates

Publication Date: 20260512
Application Date: 20251229

Claims (10)

1. A travel planning and intelligent navigation system based on a multi-mode large model is characterized by comprising the following modules: the heterogeneous data unified access module is used for uniformly receiving heterogeneous original data streams from a user side and Internet services, and carrying out protocol adaptation, format standardization and preliminary verification; The multi-mode semantic alignment and enhancement module is responsible for carrying out depth feature extraction and cross-mode semantic fusion on multi-mode heterogeneous signals submitted by users to generate uniform and context enhanced intention expression vectors; The structured intention and situation awareness module is used for converting the intention representation vector into a structured constraint set which can be directly called by the optimization engine, fusing external space-time situation information and constructing a complete planning initial condition; the dynamic space-time environment map construction module is used for fusing the static attribute of the interest point with the real-time environment signal to construct a high-dimensional, time-varying and computable environment state model; the accommodation-traffic-activity joint optimization engine module is used for synchronously optimizing accommodation site selection, daily traffic path and activity arrangement under the double conditions of user intention constraint and dynamic space-time environment map; And the situation self-adaptive closed-loop navigation module is used for dynamically adjusting, guiding and optimizing an initial travel scheme based on real-time environment change and user behavior feedback in the actual travel process of the user.
2. The multi-modal large model-based trip planning and intelligent tour system according to claim 1, wherein the heterogeneous data unified access module comprises: the interest point static attribute presetting unit is used for loading the full amount of interest point basic data of the target city in an initialization stage and taking the full amount of interest point basic data as a geographic basis of the journey planning; The user request standardization unit is used for carrying out unified analysis, structured packaging and semantic reservation on the original travel demands submitted by the user through a plurality of interaction channels; And the external environment data active pulling unit is used for automatically and parallelly calling a plurality of Internet open interfaces by the system after the user requests to trigger so as to acquire dynamic environment data related to the travel space-time range.
3. The multi-modal large model based trip planning and intelligent tour system of claim 2 wherein the interaction channel in the user request normalization unit comprises: A natural language channel for receiving free description in text or voice form; a visual reference channel, which allows a user to upload scenic spots, accommodation spots or street view images as semantic anchor points; A structured parameter channel, namely providing a form type input interface and forcedly collecting key planning parameters; And the historical behavior authorization channel is used for interfacing a third party platform or a local user database and pulling a historical travel record on the premise of definite authorization of a user.
4. The multi-modal large model-based trip planning and intelligent tour system according to claim 2, wherein the external environment data actively pulling the environment data in the unit comprises: meteorological data, namely acquiring precipitation probability, air temperature, wind power level and the like of a target city, and storing the data in a time sequence form; Traffic network data, namely obtaining geographic coordinates of all interest points in an area, congestion conditions and public transportation line topology; scenic spot activity data, namely acquiring the current day closing time of each scenic spot, the starting time, the ending time and the temporary closing state of the scenic spot activity; And acquiring real-time price, state and whether to provide baggage deposit attribute of the candidate accommodation.
5. The multi-modal large model-based trip planning and intelligent tour system according to claim 1, wherein the multi-modal semantic alignment and enhancement module comprises a text-to-speech semantic coding unit and a visual semantic enhancement unit; The visual semantic enhancement unit is used for carrying out high-level semantic analysis on an uploaded image of a user and carrying out dynamic fusion with text semantics, and also comprises two auxiliary tasks, wherein the auxiliary tasks are as follows: A scene classification sub-network for providing high-level context information to help the system to further understand scene types represented by the uploaded pictures of the user; And the key object detection sub-network is used for identifying local elements with discriminant in the image and increasing the understanding depth of the image content.
6. The multi-modal large model-based trip planning and intelligent tour system of claim 1 wherein the structured intent and context awareness module comprises the following elements: The behavior preference constraint extraction unit is used for converting natural language expression into formal preference constraint conditions through a predefined rule template and large model joint extraction technology, and identifying structural requirements of a user on the traveling process; The luggage sensitivity modeling unit automatically deduces subjective rejection degree of a user on replacement accommodation points through semantic analysis, accesses a lightweight fully-connected regression head on the basis of schematic representation vectors, and outputs luggage sensitivity after supervision and fine adjustment; and the season-event context fusion unit is used for automatically judging whether special climates or social events exist according to the travel date and the destination and generating the context of the global context label.
7. The multi-modal large model-based trip planning and intelligent tour system according to claim 6, wherein the behavior preference constraint extraction unit comprises: time anchored constraints, identifying a user's time mandatory requirements for a particular activity; Rhythm control type constraint, which reflects the user's desire for daily activity intensity; movement restriction type constraints, identifying user restrictions on traffic patterns or physical exertion.
8. The multi-modal large model-based trip planning and intelligent tour system according to claim 1, wherein the dynamic space-time environmental map construction module comprises: The weather disturbance modeling unit is used for calculating a single-day comprehensive trip adverse index based on multidimensional hour-by-hour meteorological elements and combining with the actual active period of the user trip, and comprehensively quantifying all-weather trip risks; a public transportation reachability dynamic modeling unit for evaluating the feasibility of reaching a scenic spot or an activity place from a certain accommodation candidate spot through the existing public transportation system before a target time on a specific date; The road congestion correction unit dynamically adjusts road traffic time according to the real-time road condition data; and a scenic spot activity time window binding unit for extracting and structuring specific start-stop time of fixed performance, navigation or time limiting experience and taking the specific start-stop time as a hard time window which is necessary to be covered by the journey.
9. The multi-modal large model based trip planning and intelligent tour system of claim 1 wherein the accommodation-traffic-activity joint optimization engine module comprises: the space-time function clustering unit is used for constructing a commute time matrix among scenic spots by taking the corrected actual transit time as a distance measure, and generating a plurality of scenic spot clusters by adopting a hierarchical clustering algorithm; generating a group of candidate accommodation strategies in a specific date interval for each scenic spot cluster, wherein the accommodation strategies comprise the steps of firstly determining a space-time coverage center of the scenic spot cluster, and then screening from a total candidate accommodation pool according to price, luggage sensitivity and traffic mode constraint conditions of a certain number of scenic spots in the scenic spot cluster; And the multi-target combined unit is used for setting a multi-target combined cost function and comprehensively measuring the quantitative indexes of the journey and accommodation scheme in five dimensions of commuting efficiency, economy, luggage burden, time waste and environmental risk.
10. The multi-modal large model-based trip planning and intelligent tour guide system according to claim 1, wherein the context adaptive closed-loop tour guide module comprises a real-time context awareness unit, a trip deviation detection and classification unit, a lightweight online re-planning engine and an adaptive prompt generation unit; the travel deviation detection and classification unit identifies inconsistent types of the current state of the user and the original scheme, judges whether re-planning needs to be triggered, and comprises three types of deviation mechanisms: Slight deviation that the current position of the user is in the electronic fence of the planned scenic spot on the same day, and the current time falls in the elastic time window of the scenic spot, which is regarded as normal journey fluctuation, and the system does not interfere; Moderate deviation, namely that a user deviates from the original journey but is not completely derailed, and the system triggers local optimization on the same day and pushes a light prompt; And (3) severely deviating that the main path of the day is completely disabled due to external events or subjective abandonment by a user, and the system starts the full-dose re-planning of the day.

Description

Journey planning and intelligent navigation system based on multi-mode large model Technical Field The invention relates to the technical field of data processing, in particular to a journey planning and intelligent navigation system based on a multi-mode large model. Background A travel planning and intelligent guiding system based on a multi-mode large model is an intelligent service system which integrates multiple-mode data such as text, images, voice, map, video and the like, utilizes a Large Language Model (LLM) and multi-mode expansion thereof such as VLMM to understand user intention, environment context and travel resources, processes the data, generates a personalized and executable multi-day travel scheme, and can output structured contents such as daily scenery spot arrangement, play duration, traffic mode, dining accommodation recommendation, product price, total budget and the like, thereby realizing full-link intelligent travel experience. The current journey planning and intelligent tour guide system based on a multi-mode large model has a certain progress in the aspect of fusing multi-source heterogeneous data such as texts, images, geographic information and POIs, but in the practical application facing complex multi-day tour scenes, significant technical limitations still exist, the core problem is that the system lacks collaborative modeling capability on three-way relations of accommodation-scenic spots and journey, so that a generating scheme is unbalanced between feasibility and user experience, in particular, the existing system usually processes accommodation spot recommendation and journey generation as independent modules, traffic cost, baggage handling frequency, user physical constraint and real-time dynamic data are not jointly considered under a unified optimization framework, the split architecture is difficult to quantitatively balance between 'fixed accommodation so as to reduce baggage handling' and 'segmented nearby entry so as to compress commute time', and in particular, sub-optimal solution of daily average moving time or overlength frequent accommodation replacement is easy to occur in a cross-region tour scene with distributed in scenic spot space, in addition, although the system relies on the large model to have strong semantic understanding capability, the system still has the effect of matching shallow key layer constraint extraction, the system is still difficult to realize the fact that the intelligent flow is fully matched with the constraint structure, and can not really realize the realization of the real-time constraint and the real-time dynamic data is completely fused with the intelligent flow, but the real-time information is still can be completely optimized, and the real-time information is not really expressed to realize the aspect of the intelligent flow is completely recommended and is fully improved, and can be completely converted. Disclosure of Invention The invention aims to provide a journey planning and intelligent tour guide system based on a multi-mode large model, which solves the problem of cooperative optimization of accommodation strategies and journey lines in multi-day multi-scenic spot tourism, and constructs an end-to-end intelligent journey planning framework integrating multi-mode perception, user intention deep understanding, space-time constraint modeling, real-time data fusion and combined optimization decision. In order to achieve the above purpose, the present invention adopts the following technical scheme: A travel planning and intelligent navigation system based on a multi-mode large model comprises the following modules: the heterogeneous data unified access module is used for uniformly receiving heterogeneous original data streams from a user side and Internet services, and carrying out protocol adaptation, format standardization and preliminary verification; The multi-mode semantic alignment and enhancement module is responsible for carrying out depth feature extraction and cross-mode semantic fusion on multi-mode heterogeneous signals submitted by users to generate uniform and context enhanced intention expression vectors; The structured intention and situation awareness module is used for converting the intention representation vector into a structured constraint set which can be directly called by the optimization engine, fusing external space-time situation information and constructing a complete planning initial condition; the dynamic space-time environment map construction module is used for fusing the static attribute of the interest point with the real-time environment signal to construct a high-dimensional, time-varying and computable environment state model; the accommodation-traffic-activity joint optimization engine module is used for synchronously optimizing accommodation site selection, daily traffic path and activity arrangement under the double conditions of user intention constraint and dynamic space-tim