CN-121980406-A - Intelligent cabin target recognition supervision system and method based on large model

Abstract

The invention discloses an intelligent cabin target recognition and supervision system and method based on a large model, which relates to the technical field of multi-source data analysis, and the invention generates a key attention model, a performance prediction model and a fault prediction model of each target behavior, and according to real-time comprehensive records in a real-time updating period, the key attention model, and determining whether the real-time comprehensive record is an important record, predicting the real-time performance score of the next updating period through a performance prediction model, judging whether the real-time updating period is an abnormal period, and determining whether the intelligent cabin target recognition system has a fault through a fault prediction model, so that the operation reliability and the safety of the intelligent cabin target recognition system are greatly improved.

Inventors

• WU YAN
• CAO QINGLAN
• Zhai Lingya

Assignees

• 南京绛门信息科技有限公司

Dates

Publication Date

20260505

Application Date

20260123

Claims (10)

1. 1. The intelligent cabin target recognition supervision method based on the large model is characterized by comprising the following steps of: Step S100, summarizing records of the same target behaviors in the history comprehensive records of the intelligent cabin target recognition system, collecting corresponding environment parameters, calculating environment scores, and then combining the environment scores with recognition accuracy to generate a key attention model aiming at each target behavior; Step 200, presetting a training period, acquiring a comprehensive record in an updating period, judging whether the comprehensive record is a key record or not by using a key attention model, counting the duty ratio characteristic of the key record and the occurrence ratio of each target behavior, calculating an operation scoring characteristic and an environment scoring trend characteristic, calculating a performance score based on the identification total duration and the identification accuracy, and generating a performance prediction model by combining the performance score with the duty ratio characteristic of the key record, the occurrence ratio of the target behavior, the operation scoring and the environment scoring trend characteristic; Step 300, predicting the performance score of the next updating period based on the performance prediction model, comparing the performance score with the performance score of the current updating period, calculating the descending amplitude, judging whether the cycle is an abnormal period according to the descending amplitude and the performance score of the next period, acquiring the performance score of the abnormal period in the characteristic time period, constructing a performance score time sequence, and generating a fault prediction model by combining the duration of the characteristic time period; And step 400, judging whether the record is the key record or not through the key attention model according to the real-time comprehensive record in the real-time updating period, predicting the real-time performance score of the next updating period through the performance prediction model, judging whether the record is an abnormal period or not, and judging whether the intelligent cabin target recognition system has a fault or not through the fault prediction model if the record is the abnormal period.
2. 2. The method for supervising the intelligent cockpit target recognition based on the large model according to claim 1, wherein the step S100 comprises the following steps: S101, arranging a plurality of video acquisition devices in an intelligent cabin, acquiring video data in the intelligent cabin through the video acquisition devices, identifying the video data by utilizing a target identification algorithm based on the video data, determining that preset target behaviors exist in the video data, acquiring environment parameters in the intelligent cabin and operation parameters of an intelligent cabin system when the target behaviors are identified, combining the environment parameters and the operation parameters, constructing comprehensive records of the target behaviors, and uploading the comprehensive records to the intelligent cabin target identification system; Step S102, collecting target behaviors corresponding to the history comprehensive records, classifying and summarizing the history comprehensive records based on the target behaviors, in a history comprehensive record set of a certain target behavior, advancing environmental parameters in a certain history comprehensive record, carrying out normalization calculation on the environmental parameters, presetting a weight of each environmental parameter, carrying out weighted summation on the environmental parameters and the corresponding weight, and calculating to obtain an environmental score; step S103, in the history comprehensive record of the target behavior, collecting feedback information of a user, comparing the feedback information with the target behavior identification result, determining the identification correctness of the target behavior based on the comparison result, marking the history comprehensive record with the identification correctness being incorrect as a history error comprehensive record, and marking the history comprehensive record with the identification correctness being correct as a history correct comprehensive record; And step S104, based on the history error comprehensive records and the history correct comprehensive records, assigning values to the history comprehensive records, taking environmental scores as input, taking the assignments of the history comprehensive records as output, training through a decision tree model, wherein the training process is to divide nodes and determine parameters of the decision tree model, complete model training and generate a key attention model of the target behavior.
3. 3. The method for supervising the intelligent cockpit target recognition based on the large model according to claim 2, wherein the step S200 includes the following steps: Step S201, selecting a plurality of continuous days as a training period, presetting an updating period of target identification, acquiring comprehensive records in the updating period, acquiring environmental parameters of the comprehensive records, calculating to obtain environmental scores, extracting target behaviors of the comprehensive records, inputting the environmental scores into a focused attention model of the target behaviors, calculating to obtain focused attention probability, presetting a focused attention probability threshold, marking comprehensive records larger than or equal to the focused attention probability threshold as focused records, and marking comprehensive records smaller than the focused attention probability threshold as non-focused records; Step S202, generating different record sets based on key and non-key records, in a certain record set, collecting operation parameters corresponding to a certain record at each time stamp, wherein the operation parameters comprise CPU occupancy rate, memory occupancy rate, communication delay and task scheduling queue length, carrying out normalization calculation on the operation parameters, presetting a weight of each operation parameter, carrying out weighted summation on the normalized operation parameters, calculating to obtain an operation score of each time stamp, summarizing the operation score of each time stamp, calculating to obtain an average value and a variance of the record operation score, carrying out normalization calculation on the average value and the variance of the operation score, combining the average value and the variance of the normalized operation score, and constructing an operation score characteristic; step S203, counting the number of key records and each target behavior in an updating period, calculating to obtain the duty ratio characteristic of the key records and the occurrence ratio of each target behavior in the updating period, summarizing the environmental score of each comprehensive record, calculating to obtain the average value and variance of the environmental score, respectively carrying out normalization calculation on the average value and variance of the environmental score, and combining the average value and variance of the normalized environmental score to construct the environmental score trend characteristic; step S204, acquiring the identification time length corresponding to each comprehensive record in the updating period, calculating to obtain the identification total time length, counting the identification accuracy of the updating period, carrying out normalization calculation on the identification total time length and the identification accuracy, carrying out weighted summation on the normalized identification total time length and the normalized identification accuracy, and calculating to obtain a performance score; Step S205, respectively carrying out normalization calculation on the duty ratio characteristic and the appearance proportion, taking the duty ratio characteristic and the appearance proportion after normalization in the kth updating period, the running scoring characteristic and the environment scoring trend characteristic as inputs, taking the performance score of the kth+1th updating period as output, and training through a random forest regression model, wherein the training process is to optimally adjust the parameters of the random forest regression model through a cross verification mode, so as to generate a performance prediction model.
4. 4. A method of supervising intelligent cockpit target recognition based on a large model according to claim 3, wherein the step S300 comprises the steps of: Step 301, acquiring normalized duty ratio features, occurrence ratios, running scoring features and environment scoring trend features in an update period, inputting the features into a performance prediction model, predicting performance scores of a next update period, calculating to obtain descending amplitude of performance scores of a current update period and the next update period, presetting a performance scoring threshold and a descending amplitude threshold, judging that the current update period is in an abnormal state if the performance scoring threshold is lower than the descending amplitude threshold, and marking the current update period as an abnormal period; Step S302, after an intelligent cabin target recognition system fails, determining an update period corresponding to the failure as a failure period, setting a time period between every two adjacent failure periods as a characteristic time period, acquiring abnormal periods in the characteristic time period, sequencing the abnormal periods according to a time sequence, constructing a performance grading time sequence, taking the performance grading time sequence as an input, taking the duration of the characteristic time period as an output, training through a long-short-period memory network model, training through an optimizer in the training process by adopting a mean square error loss function, and adjusting parameters of the long-short-period memory network model to generate a failure prediction model.
5. 5. The method for supervising the intelligent cockpit target recognition based on the large model according to claim 4, wherein the step S400 comprises the following steps: Step S401, in a real-time updating period, acquiring a target behavior identified by a real-time comprehensive record, collecting real-time environment parameters of the real-time comprehensive record, calculating to obtain a real-time environment score, inputting the real-time environment score into a key attention model of the target behavior, calculating to obtain a key attention probability, and determining whether the real-time comprehensive record is a key record; Step S402, acquiring normalized duty ratio characteristics, occurrence proportion, operation scoring characteristics and environment scoring trend characteristics in a real-time updating period, inputting the normalized duty ratio characteristics, occurrence proportion, operation scoring characteristics and environment scoring trend characteristics into a performance prediction model, and predicting real-time performance scores of the next updating period; Step S403, judging whether the real-time updating period is an abnormal period or not based on the real-time performance score, if not, executing step S401, if so, constructing a performance score time sequence by the performance score of the abnormal period and the performance score of the previous abnormal period, inputting the performance score time sequence into a fault prediction model, calculating to obtain a fault prediction score, presetting a fault prediction score threshold, and if the fault prediction score threshold is exceeded, reminding a worker that an intelligent cabin target recognition system has a fault and overhauling the intelligent cabin target recognition system.
6. 6. A large model-based intelligent cabin target recognition monitoring system for implementing the large model-based intelligent cabin target recognition monitoring method as set forth in any one of claims 1 to 5, wherein the system comprises a focused attention model module, a performance prediction model module, a fault prediction model module and a fault diagnosis module; summarizing records of the same target behaviors in the history comprehensive records of the intelligent cabin target recognition system, collecting corresponding environment parameters, calculating environment scores, and then combining the environment scores with recognition accuracy to generate a focus attention model aiming at each target behavior; The performance prediction model module is used for presetting a training period, acquiring comprehensive records in an updating period, judging whether the comprehensive records are important records or not by utilizing an important attention model, calculating the duty ratio characteristic of the important records and the occurrence ratio of each target behavior, calculating the operation scoring characteristic and the environment scoring trend characteristic, calculating the performance score based on the identification total duration and the identification accuracy, and generating a performance prediction model by combining the performance scoring with the duty ratio characteristic of the important records, the occurrence ratio of the target behaviors, the operation scoring trend characteristic and the environment scoring trend characteristic; The fault prediction model module predicts the performance score of the next update period based on the performance prediction model, compares the performance score with the performance score of the current update period, calculates the descending amplitude, judges whether the cycle is an abnormal period according to the descending amplitude and the performance score of the next period, acquires the performance score of the abnormal period in the characteristic time period, constructs a performance score time sequence, and combines the duration of the characteristic time period to generate a fault prediction model; And the fault diagnosis module judges whether the record is a key record or not through a key attention model according to the real-time comprehensive record in the real-time update period, predicts the real-time performance score of the next update period through a performance prediction model, judges whether the record is an abnormal period or not, and judges whether the intelligent cabin target recognition system has faults or not through the fault prediction model if the record is the abnormal period.
7. 7. The large model-based intelligent cockpit target recognition supervision system according to claim 6, wherein the focused attention model module comprises a computing environment scoring unit and a focused attention model generating unit: the computing environment scoring unit is used for collecting target behaviors corresponding to the history comprehensive records, classifying and summarizing the history comprehensive records based on the target behaviors, advancing environmental parameters in a certain history comprehensive record in a history comprehensive record set of a certain target behavior, carrying out normalized calculation on the environmental parameters, presetting a weight of each environmental parameter, carrying out weighted summation on the environmental parameters and the corresponding weights, and calculating to obtain an environment score; The key attention model generating unit is used for collecting feedback information of a user in a history comprehensive record of the target behavior, comparing the feedback information with a target behavior identification result, determining identification correctness of the target behavior based on the comparison result, marking the history comprehensive record with the identification correctness being incorrect as a history error comprehensive record, marking the history comprehensive record with the identification correctness being correct as a history correct comprehensive record, assigning values to the history comprehensive record based on the history error comprehensive record and the history correct comprehensive record, taking environmental scores as input, assigning values to the history comprehensive record as output, training through a decision tree model, and performing node division and parameter determination for the decision tree model in the training process, completing model training, and generating the key attention model of the target behavior.
8. 8. The intelligent cockpit target identification and supervision system based on a large model according to claim 6, wherein the performance prediction model module comprises a judgment key recording unit and a performance prediction model generation unit: Selecting a plurality of continuous days as a training period, presetting an updating period of target identification, acquiring comprehensive records in the updating period, acquiring environment parameters of the comprehensive records, calculating to obtain environment scores, extracting target behaviors of the comprehensive records, inputting the environment scores into a key attention model of the target behaviors, calculating to obtain key attention probability, presetting a key attention probability threshold, marking the comprehensive records larger than or equal to the key attention probability threshold as key records, and marking the comprehensive records smaller than the key attention probability threshold as non-key records; The generation performance prediction model unit: generating different record sets based on key and non-key records, in a certain record set, collecting operation parameters corresponding to a certain record at each time stamp, wherein the operation parameters comprise CPU occupancy rate, memory occupancy rate, communication delay and task scheduling queue length, carrying out normalization calculation on the operation parameters, presetting a weight of each operation parameter, carrying out weighted summation on the normalized operation parameters, calculating to obtain an operation score of each time stamp, summarizing the operation score of each time stamp, calculating to obtain an average value and a variance of the record operation score, carrying out normalization calculation on the average value and the variance of the operation score, combining the average value and the variance of the normalized operation score to construct an operation score feature, counting the number of key records and each target behavior in an updating period, calculating to obtain the duty ratio characteristic of key record and the occurrence ratio of each target behavior in the updating period, summarizing the environmental scores of each comprehensive record, calculating to obtain the average value and variance of the environmental scores, respectively carrying out normalization calculation on the average value and variance of the environmental scores, combining the average value and variance of the normalized environmental scores to construct environmental score trend characteristics, obtaining the identification time length corresponding to each comprehensive record in the updating period, calculating to obtain the identification total time length, counting the identification accuracy of the updating period, carrying out normalization calculation on the identification total time length and the identification accuracy, carrying out weighted summation on the normalized identification total time length and the identification accuracy, calculating to obtain performance scores, respectively carrying out normalization calculation on the duty ratio characteristic and the occurrence ratio, carrying out normalization calculation on the duty ratio characteristic and the occurrence ratio in the kth updating period, and the running scoring feature and the environment scoring trend feature are used as inputs, the performance score of the (k+1) th updating period is used as output, the training is carried out through the random forest regression model, and the training process is to carry out optimization adjustment on parameters of the random forest regression model in a cross verification mode, so that a performance prediction model is generated.
9. 9. The large model-based intelligent cockpit target recognition supervision system according to claim 6, wherein the fault prediction model module comprises a determining abnormal period unit and a generating fault prediction model unit: The abnormal period determining unit is used for obtaining normalized duty ratio characteristics, occurrence ratio, running scoring characteristics and environment scoring trend characteristics in an updating period, inputting the duty ratio characteristics, the occurrence ratio, the running scoring characteristics and the environment scoring trend characteristics into a performance prediction model, predicting performance scores of the next updating period, calculating to obtain descending amplitude of the performance scores of the current updating period and the next updating period, presetting a performance scoring threshold value and a descending amplitude threshold value, judging that the current updating period is in an abnormal state if the performance scoring threshold value is lower than the descending amplitude threshold value and exceeds the descending amplitude threshold value, and marking the current updating period as an abnormal period; and the fault prediction model generating unit is used for determining an update period corresponding to the fault as a fault period after the intelligent cabin target recognition system breaks down, setting a time period between every two adjacent fault periods as a characteristic time period, acquiring abnormal periods in the characteristic time period, sequencing the abnormal periods according to a time sequence, constructing a performance grading time sequence, taking the performance grading time sequence as an input, taking the duration of the characteristic time period as an output, training through a long-short-period memory network model, training through an optimizer in the training process by adopting a mean square error loss function, adjusting parameters of the long-short-period memory network model, and generating the fault prediction model.
10. 10. The large model-based intelligent cockpit target recognition supervision system according to claim 6, wherein the fault diagnosis module comprises a real-time performance scoring unit and a fault determining unit: the calculation real-time performance scoring unit is used for acquiring target behaviors identified by a real-time comprehensive record in a real-time updating period, collecting real-time environment parameters of the real-time comprehensive record, calculating to obtain a real-time environment score, inputting the real-time environment score into a focused attention model of the target behaviors, calculating to obtain a focused attention probability, determining whether the real-time comprehensive record is a focused record, acquiring normalized duty ratio characteristics, appearance proportion, operation scoring characteristics and environment scoring trend characteristics in the real-time updating period, inputting the normalized duty ratio characteristics, appearance proportion, operation scoring characteristics and environment scoring trend characteristics into a performance prediction model, and predicting real-time performance scores of the next updating period; And the fault determining unit judges whether the real-time updating period is an abnormal period or not based on the real-time performance score, if the real-time updating period is not the abnormal period, the step S401 is executed, if the real-time updating period is not the abnormal period, the performance score of the abnormal period and the performance score of the abnormal period before are constructed, a performance score time sequence is input into a fault prediction model, the fault prediction score is obtained through calculation, a fault prediction score threshold value is preset, and if the fault prediction score threshold value is exceeded, a worker is reminded that an intelligent cabin target recognition system has faults and overhauls.

Description

Intelligent cabin target recognition supervision system and method based on large model Technical Field The invention relates to the technical field of multi-source data analysis, in particular to an intelligent cabin target identification supervision system and method based on a large model. Background Along with the continuous improvement of the intelligent degree of automobiles, the intelligent cabin is developed into a comprehensive vehicle-mounted system integrating various sensors, multiple computing platforms and multiple service applications from an early functional unit taking man-machine interaction as a core, the intelligent cabin system at the present stage is designed to focus on information acquisition, fusion and result display, an operation monitoring and target recognition mechanism of the intelligent cabin system is generally judged based on a single perception model or a fixed rule, when the system load frequently fluctuates, the functional modules are dynamically switched, or multiple types of anomalies occur simultaneously, the existing monitoring mode is difficult to form continuous and complete system state cognition, a monitoring result is easy to present discrete characteristics, and a monitoring system is difficult to accurately recognize whether the system is in a performance degradation or abnormal evolution stage; in the prior art, monitoring or identification results are generally presented in a form of static state information, manual analysis is mainly relied on, continuous modeling and unified semantic hierarchy interpretation mechanisms for a system state change process are lacked, dynamic evaluation and early warning for a system operation health state are difficult to support, under the application scenes of multi-source data high-frequency acquisition and rapid change of operation working conditions, the traditional method is difficult to timely perceive abnormal accumulation and evolution trend, short-time fluctuation and potential systematic risks cannot be effectively distinguished, so that fault identification hysteresis is easy to be caused, and the stability and safety of an intelligent cabin system are influenced; Based on the above, it is necessary to provide a large-model-based intelligent cabin target recognition monitoring system and method, which perform unified semantic understanding on multi-source perception data and system operation data, perform continuous modeling and reasoning analysis on system state changes, realize comprehensive judgment on system operation health, abnormal evolution path and potential failure period, and dynamically adjust a monitoring strategy and an intervention mechanism according to the comprehensive judgment, so as to improve reliability, predictability and overall intelligent level of an intelligent cabin system. Disclosure of Invention The invention aims to provide an intelligent cabin target recognition and supervision system and method based on a large model, so as to solve the problems in the prior art. In order to solve the technical problems, the invention provides the technical scheme that the intelligent cabin target identification supervision method based on the large model comprises the following steps: Step S100, summarizing records of the same target behaviors in the history comprehensive records of the intelligent cabin target recognition system, collecting corresponding environment parameters, calculating environment scores, and then combining the environment scores with recognition accuracy to generate a key attention model aiming at each target behavior; Step 200, presetting a training period, acquiring a comprehensive record in an updating period, judging whether the comprehensive record is a key record or not by using a key attention model, counting the duty ratio characteristic of the key record and the occurrence ratio of each target behavior, calculating an operation scoring characteristic and an environment scoring trend characteristic, calculating a performance score based on the identification total duration and the identification accuracy, and generating a performance prediction model by combining the performance score with the duty ratio characteristic of the key record, the occurrence ratio of the target behavior, the operation scoring and the environment scoring trend characteristic; Step 300, predicting the performance score of the next updating period based on the performance prediction model, comparing the performance score with the performance score of the current updating period, calculating the descending amplitude, judging whether the cycle is an abnormal period according to the descending amplitude and the performance score of the next period, acquiring the performance score of the abnormal period in the characteristic time period, constructing a performance score time sequence, and generating a fault prediction model by combining the duration of the characteristic time period; And step 400