Search

CN-122019202-A - Lightweight real-time image recognition system and method for intelligent terminal

CN122019202ACN 122019202 ACN122019202 ACN 122019202ACN-122019202-A

Abstract

The invention discloses an intelligent terminal-oriented lightweight real-time image recognition system and method, wherein an algorithm dynamic sensing module collects hardware states of a terminal to generate algorithm level data, a module linkage control center generates a scheduling instruction according to the algorithm level data, a lightweight attention module generates an attention weight matrix according to the scheduling instruction, a layered feature extraction module triggers feature extraction of a corresponding semantic level according to the scheduling instruction, the attention weight matrix is fused into a convolution calculation process to output a feature map, a recognition result output module processes the feature map to obtain a recognition result and feeds back confidence coefficient data, a small sample self-learning module generates new scene feature parameters through measurement and learning, and the module linkage control center drives attention module parameter iteration according to the new scene feature parameters and carries out incremental updating on fine-grained semantic feature layers. The invention realizes terminal side calculation force self-adaption, scene customization identification and light self-learning.

Inventors

  • ZHANG HAIQIANG
  • ZHANG LIJUAN
  • LUO ZHICONG

Assignees

  • 浙江常青树信息技术有限责任公司
  • 青岛凯兴电子技术有限公司
  • 浙江蔚蓝芯片科技有限公司

Dates

Publication Date
20260512
Application Date
20260415

Claims (10)

  1. 1. The utility model provides a lightweight real-time image recognition system towards intelligent terminal which characterized in that includes: the system comprises a module linkage control center, a calculation power dynamic sensing module, a light attention module, a layered characteristic extraction module, a small sample self-learning module and a recognition result output module, wherein the calculation power dynamic sensing module, the light attention module, the layered characteristic extraction module, the small sample self-learning module and the recognition result output module are respectively connected with the module linkage control center; The power calculation dynamic sensing module is used for acquiring hardware state parameters of the intelligent terminal, processing the hardware state parameters to obtain terminal power calculation grade data and sending the terminal power calculation grade data to the module linkage control center; The module linkage control center is used for generating and issuing a scheduling instruction to the light attention module and the layered feature extraction module based on a preset linkage threshold value table according to the terminal calculation power level data, driving the light attention module to iterate parameters according to the new scene feature parameters sent by the small sample self-learning module, and controlling the layered feature extraction module to incrementally update the fine granularity semantic feature layer; The light attention module is used for dynamically adjusting the attention calculation range according to the scheduling instruction, generating an attention weight matrix and sending the attention weight matrix to the hierarchical feature extraction module; The hierarchical feature extraction module is used for triggering feature extraction of a corresponding semantic hierarchy according to the scheduling instruction, fusing the attention weight matrix into a convolution calculation process of feature extraction, and outputting a feature map to the recognition result output module; The recognition result output module is used for processing the feature map to obtain a recognition result and feeding confidence coefficient data of the recognition result back to the module linkage control center; The small sample self-learning module is used for processing the collected small sample image data of the new scene through measurement learning, obtaining the characteristic parameters of the new scene and sending the characteristic parameters to the module linkage control center.
  2. 2. The intelligent terminal-oriented lightweight real-time image recognition system according to claim 1, wherein the module linkage control center comprises: the system comprises a threshold storage unit, a linkage threshold generation unit, a characteristic extraction unit and a characteristic extraction unit, wherein the threshold storage unit is used for storing a preset linkage threshold table and a preset industrial scene basic characteristic parameter library, the linkage threshold table comprises a mapping relation between a calculation power level, an attention calculation range and a characteristic extraction level, and the industrial scene basic characteristic parameter library comprises target characteristic parameters of a plurality of typical intelligent manufacturing scenes; The scheduling decision unit is connected with the threshold storage unit and is used for receiving the terminal calculation force grade data sent by the calculation force dynamic sensing module, inquiring the linkage threshold table according to the terminal calculation force grade data, determining an attention calculation range and a feature extraction level matched with the current calculation force grade, generating a scheduling instruction based on a determination result, and issuing the scheduling instruction to the light attention module and the layered feature extraction module; The parameter evolution unit is connected with the threshold storage unit and is used for receiving the new scene characteristic parameters sent by the small sample self-learning module, storing the new scene characteristic parameters into the industrial scene basic characteristic parameter library, updating the mapping relation related to the new scene characteristic parameters in the linkage threshold table according to the new scene characteristic parameters, driving the light attention module to perform parameter iteration, and controlling the hierarchical characteristic extraction module to perform incremental updating on fine-granularity semantic characteristic layers.
  3. 3. The intelligent terminal-oriented lightweight real-time image recognition system according to claim 2, wherein the module linkage control center further comprises: the power calculation monitoring unit is used for receiving the terminal power calculation grade data sent by the power calculation dynamic sensing module and monitoring the change trend of the terminal power calculation grade data in real time; The emergency adaptation unit is connected with the power calculation monitoring unit and the scheduling decision unit and is used for triggering a power calculation emergency adaptation strategy when the power calculation monitoring unit monitors that the change amplitude of the power calculation grade data of the terminal exceeds a preset abrupt change threshold value, forcibly downwards adjusting the current power calculation grade by one grade, and instructing the scheduling decision unit to regenerate a scheduling instruction according to the downwards adjusted power calculation grade, and sending the scheduling instruction to the light attention module and the layered feature extraction module so as to freeze non-core attention calculation and switch to feature extraction of lower layers; And the restoration judging unit is connected with the power calculation monitoring unit and the emergency adaptation unit and is used for continuously monitoring the power calculation grade data of the terminal after the emergency adaptation unit triggers the emergency adaptation strategy, and canceling the emergency adaptation strategy and restoring to the conventional scheduling mode of the scheduling decision unit when the power calculation grade data of the terminal is restored to the normal range and stably exceeds a preset stability time threshold.
  4. 4. The intelligent terminal-oriented lightweight real-time image recognition system of claim 1, wherein the computing power dynamic perception module comprises: The parameter acquisition unit is used for acquiring hardware state parameters of the intelligent terminal in real time based on a system application programming interface of the intelligent terminal and adjusting the acquisition frequency according to a preset acquisition frequency threshold; The parameter processing unit is connected with the parameter acquisition unit and is used for carrying out normalization processing on the hardware state parameters, dividing the processed hardware state parameters into a plurality of calculation force levels according to a preset calculation force level threshold value and generating the terminal calculation force level data; The preset calculation force level threshold is calibrated based on hardware configuration of the intelligent terminal and preset industrial scene identification requirements, and manual fine adjustment of a terminal side is supported.
  5. 5. The intelligent terminal-oriented lightweight real-time image recognition system of claim 1, wherein the lightweight attention module comprises: The channel attention unit is used for receiving the scheduling instruction issued by the module linkage control center, dynamically determining a target high-frequency characteristic channel range according to the scheduling instruction, and distributing attention weights to characteristic channels in the target high-frequency characteristic channel range to generate channel attention weights; The space attention unit is used for receiving the scheduling instruction issued by the module linkage control center, dynamically determining a typical space target area range according to the scheduling instruction, performing attention mask calculation on the space position in the typical space target area range, and generating a space attention mask; The weight fusion unit is respectively connected with the channel attention unit and the space attention unit and is used for fusing the channel attention weight and the space attention mask, generating the attention weight matrix and sending the attention weight matrix to the hierarchical feature extraction module; the target high-frequency characteristic channel range and the typical space target area range are defined based on a preset industrial scene priori knowledge base, and the calculation ranges of the channel attention unit and the space attention unit are dynamically adjusted according to the scheduling instruction.
  6. 6. The intelligent terminal-oriented lightweight real-time image recognition system of claim 1, wherein the hierarchical feature extraction module comprises: the hierarchy control unit is used for receiving the scheduling instruction issued by the module linkage control center, determining the semantic hierarchy of the feature extraction according to the scheduling instruction, and generating a hierarchy trigger signal; The feature extraction unit is connected with the hierarchy control unit and is used for receiving the hierarchy trigger signal and the attention weight matrix sent by the light attention module, triggering a feature extraction network corresponding to a semantic hierarchy according to the hierarchy trigger signal, fusing the attention weight matrix into a convolution calculation process of the feature extraction network and outputting the feature map; The data interaction unit is connected with the feature extraction unit and is used for extracting intermediate data generated by the feature extraction unit in the feature extraction process, the intermediate data are sent to the module linkage control center, and the intermediate data are used for parameter iteration of the small sample self-learning module.
  7. 7. The intelligent terminal-oriented lightweight real-time image recognition system according to claim 6, wherein the feature extraction unit comprises: The bottom layer feature extraction subunit is used for extracting bottom layer contour features of the industrial image and corresponds to a preset low calculation power level requirement; the middle layer feature extraction subunit is connected with the bottom layer feature extraction subunit and is used for extracting middle layer texture features based on the bottom layer profile features and corresponds to a preset middle calculation force level requirement; The high-level feature extraction subunit is connected with the middle-level feature extraction subunit and is used for extracting fine-grained semantic features based on the middle-level texture features and corresponds to a preset high calculation force level requirement; And the hierarchical control unit triggers any one or more of the bottom layer feature extraction subunit, the middle layer feature extraction subunit and the high layer feature extraction subunit to execute feature extraction according to the scheduling instruction, and fuses the attention weight matrix into a convolution calculation process of the subunit triggered to execute.
  8. 8. The intelligent terminal-oriented lightweight real-time image recognition system according to claim 1, wherein the recognition result output module comprises: the classification detection unit is used for receiving the feature map output by the layered feature extraction module, processing the feature map through a preset lightweight class detection head and generating a recognition result, wherein the recognition result comprises target category, defect position and confidence data; The confidence coefficient judging unit is connected with the classification detecting unit and is used for acquiring the confidence coefficient data in the identification result, comparing the confidence coefficient data with a preset confidence coefficient threshold value, and generating an iteration trigger signal and sending the iteration trigger signal to the module linkage control center when the confidence coefficient data is lower than the confidence coefficient threshold value so as to trigger the small sample self-learning module to perform iterative optimization again; and the result output unit is connected with the classification detection unit and is used for outputting the identification result to a local storage medium and an industrial local area network transmission interface and adapting to the industrial control and data tracing requirements of the intelligent manufacturing production line.
  9. 9. The intelligent terminal-oriented lightweight real-time image recognition system of claim 1, wherein the small sample self-learning module comprises: The image preprocessing unit is used for receiving new scene small sample image data acquired by the terminal, preprocessing the new scene small sample image data and generating preprocessed small sample image data, wherein the number of the new scene small sample image data is smaller than a preset small sample number threshold value; The feature mining unit is connected with the image preprocessing unit and used for carrying out feature clustering on the preprocessed small sample image data based on measurement learning, mining a target high-frequency feature channel and a typical space target area corresponding to the new scene small sample image data, generating the new scene feature parameters and sending the new scene feature parameters to the module linkage control center; The parameter iteration unit is connected with the module linkage control center and used for carrying out incremental updating on the parameter table of the lightweight attention module according to the driving of the module linkage control center, carrying out incremental updating on the convolution kernel of the fine granularity semantic feature layer of the layered feature extraction module and simultaneously freezing the parameters of the bottom contour feature layer and the middle texture feature layer of the layered feature extraction module; when the parameter iteration unit performs incremental updating, the proportion of the frozen model parameters to the total model parameters is not lower than a preset parameter freezing proportion threshold, and the time consumption of single iteration is smaller than a preset fine tuning time consumption threshold.
  10. 10. The lightweight real-time image recognition method for the intelligent terminal, which is applied to the lightweight real-time image recognition system for the intelligent terminal as claimed in any one of claims 1 to 9, is characterized by comprising the following steps: Step S1, the power calculation dynamic sensing module collects hardware state parameters of an intelligent terminal, processes the hardware state parameters to obtain terminal power calculation grade data, and sends the terminal power calculation grade data to the module linkage control center; Step S2, the module linkage control center generates and transmits a scheduling instruction to the light attention module and the layered feature extraction module based on a preset linkage threshold table according to the terminal calculation power level data, and drives the light attention module to perform parameter iteration according to the new scene feature parameters transmitted by the small sample self-learning module, and controls the layered feature extraction module to perform incremental update on the fine granularity semantic feature layer; Step S3, the light attention module dynamically adjusts the attention calculation range according to the scheduling instruction, generates an attention weight matrix and sends the attention weight matrix to the hierarchical feature extraction module; Step S4, triggering feature extraction of a corresponding semantic hierarchy by the hierarchical feature extraction module according to the scheduling instruction, and fusing the attention weight matrix into a convolution calculation process of feature extraction to output a feature map to the recognition result output module; S5, the recognition result output module processes the feature map to obtain a recognition result, and feeds confidence coefficient data of the recognition result back to the module linkage control center; And S6, the small sample self-learning module processes the collected small sample image data of the new scene through measurement learning, obtains the characteristic parameters of the new scene and sends the characteristic parameters to the module linkage control center.

Description

Lightweight real-time image recognition system and method for intelligent terminal Technical Field The invention relates to the technical field of image recognition, in particular to a lightweight real-time image recognition system and method for an intelligent terminal. Background Along with the continuous development of intelligent manufacturing, industrial automation and edge intelligent technology, intelligent terminals such as industrial flat plates, embedded acquisition terminals, inspection terminals and the like are widely applied to industrial image recognition scenes such as part detection, material recognition, product tracing, equipment inspection and the like. The application generally requires the intelligent terminal to directly complete the tasks of image acquisition, processing and recognition under the field environment, so that higher requirements are put on the instantaneity, the light weight, the localization processing capability and the recognition accuracy of an image recognition system. However, the intelligent terminal is limited by hardware resources, power consumption level and operation environment, and in practical application, multiple tasks such as data acquisition, communication transmission, control interaction and the like are often required to be simultaneously undertaken, so that the available processing resources of the terminal have fluctuation. Under the condition, the image recognition task is difficult to always consider the processing efficiency and the recognition performance under different running states, and the problem that the response speed and the recognition effect are difficult to balance easily occurs, so that the application effect of the system in an actual industrial scene is influenced. In addition, the image data of the industrial site generally has the characteristics of complex background, multiple interference factors, fine target characteristics and the like, is easily influenced by factors such as illumination change, reflection, shielding, vibration, fouling and the like, causes larger fluctuation of image quality, and further increases the difficulty of recognition processing. Under such a complex application environment, the existing image recognition scheme is difficult to simultaneously consider the operation cost and the recognition precision, and particularly in the fine-grained target recognition or complex scene recognition task, the stability and the adaptability of the system still need to be improved. In addition, in intelligent manufacturing scenarios, with product switching, operating condition changes, or the continued appearance of new types of targets, image recognition tasks are often faced with strong scene change demands. Especially under the condition that the number of new scene samples is limited, the existing scheme often has the problems of long adaptation period and low adjustment efficiency, and is difficult to meet the application requirements of quick on-site deployment and continuous update in time. Further, existing related schemes generally lack effective overall coordination in terms of terminal resource adaptation, recognition performance maintenance, scene change response, and the like, resulting in systems that remain lacking overall robustness, flexibility, and continuous adaptation capability in complex, dynamic industrial application environments. Therefore, a lightweight real-time image recognition technical scheme suitable for intelligent terminal deployment is needed, so that recognition instantaneity and recognition accuracy are considered under the condition of limited resources, and the adaptability of the system to complex industrial scenes and new scene changes is improved, so that the actual requirements of intelligent manufacturing field application are better met. Disclosure of Invention Aiming at the defects existing in the prior art, the invention aims to provide a lightweight real-time image recognition system and method for an intelligent terminal, which are used for realizing real-time efficient, accurate and stable image recognition and self-adaptive update under the conditions of dynamic change of terminal computing power, complex industrial scene and small sample of new scene. In order to achieve the purpose, the invention provides the following technical scheme that the lightweight real-time image recognition system for the intelligent terminal comprises: the system comprises a module linkage control center, a calculation power dynamic sensing module, a light attention module, a layered characteristic extraction module, a small sample self-learning module and a recognition result output module, wherein the calculation power dynamic sensing module, the light attention module, the layered characteristic extraction module, the small sample self-learning module and the recognition result output module are respectively connected with the module linkage control center; The power calculation d