CN-121996022-A - Electro-optical mapping system and method for electronic intelligent model

CN121996022ACN 121996022 ACN121996022 ACN 121996022ACN-121996022-A

Abstract

The application provides an electro-optical mapping system and method for an electronic intelligent model, wherein the system comprises a semantic feature extraction module, an optical parameter generation module and an optical calculation module, wherein the semantic feature extraction module is used for carrying out semantic feature coding on input images, texts or multi-mode data and outputting semantic feature representation representing the calculation capability of the electronic intelligent model, the optical parameter generation module is used for generating optical calculation parameters for driving the optical calculation module according to the semantic feature representation under the condition of introducing physical constraint conditions of a target optical calculation module, and the optical calculation module is used for executing optical calculation based on the optical calculation parameters so as to reproduce the corresponding calculation capability of the electronic intelligent model in the aspect of functions or calculation behaviors and realize the mapping of the electronic intelligent model to the optical calculation system. The application can realize the efficient mapping and reliable deployment of the electronic intelligent model to the optical computing system without directly mapping the complete digital weight of the electronic model.

Inventors

CHEN YITONG
LI XINYUE
ZHAI GUANGTAO
TANG MIN

Assignees

上海交通大学

Dates

Publication Date: 20260508
Application Date: 20260310

Claims (10)

1. An electro-optical mapping system facing an electronic intelligent model is characterized by comprising an electronic side module, a cross-domain conversion module and an optical calculation module; the electronic side module is used for carrying out semantic feature coding on the input image, text or multi-mode data and outputting semantic feature representation representing the semantic understanding capability of the electronic intelligent model; The cross-domain conversion module is used for generating optical calculation parameters for driving the optical calculation module according to the semantic feature representation under the condition of introducing physical constraint conditions of the optical calculation module; And the optical calculation module is used for executing optical calculation based on the optical calculation parameters so as to reproduce the corresponding task processing capacity of the electronic intelligent model at the functional or calculation behavior level and realize the mapping of the electronic intelligent model to the optical calculation module.
2. The electronic intelligent model oriented electro-optic mapping system of claim 1, wherein the electronic side module is a feature extraction component in a pre-trained electronic intelligent model that remains frozen or only undergoes limited parameter updates during the electro-optic mapping process.
3. The electronic intelligent model-oriented electro-optical mapping system of claim 1, wherein there is no requirement for an element-by-element one correspondence between the electronic intelligent model numerical weights and the optical computing parameters of the optical computing module.
4. The electronic smart model oriented electro-optic mapping system of claim 1 wherein said physical constraints include one or more of a phase modulation range, an amplitude modulation range, a parameter quantization accuracy, a propagation loss range, or a noise level range associated with said optical computation module.
5. The electronic smart model oriented electro-optic mapping system of claim 1 wherein said optical computational parameters include optical weight parameters, phase parameters, amplitude parameters, or combinations thereof for characterizing optical computational behavior.
6. The electronic intelligent model-oriented electro-optic mapping system of claim 1, wherein the cross-domain conversion module comprises: the mapping unit inputs the semantic feature representation to a parameter generation model or a mapping function to obtain an initial parameter vector; The matching and rearranging unit performs dimension matching and rearranging on the initial parameter vector to enable the initial parameter vector to be consistent with an adjustable parameter interface of the target light computing module in dimension and structure; And the constraint unit performs physical constraint mapping on the rearranged initial parameter vector to enable the generated optical calculation parameters to meet the physical constraint condition, so as to obtain the optical calculation parameters which can be used for driving the optical calculation module.
7. The electronic intelligent model oriented electro-optic mapping system of claim 1, further comprising a deployment adaptation module; The deployment adaptation module is used for adjusting the optical calculation parameters based on the real optical path output result of the optical calculation module in the system deployment stage so as to realize the consistency of the output of the optical calculation module and a downstream interface.
8. An electro-optical mapping method for an electronic intelligent model is characterized by comprising the following steps: Carrying out semantic feature coding on the input image, text or multi-mode data, and outputting semantic feature representation representing the semantic understanding capability of the electronic intelligent model; providing an optical computing module, and generating optical computing parameters for driving the optical computing module according to the semantic feature representation under the condition of physical constraint conditions of the optical computing module; And executing optical calculation based on the optical calculation parameters so as to reproduce the corresponding task processing capacity of the electronic intelligent model at the functional or calculation behavior level and realize the mapping of the electronic intelligent model to the optical calculation module.
9. A terminal comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor is operable to run the system of any one of claims 1-7 or to perform the method of claim 8 when the program is executed by the processor.
10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor is operable to run the system of any of claims 1-7 or to perform the method of claim 8.

Description

Electro-optical mapping system and method for electronic intelligent model Technical Field The application relates to the technical fields of optical computation, electronic computation and artificial intelligence, in particular to an electro-optical mapping system and method oriented to an electronic intelligent model. Background In recent years, electronic intelligent models based on deep learning have made remarkable progress in the fields of computer vision, natural language processing, multi-modal understanding and the like. Such models typically rely on large-scale electronic computing platforms for training and reasoning, with computing power supported mainly by high-dimensional parameters and complex computing structures, placing high demands on computing power and energy consumption in the actual deployment process. Meanwhile, optical computing, which is an emerging computing paradigm, is gradually considered as an important approach to break through traditional electronic computing bottlenecks due to its natural advantages in terms of parallelism, bandwidth and energy efficiency. By reasonably designing physical processes such as light propagation, interference, modulation and the like, the optical computing system can complete matrix operation or signal processing tasks in a specific form under the conditions of extremely low delay and power consumption, and has potential application value in accelerating artificial intelligence reasoning. However, existing optical computing systems are often limited by physical implementation conditions, with significant differences in their computational structure and parametric form from electronic smart models. High precision numerical weights, complex nonlinear structures, and flexible computational topologies in electronic models are difficult to map directly into an optics parameter space with phase range, modulation accuracy, and noise constraints. This mismatch across computing paradigms makes efficient and reliable deployment of electronic smart models onto optical computing systems difficult. To alleviate the above problems, studies have attempted to re-design optical networks by simplifying the model structure or for specific tasks, but often require re-training the model or are only suitable for a single application scenario, making it difficult to multiplex the general computational power already developed in the electronic smart model. Meanwhile, a systematic method for describing the mapping relation between the electronic model and the optical calculation is lacking, and further application of the optical calculation in complex intelligent tasks is restricted. Therefore, a new technical scheme is needed, and the problems of cross-domain mapping and physical constraint existing in the mapping process of the electronic model to the optical computing system can be solved on the premise of fully multiplexing the existing computing capacity of the electronic intelligent model, so that efficient deployment and stable operation of the electronic intelligent model on the optical computing platform are realized. Through searching, china patent application No. 202010255884.0 discloses a diffraction depth neural network system based on a residual error network, which optimizes a network structure by introducing a residual error connection module so as to improve the performance of an optical system. However, the way this technology improves performance relies on de novo training of the optical model, a substantial distinction from the technical route employed by the present application in which the electronic model is deployed to the optical platform map. Disclosure of Invention In view of the shortcomings/drawbacks of the prior art, it is an object of the present application to provide an electro-optical mapping system and method for an electronic intelligent model. The application provides an electro-optical mapping system oriented to an electronic intelligent model, which comprises an electronic side module, a cross-domain conversion module and an optical calculation module; the electronic side module is used for carrying out semantic feature coding on the input image, text or multi-mode data and outputting semantic feature representation representing the semantic understanding capability of the electronic intelligent model; The cross-domain conversion module is used for generating optical calculation parameters for driving the optical calculation module according to the semantic feature representation under the condition of introducing physical constraint conditions of the optical calculation module; And the optical calculation module is used for executing optical calculation based on the optical calculation parameters so as to reproduce the corresponding task processing capacity of the electronic intelligent model at the functional or calculation behavior level and realize the mapping of the electronic intelligent model to the optical calculation modul