CN-121998872-A - Design method and device of computing imaging system and electronic equipment

CN121998872ACN 121998872 ACN121998872 ACN 121998872ACN-121998872-A

Abstract

The embodiment of the invention provides a design method and device of a computing imaging system and electronic equipment. The method comprises the steps of respectively determining degradation parameters of light rays after the light rays pass through first simplified optical systems under first working conditions, respectively degrading a clear sample image by utilizing the first degradation parameters, respectively repairing the first degradation images by utilizing the first degradation parameters, determining the corresponding first simplified optical systems of which the first repairing images meet preset repairable conditions, determining an optical structure parameter threshold value which comprises optical structure parameters of the repairable simplified optical systems as a repairable boundary, designing a second simplified optical system with optical structure parameters within the repairable boundary, and training to obtain a target neural network for repairing the images acquired by the second simplified optical system. The feasibility of the designed computational imaging system can be improved.

Inventors

SHAO XIAOPENG
WEI SHIJIE
WU TENGFEI
ZHANG YINUO
CHEN YUTONG

Assignees

中国科学院西安光学精密机械研究所

Dates

Publication Date: 20260508
Application Date: 20260408

Claims (10)

1. A method of designing a computational imaging system, the method comprising: Determining degradation parameters of light rays passing through each first simplified optical system under each first working condition respectively as first degradation parameters of each first simplified optical system, wherein the degradation parameters are used for representing wavefront aberration of the light rays; Respectively degrading the clear sample image by using each first degradation parameter to obtain a first degradation image corresponding to each first simplified optical system; repairing each first degradation image by using each first degradation parameter to obtain each corresponding first repair image of each first simplified optical system; Determining a first simplified optical system, corresponding to the first repair image, meeting a preset repairable condition, as a repairable simplified optical system; Determining an optical structure parameter threshold value including an optical structure parameter of each of the repairable simplified optical systems as a restorable boundary; And designing a second simplified optical system with optical structure parameters within the restorable boundary, and training to obtain a target neural network for repairing the image acquired by the second simplified optical system.
2. The method of claim 1, wherein the training results in a target neural network for repairing the image acquired by the second simplified optical system, comprising: And training by using a real pairing data set to obtain a target neural network, wherein the real pairing data set comprises a plurality of sample data, each sample data comprises a second degradation image obtained by a second simplified optical system for acquiring a sample environment under a second working condition, a second degradation parameter and a reference image obtained by a reference optical system for acquiring the sample environment, the second degradation parameter is a degradation parameter of light after passing through the second simplified optical system under the second working condition, second degradation images included in different sample data are different, and different second degradation images are obtained by acquiring different sample environments under different second working conditions.
3. The method of claim 2, wherein training with the true pairing dataset results in a target neural network, comprising: inputting a second degradation image and a second degradation parameter in the sample data into an original neural network to obtain an image output by the original neural network as a second repair image; Degrading the second repair image by using the second degradation parameter to obtain a third degradation image; Calculating a difference between the second repair image and a reference image in the sample data as a first difference, and calculating a difference between the third degraded image and the second degraded image as a second difference; and adjusting model parameters of the original neural network towards the direction of reducing the first difference and the second difference to obtain a target neural network.
4. The method of claim 1, wherein the predetermined repairable condition comprises a peak signal-to-noise ratio of the first repair image being higher than a predetermined first threshold and/or a structural similarity between the first repair image and the sample sharp image being higher than a predetermined second threshold.
5. A computational imaging method based on a simplified optical system, the method comprising: acquiring a fourth degradation image and a fourth working condition acquired by a second simplified optical system, wherein the fourth working condition is the working condition of the second simplified optical system when the fourth degradation image is acquired; Determining a degradation parameter of the light after passing through the second simplified optical system under the fourth working condition as a third degradation parameter; Inputting the fourth degradation image and the third degradation parameter into a target neural network to obtain a repair image output by the target neural network as an imaging result; Wherein the second simplified optical system and the target neural network are obtained in advance by the design method of the computational imaging system according to any one of claims 1 to 4.
6. A design apparatus for a computational imaging system, the apparatus comprising: The first parameter determining module is used for determining degradation parameters of light rays after passing through each first simplified optical system under each first working condition respectively as first degradation parameters of each first simplified optical system, wherein the degradation parameters are used for representing wavefront aberration of the light rays; the degradation module is used for respectively degrading the clear sample image by utilizing the first degradation parameters to obtain first degradation images corresponding to the first simplified optical systems; The restoration module is used for restoring each first degradation image by utilizing each first degradation parameter to obtain each corresponding first restoration image of each first simplified optical system; The screening module is used for determining a first simplified optical system, corresponding to the first repair image, meeting a preset repairable condition, and taking the first simplified optical system as a repairable simplified optical system; A boundary determining module for determining an optical structure parameter threshold value including an optical structure parameter of each of the repairable simplified optical systems as a restorable boundary; The design module is used for designing a second simplified optical system with optical structure parameters within the restorable boundary and training a target neural network for repairing an image acquired by the second simplified optical system.
7. The apparatus of claim 6, wherein the design module trains a target neural network for repairing the image acquired by the second simplified optical system, comprising: training with a true paired data set to obtain a target neural network, wherein the true paired data set comprises a plurality of sample data, each sample data comprises a second degradation image obtained by a second simplified optical system for acquiring a sample environment under a second working condition, a second degradation parameter and a reference image obtained by a reference optical system for acquiring the sample environment, the second degradation parameter is a degradation parameter of light after passing through the second simplified optical system under the second working condition, second degradation images included in different sample data are different, different second degradation images are obtained by acquiring different sample environments under different second working conditions, and/or, The design module trains by using a real pairing data set to obtain a target neural network, and the design module comprises the following steps: inputting a second degradation image and a second degradation parameter in the sample data into an original neural network to obtain an image output by the original neural network as a second repair image; Degrading the second repair image by using the second degradation parameter to obtain a third degradation image; Calculating a difference between the second repair image and a reference image in the sample data as a first difference, and calculating a difference between the third degraded image and the second degraded image as a second difference; Adjusting model parameters of the original neural network in a direction of reducing the first difference and the second difference to obtain a target neural network, and/or, The preset repairable condition comprises that the peak signal-to-noise ratio of the first repair image is higher than a preset first threshold value, and/or the structural similarity between the first repair image and the clear sample image is higher than a preset second threshold value, and/or, The operating conditions include one or more of a temperature drift amount, a tuning drift amount, a vibration impact amount, a focal plane offset amount, and a field of view position, and/or, The degradation parameters include one or more of an aberration parameter, an equivalent pupil, a spatially varying point spread function.
8. A computational imaging device based on a simplified optical system, the device comprising: The input module is used for acquiring a fourth degradation image acquired by the second simplified optical system and a fourth working condition, wherein the fourth working condition is the working condition of the second simplified optical system when the fourth degradation image is acquired; a second parameter determining module, configured to determine a degradation parameter of the light after passing through the second simplified optical system under the fourth working condition, as a third degradation parameter; the algorithm module is used for inputting the fourth degradation image and the third degradation parameter into a target neural network to obtain a repair image output by the target neural network as an imaging result; Wherein the second simplified optical system and the target neural network are obtained in advance by the design method of the computational imaging system according to any one of claims 1 to 4.
9. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus; A memory for storing a computer program; A processor for carrying out the method steps of any one of claims 1-4 or 5 when executing a program stored on a memory.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-4 or 5.

Description

Design method and device of computing imaging system and electronic equipment Technical Field The present invention relates to the field of optical technologies, and in particular, to a design method and apparatus for a computing imaging system, and an electronic device. Background In application scenes such as remote sensing imaging, high-resolution cameras, unmanned aerial vehicle loading and mobile platform imaging, an imaging system is generally required to realize high resolution, wide field of view and stable image quality under the constraint of limited size, weight and power consumption, and is also suitable for complex working conditions such as temperature drift, vibration impact and the like. In order to meet the requirements, the traditional optical imaging system mostly adopts a plurality of high-precision lens combinations, introduces advanced elements such as aspheric surfaces and the like, and controls aberration to improve imaging quality by matching with strict machining and assembling tolerances, but the engineering burden such as complex structure, high manufacturing, detecting and adjusting cost, long period and the like is caused, so that the miniaturization and large-scale deployment of the optical system are limited. With the enhancement of low cost, light weight and rapid iteration demands, the computing imaging system gradually forms the development direction of 'physical end simplification+computing compensation', namely, the front end is allowed to acquire necessary information by adopting a simplified optical structure which is more friendly in processing, adjusting and adjusting, and the algorithm end performs compensation in a digital domain. In the prior engineering practice, front end design and algorithm compensation are always relatively fractured, wherein the front end is mainly optimized by taking traditional image quality indexes (such as MTF, point column diagram and RMS wave aberration) as targets, and the algorithm end trains a restoration model under the preset degradation condition, so that the front end design and algorithm compensation lack a unified design and evaluation closed loop, and further the designed computing imaging system cannot be normally realized because the front end design and algorithm compensation cannot be mutually matched. Disclosure of Invention The embodiment of the invention aims to provide a design method and device of a computing imaging system and electronic equipment, so as to improve the feasibility of the computing imaging system obtained by design. The specific technical scheme is as follows: In a first aspect of the present application, there is provided a method of designing a computational imaging system, the method comprising: Determining degradation parameters of light rays passing through each first simplified optical system under each first working condition respectively as first degradation parameters of each first simplified optical system, wherein the degradation parameters are used for representing wavefront aberration of the light rays; Respectively degrading the clear sample image by using each first degradation parameter to obtain a first degradation image corresponding to each first simplified optical system; repairing each first degradation image by using each first degradation parameter to obtain each corresponding first repair image of each first simplified optical system; Determining a first simplified optical system, corresponding to the first repair image, meeting a preset repairable condition, as a repairable simplified optical system; Determining an optical structure parameter threshold value including an optical structure parameter of each of the repairable simplified optical systems as a restorable boundary; And designing a second simplified optical system with optical structure parameters within the restorable boundary, and training to obtain a target neural network for repairing the image acquired by the second simplified optical system. In one possible embodiment, the training results in a target neural network for repairing the image acquired by the second simplified optical system, comprising: And training by using a real pairing data set to obtain a target neural network, wherein the real pairing data set comprises a plurality of sample data, each sample data comprises a second degradation image obtained by a second simplified optical system for acquiring a sample environment under a second working condition, a second degradation parameter and a reference image obtained by a reference optical system for acquiring the sample environment, the second degradation parameter is a degradation parameter of light after passing through the second simplified optical system under the second working condition, second degradation images included in different sample data are different, and different second degradation images are obtained by acquiring different sample environments under different second working condit