CN-121995616-A - Image generation method, microscope device, and storage medium

CN121995616ACN 121995616 ACN121995616 ACN 121995616ACN-121995616-A

Abstract

The embodiment of the application provides an image generation method, a microscope device and a storage medium, which relate to the technical field of image processing, and the method comprises the steps of respectively acquiring observation images of an object to be observed, which are acquired by a microscope according to a specified multiplying power in the illumination direction of at least three preset light sources; the method comprises the steps of determining illumination directions and illumination intensities of pixel positions in each observation image, calculating an initial normal vector of each pixel position in each observation image by combining the pixel value, the illumination directions and the illumination intensities of the pixel positions, wherein the modular length of the initial normal vector of any pixel position represents a diffuse reflection coefficient of a real position represented by the pixel position, calculating an initial gray value of the pixel position based on a vector representing a specified observation direction and the initial normal vector of the pixel position, and mapping the initial gray value of the pixel position to a gray value range to obtain a target gray value of the pixel position. The method can effectively observe the detailed structure of the object to be observed.

Inventors

XU BAOBEI
CHEN WEIJIE
XIE DI

Assignees

杭州海康威视数字技术股份有限公司

Dates

Publication Date: 20260508
Application Date: 20241106

Claims (10)

1. An image generation method, the method comprising: respectively acquiring observation images of an object to be observed, which are acquired by a microscope according to a specified multiplying power in the illumination direction of at least three preset light sources; Determining the illumination direction and illumination intensity of each pixel position in the observation image under the illumination direction of each preset light source aiming at each preset light source; Calculating an initial normal vector of each pixel position by combining a pixel value, an illumination direction and illumination intensity of the pixel position in an observation image under the illumination direction of each preset light source according to each pixel position, wherein the modular length of the initial normal vector of any pixel position represents the diffuse reflection coefficient of the real position on the surface of the object to be observed, which is represented by the pixel position; Calculating an initial gray value for the pixel location based on the direction vector representing the specified viewing direction and the initial normal vector for the pixel location; And mapping the initial gray value of the pixel position to a gray value range to obtain a target gray value of the pixel position in the gray image of the object to be observed in the appointed observation direction.
2. The method of claim 1, wherein prior to calculating the initial gray value for the pixel location based on the direction vector representing the specified viewing direction and the initial normal vector for the pixel location, the method further comprises: Determining the depth value of each pixel position in the depth image of the object to be observed based on a preset energy function and an initial normal vector of each pixel position; Converting the pixel coordinates and the depth values of each pixel position according to conversion coefficients between a pre-calibrated pixel coordinate system and a world coordinate system to obtain world coordinates corresponding to each pixel position, wherein a first coordinate axis of the world coordinate system is perpendicular to a plane of an object carrying platform of the microscope; Performing plane fitting based on world coordinates corresponding to each pixel position to obtain a plane to be corrected; Calculating a rotation matrix between the normal vector of the plane to be corrected and the direction of the first coordinate axis; The calculating the initial gray value of the pixel position based on the direction vector representing the appointed observation direction and the initial normal vector of the pixel position comprises: calculating the product of the rotation matrix and the initial normal vector of the pixel position to obtain the normal vector to be utilized of the pixel position; And calculating the product of the direction vector representing the appointed observation direction and the normal vector to be utilized of the pixel position to obtain the initial gray value of the pixel position.
3. The method according to claim 2, wherein the performing plane fitting based on world coordinates corresponding to each pixel position to obtain a plane to be corrected includes: Randomly selecting a first number of world coordinates from world coordinates corresponding to each pixel location; performing plane fitting according to the selected world coordinates to obtain current alternative planes, and returning to the step of randomly selecting the first number of world coordinates from the world coordinates corresponding to each pixel position until the preset number of alternative planes are obtained by fitting; The method comprises the steps of obtaining the number of the inner points of each alternative plane as the number of the points corresponding to each alternative plane, wherein the inner points of any alternative plane represent the positions represented by world coordinates, the distance between the inner points and the alternative plane is smaller than a preset distance threshold value; and determining a plane to be corrected based on the candidate plane with the largest number of corresponding points.
4. A method according to claim 3, wherein said determining a plane to be corrected based on the candidate plane for which the corresponding number of points is the largest comprises: and performing plane fitting according to the world coordinates of the inner points of the candidate plane with the largest number of corresponding points to obtain a plane to be corrected.
5. The method according to claim 2, characterized in that the preset energy function is as follows: J(z)=∫∫((z u -p) 2 +(z v -q) 2 )dudv+λ(∑|z u |+|z v |) Wherein J (z) represents energy of the depth image, z u and z v represent gradients of the depth image in two coordinate axis directions of a pixel coordinate system, p=n x /n z ,q＝n y /n z ,(n x ,n y ,n z represents unit normal vector of each pixel position, and λ represents regularization constraint intensity coefficient.
6. The method according to claim 1, wherein determining, for each preset light source, the illumination direction and the illumination intensity of each pixel position in the observed image in the illumination direction of the preset light source comprises: For each preset light source, calculating the illumination direction of each pixel position in an observation image under the illumination direction of the preset light source according to a first formula, wherein the first formula is as follows: K (u, v) represents the illumination direction of the pixel position (u, v), m represents the coordinate of the preset light source under the world coordinate system, p (u, v) represents the coordinate of the real position on the surface of the object to be observed, which is represented by the pixel position (u, v); Under the condition that the specified multiplying power is larger than a preset multiplying power threshold value, calculating the illumination intensity of each pixel position in an observation image of each preset light source in the illumination direction of the preset light source according to a second formula, wherein the second formula is as follows: K (u, v) represents the illumination direction of the pixel position (u, v), m represents the coordinate of the preset light source under the world coordinate system, p (u, v) represents the coordinate of the real position on the surface of the object to be observed, represented by the pixel position (u, v), s (u, v) represents the illumination intensity of the pixel position (u, v), ψ represents the illumination intensity at the position of the preset light source, n s represents the main direction of the preset light source; Under the condition that the appointed multiplying power is not larger than the preset multiplying power threshold value, uniformly dividing an observation image of each preset light source in the illumination direction of the preset light source in a grid matrix mode to obtain a second number of rectangular areas with the same size; Measuring the illumination intensity of the real position on the surface of the object to be observed, which is represented by the center of each rectangular area, and taking the illumination intensity as the illumination intensity corresponding to the rectangular area; And carrying out linear interpolation based on the illumination intensities corresponding to the rectangular areas obtained by dividing to obtain the illumination intensity of each pixel position in the observation image in the illumination direction of the preset light source.
7. The method according to claim 1, wherein mapping the initial gray value of the pixel position to the gray value range to obtain the target gray value of the pixel position in the gray image of the object to be observed in the specified observation direction comprises: Mapping the initial gray value of the pixel position to a gray value range according to a mapping formula to obtain a target gray value of the pixel position in the gray image of the object to be observed in the appointed observation direction, wherein the mapping formula is as follows: G (u, v) represents a target gradation value of a pixel position (u, v) in the gradation image of the object to be observed in the specified observation direction, G c (u, v) represents an initial gradation value of the pixel position (u, v), and max (abs (G c )) represents a maximum value among absolute values of the initial gradation values of the pixel positions.
8. The method according to claim 1, wherein for each pixel position, calculating an initial normal vector of the pixel position in combination with a pixel value, an illumination direction and an illumination intensity of the pixel position in the observed image under the illumination direction of each preset light source comprises: Calculating the brightness value of each pixel position in the illumination direction of each preset light source based on the pixel value of the pixel position in the observation image in the illumination direction of each preset light source; According to a diffuse reflection formula, combining the brightness value, the illumination direction and the illumination intensity of the pixel position in the illumination direction of each preset light source, solving the product of the diffuse reflection coefficient of the real position on the surface of the object to be observed, represented by the pixel position, and the unit normal vector to obtain the initial normal vector of the pixel position, wherein the diffuse reflection formula is as follows: I i (u,v)＝s i (u,v)·l i (u,v)·N(u,v) s.t. N(u,v)=ρ(u,v)·n(u,v) I i (u, v) represents the brightness value of the pixel position (u, v) in the illumination direction of the ith preset light source, l i (u,v)、s i (u, v) represents the illumination direction and illumination intensity of the pixel position (u, v) in the observation image in the illumination direction of the ith preset light source respectively, N (u, v) represents the initial normal vector of the pixel position (u, v), ρ (u, v) represents the diffuse reflection coefficient of the real position on the surface of the object to be observed represented by the pixel position (u, v), and N (u, v) represents the unit normal vector of the real position on the surface of the object to be observed represented by the pixel position (u, v).
9. The microscope device is characterized by comprising a microscope and at least three preset light sources, wherein an image sensor and a processor are arranged in the microscope; The image sensor is used for acquiring an observation image of an object to be observed under the condition that the object to be observed is observed according to a specified multiplying power in the illumination directions of the at least three preset light sources respectively, and sending the acquired observation image to the processor; the processor being configured to perform the method of any of claims 1-8.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a computer program which, when executed by a processor, implements the method of any of claims 1-8.

Description

Image generation method, microscope device, and storage medium Technical Field The present application relates to the field of image processing technology, and in particular, to an image generation method, a microscope device, and a storage medium. Background In the industrial production scene, it is often necessary to observe the tiny parts in the production process, so as to detect defects of the tiny parts or observe the processing state of the tiny parts. For example, microscopic parts can be observed with a microscope. However, the detailed structure of the surface of the micro part is often difficult to observe, so how to effectively observe the detailed structure of the micro part (may be referred to as an object to be observed) is a problem to be solved. Disclosure of Invention An object of an embodiment of the present application is to provide an image generating method, a microscope apparatus, and a storage medium, so as to effectively observe a detailed structure of an object to be observed in a specified observation direction. The specific technical scheme is as follows: in a first aspect of an embodiment of the present application, there is provided an image generating method, including: respectively acquiring observation images of an object to be observed, which are acquired by a microscope according to a specified multiplying power in the illumination direction of at least three preset light sources; Determining the illumination direction and illumination intensity of each pixel position in the observation image under the illumination direction of each preset light source aiming at each preset light source; Calculating an initial normal vector of each pixel position by combining a pixel value, an illumination direction and illumination intensity of the pixel position in an observation image under the illumination direction of each preset light source according to each pixel position, wherein the modular length of the initial normal vector of any pixel position represents the diffuse reflection coefficient of the real position on the surface of the object to be observed, which is represented by the pixel position; Calculating an initial gray value for the pixel location based on the direction vector representing the specified viewing direction and the initial normal vector for the pixel location; And mapping the initial gray value of the pixel position to a gray value range to obtain a target gray value of the pixel position in the gray image of the object to be observed in the appointed observation direction. Optionally, before the calculating the initial gray value of the pixel position based on the direction vector representing the specified observation direction and the initial normal vector of the pixel position, the method further comprises: Determining the depth value of each pixel position in the depth image of the object to be observed based on a preset energy function and an initial normal vector of each pixel position; Converting the pixel coordinates and the depth values of each pixel position according to conversion coefficients between a pre-calibrated pixel coordinate system and a world coordinate system to obtain world coordinates corresponding to each pixel position, wherein a first coordinate axis of the world coordinate system is perpendicular to a plane of an object carrying platform of the microscope; Performing plane fitting based on world coordinates corresponding to each pixel position to obtain a plane to be corrected; Calculating a rotation matrix between the normal vector of the plane to be corrected and the direction of the first coordinate axis; The calculating the initial gray value of the pixel position based on the direction vector representing the appointed observation direction and the initial normal vector of the pixel position comprises: calculating the product of the rotation matrix and the initial normal vector of the pixel position to obtain the normal vector to be utilized of the pixel position; And calculating the product of the direction vector representing the appointed observation direction and the normal vector to be utilized of the pixel position to obtain the initial gray value of the pixel position. Optionally, performing plane fitting based on world coordinates corresponding to each pixel position to obtain a plane to be corrected, including: Randomly selecting a first number of world coordinates from world coordinates corresponding to each pixel location; performing plane fitting according to the selected world coordinates to obtain current alternative planes, and returning to the step of randomly selecting the first number of world coordinates from the world coordinates corresponding to each pixel position until the preset number of alternative planes are obtained by fitting; The method comprises the steps of obtaining the number of the inner points of each alternative plane as the number of the points corresponding to each alternative plane,