Search

EP-3360111-B1 - METHODS AND SYSTEMS FOR FUSION DISPLAY OF THERMAL INFRARED AND VISIBLE IMAGE

EP3360111B1EP 3360111 B1EP3360111 B1EP 3360111B1EP-3360111-B1

Inventors

  • LU, WEI
  • LI, QIANKUN

Dates

Publication Date
20260513
Application Date
20160927

Claims (7)

  1. A method comprising: obtaining a thermal infrared image and a visible image; receiving a fusion parameter corresponding to a color space, wherein the color space is an HSI space and the fusion parameter corresponding to the color space includes a hue value, a saturation value, and an intensity value; generating, based on the fusion parameter, a fused image of the thermal infrared image and the visible image, wherein the hue value is determined according to the thermal infrared image, the saturation value is determined according to the thermal infrared image and the visible image, and the intensity value is determined based the visible image; receiving a regulation parameter, the regulation parameter including a color scheme and a supplementary color scheme, wherein the color scheme defines a section of a color circle and includes a first hue value (h 1 ), a second hue value (h 2 ), and a direction between the first hue value and the second hue value, the supplementary color scheme converts the color scheme into a black-and-white decoration, wherein the highest temperature is defined as white and the lowest temperature is defined as black; and adjusting the fused image according to the regulation parameter including: gradually changing a saturation value of a first color corresponding to the first hue value to 0 and meanwhile adding a half one color block when compared with the color scheme to convert the first color to white, the color block being a color interval on the color circle which is divided by color and which represents the hue of the color space, and gradually changing an intensity value of a second color corresponding to the second hue value to 0 to convert the second color to black.
  2. The method of claim 1, wherein: the adjusting of the fused image according to the regulation parameter includes adjusting a color display of the fused image according to the color scheme.
  3. The method of any one of claim 1-2, wherein the regulation parameter further includes a partial contrast and the partial contrast is used to adjust a contrast level of a region of interest, ROI, in the fused image, and the adjusting a contrast level of a region of interest in the fused image, further comprising selecting a region of interest in the fused image; magnifying the ROI; and adjusting the contrast level of the ROI.
  4. A system comprising: an imaging device configured to obtain a thermal infrared image and a visible image; one or more processors that are configured to execute instructions to: receive a fusion parameter corresponding to a color space, wherein the color space is an HSI space and the fusion parameter corresponding to the color space includes a hue value, a saturation value, and an intensity value; generate, based on the fusion parameter, a fused image of the thermal infrared image and the visible image, wherein the hue value is determined according to the thermal infrared image, the saturation value is determined according to the thermal infrared image and the visible image, and the intensity value is determined based the visible image; receive a regulation parameter, the regulation parameter including a color scheme and a supplementary color scheme, wherein the color scheme defines a section of a color circle and includes a first hue value (h 1 ), a second hue value (h 2 ), and a direction between the first hue value and the second hue value, the supplementary color scheme converts the color scheme into a black-and-white decoration, wherein the highest temperature is defined as white and the lowest temperature is defined as black; and adjust the fused image according to the regulation parameter including: gradually changing a saturation value of a first color corresponding to the first hue value to 0 and meanwhile adding a half one color block when compared with the color scheme to convert the first color to white, the color block being a color interval on the color circle which is divided by color and which represents the hue of the color space, and gradually changing an intensity value of a second color corresponding to the second hue value to 0 to convert the second color to black.
  5. The system of claim 4, wherein: the adjusting of the fused image according to the regulation parameter includes adjusting a color display of the fused image according to the color scheme.
  6. The system of any one of claim 4-5, wherein the regulation parameter further includes a partial contrast and the partial contrast is used to adjust a contrast level of a region of interest in the fused image, and wherein to adjust a contrast level of a region of interest in the fused image, the one or more processors are further configured to: select a region of interest in the fused image; magnify the ROI; and adjust the contrast level of the ROI.
  7. A non-transitory computer readable medium embodying a computer program product, the computer program product comprising instructions configured to cause a processor to perform a method comprising: obtaining a thermal infrared image and a visible image; receiving a fusion parameter corresponding to a color space, wherein the color space is an HSI space and the fusion parameter corresponding to the color space includes a hue value, a saturation value, and an intensity value; generating, based on the fusion parameter, a fused image of the thermal infrared image and the visible image, wherein the hue value is determined according to the thermal infrared image, the saturation value is determined according to the thermal infrared image and the visible image, and the intensity value is determined based the visible image; receiving a regulation parameter, the regulation parameter including a color scheme and a supplementary color scheme, wherein the color scheme defines a section of a color circle and includes a first hue value (h 1 ), a second hue value (h 2 ), and a direction between the first hue value and the second hue value, the supplementary color scheme converts the color scheme into black-and-white decoration, wherein the highest temperature is defined as white and the lowest temperature is defined as black; and adjusting the fused image according to the regulation parameter including: gradually changing a saturation value of a first color corresponding to the first hue value to 0 and meanwhile adding a half one color block when compared with the color scheme to convert the first color to white, the color block being a color interval on the color circle which is divided by color and which represents the hue of the color space, and gradually changing an intensity value of a second color corresponding to the second hue value to 0 to convert the second color to black.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS TECHNICAL FIELD This present disclosure generally relates to image fusion, and more particularly, relates to methods and systems for image fusion of thermal infrared and visible images. BACKGROUND Thermal imaging devices can be used to obtain temperature information of a target scene. The obtained temperature information may then be quantized and normalized to some discrete values range from 0 to 255 (i.e., [0, 255]). A grayscale image may then be generated according to the discrete values. In order to show distribution characteristics of temperature of the target scene, a color image may be obtained based on a temperature quantized value-hue look-up table. However, the color image may lack of edge and/or other details due to its low resolution. Visible imaging devices can be used to capture visible light information of a target scene. Generally, a visible image may have higher resolution compared with the grayscale image generated by a thermal imaging device. If different information in the thermal infrared image and the visible image can be combined together, and displayed in a fused image, it may make full use of advantages of both thermal imaging device and visible imaging device. By this way, for a user, distribution characteristics of temperature and detailed information of the target scene may be acquired at the same time in a single fused image. Although there were plenty of disclosures focusing on image registration, that may determine an overlapping area between a thermal infrared image and a visible image, disclosures such as how to merge images' information in an appropriate way, or how to display the fused image in a personalized way are still in need. EP 2921995 A1 discloses a method for edge detection and presentation in thermal images. WO 2013144298A1 discloses a method for enabling improved analysis and interpretation of associated image data in an infrared (IR) image and a visible light (VL) image depicting a real world scene, captured using a thermography arrangement comprising an IR imaging system and a VL imaging system, the method comprising: capturing an IR image de picting the scene, having a first field of view (FOV); capturing a VL image depicting the scene, having a second FOV; processing at least one of the VL image and the IR image such that the FOV represented in the VL image sub stantially corresponds to the FOV represented in the IR image; associating the resulting IR and VL images to provide associated images; and enabling a user to access the associated images for display, wherein analysis and inter pretation of the associated images, having image pair content, is intuitive as the resulting VL image and the resulting IR image after associating represent the same FOV. SONGFENG YIN ET AL: "One color contrast enhanced infrared and visible image fusion method", INFRARED PHYSICS & TECHNOLOGY discloses that "based on the characteristics of the IR image, we introduce a ratio of local to global divergence of the IR image to improve the color contrast. The enhancement ratios for both hot and cold targets are larger than one, while it tends to one for the background. As a result, the proposed method pops out both hot and cold targets in color, where hot targets will appear intense red, and cold targets will appear cyan. Subjective results show visible color contrast enhancement effects. Target detection experiments through hue and saturation components of the fused image show an improvement in the hit rate for target detection, owing to larger color distance between the target and the background." Roger Schmidt: "Benefits of IR/visible fusion", Proceedings of SPIE discloses that "thermographers have always wanted an infrared camera that produces images with both a wide field of view (FOV) and detailed spatial resolution. An infrared-only camera with this combination is prohibitively expensive for most applications. A less expensive way of providing both features in a single camera is to blend a wide FOV visible image with a smaller FOV infrared image. The major benefit of this combination is that thermographers can pinpoint and identify infrared problem areas in a clear visible picture." US 20120262584A1 discloses that "a method according to an embodiment comprises: receiving a visual image and an infrared (IR) image of a scene and for a portion of said IR image extracting high spatial frequency content from a corresponding portion of said visual image. The method according to the embodiment further comprises combining said extracted high spatial frequency content from said portion of the visual image with said portion of the IR image, to generate a combined image, wherein the contrast and/or resolution in the portion of the IR image is increased compared to the contrast and/or resolution of said received IR image." SUMMARY The invention is set out in the appended set of claims. Additional features will be set forth in part in the description w