Search

CN-115471643-B - Fish-eye image unfolding method and device

CN115471643BCN 115471643 BCN115471643 BCN 115471643BCN-115471643-B

Abstract

The application provides a method and a device for expanding a fisheye image, which relate to the technical field of image processing and can be used for adjusting the expanding initial position of the fisheye image so as to improve the image effect of the fisheye image based on the expanding and the accuracy of target detection. The method comprises the steps of firstly expanding an original fisheye image based on a starting position to obtain a first expanded image of the original fisheye image. And then splicing the tail end of the first unfolded image with the head end of the copied image to obtain a spliced image of the original fish-eye image, wherein the copied image is an image obtained by copying the first unfolded image. And then carrying out target detection on the spliced image, and determining whether the segmented image of the detection object exists in the first unfolding image. And if the segmented image of the detection object exists in the first expansion image, adjusting the initial position, expanding the original fisheye image based on the adjusted position to obtain a second expansion image, wherein the second expansion image does not exist the segmented image of the detection object.

Inventors

  • MAO ZHUANG
  • HU CHUNBO
  • PAN YAN

Assignees

  • 杭州海康威视数字技术股份有限公司

Dates

Publication Date
20260505
Application Date
20220811

Claims (16)

  1. 1. A method of expanding a fisheye image, the method comprising: Expanding an original fisheye image based on a starting position to obtain a first expanded image of the original fisheye image, wherein the expanding starting end of the original fisheye image corresponds to the head end of the first expanded image, and the expanding ending end of the original fisheye image corresponds to the tail end of the first expanded image; Splicing the tail end of the first unfolded image with the head end of the copied image to obtain a spliced image of the original fisheye image, wherein the copied image is an image obtained by copying the first unfolded image; performing target detection on the spliced images to obtain one or more target detection frames, wherein one target detection frame comprises an image of a detection object; If the overlapping part exists between the region of any one of the one or more target detection frames on the spliced image and the splicing position, determining that the image of the segmented detection object exists in the first unfolding image, wherein the splicing position is the splicing position of the first unfolding image and the copy image in the spliced image; and if the first expansion image contains the image of the segmented detection object, adjusting the initial position, expanding the original fisheye image based on the adjusted position to obtain a second expansion image, wherein the second expansion image does not contain the image of the segmented detection object.
  2. 2. The method of claim 1, wherein stitching the trailing end of the first unfolded image with the leading end of the copied image comprises: completely copying the first unfolded image to obtain a first copied image; and splicing the tail end of the first unfolded image with the head end of the first copied image to obtain a spliced image of the original fisheye image.
  3. 3. The method of claim 1, wherein stitching the tail end of the first expanded image with the head end of the duplicate image comprises: Starting from the head end of the first unfolded image, partially copying the first unfolded image to obtain a second copied image, wherein the width of the second copied image is larger than or equal to a first preset width threshold value; And splicing the tail end of the first unfolded image with the head end of the second copied image to obtain a spliced image of the original fisheye image.
  4. 4. A method according to any one of claims 1 to 3, wherein said performing object detection on said stitched image results in one or more object detection boxes, comprising: and performing target detection on all the images of the spliced image to obtain one or more target detection frames.
  5. 5. The method according to claim 1 or 2, wherein performing object detection on the stitched image to obtain one or more object detection frames comprises: And performing target detection on a first sub-image in the spliced image to obtain one or more target detection frames, wherein the first sub-image comprises the splicing position of the first unfolded image and the copied image, the first sub-image comprises all images of the first unfolded image and a second sub-image in the copied image, and the width of the second sub-image is larger than or equal to a first preset width threshold value.
  6. 6. The method of claim 5, wherein the first preset width threshold is a preset maximum width value of the target detection frame, the preset maximum width value being a width value determined based on a maximum imaging size of the detection object in the original fisheye image.
  7. 7. A method according to any one of claims 1 to 3, wherein said performing object detection on said stitched image results in one or more object detection boxes, comprising: Dividing the spliced image into one or more third sub-images based on a preset width, wherein the width of a repeated image between every two adjacent third sub-images is larger than or equal to a second preset width threshold value; Respectively carrying out target detection on the one or more third sub-images to obtain one or more target detection frames; If there is a superposition portion between the region where any one of the one or more target detection frames is located on the stitched image and the stitching position, determining that there is an image of the segmented detection object in the first expanded image includes: And if the overlapping part exists between the region of any one of the one or more target detection frames on the third sub-image and the splicing position, determining that the image of the segmented detection object exists in the first unfolding image.
  8. 8. The method of claim 7, wherein the aspect ratio of the third sub-image is 3:2, 4:3, 16:9, or 1:1.
  9. 9. A method according to any one of claims 1 to 3,6 or 8, further comprising: Determining a first detection object; and positioning the first detection object in real time and displaying the image of the first detection object.
  10. 10. The method of any one of claims 1 to 3, 6 or 8, wherein the original fisheye image is an image captured by an image capture device, the image capture device comprising a fisheye lens and a plurality of audio capture components arranged in an array, a center point of the fisheye lens coinciding with an arrangement center point of the plurality of audio capture components, the plurality of audio capture components configured to capture audio information in an environment in which the image capture device is placed.
  11. 11. A fish-eye image unfolding apparatus, characterized in that the apparatus comprises: the device comprises a starting module, a developing module and a developing module, wherein the starting module is used for developing an original fisheye image based on a starting position to obtain a first developed image of the original fisheye image; The splicing module is used for splicing the tail end of the first unfolded image with the head end of the copied image to obtain a spliced image of the original fisheye image, and the copied image is an image obtained by copying the first unfolded image; the detection module is used for carrying out target detection on the spliced images to obtain one or more target detection frames, wherein one target detection frame comprises an image of a detection object; The processing module is used for determining that the first unfolded image is the image of the segmented detection object if the region of any one of the one or more target detection frames on the spliced image has a superposition part with a splicing position, wherein the splicing position is the splicing position of the first unfolded image and the copied image in the spliced image; The processing module is further configured to adjust the starting position if the first expanded image has an image of the segmented detection object, and expand the original fisheye image based on the adjusted position to obtain a second expanded image, where the second expanded image does not have the image of the segmented detection object.
  12. 12. The apparatus of claim 11, wherein the splicing module is specifically configured to: completely copying the first unfolded image to obtain a first copied image; Splicing the tail end of the first unfolded image with the head end of the first copied image to obtain a spliced image of the original fisheye image; The splicing module is further specifically configured to: Starting from the head end of the first unfolded image, partially copying the first unfolded image to obtain a second copied image, wherein the width of the second copied image is larger than or equal to a first preset width threshold value; Splicing the tail end of the first unfolded image with the head end of the second copied image to obtain a spliced image of the original fisheye image; The detection module is specifically configured to: performing target detection on all the images of the spliced images to obtain one or more target detection frames, wherein one target detection frame comprises an image of a detection object; If the overlapping part exists between the region of any one of the one or more target detection frames on the spliced image and the splicing position, determining that the image of the segmented detection object exists in the first unfolding image, wherein the splicing position is the splicing position of the first unfolding image and the copy image in the spliced image; the detection module is also specifically configured to: Performing target detection on a first sub-image in the spliced image to obtain one or more target detection frames, wherein one target detection frame comprises an image of a detection object, the first sub-image comprises a splicing position of the first unfolded image and the copied image, the first sub-image comprises all images of the first unfolded image and a second sub-image in the copied image, and the width of the second sub-image is larger than or equal to a first preset width threshold value; If the overlapping part exists between the region of any one of the one or more target detection frames on the spliced image and the splicing position, determining that the image of the segmented detection object exists in the first unfolding image, wherein the splicing position is the splicing position of the first unfolding image and the copy image in the spliced image; The first preset width threshold value is a preset maximum width value of the target detection frame, and the preset maximum width value is a width value determined based on the maximum imaging size of the detection object in the original fisheye image; the detection module is also specifically configured to: Dividing the spliced image into one or more third sub-images based on a preset width, wherein the width of a repeated image between every two adjacent third sub-images is larger than or equal to a second preset width threshold value; Respectively carrying out target detection on the one or more third sub-images to obtain one or more target detection frames, wherein one target detection frame comprises an image of a detection object; if the overlapping part exists between the region of any one of the one or more target detection frames on the third sub-image and the splicing position, determining that the image of the segmented detection object exists in the first unfolding image, wherein the splicing position is the splicing position of the first unfolding image and the copy image in the splicing image; the aspect ratio of the third sub-image is 3:2, 4:3, 16:9 or 1:1; the processing module is further configured to: Determining a first detection object; positioning the first detection object in real time and displaying an image of the first detection object; The original fisheye image is an image acquired by image acquisition equipment, the image acquisition equipment comprises a fisheye lens and a plurality of audio acquisition components arranged in an array, the center point of the fisheye lens coincides with the arrangement center point of the plurality of audio acquisition components, and the plurality of audio acquisition components are used for acquiring audio information in the placement environment of the image acquisition equipment.
  13. 13. An image acquisition apparatus, characterized in that the image acquisition apparatus comprises: The fish-eye lens is used for shooting an original fish-eye image; A processor for performing the method of unfolding a fisheye image according to any one of claims 1 to 10; And the display is used for displaying the fish eye image processed by the processor in real time.
  14. 14. The image capture device of claim 13, further comprising a plurality of audio capture components arranged in an array, the plurality of audio capture components configured to capture audio information; The processor is further used for performing sound source positioning processing on the audio information to determine the position of a first detection object, wherein the first detection object is an object which is speaking; The display is further used for displaying the image of the first detection object in real time in the fish-eye image processed by the processor.
  15. 15. An electronic device comprising a memory and a processor, the memory and the processor coupled, the memory for storing computer program code, the computer program code comprising computer instructions; wherein the computer instructions, when executed by the processor, cause the electronic device to perform the method of unfolding a fisheye image according to any one of claims 1-10.
  16. 16. A computer readable storage medium storing computer instructions that, when run on an electronic device, cause the electronic device to perform the method of unfolding a fisheye image according to any one of claims 1-10.

Description

Fish-eye image unfolding method and device Technical Field The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for expanding a fisheye image. Background The fish-eye camera is an ultra-wide angle camera for simulating the effect of fish looking up on the water surface, and has the advantages of large field angle, multiple accommodating scenes, suitability for shooting in a narrow space and the like. The fish-eye camera is used for collecting images, and the shooting angle of view is far larger than that of a common lens, so that the information of the collected images is more abundant. Therefore, the fish-eye camera is widely applied in the fields of virtual and real scene technology, robot navigation, visual monitoring, intelligent auxiliary driving and the like. Based on the fisheye image acquired by the fisheye camera, deformation to a certain extent can be generated at the edge of the image. And the information quantity near the center of the fish-eye image is maximum, the deformation is minimum, and the deformation gradually increases along with the increase of the radius, the information quantity is reduced. Thus, the information obtained by directly observing the fisheye image with the naked eye of the user is not accurate, and the effect of image recognition based on the fisheye image is also poor. Therefore, in the actual use process of the fisheye camera, the collected fisheye image is usually unfolded according to the characteristics of the camera. However, if the proper initial unfolding position is not determined in the unfolding process of the fisheye image, the image of the detection object is segmented at two ends of the unfolded image, so that the image effect of the unfolded fisheye image is poor and the accuracy of target detection is affected. Disclosure of Invention The application provides a method and a device for expanding a fisheye image, which can be used for adjusting the initial expanding position of the fisheye image so as to improve the image effect of the fisheye image based on the expansion and the accuracy of target detection. In order to achieve the technical purpose, the application adopts the following technical scheme: According to the first aspect, the application provides a method for expanding a fisheye image, which specifically comprises the steps of expanding an original fisheye image based on a starting position to obtain a first expanded image of the original fisheye image, wherein an expansion starting end of the original fisheye image corresponds to a head end of the first expanded image, and an expansion ending end of the original fisheye image corresponds to a tail end of the first expanded image. And then splicing the tail end of the first unfolded image with the head end of the copied image to obtain a spliced image of the original fish-eye image, wherein the copied image is an image obtained by copying the first unfolded image. And then carrying out target detection on the spliced image, and determining whether the segmented image of the detection object exists in the first unfolding image. And if the segmented image of the detection object exists in the first expansion image, adjusting the initial position, expanding the original fisheye image based on the adjusted position to obtain a second expansion image, wherein the second expansion image does not exist the segmented image of the detection object. The method for expanding the fisheye image has the advantages that the method firstly performs pre-expansion processing on the original fisheye image based on the preset initial position to obtain a preliminary expansion result. And based on the annular characteristic of the 360-degree annular fisheye image during expansion, the image at the head end and the image at the tail end of the first expansion image can be spliced to obtain an image at the initial position in the original fisheye image. Therefore, a spliced image of the preliminary expanded image can be obtained through a copying and splicing process, and then detection processing is carried out on the spliced image so as to determine whether the expanded image based on the initial position can appear that the image of the detection object is segmented at two ends of the expanded image, namely, the situation that the segmented image of the detection object exists in the first expanded image. The determining process combines the copying and splicing of the images, is simple to operate and easy to realize, and has practicability. Furthermore, when it is determined that the first expanded image has the image of the segmented detection object, the initial position of the image expansion operation can be adjusted according to the target detection result, and the initial fisheye image is expanded based on the adjusted initial position, so that the obtained expanded image has a better image effect, the viewing experience of the user