Search

EP-3786844-B1 - IMAGE PROCESSING SYSTEM, IMAGE PROCESSING METHOD, AND PROGRAM

EP3786844B1EP 3786844 B1EP3786844 B1EP 3786844B1EP-3786844-B1

Inventors

  • CHAE, Yeongnam

Dates

Publication Date
20260506
Application Date
20180706

Claims (8)

  1. An image processing system comprising: obtaining means for obtaining a captured image of a document that includes a fixed part and an un-fixed part, the document being captured by an image reader or an image capture device, wherein the fixed part is a part where content is fixed, and common to all of the documents, wherein the un-fixed part is a part where content is not fixed, and is not common to two documents or more documents; first shaping means for adjusting the shape of the captured image based on a feature point group extracted from the fixed part of the document in a sample image and a feature point group extracted from the fixed part of the document in the captured image so as to obtain a first shaped image, wherein the adjusting can correct nonliner skew, wherein the sample image is an image in a state where the document is captured from a front direction or a substantially front direction, wherein the front direction is a direction perpendicular to a surface of the document, wherein the angle between the substantially front direction and the surface of the document is 80° or more; detecting means for detecting a feature part of the fixed part from the first shaped image; wherein the detecting means detects the feature part from the first shaped image based on template matching using a template image of a feature part in the sample image, the image processing system further comprising: second shaping means for correcting only linear skew by aligning the first shaped image such that a positional relationship of the feature part detected by the detecting means is aligned with the matched feature part in the sample image, so as to obtain a second shaped image.
  2. The image processing system according to claim 1, wherein the detecting means detects the feature part from an area that is located in the first shaped image and includes a predetermined position.
  3. The image processing system according to any one of claims 1 or 2, wherein the detecting means detects a plurality of feature parts, and the second shaping means shapes the first shaped image such that positions of the plurality of feature parts are respectively aligned with predetermined positions corresponding to the plurality of feature parts so as to obtain the second shaped image.
  4. The image processing system according to claim 3, wherein the plurality of feature parts are disposed near edges of the document.
  5. The image processing system according to any one of claims 1 to 4, wherein the plurality of feature parts are characters or symbols in the fixed part.
  6. The image processing system according to any one of claims 1 to 5, further comprising execution means that executes optical character recognition on the second shaped image.
  7. An image processing method comprising: an obtaining step of obtaining a captured image of a document that includes a fixed part and an un-fixed part, the document being captured by an image reader or an image capture device, wherein the fixed part is a part where content is fixed, and common to all of the documents, wherein the un-fixed part is a part where content is not fixed, and is not common to two documents or more documents; a first shaping step of adjusting the shape of the captured image based on a feature point group extracted from the fixed part of the document in a sample image and a feature point group extracted from the fixed part of the document in the captured image so as to obtain a first shaped image, wherein the adjusting can correct nonlinear skew, wherein the sample image is an image in a state where the document is captured from a front direction or a substantially front direction, wherein the front direction is a direction perpendicular to a surface of the document, wherein the angle between the substantially front direction and the surface of the document is 80° or more; a detecting step of detecting a feature part of the fixed part from the first shaped image; wherein in the detecting step the feature part from the first shaped image is detected based on template matching using a template image of a feature part in the sample image, the image processing method further comprising: a second shaping step for correcting only linear skew by aligning the first shaped image such that a positional relationship of the feature part detected by the detecting means is aligned with the matched feature part in the sample image, so as to obtain a second shaped image.
  8. A computer-readable information storage medium for storing a program for causing a computer to function as: obtaining means for obtaining a captured image of a document that includes a fixed part and an un-fixed part, the document being captured by an image reader or an image capture device, wherein the fixed part is a part where content is fixed, and common to all of the documents, wherein the un-fixed part is a part where content is not fixed, and is not common to two documents or more documents; first shaping means for adjusting the shape of the captured image based on a feature point group extracted from the fixed part of the document in a sample image and a feature point group extracted from the fixed part of the document in the captured image so as to obtain a first shaped image, wherein the adjusting can correct nonlinear skew, wherein the sample image is an image in a state where the document is captured from a front direction or a substantially front direction, wherein the front direction is a direction perpendicular to a surface of the document, wherein the angle between the substantially front direction and the surface of the document is 80° or more; detecting means for detecting a feature part of the fixed part from the first shaped image; wherein the detecting means detects the feature part from the first shaped image based on template matching using a template of a feature part in the sample image, and further causing the computer to function as: second shaping means for correcting only linear skew by aligning the first shaped image such that a positional relationship of the feature part detected by the detecting means is aligned with the matched feature part in the sample image, so as to obtain a second shaped image.

Description

Technical Field The one embodiment of the present invention relates to an image processing system, an image processing method, and a program. Background Art There are known techniques for analyzing an image of a document captured by a scanner or a camera. For example, Patent Literature 1 describes a device that searches for a reference mark, which is printed on a predetermined position in a document as a feature part, in a direction oblique to the captured image. In this device, the skew of the document in the captured image is corrected based on the position of the detected reference mark, and then optical character recognition is performed. Citation List Patent Literature Patent Literature 1: JP2001-014427A Summary of Invention Technical Problem However, how the document is skewed or distorted is different depending on captured images, and thus, even the feature part is searched obliquely in the captured image as in the Patent Literature 1, sometimes the feature part may not be detected well. In this case, the feature part needs to be searched repeatedly by changing the directions, which takes much time for performing image processing. One or more embodiments of the present invention have been conceived in view of the above, and an object thereof is to provide an image processing system, an image processing method, and a program capable of speeding up image processing. Solution to Problem In order to solve the above described issues, an image processing system according to the one embodiment of the present invention is defined in claim 1. An image processing method according to the one embodiment of the present invention is defined in claim 7. A program according to the one embodiment of the present invention is defined in claim 8. Effects of the Invention According to the present invention, it is possible to speed up image processing. Brief Description of the Drawings FIG. 1 is a diagram illustrating an overall configuration of an image processing system;FIG. 2 is a diagram showing how a driver's license is captured;FIG. 3 is a diagram illustrating an example of a captured image;FIG. 4 is a functional block diagram showing an example of functions implemented in the image processing system;FIG. 5 is a diagram illustrating an example of a sample image;FIG. 6 is a diagram illustrating an example of data storage of the sample image database;FIG. 7 is a diagram illustrating examples of template images stored in the sample image database;FIG. 8 is a diagram illustrating how a captured image is shaped;FIG. 9 is a diagram illustrating how a captured image is shaped;FIG. 10 is a diagram illustrating an example of template matching;FIG. 11 is a diagram illustrating processing of a second shaping unit;FIG. 12 is a diagram illustrating processing of the second shaping unit;FIG. 13 is a diagram illustrating an example of characters extracted by optical character recognition;FIG. 14 is a flow chart showing an example of processing executed in the image processing system;FIG. 15 is a diagram illustrating an example of the captured image;FIG. 16 is a functional block diagram showing an example of functions implemented in the image processing system;FIG. 17 is a diagram showing how a substitute feature part is searched;FIG. 18 is a diagram showing how a substitute feature part is detected from a sample image;FIG. 19 is a diagram showing how a shaped image is obtained in embodiment 2;FIG. 20 is a diagram showing how a shaped image is obtained in embodiment 2;FIG. 21 is a diagram illustrating an example of characters extracted by optical character recognition;FIG. 22 is a flow chart showing an example of processing executed in the image processing system;FIG. 23 is a flow chart showing an example of processing executed in the image processing system;FIG. 24 is a diagram explaining processing in variation (1) of embodiment 2; andFIG. 25 is a diagram explaining processing in variation (2) of embodiment 2. Description of Embodiments [1. Embodiment 1] An embodiment of the image processing system according to the one embodiment of the present invention will be described below. [1-1. Overall configuration of image processing system] FIG. 1 is a diagram illustrating an overall configuration of the image processing system. As shown in FIG. 1, the image processing system S includes a user terminal 10, an image reader 20, and a server 30. The user terminal 10 and the server 30 are connectable to the network, such as the Internet. FIG. 1 shows one user terminal 10, one image reader 20, and one server 30, although the number of each of them may be two or more. The image reader 20 may be connectable to the network. The user terminal 10 is a computer operated by a user, such as a mobile phone (including a smartphone), a portable information terminal (including a tablet computer), and a personal computer. As shown in FIG. 1, the user terminal 10 includes a control unit 11, a storage unit 12, a communication unit 13, an operation unit 14,