CN-116547708-B - Object classification device, object classification system, and object classification method

Abstract

Representative feature amounts, which are representative feature amounts determining the type or state of the object, are extracted, the type or state of the object is determined based on the representative feature amounts, the object is classified, and the classification result and the representative feature amounts of the object are associated with the image and output.

Inventors

• KAKINOMOTO HIROO
• HATORI HIDEHARU
• Ban Jizhuo

Assignees

• 株式会社日立高新技术

Dates

Publication Date

20260512

Application Date

20211112

Priority Date

20201208

Claims (15)

1. 1. An object classification device for classifying objects in an image to determine the type or state of the objects, characterized in that, The object classification device has: an object region calculation unit that calculates an object region of the object in the image; A feature amount selection unit that selects a feature amount of the object using the object region and outputs a feature amount selection signal; a feature amount extraction unit that extracts a plurality of feature amounts from the image based on the feature amount selection signal; A feature quantity classification unit that classifies a plurality of the feature quantities to extract a representative feature quantity that is a representative feature quantity that determines the type or state of the object; An object classification unit that classifies the object based on the type or the state of the object determined by the representative feature quantity, and outputs a classification result of the object, and An output unit that outputs the classification result and the representative feature quantity of the object in association with the image, The feature amount selection unit selects a part of the feature amounts from the plurality of feature amounts using the target region, and outputs a feature amount selection signal including the type and size of the feature amounts and region information for extracting the feature amounts.
2. 2. The object classification apparatus of claim 1, wherein, The feature extraction unit extracts the feature corresponding to the target portion, The object classification section classifies the object based on the combination of the representative feature amounts, The output unit outputs the classification result of the object and the representative feature quantity in association with each other for each representative feature quantity.
3. 3. The object classification apparatus of claim 1, wherein, The feature amount classification unit extracts the representative feature amount of the object based on the object region calculated by the object region calculation unit, the feature amount selection signal calculated by the feature amount selection unit, and the feature amount extracted by the feature amount extraction unit.
4. 4. The object classification apparatus of claim 1, wherein, The feature amount selection unit selects a type of the feature amount to be applied to the image, based on a size of the target area.
5. 5. The object classification apparatus of claim 1, wherein, The object region calculating unit estimates a distance image to separate the objects that are in contact with each other.
6. 6. The object classification apparatus of claim 1, wherein, The object classification unit classifies the object by referring to an object classification table in which the representative feature amount is stored in association with the type or state of the object.
7. 7. An object classification system, comprising: The object classification apparatus of claim 1; an image capturing device capturing the image input to the object classifying device, and And a display device that displays the classification result of the object in the image in association with the representative feature quantity.
8. 8. An object classification system, comprising: The object classification apparatus of claim 1; a photographing device photographing the image input to the object classifying device; an input device that accepts operation information of a user; GUI generating means for generating a GUI for highlighting a predetermined object based on at least one of the classification result and the representative feature amount of the object selected by the user from the input means, and And a display device that displays the GUI generated by the GUI generating device.
9. 9. The object classification system of claim 8, wherein, The GUI generating means generates the GUI having a sensitivity adjustment bar capable of adjusting the extraction sensitivity of the representative feature quantity.
10. 10. An object classification system, comprising: The object classification apparatus of claim 1; a photographing device photographing the image input to the object classifying device; a statistical information calculating means for calculating statistical information including the number or distribution of the objects based on the classification result and the representative feature quantity of the objects output from the object classifying means, and And a display device that displays the statistical information output by the statistical information calculation device.
11. 11. An object classification method for classifying objects in an image to determine the type or state of the objects, characterized in that, The object classification method has the steps of: An object region calculation step of calculating an object region of the object within the image; A feature amount selection step of selecting a feature amount of the object using the object region and outputting a feature amount selection signal; a feature amount extraction step of extracting a plurality of feature amounts from the image in accordance with the feature amount selection signal; a feature quantity classification step of classifying a plurality of the feature quantities to extract a representative feature quantity, which is a representative feature quantity that determines the type or state of the object; An object classification step of classifying the object based on the type or the state of the object determined by the representative feature quantity, outputting a classification result of the object, and An output step of outputting the classification result and the representative feature quantity of the object in association with the image, In the feature amount selecting step, the feature amount selecting signal including the type and size of the feature amount and the region information for extracting the feature amount is outputted by selecting a part of the feature amount from the plurality of feature amounts using the target region.
12. 12. The method of classifying objects according to claim 11, In the feature amount extraction step, the feature amount corresponding to the portion of the object is extracted, In the object classifying step, the object is classified based on the combination of the representative feature amounts, In the outputting step, the classification result of the object and the representative feature quantity are associated and outputted for each of the representative feature quantities in the combination of the representative feature quantities.
13. 13. The method of classifying objects according to claim 11, In the feature quantity classification step, the representative feature quantity of the object is extracted based on the object region calculated in the object region calculation step, the feature quantity selection signal calculated in the feature quantity selection step, and the feature quantity extracted in the feature quantity extraction step.
14. 14. The method of classifying objects according to claim 11, In the feature amount selection step, a type of the feature amount applied to the image is selected according to a size of the object region.
15. 15. The method of classifying objects according to claim 11, In the object region calculating step, the objects that are in contact with each other are separated by estimating a distance image.

Description

Object classification device, object classification system, and object classification method Technical Field The invention relates to an object classification device, an object classification system, and an object classification method. Background In general, an optical microscope, an electron microscope, or the like is used for observing cells and materials. However, visual inspection of microscopic images is time consuming and requires expertise in many cases. Therefore, in order to assist in evaluation of cells or materials using a microscope image, a technique for automating a part of the processing using an image processing has been developed. For example, patent document 1 discloses a method for determining the state of a cell using time variations in morphological characteristics of 2 different types of cells. However, the surface of the cell or material is often amorphous, and may not be accurately determined by the feature amount designed by hand alone. On the other hand, in recent years, an example has been reported in which accuracy of microscopic image analysis is improved as compared with the conventional technique by using machine learning such as deep learning. In the case of deep learning, the feature quantity is automatically learned mechanically, and thus a feature quantity which is difficult to design manually can sometimes be obtained. Prior art literature Patent literature Patent document 1 Japanese patent application laid-open No. 2011-229409 Disclosure of Invention Problems to be solved by the invention By generating the classifier of the object using deep learning, the feature quantity effective for improving the classification accuracy can be automatically learned as compared with the manual design feature quantity. However, in the case of using a general deep learning classifier, the output obtained by the user is only the classification result. For example, when a microscope image is observed by the human eye, even if a feature considered to be effective for classification exists at a specific part within an object, it is uncertain whether or not the feature quantity is correctly learned and extracted from the part. Therefore, it is difficult to classify objects within an image with high accuracy. The object of the present invention is to accurately classify an object in an image in an object classification device. Means for solving the problems An object classification device according to one aspect of the present invention is an object classification device for classifying an object in an image to determine a type or a state of the object, the object classification device including an object region calculation unit for calculating an object region of the object in the image, a feature amount selection unit for selecting a feature amount of the object using the object region and outputting a feature amount selection signal, a feature amount extraction unit for extracting a plurality of the feature amounts from the image based on the feature amount selection signal, a feature amount classification unit for classifying the plurality of the feature amounts to extract a representative feature amount which is a representative feature amount for determining the type or the state of the object, an object classification unit for classifying the object based on the representative feature amount to determine the type or the state of the object, and an output unit for outputting a classification result of the object in association with the image. Effects of the invention According to one aspect of the present invention, in an object classification device, objects in an image can be classified with high accuracy. Drawings Fig. 1 shows an example of a hardware configuration of the object classification system of embodiment 1. Fig. 2 shows an example of a functional block diagram of the object classification apparatus according to embodiment 1. Fig. 3 shows an example of a microscopic image obtained by photographing ES cells. Fig. 4 shows an example of the result of the region division application. Fig. 5 shows an example of separation of contact objects estimated based on distance images. Fig. 6 shows an example of feature extraction of nuclei. FIG. 7 shows an example of a cell membrane region and a cytoplasmic region. Fig. 8 shows an example of the calculation of the representative feature quantity of the object. Fig. 9 shows an example of a correspondence table between feature amounts and object classes. Fig. 10 shows an example of display of the object classification result in embodiment 1. Fig. 11 shows an example of a hardware configuration of the object search system of embodiment 2. Fig. 12A shows an example of the GUI of embodiment 2. Fig. 12B shows an example of the GUI of embodiment 2. Fig. 13 shows an example of a hardware configuration of the object statistical information calculation system of embodiment 3. Fig. 14 shows an example of the GUI of example 3. Fig. 15 shows an exampl