CN111192271B - Image segmentation method and device - Google Patents
Image segmentation method and device Download PDFInfo
- Publication number
- CN111192271B CN111192271B CN201811355417.4A CN201811355417A CN111192271B CN 111192271 B CN111192271 B CN 111192271B CN 201811355417 A CN201811355417 A CN 201811355417A CN 111192271 B CN111192271 B CN 111192271B
- Authority
- CN
- China
- Prior art keywords
- image
- feature
- indication
- query
- segmented
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003709 image segmentation Methods 0.000 title claims abstract description 68
- 238000000034 method Methods 0.000 title claims abstract description 62
- 239000011159 matrix material Substances 0.000 claims abstract description 65
- 238000012545 processing Methods 0.000 claims description 72
- 239000013598 vector Substances 0.000 claims description 52
- 238000010586 diagram Methods 0.000 claims description 27
- 238000011176 pooling Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 238000010606 normalization Methods 0.000 abstract description 6
- 230000011218 segmentation Effects 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 7
- 238000013528 artificial neural network Methods 0.000 description 6
- 238000002372 labelling Methods 0.000 description 6
- 238000003062 neural network model Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 241000282414 Homo sapiens Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000035479 physiological effects, processes and functions Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application provides an image segmentation method and device, which are characterized in that an indication image containing a target object and a query image to be segmented are encoded, image features extracted after encoding are used for comparison, and then the obtained feature distribution matrix is matched with the extracted image features for decoding, so that segmented images of the target object are segmented from the query image, normalization or size constraint is not needed to be carried out on the images to be segmented, the corresponding target object can be segmented from the query image to be segmented by using a small amount of data through comparison of the two images, the speed is high, the robustness is good, and the adaptability is strong.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image segmentation method and apparatus.
Background
Image processing technology is a interdisciplinary field, along with the continuous development of scientific technology, image processing technology is also developed in a long term, and image processing and analysis gradually form a scientific system of own, so that the image processing technology brings about wide attention to people in all aspects and industries. First, vision is the most important perception means for human beings, and images are the basis of vision, so image processing is an effective tool for studying vision perception by students in many fields such as psychology, physiology, computer science and the like. Second, image processing is in ever-increasing demand in large-scale applications such as military, remote sensing, weather, and the like.
The existing image segmentation technology generally utilizes a neural network, such as a U-net neural network, to perform end-to-end semantic segmentation on a picture, namely, the neural network is enabled to learn the mapping from an RGB image space to a segmented image, the method can effectively segment the image, but a large amount of labeling data is needed to be performed on the image to play a role, the data amount in the processing process is large, the cost for obtaining the labeling data is very high, the time and the labor are consumed, and in practical application, a large amount of data labeling is not easy to be performed on some images, such as the medical field, the safety field and the like.
Disclosure of Invention
In view of the above, the present application provides an image segmentation method and apparatus, so as to facilitate the segmentation of a corresponding object in an image to be segmented by a small amount of object data, with fast speed, good robustness and strong adaptability.
The embodiment of the application provides an image segmentation method, which comprises the following steps:
encoding the query image and the indication image corresponding to the query image by using the same encoding mode, wherein the indication image and the query image contain the same kind of target objects;
extracting first image features of the coded query image and extracting second image features of the coded indication image;
based on the first image features and the second image features, performing feature comparison on the query image and the indication image, and determining a feature distribution matrix of a target object in the indication image in the query image;
and decoding according to the characteristic distribution matrix and the first image characteristic to obtain a segmented image of the target object segmented from the query image.
Further, the determining the feature distribution matrix of the target object in the indication image in the query image based on the first image feature and the second image feature by comparing the features of the query image and the indication image includes:
determining a feature distribution map of the query image based on the first image features;
determining a feature vector of the indication image based on the second image feature;
and determining a feature distribution matrix of the indication image based on the feature distribution map of the query image and the image feature vector of the indication image.
Further, the determining, based on the second image feature, a feature vector of the indication image includes:
generating a feature distribution map of the indication image based on the second image feature;
and carrying out maximum pooling treatment on the characteristic distribution diagram of the indication image to obtain the characteristic vector of the indication image.
Further, the determining the feature distribution matrix of the indication image based on the feature distribution map of the query image and the feature vector of the indication image includes:
performing inner product processing on the feature vector of the indication image and the feature vector at each position in the feature distribution diagram of the query image;
and normalizing the result of the inner product processing to obtain the characteristic distribution matrix of the indication image.
Further, the decoding process according to the feature distribution matrix and the first image feature, to obtain a segmented image of the target object segmented from the query image, includes:
performing product processing on the characteristic distribution matrix and the second image characteristic;
the result after the product processing is connected in parallel with the first image feature and then is decoded;
and determining a segmented image of the target object obtained by carrying out image segmentation on the query image based on the result after the decoding processing.
An embodiment of the present application provides an image segmentation apparatus including:
the coding module is used for respectively coding the query image and the indication image corresponding to the query image in the same coding mode, wherein the indication image and the query image contain the same type of target objects;
the extraction module is used for extracting the first image characteristics of the coded query image and extracting the second image characteristics of the coded indication image;
the comparison module is used for comparing the features of the query image and the indication image based on the first image features and the second image features, and determining a feature distribution matrix of a target object in the indication image in the query image;
and the decoding module is used for carrying out decoding processing according to the characteristic distribution matrix and the first image characteristic to obtain a segmented image of the target object segmented from the query image.
Further, the comparison module includes:
a first determining unit configured to determine a feature distribution map of the query image based on the first image feature;
a second determining unit configured to determine a feature vector of the instruction image based on the second image feature;
and a third determining unit, configured to determine a feature distribution matrix of the indication image based on a feature distribution map of the query image and an image feature vector of the indication image.
Further, the second determining unit is specifically configured to:
generating a feature distribution map of the indication image based on the second image feature;
and carrying out maximum pooling treatment on the characteristic distribution diagram of the indication image to obtain the characteristic vector of the indication image.
Further, the third determining unit is specifically configured to:
performing inner product processing on the feature vector of the indication image and the feature vector at each position in the feature distribution diagram of the query image;
and normalizing the result of the inner product processing to obtain the characteristic distribution matrix of the indication image.
Further, the decoding module includes:
the first processing unit is used for performing product processing on the characteristic distribution matrix and the second image characteristic;
the second processing unit is used for parallelly connecting the result after the product processing with the first image characteristic and then carrying out decoding processing;
and a fourth determining unit configured to determine a segmented image of the target object obtained by image segmentation of the query image based on a result of the decoding process.
The embodiment of the application also provides electronic equipment, which comprises: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory in communication via the bus when the electronic device is running, the machine-readable instructions when executed by the processor performing the steps of the image segmentation method as described above.
The embodiments of the present application also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the image segmentation method as described above.
According to the image segmentation method and device provided by the embodiment of the application, the query image and the indication image corresponding to the query image are respectively encoded in the same encoding mode, and the indication image and the query image contain the same type of target objects; extracting first image features of the coded query image and extracting second image features of the coded indication image; based on the first image features and the second image features, performing feature comparison on the query image and the indication image, and determining a feature distribution matrix of a target object in the indication image in the query image; and decoding according to the characteristic distribution matrix and the first image characteristic to obtain a segmented image of the target object segmented from the query image. Compared with the image segmentation method in the prior art, the method has the advantages that the indication image containing the target object and the query image to be segmented are encoded, the image features extracted after encoding are used for comparison, and then the obtained feature distribution matrix is matched with the extracted image features for decoding, so that the segmented image of the target object is segmented from the query image, normalization or size constraint of the image to be segmented is not needed, the corresponding target object can be segmented from the query image to be segmented by using a small amount of data through comparison of the two images, the speed is high, the robustness is good, and the adaptability is strong.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a diagram of a system architecture in one possible application scenario;
FIG. 2 is a flowchart of an image segmentation method according to an embodiment of the present application;
FIG. 3 is a flowchart of an image segmentation method according to another embodiment of the present application;
FIG. 4 is a schematic diagram of an image segmentation model;
FIG. 5 is a schematic illustration of the feature alignment shown in FIG. 4;
FIG. 6 is a block diagram of an image segmentation apparatus according to an embodiment of the present application;
FIG. 7 is a block diagram of the alignment module shown in FIG. 6;
FIG. 8 is a block diagram of the decoding module shown in FIG. 6;
fig. 9 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art without making any inventive effort, are intended to be within the scope of the present application.
First, an application scenario to which the present application is applicable will be described. The method can be applied to the technical field of image segmentation, and can segment the target object image from a given image, and has the advantages of high speed, good robustness and strong adaptability. Referring to fig. 1, fig. 1 is a system diagram in the application scenario. As shown in fig. 1, the system includes an image segmentation device, an indication image and a query image, the image segmentation device may acquire the indication image and the query image from the indication image and the query image, respectively, and image-segment the query image using the indication image.
According to research, the traditional image segmentation technology generally utilizes a neural network, such as a U-net neural network, to perform end-to-end semantic segmentation on the image, namely, the neural network is enabled to learn the mapping from RGB image space to segmented images, the method can effectively segment the image, but a large amount of labeling data is needed to be performed on the image to play a role, the data amount in the processing process is large, the cost for acquiring the labeling data is high, the time and the labor are consumed, and in practical application, a large amount of data labeling is not easy to be performed on some images, such as the medical field, the safety field and the like.
Based on the above, the embodiment of the application provides an image segmentation method and device, which can find out similar areas by comparing the image characteristics of a query image and an indication image so as to realize the segmentation of a target object image, so that a small amount of data can be used for segmenting a corresponding target object in the query image to be segmented, and the method and device are high in speed, good in robustness and strong in adaptability.
Referring to fig. 2, fig. 2 is a flowchart of an image segmentation method according to an embodiment of the application. As shown in fig. 2, the image segmentation method provided by the embodiment of the application includes:
step 201, encoding the query image and the indication image corresponding to the query image by using the same encoding mode, wherein the indication image and the query image contain the same kind of target objects.
In this step, when the image segmentation device for segmenting the image needs to segment the target object from the query image to be segmented, the image segmentation device may first obtain the query image and the instruction image corresponding to the query image, and encode the query image and the instruction image by using the same encoding method.
The indication image and the query image contain the same kind of target objects, and specifically, the indication image may be a target object indication image of the same kind of target objects that need to be segmented from the query image.
The query image and the indication image are encoded, and a deep learning segmentation neural network model may be used for encoding, for example, a U-net segmentation network model, a deconvolution neural network model, and the like, where the segmentation neural network model may include an encoding network and a decoding network.
In this way, the query image and the indication image are encoded by using the same encoding mode, parameters of the encoding network are shared, and data structures, attributes and the like can be unified, so that subsequent feature extraction, comparison and data processing are facilitated.
Step 202, extracting first image features of the coded query image, and extracting second image features of the coded indication image.
In this step, after encoding the query image and the instruction image, the image segmentation apparatus may further perform feature extraction processing on the encoded query image and the encoded instruction image to extract a first image feature of the query image and a second image feature of the instruction image.
In this way, since the same encoding method is used for encoding, the same feature extraction mode is used for extracting the features, and the workload of the image segmentation apparatus can be reduced.
And 203, comparing the features of the query image and the indication image based on the first image features and the second image features, and determining a feature distribution matrix of the target object in the indication image in the query image.
In this step, after the image segmentation device extracts the first image feature and the second image feature, the image segmentation device may perform feature comparison on the query image and the indication image according to the first image feature and the second image feature, so as to implement a process of fusing information of the query image and the indication image, thereby determining a feature distribution matrix of a target object in the indication image in the query image.
Therefore, by comparing the image characteristics of the query image and the indication image and determining the characteristic distribution matrix, the information fusion of the query image and the indication image can be realized, and the similar area of the query image and the indication image can be quickly found.
And 204, performing decoding processing according to the feature distribution matrix and the first image features to obtain a segmented image of the target object segmented from the query image.
In this step, after the feature distribution matrix is determined, the image segmentation apparatus may perform decoding processing on the feature distribution matrix and the first image feature according to the feature distribution matrix and the first image feature, so as to obtain a segmented image of the target object segmented from the query image.
Thus, after the similar areas of the query image and the indication image are found, the target image can be effectively segmented through decoding processing.
According to the image segmentation method provided by the embodiment of the application, the query image and the indication image corresponding to the query image are respectively encoded in the same encoding mode, and the indication image and the query image contain the same type of target objects; extracting first image features of the coded query image and extracting second image features of the coded indication image; based on the first image features and the second image features, performing feature comparison on the query image and the indication image, and determining a feature distribution matrix of a target object in the indication image in the query image; and decoding according to the characteristic distribution matrix and the first image characteristic to obtain a segmented image of the target object segmented from the query image.
Compared with the image segmentation method in the prior art, the method has the advantages that the indication image containing the target object and the query image to be segmented are encoded, the image features extracted after encoding are used for comparison, and then the obtained feature distribution matrix is matched with the extracted image features for decoding, so that the segmented image of the target object is segmented from the query image, normalization or size constraint of the image to be segmented is not needed, the corresponding target object can be segmented from the query image to be segmented by using a small amount of data through comparison of the two images, the speed is high, the robustness is good, and the adaptability is strong.
Referring to fig. 3, fig. 3 is a flowchart of an image segmentation method according to another embodiment of the present application. As shown in fig. 3, the image segmentation method provided by the embodiment of the application includes:
step 301, encoding a query image and an indication image corresponding to the query image by using the same encoding mode, wherein the indication image and the query image contain the same kind of target objects.
Step 302, extracting a first image feature of the coded query image, and extracting a second image feature of the coded indication image.
Step 303, comparing features of the query image and the indication image based on the first image feature and the second image feature, and determining a feature distribution matrix of the target object in the indication image in the query image.
And 304, multiplying the characteristic distribution matrix and the second image characteristic.
In this step, after the image segmentation apparatus determines the feature distribution matrix and the second image feature, the image segmentation apparatus may prepare to decode the feature and the feature distribution after the comparison, specifically, may first perform product processing on the feature distribution matrix and the second image feature, so as to multiply elements in the feature distribution matrix with the second image feature of the indication image one by one, and perform association.
And 305, parallelly connecting the result after the product processing with the first image feature, and then performing decoding processing.
In this step, the image segmentation apparatus may perform a product process on the feature distribution matrix and the second image feature, and then input a result of the product process and the first image feature in parallel to a decoding network of the segmented neural network model to perform a decoding process, so as to obtain a decoded processing result.
And 306, determining a segmented image of the target object obtained by performing image segmentation on the query image based on the decoded result.
In this step, the image segmentation device performs decoding processing after the result of the product processing is connected in parallel with the first image feature, and after the result of the decoding processing is obtained, the segmented image of the target object segmented after the image segmentation of the query image can be determined according to the result.
The descriptions of steps 301 to 303 may refer to the descriptions of steps 201 to 203, which are not described herein.
Further, step 303 includes:
determining a feature distribution map of the query image based on the first image features; determining a feature vector of the indication image based on the second image feature; and determining a feature distribution matrix of the indication image based on the feature distribution map of the query image and the image feature vector of the indication image.
In this step, the image segmentation apparatus may determine a query image feature distribution map of the query image using the first image feature after the first image feature is extracted, and may determine a feature vector of the indication image using the second image feature, and the image segmentation apparatus may determine a feature distribution matrix of the indication image by feature comparison based on the first image feature and the second image feature after the first image feature and the second image feature are determined.
Wherein each element in the feature distribution matrix reflects the distribution of the target objects contained in the indication image on the query image.
Specifically, in some specific embodiments, the determining, based on the second image feature, a feature vector of the indication image includes:
generating a feature distribution map of the indication image based on the second image feature; and carrying out maximum pooling treatment on the characteristic distribution diagram of the indication image to obtain the characteristic vector of the indication image.
In this step, the image segmentation apparatus may generate a feature distribution map of the instruction image by feature tracing or the like based on the second image feature, and then perform a maximum pooling process on the feature distribution map of the instruction image, so as to obtain a feature vector of the instruction image.
Wherein, the feature vector of the indication image may be a feature vector with a spatial resolution of 1×1.
Specifically, in some specific embodiments, the determining the feature distribution matrix of the indication image based on the feature distribution map of the query image and the feature vector of the indication image includes:
performing inner product processing on the feature vector of the indication image and the feature vector at each position in the feature distribution diagram of the query image; and normalizing the result of the inner product processing to obtain the characteristic distribution matrix of the indication image.
In this step, after determining the feature distribution map of the query image and the feature vector of the indication image, the image segmentation apparatus may correlate the feature vector of the indication image with the feature distribution map of the query image, perform inner product processing, and then normalize the result of the inner product processing by using a normalization exponential function, such as a softmax function, so as to obtain a feature distribution matrix of the indication image.
Wherein the feature distribution matrix dimension is a matrix of S x S dimensions.
For example, referring to fig. 4 and 5 as well, fig. 4 is a schematic diagram of an image segmentation model, and fig. 5 is a schematic diagram of feature alignment shown in fig. 4. As shown in fig. 4 and fig. 5, when the image segmentation apparatus needs to segment the target object 11 in the query image 10, an indication image 20 containing the target object 11 may be obtained, a first image feature of the query image 10 and a second image feature of the indication image 20 are extracted by means of feature extraction or the like, then a deep learning segmentation neural network model is used, for example, a typical U-net segmentation network structure may be used, and the query image 10 and the indication image 20 may be encoded by using the same encoding method, specifically, the first image feature and the second image feature may be encoded, and then multiple convolution computation is performed on the encoded first image feature and the encoded second image feature by using the segmentation neural network model, so as to obtain a first feature map 12 of the query image 10 and a second feature map 21 of the indication image 20 successively.
The first feature map 12 and the second feature map 21 may be represented by using feature vectors with spatial resolution of s×s×c, and may represent a distribution of features on the query image 10 according to the first feature map 12 of the query image 10.
Wherein S is the spatial resolution and C is the channel number.
In the feature comparison stage of the query image 10 and the indication image 20, the second feature map 21 of the indication image 20 may be subjected to global maximum pooling processing to obtain feature vectors 22 with a spatial resolution of 1×1, then the feature vectors 22 are respectively used to make inner products with feature vectors at each position in the first feature map 12 of the query image 10, and normalized by a softmax function to obtain a feature distribution matrix 23 of the indication image 20 in s×s dimension, each element in the feature distribution matrix 23 reflects the distribution condition of the target object contained in the indication image 20 on the query image 10, finally the elements in the feature distribution matrix 23 are multiplied by the feature vectors 22 of the indication image 20 one by one, the multiplied result is parallel connected with the first feature map 12 of the query image 10 and then is input to a decoding network as decoding features for decoding processing, for example, the segmented image 30 containing the target object may be output by a calculation mode such as deconvolution.
In the present embodiment, four convolution calculations are taken as an example, but the present application is not limited thereto, and in other embodiments, the number of convolution calculations may be set according to actual needs.
According to the image segmentation method provided by the embodiment of the application, the query image and the indication image corresponding to the query image are respectively encoded in the same encoding mode, and the indication image and the query image contain the same type of target objects; extracting first image features of the coded query image and extracting second image features of the coded indication image; based on the first image features and the second image features, performing feature comparison on the query image and the indication image, and determining a feature distribution matrix of a target object in the indication image in the query image; performing product processing on the characteristic distribution matrix and the second image characteristic; the result after the product processing is connected in parallel with the first image feature and then is decoded; and determining a segmented image of the target object obtained by carrying out image segmentation on the query image based on the result after the decoding processing.
Compared with the image segmentation method in the prior art, the method has the advantages that the indication image containing the target object and the query image to be segmented are encoded, the image features extracted after encoding are used for comparison, and the obtained feature distribution matrix is matched with the extracted image features for carrying out multiplication and decoding processing, so that the segmented image of the target object is determined from the query image, therefore, the normalization or the size constraint of the image to be segmented is not needed, the corresponding target object can be segmented in the query image to be segmented by using a small amount of data through comparison of the two images, the speed is high, the robustness is good, and the adaptability is strong.
Referring to fig. 6, fig. 6 is a block diagram of an image segmentation apparatus according to an embodiment of the present application, fig. 7 is a block diagram of the comparison module shown in fig. 6, and fig. 8 is a block diagram of the decoding module shown in fig. 6. As shown in fig. 6, the image segmentation apparatus 600 includes:
the encoding module 610 is configured to encode an inquiry image and an indication image corresponding to the inquiry image by using the same encoding mode, where the indication image and the inquiry image include the same type of target object.
An extraction module 620, configured to extract a first image feature of the encoded query image and extract a second image feature of the encoded indication image.
And a comparison module 630, configured to perform feature comparison on the query image and the indication image based on the first image feature and the second image feature, and determine a feature distribution matrix of the target object in the indication image in the query image.
And the decoding module 640 is configured to perform decoding processing according to the feature distribution matrix and the first image feature, so as to obtain a segmented image of the target object segmented from the query image.
Further, as shown in fig. 7, the comparison module 630 includes:
a first determining unit 631 for determining a feature distribution map of the query image based on the first image feature.
A second determining unit 632 is configured to determine a feature vector of the indication image based on the second image feature.
A third determining unit 633, configured to determine a feature distribution matrix of the indication image based on a feature distribution map of the query image and an image feature vector of the indication image.
Further, the second determining unit 632 is specifically configured to:
generating a feature distribution map of the indication image based on the second image feature; and carrying out maximum pooling treatment on the characteristic distribution diagram of the indication image to obtain the characteristic vector of the indication image.
Further, the third determining unit 633 is specifically configured to:
performing inner product processing on the feature vector of the indication image and the feature vector at each position in the feature distribution diagram of the query image; and normalizing the result of the inner product processing to obtain the characteristic distribution matrix of the indication image.
Further, as shown in fig. 8, the decoding module 640 includes:
the first processing unit 641 is configured to perform product processing on the feature distribution matrix and the second image feature.
And a second processing unit 642, configured to perform decoding processing after connecting the result after the product processing with the first image feature in parallel.
A fourth determining unit 643, configured to determine a segmented image of the target object obtained by image segmentation of the query image based on a result of the decoding process.
The image segmentation apparatus 600 in this embodiment may implement all the method steps of the image segmentation method in the embodiment shown in fig. 2 and 3, and may achieve the same effects, which are not described herein.
The image segmentation device provided by the embodiment of the application respectively carries out coding processing on the query image and the indication image corresponding to the query image by using the same coding mode, wherein the indication image and the query image contain the same type of target objects; extracting first image features of the coded query image and extracting second image features of the coded indication image; based on the first image features and the second image features, performing feature comparison on the query image and the indication image, and determining a feature distribution matrix of a target object in the indication image in the query image; and decoding according to the characteristic distribution matrix and the first image characteristic to obtain a segmented image of the target object segmented from the query image.
Compared with the image segmentation method in the prior art, the method has the advantages that the indication image containing the target object and the query image to be segmented are encoded, the image features extracted after encoding are used for comparison, and then the obtained feature distribution matrix is matched with the extracted image features for decoding, so that the segmented image of the target object is segmented from the query image, normalization or size constraint of the image to be segmented is not needed, the corresponding target object can be segmented from the query image to be segmented by using a small amount of data through comparison of the two images, the speed is high, the robustness is good, and the adaptability is strong.
Referring to fig. 9, fig. 9 is a block diagram of an electronic device according to an embodiment of the application. As shown in fig. 9, the electronic device 900 includes a processor 910, a memory 920, and a bus 930.
The memory 920 stores machine-readable instructions executable by the processor 910, when the electronic device 900 is running, the processor 910 communicates with the memory 920 through the bus 930, and when the machine-readable instructions are executed by the processor 910, the steps of the image segmentation method in the method embodiments shown in fig. 2 and fig. 3 may be executed, and a specific implementation may refer to a method embodiment and will not be described herein.
The embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the steps of the image segmentation method in the method embodiments shown in fig. 2 and fig. 3 may be executed, and a specific implementation manner may refer to the method embodiment and will not be described herein.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Finally, it should be noted that: the above examples are only specific embodiments of the present application, and are not intended to limit the scope of the present application, but it should be understood by those skilled in the art that the present application is not limited thereto, and that the present application is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (8)
1. An image segmentation method, the method comprising:
encoding the query image and the indication image corresponding to the query image by using the same encoding mode, wherein the indication image and the query image contain the same kind of target objects;
extracting first image features of the coded query image and extracting second image features of the coded indication image;
determining a feature vector at each location in a feature distribution map of the query image based on the first image feature;
determining a feature vector of the indication image based on the second image feature;
determining a feature distribution matrix of the indication image based on the feature vector at each position in the feature distribution map of the query image and the feature vector of the indication image;
and carrying out product processing on the characteristic distribution matrix and the second image characteristic, and carrying out decoding processing after the product processing result is connected with the first image characteristic in parallel to obtain a segmented image of the target object segmented from the query image.
2. The method of claim 1, wherein the determining the feature vector of the indication image based on the second image feature comprises:
generating a feature distribution map of the indication image based on the second image feature;
and carrying out maximum pooling treatment on the characteristic distribution diagram of the indication image to obtain the characteristic vector of the indication image.
3. The method of claim 1, wherein the determining the feature distribution matrix of the indicator image based on the feature vector at each location in the feature distribution map of the query image and the feature vector of the indicator image comprises:
performing inner product processing on the feature vector of the indication image and the feature vector at each position in the feature distribution diagram of the query image;
and normalizing the result of the inner product processing to obtain the characteristic distribution matrix of the indication image.
4. The method of claim 1, wherein the obtaining a segmented image of the object segmented from the query image based on the feature distribution matrix, the first image feature, and the second image feature comprises:
performing product processing on the characteristic distribution matrix and the second image characteristic;
the result after the product processing is connected in parallel with the first image feature and then is decoded;
and determining a segmented image of the target object obtained by carrying out image segmentation on the query image based on the result after the decoding processing.
5. An image segmentation apparatus, characterized in that the image segmentation apparatus comprises:
the coding module is used for respectively coding the query image and the indication image corresponding to the query image in the same coding mode, wherein the indication image and the query image contain a similar object;
the extraction module is used for extracting the first image characteristics of the coded query image and extracting the second image characteristics of the coded indication image;
the comparison module is used for determining a feature vector at each position in a feature distribution diagram of the query image based on the first image features, determining a feature vector of the indication image based on the second image features, and determining a feature distribution matrix of the indication image based on the feature vector at each position in the feature distribution diagram of the query image and the feature vector of the indication image;
and the decoding module is used for carrying out product processing on the characteristic distribution matrix and the second image characteristic, connecting the result after the product processing with the first image characteristic in parallel, and then carrying out decoding processing to obtain the segmented image of the target object segmented from the query image.
6. The image segmentation apparatus as set forth in claim 5, wherein the comparison module is configured to:
generating a feature distribution map of the indication image based on the second image feature;
and carrying out maximum pooling treatment on the characteristic distribution diagram of the indication image to obtain the characteristic vector of the indication image.
7. The image segmentation apparatus as set forth in claim 5, wherein the comparison module is configured to:
performing inner product processing on the feature vector of the indication image and the feature vector at each position in the feature distribution diagram of the query image;
and normalizing the result of the inner product processing to obtain the characteristic distribution matrix of the indication image.
8. The image segmentation apparatus as set forth in claim 5, wherein the decoding module comprises:
the first processing unit is used for performing product processing on the characteristic distribution matrix and the second image characteristic;
the second processing unit is used for parallelly connecting the result after the product processing with the first image characteristic and then carrying out decoding processing;
and a fourth determining unit configured to determine a segmented image of the target object obtained by image segmentation of the query image based on a result of the decoding process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811355417.4A CN111192271B (en) | 2018-11-14 | 2018-11-14 | Image segmentation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811355417.4A CN111192271B (en) | 2018-11-14 | 2018-11-14 | Image segmentation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111192271A CN111192271A (en) | 2020-05-22 |
| CN111192271B true CN111192271B (en) | 2023-08-22 |
Family
ID=70710479
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811355417.4A Active CN111192271B (en) | 2018-11-14 | 2018-11-14 | Image segmentation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111192271B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111982911B (en) * | 2020-07-10 | 2023-08-25 | 深圳先进技术研究院 | Method and device for detecting defects of circuit board, terminal equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101447020A (en) * | 2008-12-12 | 2009-06-03 | 北京理工大学 | Pornographic image recognizing method based on intuitionistic fuzzy |
| CN101667291A (en) * | 2009-09-25 | 2010-03-10 | 西安电子科技大学 | High-spectrum image segmentation method based on pixel space information |
| CN101996245A (en) * | 2010-11-09 | 2011-03-30 | 南京大学 | Form feature describing and indexing method of image object |
| CN107844735A (en) * | 2016-09-19 | 2018-03-27 | 北京眼神科技有限公司 | Authentication method and device for biological characteristics |
| CN107895371A (en) * | 2017-11-24 | 2018-04-10 | 常州大学 | Textile flaw detection method based on peak value coverage values and Gabor characteristic |
| WO2018076212A1 (en) * | 2016-10-26 | 2018-05-03 | 中国科学院自动化研究所 | De-convolutional neural network-based scene semantic segmentation method |
-
2018
- 2018-11-14 CN CN201811355417.4A patent/CN111192271B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101447020A (en) * | 2008-12-12 | 2009-06-03 | 北京理工大学 | Pornographic image recognizing method based on intuitionistic fuzzy |
| CN101667291A (en) * | 2009-09-25 | 2010-03-10 | 西安电子科技大学 | High-spectrum image segmentation method based on pixel space information |
| CN101996245A (en) * | 2010-11-09 | 2011-03-30 | 南京大学 | Form feature describing and indexing method of image object |
| CN107844735A (en) * | 2016-09-19 | 2018-03-27 | 北京眼神科技有限公司 | Authentication method and device for biological characteristics |
| WO2018076212A1 (en) * | 2016-10-26 | 2018-05-03 | 中国科学院自动化研究所 | De-convolutional neural network-based scene semantic segmentation method |
| CN107895371A (en) * | 2017-11-24 | 2018-04-10 | 常州大学 | Textile flaw detection method based on peak value coverage values and Gabor characteristic |
Non-Patent Citations (1)
| Title |
|---|
| 周雪燕 ; 孔梦荣 ; .多尺度纹理图像数据抗干扰信息映射方法研究.微电子学与计算机.2017,(第07期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111192271A (en) | 2020-05-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110472090B (en) | Image retrieval method based on semantic tags, related device and storage medium | |
| CN112465828B (en) | Image semantic segmentation method and device, electronic equipment and storage medium | |
| CN110532884B (en) | Pedestrian re-recognition method, device and computer readable storage medium | |
| WO2022105125A1 (en) | Image segmentation method and apparatus, computer device, and storage medium | |
| CN114663670A (en) | Image detection method and device, electronic equipment and storage medium | |
| CN110866469B (en) | Facial five sense organs identification method, device, equipment and medium | |
| CN116311214B (en) | License plate recognition method and device | |
| CN111915618B (en) | Peak response enhancement-based instance segmentation algorithm and computing device | |
| CN113591566A (en) | Training method and device of image recognition model, electronic equipment and storage medium | |
| CN113177449A (en) | Face recognition method and device, computer equipment and storage medium | |
| CN115577768A (en) | Semi-supervised model training method and device | |
| CN111126049B (en) | Object relation prediction method, device, terminal equipment and readable storage medium | |
| Noor et al. | Multi-scale gradient image super-resolution for preserving SIFT key points in low-resolution images | |
| CN111192271B (en) | Image segmentation method and device | |
| CN114549859A (en) | Bone age prediction method, bone age prediction device, electronic equipment and storage medium | |
| CN114120454A (en) | Training method and device of living body detection model, electronic equipment and storage medium | |
| CN113516697A (en) | Image registration method and device, electronic equipment and computer-readable storage medium | |
| CN112580666A (en) | Image feature extraction method, training method, device, electronic equipment and medium | |
| CN116977714A (en) | Image classification method, apparatus, device, storage medium, and program product | |
| CN117218400A (en) | Image recognition model training method and device, electronic equipment and storage medium | |
| CN115810152A (en) | Remote sensing image change detection method and device based on graph convolution and computer equipment | |
| CN115879455A (en) | Word emotion polarity prediction method and device, electronic equipment and storage medium | |
| CN113610856A (en) | Method and device for training image segmentation model and image segmentation | |
| CN113283241A (en) | Text recognition method and device, electronic equipment and computer readable storage medium | |
| CN111325068A (en) | Video description method and device based on convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231124 Address after: No. 16 Hongzehu West Road, Hongze County, Huai'an City, Jiangsu Province, 223100 Patentee after: Suqian Public Security Bureau Sucheng Branch Patentee after: Watrix Technology (Beijing) Co.,Ltd. Address before: 101500 Room 501-1753, Development Zone Office Building, No. 8 Xingsheng South Road, Miyun District Economic Development Zone, Beijing (Central Office Area of Economic Development Zone) Patentee before: Watrix Technology (Beijing) Co.,Ltd. |
|
| CP02 | Change in the address of a patent holder | ||
| CP02 | Change in the address of a patent holder |
Address after: No. 16 Hongzehu West Road, Sucheng District, Suqian City, Jiangsu Province, 223800 Patentee after: Suqian Public Security Bureau Sucheng Branch Patentee after: Watrix Technology (Beijing) Co.,Ltd. Address before: No. 16 Hongzehu West Road, Hongze County, Huai'an City, Jiangsu Province, 223100 Patentee before: Suqian Public Security Bureau Sucheng Branch Patentee before: Watrix Technology (Beijing) Co.,Ltd. |