CN112749702B - Image recognition method, device, terminal and storage medium - Google Patents

Image recognition method, device, terminal and storage medium Download PDF

Info

Publication number
CN112749702B
CN112749702B CN201911048119.5A CN201911048119A CN112749702B CN 112749702 B CN112749702 B CN 112749702B CN 201911048119 A CN201911048119 A CN 201911048119A CN 112749702 B CN112749702 B CN 112749702B
Authority
CN
China
Prior art keywords
sub
image
images
category
image set
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911048119.5A
Other languages
Chinese (zh)
Other versions
CN112749702A (en
Inventor
吴秦龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Mobile Communications Group Co Ltd
China Mobile Suzhou Software Technology Co Ltd
Original Assignee
China Mobile Communications Group Co Ltd
China Mobile Suzhou Software Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Mobile Communications Group Co Ltd, China Mobile Suzhou Software Technology Co Ltd filed Critical China Mobile Communications Group Co Ltd
Priority to CN201911048119.5A priority Critical patent/CN112749702B/en
Publication of CN112749702A publication Critical patent/CN112749702A/en
Application granted granted Critical
Publication of CN112749702B publication Critical patent/CN112749702B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the application provides an image identification method, an image identification device, a terminal and a storage medium, wherein the method comprises the following steps: dividing an image to be predicted to generate a first sub-image set; based on the picture content of the first sub-image, adjusting the number of the first sub-images in the first sub-image set to obtain a second sub-image set; identifying the picture content of a second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs; the category meeting the preset condition in the identification result is used as the target category of the image to be predicted; therefore, the number of images for image recognition can be expanded, the accuracy of image recognition is improved, and the application scene of the image recognition method is expanded.

Description

Image recognition method, device, terminal and storage medium
Technical Field
The present application relates to image processing, and relates to, but is not limited to, an image recognition method, apparatus, terminal, and storage medium.
Background
Most of the plant diseases and insect pests identification technologies are realized based on classification algorithms, and support vector machines or deep convolutional neural networks are generally selected as classification algorithms in the realization process. In the training stage, a large number of plant diseases and insect pests pictures are used as training data to generate a training model; and in the prediction stage, extracting the characteristics of each picture by using a trained model, and then outputting the plant diseases and insect pests corresponding to the pictures. Because the whole picture is used as the input of the neural network to carry out the classification model training, the local characteristics of the data can not be well learned and the image can not be accurately identified under the conditions that the lesion part in the image occupies smaller area and the image shooting angle is changeable and complex.
Disclosure of Invention
The embodiment of the application provides an image recognition method, an image recognition device, a terminal and a storage medium, which can improve the accuracy of image recognition.
The technical scheme of the embodiment of the application is realized as follows:
the embodiment of the application provides an image identification method, which comprises the following steps:
dividing an image to be predicted to generate a first sub-image set;
based on the picture content of the first sub-image, adjusting the number of the first sub-images in the first sub-image set to obtain a second sub-image set;
identifying the picture content of a second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs;
and taking the category meeting the preset condition in the identification result as the target category of the image to be predicted.
An embodiment of the present application provides an image recognition apparatus, including:
the segmentation module is used for segmenting the image to be predicted to generate a first sub-image set;
the adjusting module is used for adjusting the number of the first sub-images in the first sub-image set based on the picture content of the first sub-images to obtain a second sub-image set;
the identification module is used for identifying the picture content of the second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs;
and the determining module is used for taking the category meeting the preset condition in the identification result as the target category of the image to be predicted.
The embodiment of the application provides a terminal, which at least comprises: a controller and a storage medium configured to store executable instructions, wherein:
the controller is configured to execute stored executable instructions configured to perform the image recognition method provided above.
Embodiments of the present application provide a computer-readable storage medium having stored therein computer-executable instructions configured to perform the image recognition method provided above.
The embodiment of the application provides an image identification method, an image identification device, a terminal and a storage medium, wherein an image to be predicted is firstly segmented to generate a first sub-image set; then, based on the picture content of the first sub-image, adjusting the number of the first sub-images in the first sub-image set to obtain a second sub-image set; identifying the picture content of a second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs; finally, the category meeting the preset condition in the identification result is used as the target category of the image to be predicted; therefore, the number of images for image recognition is expanded by dividing the image to be predicted into a plurality of sub-images, the duty ratio of the second sub-image in the second sub-image set is improved by adjusting the number of the first sub-images, the accuracy of image recognition is improved, and the application scene of the image recognition method is enlarged.
Drawings
In the drawings (which are not necessarily drawn to scale), like numerals may describe similar components in different views. Like reference numerals having different letter suffixes may represent different instances of similar components. The drawings illustrate generally, by way of example and not by way of limitation, various embodiments discussed herein.
FIG. 1 is a schematic diagram of an implementation flow of an image recognition method according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of another implementation of the image recognition method according to the embodiment of the present application;
FIG. 3 is a flowchart illustrating another implementation of the image recognition method according to an embodiment of the present disclosure;
FIG. 4 is a flowchart illustrating another implementation of the image recognition method according to an embodiment of the present application;
FIG. 5 is a schematic diagram of image segmentation of an image to be predicted according to an embodiment of the present application;
FIG. 6 is a schematic diagram illustrating the structure of an image recognition device according to an embodiment of the present disclosure;
fig. 7 is a schematic diagram of a composition structure of a terminal according to an embodiment of the present application.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
In the following description, suffixes such as "module", "component", or "unit" for representing elements are used only for facilitating the description of the present application, and are not of specific significance per se. Thus, "module," "component," or "unit" may be used in combination.
The terminal may be implemented in various forms. For example, terminals described in the present application may include mobile terminals such as cell phones, tablet computers, notebook computers, palm computers, personal digital assistants (Personal Digital Assistant, PDA), portable media players (Portable Media Player, PMP), navigation devices, wearable terminals, smart bracelets, pedometers, and stationary terminals such as digital TVs, desktop computers, and the like.
The following description will be given taking a mobile terminal as an example, and those skilled in the art will understand that the configuration according to the embodiment of the present application can be applied to a fixed type terminal in addition to elements particularly used for a moving purpose
An embodiment of the present application provides an image recognition method, and fig. 1 is a schematic implementation flow chart of the image recognition method according to the embodiment of the present application, as shown in fig. 1, where the image recognition method includes the following steps:
step S101: the image to be predicted is segmented, and a first sub-image set is generated.
Here, the image to be predicted is an image of any size photographed by the terminal, which is subject to detection or recognition. The segmentation of the picture to be predicted refers to the division of the picture to be predicted into a number of specific first sub-pictures, which constitute the first set of sub-pictures. Such as: given an image, the pixel size of the image is 640 x 480, and the image is cut into a plurality of sub-images with the pixel size of 160 x 120 according to the pixel information of the image. Thus, one image can be divided into a plurality of sub-images, the number of images for image recognition is expanded, and the accuracy of image recognition is improved.
Step S102: and adjusting the number of the first sub-images in the first sub-image set based on the picture content of the first sub-images to obtain a second sub-image set.
After the image to be predicted is segmented, a plurality of first sub-images are obtained, wherein each sub-image has corresponding picture content. The step S102 may be understood as classifying the first sub-images according to the picture content of the first sub-images, and adjusting the number of the first sub-images in the first sub-image set according to the ratio of the number of the sub-images of the non-pest type to the number of the sub-images of the pest type in the first sub-image set to obtain the second sub-image set. The adjusting the number of the first sub-images may be reducing the number of the first sub-images or increasing the number of the first sub-images. In this way, the duty ratio of the second sub-image for image recognition in the second sub-image is improved, and the accuracy of image recognition is further improved.
Step S103: and identifying the picture content of the second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs.
In some possible implementations, a neural network is used to identify the picture content of the second sub-image in the second sub-image set, so as to obtain an identification result of the category to which the second sub-image belongs. The recognition result may be characterized as a probability that the category to which the second sub-image belongs may be a certain category in the preset category library, or a confidence level that the category to which the second sub-image belongs may be a certain category in the preset category library.
Step S104: and taking the category meeting the preset condition in the identification result as the target category of the image to be predicted.
Here, the preset conditions may be preset according to application scenes of the image recognition method, for example: the recognition result meets a certain threshold, and the recognition result is the maximum value of probability or confidence.
In the embodiment of the application, the number of the images for image recognition is expanded by dividing the image to be predicted into a plurality of sub-images, and the duty ratio of the second sub-image in the second sub-image set is improved by adjusting the number of the first sub-images in the first sub-image set, so that the accuracy of image recognition is improved, and the application scene of the image recognition method is enlarged.
The embodiment of the application provides an image recognition method, which is specifically described by the following steps:
step S111: dividing an image to be predicted into a plurality of first sub-images with preset pixel size according to a preset step length to obtain the first sub-image set.
Here, the preset step length may be preset as a step length in both the x-axis direction and the y-axis direction, where the length of the x-axis direction step length and the y-axis direction step length may be the same or different.
In the process of dividing the image to be predicted, the image to be predicted is sequentially moved along the x-axis direction or the y-axis direction according to the step length of the image in the x-axis direction and the y-axis direction according to the pixel information of the image, and a plurality of first sub-images which accord with the preset pixel size are intercepted, and form a first sub-image set. In this way, the number of images for image recognition is expanded.
Step S112: and adjusting the number of the first sub-images in the first sub-image set based on the picture content of the first sub-images to obtain a second sub-image set.
Step S113: and identifying the picture content of the second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs.
Step S114: and taking the category meeting the preset condition in the identification result as the target category of the image to be predicted.
In the embodiment of the application, the image to be predicted is intercepted into a plurality of sub-images according to the pixel information of the image, so that the number of the images for image recognition is increased, the accuracy of image recognition is improved, and the application scene of the image recognition method is enlarged.
An embodiment of the present application provides an image recognition method, and fig. 2 is a schematic flow chart of another implementation of the image recognition method according to the embodiment of the present application, as shown in fig. 2, and is described in connection with the method shown in fig. 2:
step S201: dividing an image to be predicted into a plurality of first sub-images with preset pixel size according to a preset step length to obtain the first sub-image set.
Step S202: and determining the category of the picture content in the first sub-image as a target area of a preset category.
Here, the preset category is a plant disease and insect pest category of the crop, the number of the categories is N, and the value of N is a positive integer. In the embodiment of the present application, the target area refers to an area where a lesion position in an image corresponding to any one of N types of plant diseases and insect pests is located. For example, the region in the first sub-image belongs to the lesion position.
Step S203: and determining the intersection ratio of the area of the first sub-image and the area of the target area.
Here, the screen content of the first sub-image is compared with the screen content of the target area, the area of the first sub-image is determined and compared with the area of the target area, and the parallel-to-cross ratio (Intersection over Union, ioU) of the area of the first sub-image and the area of the target area is obtained and is denoted as S. Here, ioU, also called overlap, ioU is calculated by the method shown below, with two rectangular boxes a and B, ioU =aζb/aζb, i.e. IoU having the value of the area ratio of the overlapping area of rectangular boxes A, B to the area of A, B union.
Step S204: and adjusting the number of the first sub-images in the first sub-image set based on the parallel-to-cross ratio to obtain a third sub-image set.
Here, the number of third sub-images is smaller than the number of first sub-images. And deleting the first sub-image corresponding to the sum cross ratio larger than and equal to the first threshold value smaller than and equal to the second threshold value from the first sub-image set to obtain the third sub-image set.
In the embodiment of the present application, the first threshold may be set to 10% and the second threshold may be set to 30%. And deleting the first sub-image corresponding to S from the first sub-image set when S is more than or equal to 10% and less than or equal to 30%, so as to obtain the third sub-image set, wherein the number of the third sub-images in the third sub-image set is smaller than that of the first sub-images in the first sub-image set. The corresponding parallel-to-cross ratio range of the third sub-image in the third sub-image set is as follows: s is less than 10% and S is more than 30%. Here, classifying the third sub-image setting tags in the third sub-image set, wherein the sub-image setting tags corresponding to S < 10% are set to 0, i.e. such sub-image is a non-pest image or a background image; setting a sub-image corresponding to S & gt 30% as N, wherein N is the type of plant diseases and insect pests, and N is E N; in this way, the interference sub-image is deleted from the first sub-image set, and the influence of the interference image in image recognition on the image recognition result is reduced.
Step S205: and adjusting the number of the third sub-images in the third sub-image set based on the picture content of the third sub-image to obtain the second sub-image set.
Here, the number of the third sub-images in the third sub-image set is adjusted according to the number of sub-images with the label of 0 and the number of sub-images with the label of n, so as to obtain the second sub-image set.
Step S206: and identifying the picture content of the second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs.
Step S207: and taking the category meeting the preset condition in the identification result as the target category of the image to be predicted.
In the embodiment of the application, the image to be predicted is segmented according to the pixel information of the image to obtain a plurality of predicted sub-images, so that the number of images for image recognition is increased; and screening the predicted sub-images according to IoU values between the predicted sub-images and target areas of preset categories, deleting the interference sub-images, reducing the influence of the interference images in image recognition on image recognition results, and improving the accuracy of image recognition.
The embodiment of the application provides an image recognition method, which is specifically described by the following steps:
step S211: the image to be predicted is segmented, and a first sub-image set is generated.
Step S212: and determining the category of the picture content in the first sub-image as a target area of a preset category.
Step S213: and determining the intersection ratio of the area of the first sub-image and the area of the target area.
Step S214: and adjusting the number of the first sub-images in the first sub-image set based on the parallel-to-cross ratio to obtain a third sub-image set.
Here, the number of third sub-images is smaller than the number of first sub-images. And deleting the first sub-image corresponding to the sum cross ratio larger than and equal to the first threshold value smaller than and equal to the second threshold value from the first sub-image set to obtain the third sub-image set.
In the embodiment of the present application, the first threshold may be set to 10% and the second threshold may be set to 30%. And deleting the first sub-image corresponding to S from the first sub-image set when S is more than or equal to 10% and less than or equal to 30%, so as to obtain the third sub-image set, wherein the number of the third sub-images in the third sub-image set is smaller than that of the first sub-images in the first sub-image set. The corresponding parallel-to-cross ratio range of the third sub-image in the third sub-image set is as follows: s is less than 10% and S is more than 30%.
Step S215: and if the sum-to-cross ratio is smaller than a first threshold value, determining that the category of the third sub-image is a non-pest category.
In one specific example, the first threshold is set to 10%. And classifying the third sub-images in the third sub-image set, wherein the sub-images corresponding to the parallel-to-cross ratio range S < 10% are set to be 0, namely the sub-images are non-pest images or background images. If the sum-to-cross ratio S of the sub-images is smaller than 10%, the label of the sub-image is 0, and the category corresponding to the sub-image is a non-disease and pest category.
Step S216: and if the sum-to-cross ratio is greater than a second threshold, determining the category of the third sub-image as a plant disease and insect pest category.
In one specific example, the second threshold is set to 30%. And setting the sub-image corresponding to the parallel-to-cross ratio S > 30% as n, wherein n is the type of the plant diseases and insect pests. And if the sum-to-intersection ratio S is more than 30%, the label of the third sub-image is n, and the category of the third sub-image is the plant diseases and insect pests category n.
Step S217: a ratio between the number of sub-images belonging to the non-pest category and to the pest category in the third set of sub-images is determined.
Here, the number of sub-images belonging to the non-pest category in the third sub-image set is calculated, the total number of sub-images belonging to any pest category in the third sub-image set is calculated, and then the ratio u of the number of sub-images of the non-pest category to the number of sub-images of the pest category is calculated.
Step S218: and adjusting the number of the third sub-images based on a preset proportion and the ratio to obtain the second sub-image set.
Here, if the ratio is greater than or equal to the preset ratio, reducing the number of sub-images belonging to the non-pest category in the third sub-image set to obtain the second sub-image set; the ratio of the number of the sub-images belonging to the non-disease and pest type to the number of the sub-images belonging to the disease and pest type in the second sub-image set is smaller than the preset ratio.
In the embodiment of the present application, if the preset ratio is set to be 1, if the ratio u is greater than or equal to 1, the number of sub-images belonging to the non-pest and disease damage category in the third sub-image set is reduced until the ratio u is less than 1. In this way, the ratio of the number of the non-pest images for image recognition to the total number of the images is lower than 50% by adjusting the number of the sub-images of the non-pest categories in the third image subset, so that the ratio of the pest images for image recognition is improved, and the accuracy of image recognition can be improved.
Step S219: and identifying the picture content of the second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs.
Step S220: and taking the category meeting the preset condition in the identification result as the target category of the image to be predicted.
In the embodiment of the application, the number of the non-pest sub-images in the prediction sub-image set is reduced, so that the duty ratio of the pest images for image recognition is improved, and the accuracy of image recognition is further improved.
An embodiment of the present application provides an image recognition method, and fig. 3 is a schematic flow chart of still another implementation of the image recognition method according to the embodiment of the present application, as shown in fig. 3, and is described in connection with the method shown in fig. 3:
step S301: the image to be predicted is segmented, and a first sub-image set is generated.
Step S302: and adjusting the number of the first sub-images in the first sub-image set based on the picture content of the first sub-images to obtain a second sub-image set.
Step S303: and identifying the picture content of the second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs.
Step S304: and determining the probability value of the category to which the picture content of the second sub-image belongs as a preset category so as to obtain the identification result.
Here, the recognition result of the category to which the second sub-image belongs is characterized as a probability value, or may be a confidence. The preset categories are plant diseases and insect pests of crops, the number of the categories is N, and the value of N is a positive integer. The category to which the picture content of the second sub-image belongs may be any one of the N preset categories, so there are a plurality of outputs of probability values or confidence degrees that the category to which the picture content of the second sub-image belongs is a preset category, that is, there are a plurality of the recognition results.
Step S305: and in the identification result, determining the category to which the picture content of the second sub-image corresponding to the maximum probability value belongs as the target category.
Here, a category to which the picture content of the second sub-image corresponding to the maximum value among the plurality of recognition results belongs is taken as a target category of the image to be predicted. The recognition result may be a probability value or a confidence level.
In the embodiment of the application, the image to be predicted is divided into a plurality of predicted sub-images, the plurality of predicted sub-images are subjected to image recognition to obtain a plurality of recognition results, the recognition results meeting the preset conditions are determined from the plurality of recognition results, and the corresponding classification is used as the target classification of the image to be predicted, so that the accuracy of image recognition is improved.
An embodiment of the present application provides an image recognition method, and fig. 4 is a schematic flowchart of still another implementation of the image recognition method according to the embodiment of the present application, and is described with reference to steps shown in fig. 4:
step S401: and performing image segmentation on the image to be predicted to generate a first sub-image.
Fig. 5 is a schematic diagram of image segmentation of an image to be predicted according to an embodiment of the present application. As shown in fig. 5, the image to be predicted is divided according to pixel size information of the image. The size of the image to be predicted is 640 x 480, the area 504 is the actual target area, namely the lesion position of the plant, other areas in the image do not contain information of the lesion of the plant, the number of the plant diseases and insect pests to be identified is N, N is a positive integer, and the plant diseases and insect pests at the lesion position in the image are N, N is E N. The image interception of the image to be predicted to generate the first sub-image specifically comprises the following steps:
the first step: and carrying out image cutting on the image to be predicted in sequence to generate a first sub-image.
In the embodiment of the application, the first sub-image can be generated by an image searching method. And sequentially intercepting images of the image to be predicted by using a rectangular frame with the pixel size of 160 x 120 according to the method that the step length of the x-axis direction is 80 pixels and the step length of the y-axis direction is 60 pixels, so that 49 first sub-images with the pixel size of 160 x 120 can be obtained. As shown in fig. 5, a region 501 is a first sub-image obtained by sequentially capturing images of an image to be predicted with a rectangular frame having a pixel size of 160×120 and a step size of 80 pixels in the x-axis direction and a step size of 60 pixels in the y-axis direction, a region 502 is a first sub-image obtained by capturing the region 501 by moving the region in the x-axis direction by 80 pixels in the step size, and a region 503 is the i-th first sub-image obtained by capturing, 1< i <49.
And a second step of: and judging the category of the first sub-image according to the intersection ratio of the area of the first sub-image and the area of the actual target area.
Specifically, each of the first sub-images obtained in the first step is compared with the actual target area, and IoU of both are calculated and denoted as S. The 49 sub-images acquired in the first step are classified into n+1 types according to the value of S. If S is less than 10%, the first sub-image does not belong to the pest and disease damage image, and a label is set for the first sub-image and is marked as 0; if S is more than 30%, the sub-image belongs to the disease and pest image, and a label is set for the sub-image and is marked as n; if S is more than or equal to 10% and less than or equal to 30%, deleting the first sub-image. Thus, a first sub-image with a size of M Zhang Xiangsu of 160 x 120 and a label class of n+1, where 0.ltoreq.M.ltoreq.49, is obtained.
In the embodiment of the application, a large amount of training data can be generated under the condition of a small amount of pest images by using an image searching method; and judging the category of the first sub-image according to the IoU value of the area of the first sub-image and the area of the actual target area, so that the interference first sub-image can be deleted, and the influence of the interference first sub-image on the image recognition effect in the image recognition process is reduced.
Step S402: and inputting the first sub-image into a neural network to obtain an image classification model of the first sub-image.
In this embodiment of the present application, performing depth feature extraction on the first sub-image by using a neural network, and further obtaining a classification model of the first sub-image includes the following steps:
step one: and resampling the acquired M first sub-images to enable the ratio of the number of samples with the label of 0 to the total training sample number to be lower than 50%, so that resampled training data, namely a second sub-image, can be obtained.
Step two: the second sub-image is input to a neural network.
In this embodiment of the present application, a Residual Network (Residual Network50, resnet 50) is used as a backbone Network, and the second sub-image is input to the Resnet50 Network, specifically, the second sub-image with the pixel size of 160×120 is input, and probabilities of n+1 categories are output.
Step three: and obtaining an image classification model through iterative training.
According to the operation of the first step, through multiple rounds of iterative training, an image classification network with high accuracy, namely an image classification model, can be obtained. And extracting depth features of a second sub-image with 160-120 pixels of the input image through the classification model, and outputting the category of the plant diseases and insect pests to which the second sub-image belongs, wherein 0 indicates that the sub-image belongs to the non-plant diseases and insect pests category.
In the embodiment of the application, the deep convolutional neural network is used as a basic network for image feature extraction, and the pre-training network is used for transfer learning, so that the image classification model with higher accuracy can be trained faster.
Step S403: inputting the image to be predicted into the image classification model, and judging the plant diseases and insect pests of the image to be predicted through the image classification model.
In this embodiment of the present application, the step of inputting the image to be predicted into the image classification model, and judging the type of the plant diseases and insect pests of the image to be predicted by using the image classification model includes:
step one: and carrying out image sequential interception on the image to be predicted to obtain a first sub-image.
Step two: the first sub-image is input to an image classification model.
Step three: and judging the plant diseases and insect pests of the image to be predicted through the image classification model.
After the first sub-image is input into an image classification model, obtaining a plant disease and insect pest type probability P corresponding to the first sub-image ij (i∈[0,63],j∈[0,N]) And corresponding the maximum probability value in the output probabilitiesCategory as predictive category L for the first sub-image i (i∈[0,63]) And the corresponding probability value is P i (i∈[0,63])。
If the prediction category L of any one of the first sub-images i (i∈[0,63]) And if the image to be predicted is 0, judging that the type of the image to be predicted is a non-disease and pest type. If the prediction category L of the sub-image i (i∈[0,63]) If the existing label is a class other than 0, the class corresponding to the maximum probability value of the probabilities in the class other than 0 is used as the class MAX (P i ) Wherein L is i ≠0。
In the embodiment of the application, in the stage of predicting the disease and pest categories of the image to be predicted, a method of searching the image in target detection is adopted to obtain local sub-images of the predicted image, the categories to which the local sub-images belong and the corresponding probabilities are identified and output through an image classification model, and finally the discrimination probabilities of all the sub-images are combined to obtain the disease and pest categories of the image to be predicted.
In the embodiment of the application, training data for image recognition is expanded through a target searching method, interference of non-pest background images is eliminated, and the proportion of pest image part data in the training data is improved; in the prediction stage, candidate images are obtained by the same method, and the recognition results of a plurality of candidate images are integrated, so that the plant disease and insect pest images can still be correctly recognized under the condition of small occupation of complex background and plant disease and insect pest images.
An embodiment of the present application provides an image recognition device, fig. 6 is a schematic structural diagram of the image recognition device according to the embodiment of the present application, as shown in fig. 6, the device 600 includes: a segmentation module 601, a first adjustment module 602, an identification module 603, and a first determination module 604, wherein:
the segmentation module 601 is configured to segment an image to be predicted, and generate a first sub-image set.
The first adjustment module 602 is configured to adjust, based on a picture content of a first sub-image, a number of the first sub-images in the first sub-image set, to obtain a second sub-image set.
The identifying module 603 is configured to identify a picture content of a second sub-image in the second sub-image set, so as to obtain an identification result of a category to which the second sub-image belongs.
The first determining module 604 is configured to take a category that satisfies a preset condition in the identification result as a target category of the image to be predicted.
In the above apparatus, the dividing module 601 includes:
the segmentation sub-module is further used for segmenting the image to be predicted into a plurality of first sub-images with the size of a preset pixel size according to a preset step length to obtain the first sub-image set.
In the above apparatus, the apparatus further includes:
the second determining module is used for determining that the category of the picture content in the first sub-image is a target area of a preset category; determining a union ratio of the area of the first sub-image and the area of the target area;
the second adjusting module is used for adjusting the number of the first sub-images in the first sub-image set based on the parallel-to-cross ratio to obtain a third sub-image set; wherein the number of third sub-images is smaller than the number of first sub-images;
and the third adjusting module is used for adjusting the number of the third sub-images in the third sub-image set based on the picture content of the third sub-image to obtain the second sub-image set.
In the above apparatus, the second adjusting module includes:
and the deleting sub-module is used for deleting the first sub-image corresponding to the sum-to-cross ratio larger than or equal to a first threshold value smaller than or equal to a second threshold value from the first sub-image set to obtain the third sub-image set.
In the above apparatus, the third adjusting module includes:
a first determining sub-module, configured to determine that the category of the third sub-image is a non-pest category if the intersection ratio is less than a first threshold;
a second determining sub-module, configured to determine, if the intersection ratio is greater than a second threshold, that the class of the third sub-image is a pest class;
a third determining sub-module, configured to determine a ratio between the number of sub-images belonging to the non-pest category and the pest category in the third sub-image set;
and the adjusting sub-module is also used for adjusting the number of the third sub-images based on a preset proportion and the ratio so as to obtain the second sub-image set.
In the above apparatus, the adjusting submodule includes:
the adjusting unit is used for reducing the number of the sub-images belonging to the non-pest categories in the third sub-image set to obtain the second sub-image set if the ratio is larger than or equal to the preset ratio; the ratio of the number of the sub-images belonging to the non-disease and pest type to the number of the sub-images belonging to the disease and pest type in the second sub-image set is smaller than the preset ratio.
In the above apparatus, the identifying module 603 includes:
and the second determining submodule is used for determining that the category to which the picture content of the second sub-image belongs is a probability value of a preset category so as to obtain the identification result.
In the above apparatus, the second determining submodule includes:
and the determining unit is used for determining the category to which the picture content of the second sub-image corresponding to the maximum probability value belongs as the target category in the identification result.
The embodiment of the application further provides an image recognition device, which comprises all the included modules, all the sub-modules and all the units included by all the modules, and can be realized by a processor in the terminal; of course, the method can also be realized by a specific logic circuit; in an implementation, the processor may be a Central Processing Unit (CPU), a Microprocessor (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like.
Correspondingly, the embodiment of the present application provides a terminal, fig. 7 is a schematic diagram of a composition structure of the terminal according to the embodiment of the present application, as shown in fig. 7, and the terminal 700 at least includes: a controller 701 and a storage medium 702 configured to store executable instructions, wherein:
the controller 701 is configured to execute stored executable instructions for implementing the provided image recognition method.
It should be noted that the description of the terminal embodiment above is similar to the description of the method embodiment above, and has similar advantageous effects as the method embodiment. For technical details not disclosed in the terminal embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.
Correspondingly, the embodiment of the application provides a computer storage medium, wherein computer executable instructions are stored in the computer storage medium, and the computer executable instructions are configured to execute the image recognition method provided by other embodiments of the application.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing embodiment numbers of the present application are merely for describing, and do not represent advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, etc.) to perform the method described in the various embodiments of the present application.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, terminals (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the claims, and all equivalent structures or equivalent processes using the descriptions and drawings of the present application, or direct or indirect application in other related technical fields are included in the scope of the claims of the present application.

Claims (11)

1. An image recognition method, wherein the method is applied to recognition of pest categories of crops, and the method comprises the following steps:
dividing an image to be predicted to generate a first sub-image set; the first sub-image set comprises a plurality of first sub-images;
classifying the first sub-images according to the picture content of the first sub-images, and adjusting the number of the first sub-images in the first sub-image set according to the ratio of the number of the sub-images in the non-disease and pest categories to the number of the sub-images in the disease and pest categories in the first sub-image set to obtain a second sub-image set;
identifying the picture content of a second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs;
and taking the category meeting the preset condition in the identification result as the target category of the image to be predicted.
2. The method of claim 1, wherein the segmenting the image to be predicted to generate the first set of sub-images comprises:
dividing an image to be predicted into a plurality of first sub-images with preset pixel size according to a preset step length to obtain the first sub-image set.
3. The method of claim 2, wherein after segmenting the image to be predicted, generating the first set of sub-images, the method further comprises:
determining the category of the picture content in the first sub-image as a target area of a preset category;
determining a union ratio of the area of the first sub-image and the area of the target area;
based on the parallel-to-cross ratio, adjusting the number of first sub-images in the first sub-image set to obtain a third sub-image set; wherein the number of third sub-images is smaller than the number of first sub-images;
correspondingly, based on the picture content of the third sub-image, the third sub-image quantity in the third sub-image set is adjusted to obtain the second sub-image set.
4. A method according to claim 3, wherein said adjusting the number of first sub-images in the first sub-image set based on said parallel-to-cross ratio to obtain a third sub-image set comprises:
and deleting the first sub-image corresponding to the sum-to-cross ratio larger than and equal to a first threshold value smaller than and equal to a second threshold value from the first sub-image set to obtain the third sub-image set.
5. A method according to claim 3, wherein before said adjusting the number of said third sub-images based on the picture content of said third sub-images, the method further comprises:
if the intersection ratio is smaller than a first threshold value, determining that the category of the third sub-image is a non-plant disease and insect pest category;
if the sum-to-intersection ratio is larger than a second threshold value, determining the category of the third sub-image as a plant disease and insect pest category;
correspondingly, determining a ratio between the number of sub-images belonging to the non-pest category and the pest category in the third sub-image set;
and adjusting the number of the third sub-images based on a preset proportion and the ratio to obtain the second sub-image set.
6. The method of claim 5, wherein adjusting the number of third sub-images based on a preset ratio and the ratio to obtain the second set of sub-images comprises:
if the ratio is greater than or equal to the preset ratio, reducing the number of the sub-images belonging to the non-pest categories in the third sub-image set to obtain the second sub-image set; the ratio of the number of the sub-images belonging to the non-disease and pest type to the number of the sub-images belonging to the disease and pest type in the second sub-image set is smaller than the preset ratio.
7. The method according to claim 1, wherein the identifying the picture content of the second sub-image in the second sub-image set to obtain the identification result of the category to which the second sub-image belongs includes:
and determining the probability value of the category to which the picture content of the second sub-image belongs as a preset category so as to obtain the identification result.
8. The method according to claim 7, wherein the step of using the category satisfying the preset condition in the recognition result as the target category of the image to be predicted includes:
and in the identification result, determining the category to which the picture content of the second sub-image corresponding to the maximum probability value belongs as the target category.
9. An image recognition device, wherein the device is applied to recognize pest categories of crops, the device comprising:
the segmentation module is used for segmenting the image to be predicted to generate a first sub-image set; the first sub-image set comprises a plurality of first sub-images;
the adjusting module is used for classifying the first sub-images according to the picture content of the first sub-images, and adjusting the number of the first sub-images in the first sub-image set according to the ratio of the number of the sub-images in the non-disease and pest categories to the number of the sub-images in the disease and pest categories in the first sub-image set to obtain a second sub-image set;
the identification module is used for identifying the picture content of the second sub-image in the second sub-image set to obtain an identification result of the category to which the second sub-image belongs;
and the determining module is used for taking the category meeting the preset condition in the identification result as the target category of the image to be predicted.
10. A terminal, the terminal comprising at least: a controller and a storage medium configured to store executable instructions, wherein:
the controller is configured to execute stored executable instructions configured to perform the image recognition method provided in any one of the preceding claims 1 to 8.
11. A computer-readable storage medium having stored therein computer-executable instructions configured to perform the image recognition method provided in any one of the preceding claims 1 to 8.
CN201911048119.5A 2019-10-30 2019-10-30 Image recognition method, device, terminal and storage medium Active CN112749702B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911048119.5A CN112749702B (en) 2019-10-30 2019-10-30 Image recognition method, device, terminal and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911048119.5A CN112749702B (en) 2019-10-30 2019-10-30 Image recognition method, device, terminal and storage medium

Publications (2)

Publication Number Publication Date
CN112749702A CN112749702A (en) 2021-05-04
CN112749702B true CN112749702B (en) 2023-04-28

Family

ID=75640951

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911048119.5A Active CN112749702B (en) 2019-10-30 2019-10-30 Image recognition method, device, terminal and storage medium

Country Status (1)

Country Link
CN (1) CN112749702B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114663418A (en) * 2022-04-06 2022-06-24 京东安联财产保险有限公司 Image processing method and device, storage medium and electronic equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107292229A (en) * 2017-05-08 2017-10-24 北京三快在线科技有限公司 A kind of image-recognizing method and device
WO2018112783A1 (en) * 2016-12-21 2018-06-28 深圳前海达闼云端智能科技有限公司 Image recognition method and device
CN109241985A (en) * 2017-07-11 2019-01-18 普天信息技术有限公司 A kind of image-recognizing method and device
CN110298405A (en) * 2019-07-03 2019-10-01 北京字节跳动网络技术有限公司 Classification recognition methods and device, storage medium and terminal

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018112783A1 (en) * 2016-12-21 2018-06-28 深圳前海达闼云端智能科技有限公司 Image recognition method and device
CN107292229A (en) * 2017-05-08 2017-10-24 北京三快在线科技有限公司 A kind of image-recognizing method and device
CN109241985A (en) * 2017-07-11 2019-01-18 普天信息技术有限公司 A kind of image-recognizing method and device
CN110298405A (en) * 2019-07-03 2019-10-01 北京字节跳动网络技术有限公司 Classification recognition methods and device, storage medium and terminal

Also Published As

Publication number Publication date
CN112749702A (en) 2021-05-04

Similar Documents

Publication Publication Date Title
CN112052787B (en) Target detection method and device based on artificial intelligence and electronic equipment
US11348249B2 (en) Training method for image semantic segmentation model and server
CN111507958B (en) Target detection method, training method of detection model and electronic equipment
CN106547744B (en) Image retrieval method and system
CN108470172B (en) Text information identification method and device
CN110569721A (en) Recognition model training method, image recognition method, device, equipment and medium
CN110378297B (en) Remote sensing image target detection method and device based on deep learning and storage medium
WO2020232909A1 (en) Pedestrian visual tracking method, model training method and device, apparatus and storage medium
CN111738357B (en) Junk picture identification method, device and equipment
CN111368636B (en) Object classification method, device, computer equipment and storage medium
CN112508975A (en) Image identification method, device, equipment and storage medium
CN112418278A (en) Multi-class object detection method, terminal device and storage medium
KR101802500B1 (en) Learning device for improving image recogntion performance and learning method thereof
CN114187311A (en) Image semantic segmentation method, device, equipment and storage medium
CN114549913B (en) Semantic segmentation method and device, computer equipment and storage medium
JP2020170495A (en) Single pixel attack sample generating method, device, facility, and storage medium
CN112101359B (en) Text formula positioning method, model training method and related device
CN111178364A (en) Image identification method and device
CN111340213B (en) Neural network training method, electronic device, and storage medium
CN115082667A (en) Image processing method, device, equipment and storage medium
CN116543261A (en) Model training method for image recognition, image recognition method device and medium
CN115731422A (en) Training method, classification method and device of multi-label classification model
CN113743443B (en) Image evidence classification and recognition method and device
CN112749702B (en) Image recognition method, device, terminal and storage medium
CN113221929A (en) Image processing method and related equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant