CN112307983A - Method and system for enhancing plant colors in image - Google Patents
Method and system for enhancing plant colors in image Download PDFInfo
- Publication number
- CN112307983A CN112307983A CN202011202232.7A CN202011202232A CN112307983A CN 112307983 A CN112307983 A CN 112307983A CN 202011202232 A CN202011202232 A CN 202011202232A CN 112307983 A CN112307983 A CN 112307983A
- Authority
- CN
- China
- Prior art keywords
- plant
- original image
- color
- plants
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000002708 enhancing effect Effects 0.000 title claims abstract description 26
- 239000003086 colorant Substances 0.000 title abstract description 19
- 241000196324 Embryophyta Species 0.000 claims description 314
- 238000012549 training Methods 0.000 claims description 13
- 238000001514 detection method Methods 0.000 claims description 11
- 238000013507 mapping Methods 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 6
- 238000012790 confirmation Methods 0.000 claims description 5
- 230000001629 suppression Effects 0.000 claims description 5
- 241001464837 Viridiplantae Species 0.000 abstract description 42
- 238000013403 standard screening design Methods 0.000 description 42
- 230000006870 function Effects 0.000 description 9
- 241000124033 Salix Species 0.000 description 8
- 229960001506 brilliant green Drugs 0.000 description 6
- HXCILVUBKWANLN-UHFFFAOYSA-N brilliant green cation Chemical compound C1=CC(N(CC)CC)=CC=C1C(C=1C=CC=CC=1)=C1C=CC(=[N+](CC)CC)C=C1 HXCILVUBKWANLN-UHFFFAOYSA-N 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 230000014509 gene expression Effects 0.000 description 6
- 235000017166 Bambusa arundinacea Nutrition 0.000 description 4
- 235000017491 Bambusa tulda Nutrition 0.000 description 4
- 244000082204 Phyllostachys viridis Species 0.000 description 4
- 235000015334 Phyllostachys viridis Nutrition 0.000 description 4
- 241000018650 Pinus massoniana Species 0.000 description 4
- 235000011610 Pinus tabuliformis Nutrition 0.000 description 4
- 240000003243 Thuja occidentalis Species 0.000 description 4
- 235000008109 Thuja occidentalis Nutrition 0.000 description 4
- 239000011425 bamboo Substances 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- DSSYKIVIOFKYAU-XCBNKYQSSA-N (R)-camphor Chemical compound C1C[C@@]2(C)C(=O)C[C@@H]1C2(C)C DSSYKIVIOFKYAU-XCBNKYQSSA-N 0.000 description 2
- 240000004144 Acer rubrum Species 0.000 description 2
- 235000011772 Acer rubrum var tomentosum Nutrition 0.000 description 2
- 235000009057 Acer rubrum var tridens Nutrition 0.000 description 2
- 244000025254 Cannabis sativa Species 0.000 description 2
- 235000009024 Ceanothus sanguineus Nutrition 0.000 description 2
- 241000218645 Cedrus Species 0.000 description 2
- 241000755729 Clivia Species 0.000 description 2
- 240000005636 Dryobalanops aromatica Species 0.000 description 2
- 244000252337 Epipremnum pinnatum Species 0.000 description 2
- 240000003553 Leptospermum scoparium Species 0.000 description 2
- 241000830535 Ligustrum lucidum Species 0.000 description 2
- 235000015459 Lycium barbarum Nutrition 0.000 description 2
- 235000005087 Malus prunifolia Nutrition 0.000 description 2
- 244000070406 Malus silvestris Species 0.000 description 2
- 241000159443 Myrcia Species 0.000 description 2
- 241001092500 Photinia x fraseri Species 0.000 description 2
- 241000218657 Picea Species 0.000 description 2
- 241001312238 Scindapsus Species 0.000 description 2
- 244000046101 Sophora japonica Species 0.000 description 2
- 235000010586 Sophora japonica Nutrition 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000000442 meristematic effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 241000894007 species Species 0.000 description 2
- 238000011282 treatment Methods 0.000 description 2
- 241000579895 Chlorostilbon Species 0.000 description 1
- 241001465754 Metazoa Species 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 229910052876 emerald Inorganic materials 0.000 description 1
- 239000010976 emerald Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention provides a method and a system for enhancing plant colors in an image, wherein the method comprises the following steps: acquiring an original image of a color to be enhanced, and detecting and identifying the type of plants in the original image by using an SSD model; further judging the color of the plant according to the value of G/R, G/B in RGB three channels in the plant area detected in the original image; judging whether the plant is the main body of the original image; if the plant is the subject of the original image; and according to the color characteristics of the plants, performing differentiated color enhancement on the plants in the original image. The invention solves the problems that the color of the plant in the image is not bright enough, and the like, and is particularly suitable for enhancing the green of the green plant in the image.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a method and a system for enhancing plant colors in an image.
Background
With the continuous improvement and development of digital camera technology, people have higher and higher requirements on image effect, and plants such as leaves or large-area grasslands and the like which are shot are expected to be more bright and popular; however, due to the inherent hardware limitation of the CMOS image sensor and the defect of the ISP algorithm, plants in the image are yellowish and not bright enough.
In the experience and demand of users, the green color of the enhancement plants accounts for the most part of the demand of color enhancement, and at present, most of the existing green enhancement algorithms judge the green color according to the difference between the channel G and the channel R, B in the scene, and then perform green enhancement, or rotate or map the pixel points in the yellow-green area in the YCbCr space to make the yellow-green area more emerald. None of the methods considers whether the plants shot by the user are green, whether the green plants are the shot subjects, and different degrees of green enhancement according to the types of the green plants, so that the higher requirements of people on the green enhancement cannot be met.
Disclosure of Invention
In view of the above-mentioned deficiencies of the prior art, the present invention provides a method and system for enhancing plant color in an image. Aims to solve the problems that the plant color in the image is yellow and not bright enough, the differential color enhancement can not be carried out, and the like.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a method for enhancing plant colors in an image, which comprises the following steps:
acquiring an original image of a color to be enhanced, and detecting and identifying the type of plants in the original image by using an SSD model;
further judging the color of the plant according to the value of G/R, G/B in RGB three channels in the plant area detected in the original image; judging whether the plant is the main body of the original image;
if the plant is the subject of the original image;
and according to the color characteristics of the plants, performing differentiated color enhancement on the plants in the original image.
In a preferred embodiment provided by the first aspect of the present invention, before the detecting and the species identifying of the plant in the original image by using the SSD model, the training method further includes:
collecting a plurality of sample images containing plants, and classifying according to the leaf shapes of the plants;
inputting the sample image into an SSD model, setting prior frames with different sizes, matching plant categories with different shapes, and generating a corresponding feature mapping chart;
the feature mapping graph is converted through a prediction convolution layer to obtain a prediction frame for distinguishing plant categories and corresponding category confidence coefficients;
and updating the SSD model parameters through the classification loss function and the classification loss function to obtain the optimal SSD model.
In a preferred embodiment of the first aspect of the present invention, the detecting and identifying the plant species in the original image comprises
Inputting an original image into a trained SSD model, performing descending order arrangement on the bounding boxes through a class confidence coefficient, and calculating the intersection and parallel ratio of the bounding boxes and a prior box if the confidence coefficient is higher than a preset value;
and obtaining the optimal prediction frame and the category of each category by using a non-maximum suppression method, and outputting an identification result.
In a preferred embodiment of the first aspect of the present invention, the determining the color of the plant includes obtaining an edge of a leaf of the plant through an SSD model, drawing a boundary region of the plant, and counting G/R of pixels of the plant in the boundary regionaveAnd G/BaveAnd judging the real color of the plant.
In a preferred embodiment of the first aspect of the present invention, the determining whether the plant is the main body of the original image is determined according to a position of the plant in the original image or/and a pixel ratio of the plant in the original image.
In a preferred embodiment provided by the first aspect of the present invention, determining whether the plant is a subject of the original image according to the position of the plant in the original image includes:
setting a point in the original image as the image center coordinate C (x, y), and setting a point in the plant area as the plant center coordinateLabel G (x)1，y1) The original image has a length and width of W, H, and the distance between the center point of the plant and the center point of the image is
If the following conditions are met, judging that the plant is the main body of the original image;
in a preferred embodiment provided by the first aspect of the present invention, determining whether a plant is a main body of an original image according to a pixel proportion of the plant in the original image includes:
calculating the number n of pixel points of the original image, and calculating the number n of pixel points of the plant region in the original image1，
If n is1And if the value is more than or equal to 0.4 x n, judging that the plant is the main body of the original image.
In a preferred embodiment provided by the first aspect of the present invention, the performing differentiated color enhancement on the plant in the original image according to the color feature of the plant includes:
obtaining input pixel value (R) of plant in original imagein，Gin，Bin) The output value after the differential color enhancement is (R)out，Gout，Bout) Then, the process of enhancing color is:
Rout＝k1*Rin+b1
Gout＝k2*Gin+b2
Bout＝k3*Bin+b3
wherein k is1，k2，k3Representing the intensity coefficients of the RGB three channels, b1，b2，b3Representing the offset of the RGB three channels.
The second aspect of the present invention provides a system for enhancing colors of plants in an image, the system comprising: an acquisition module, an SSD model, a color confirmation module, a main body judgment module and a color enhancement module, wherein,
the acquisition module is used for acquiring an original image of the color to be enhanced;
the SSD model is used for detecting and identifying the types of plants in the original image;
the color confirmation module is used for further judging the color of the plant according to the value of G/R, G/B in RGB three channels in the plant area detected in the original image;
the main body judging module is used for judging whether the plant is a main body of the original image;
if the plant is the main body of the original image, the plant in the original image is subjected to differential color enhancement through the color enhancement module according to the color characteristics of the plant.
In a preferred aspect provided by the second aspect of the present invention, before identifying the plant in the original image using the SSD model, training the SSD model is further included; training the SSD model comprises:
collecting a plurality of sample images containing plants, and classifying according to the leaf shapes of the plants;
inputting the sample image into an SSD model, setting prior frames with different sizes, matching plant categories with different shapes, and generating a corresponding feature mapping chart;
the feature mapping graph is converted through a prediction convolution layer to obtain a prediction frame for distinguishing plant categories and corresponding category confidence coefficients;
and updating the SSD model parameters through the classification loss function and the classification loss function to obtain the optimal SSD model.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a method and a system for enhancing plant colors in an image, wherein the method comprises the following steps: acquiring an original image of a color to be enhanced, and detecting and identifying the type of plants in the original image by using an SSD model; further judging the color of the plant according to the value of G/R, G/B in RGB three channels in the plant area detected in the original image; judging whether the plant is the main body of the original image; if the plant is the subject of the original image; and according to the color characteristics of the plants, performing differentiated color enhancement on the plants in the original image. The invention solves the problems that the color of the plant in the image is not bright enough, and the like, and is particularly suitable for enhancing the green of the green plant in the image.
Drawings
FIG. 1 is a schematic block diagram of a process for enhancing plant colors in an image according to the present invention.
FIG. 2 is a schematic block diagram of a process for training an SSD model in accordance with the present invention.
FIG. 3 is a block diagram of a system for enhancing plant colors in an image according to the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
In order to make the objects, technical solutions and effects of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It should be noted that the scheme of the present invention is applicable to terminals with shooting functions, such as digital cameras, smart phones, tablet computers, PC devices, and the like; the color of the picture to be shot and the color of the shot picture can be enhanced by utilizing the scheme; in addition, since various demands are often made on the image capturing of the green plant in view of receiving the information fed back by the user, the following embodiments will describe the color enhancement in detail by taking the image to be captured and the green enhancement of the plant as an example.
It can be understood that the scheme is not only for enhancing the green color of the plant, but also for enhancing other colors such as red, yellow and the like of the plant; of course, it is understood that the present solution is applicable to color enhancement of other subjects such as animals and clothes, as well as color enhancement of plants.
Referring to fig. 1, a first aspect of the present invention provides a method for enhancing plant color in an image, the method comprising the steps of:
s10, acquiring an original image of the color to be enhanced, and detecting and identifying the type of the plant in the original image by using the SSD model;
s20, further judging the color of the plant according to the value of G/R, G/B in RGB three channels in the plant area detected in the original image; judging whether the plant is the main body of the original image;
s30, if the plant is the main body of the original image;
and S40, performing differentiated color enhancement on the plants in the original image according to the color characteristics of the plants.
Among them, the ssd (single Shot multi box detector) model is a single-stage object detector. Different from a two-stage detection method, the single-stage target detection does not perform region recommendation, but directly regresses a boundary box and classification probability of a target from a feature map; SSDs exploit this single-phase detection idea and improve it: and detecting the target with the corresponding scale on the feature maps with different scales.
And detecting and classifying the green plants by using an SSD model detection algorithm, and judging whether the class of the green plants in the shot picture is willow-leaf-shaped plants, needle-leaf-shaped plants, round or elliptical-tip-shaped plants, wide-egg-shaped plants or narrow-egg-shaped plants according to the outline shape of the plant leaves.
After the plant species is judged according to the plant leaf contour shape, the color of the plant in the original image is further judged according to the value of G/R, G/B in RGB three channels in the plant region detected in the original image.
Judging whether the plant is a main shooting object in shooting according to the area ratio of the plant in the original image or/and the position relation of the plant in the original image; if so,
performing differential color enhancement according to leaf color characteristics of different leaf-shaped plants, such as willow leaf-shaped plants (such as grassland, bamboo, etc.), namely enhancing the leaf color of the plants to be yellow green; for example, coniform plants (such as Chinese pine, arborvitae, etc.) should have blue-green leaves, while oval and oval leaves (such as glossy privet, scindapsus, etc.) should have brilliant green leaves.
In conclusion, the method solves the problems that the colors of plants in the image are not bright enough and the like, and is particularly suitable for enhancing the green color of green plants in the image.
Referring to fig. 2, in a preferred embodiment provided by the first aspect of the present invention, before the detecting and the species identifying of the plant in the original image by using the SSD model, the training method further includes the following steps:
s11, collecting a plurality of sample images containing plants, and classifying according to the leaf shapes of the plants;
s12, inputting the sample image into the SSD model, setting prior frames with different sizes, matching plant categories with different shapes, and generating corresponding feature maps;
s13, converting the feature mapping graph through a prediction convolutional layer to obtain a prediction frame for distinguishing plant categories and corresponding category confidence;
and S14, updating the SSD model parameters through the classification loss function and the classification loss function to obtain the optimal SSD model.
The preparation work of the data sample collects a plurality of sample images containing common plants, and it can be understood that the larger the collected base number is, the better the plant types are; the plants in these sample images are then labeled and can be classified into four categories: willow leaf shaped plant (such as grass, bamboo leaf, Clivia, etc.); coniform plants (such as Chinese pine, spruce, cedar, arborvitae, etc.); round (including oval and elliptical tip) leaf plants (such as fructus Ligustri Lucidi, fructus Gardeniae, Sophora japonica, and Cinnamomum camphora); oval (including wide and narrow oval) leafy plants (e.g., scindapsus aureus, myrcia, crabapple, tea tree, etc.);
therefore, the type of the plant in the sample image can be preliminarily judged, and the conventional color of the plant can be judged according to the type of the plant; of course, most plants have a meristematic and wilting stage, and some plants have green leaves in the growing stage, but in the wilting stage, the leaves of these plants may have a yellow color, and therefore, further color measurement of the plants in the original image is required.
In a preferred embodiment of the first aspect of the present invention, the detecting and identifying the plant species in the original image comprises
Inputting an original image into a trained SSD model, performing descending order arrangement on the bounding boxes through a class confidence coefficient, and calculating the intersection and parallel ratio of the bounding boxes and a prior box if the confidence coefficient is higher than a preset value;
and obtaining the optimal prediction frame and the category of each category by using a non-maximum suppression method, and outputting an identification result.
Inputting a plant picture (original image) to be recognized into a trained SSD model, sequencing the bounding boxes in a descending order according to class confidence, calculating the intersection and comparison between the bounding boxes and a prior frame, obtaining the optimal prediction frame and the class of each class by using a non-maximum inhibition method, wherein the percentage of the bounding boxes is higher than 80%; namely four categories of plants: willow leaf shaped plants; needle-leaved plants; oval or oval tip plants; wide-egg or narrow-egg plants.
In a preferred embodiment of the first aspect of the present invention, the determining the color of the plant includes obtaining an edge of a leaf of the plant through an SSD model, drawing a boundary region of the plant, and counting G/R of pixels of the plant in the boundary regionaveAnd G/BaveAnd judging the real color of the plant.
It can be understood that, because the intelligent recognition algorithm has certain misjudgment, and the growth period or the withering period of the plants, in order to further improve the accuracy of green plant recognition, when the plants with various types of plants and other colors, such as red maple leaves or photinia fraseri, are excluded from the image, it is necessary to further judge the real color expression of the green plants or other non-green plants, and further determine whether the plants are the green plants of the category according to the value of G/R, G/B in the RGB three channels in the detection area of the green plants in the image. The edge of green leaf is obtained through the SSD model of the above-mentioned training that advances, draws out the boundary region of the green plant that detects, and statistics green plant pixel's G/Rave and G/Bave in this boundary region need satisfy following condition:
thrg/r and Thrg/b are not suitable to be set too large, otherwise green plants are wrongly judged as non-green plants. Because the green expressions of the leaf colors of different kinds of green plants are different, the real green expressions of different plants can be enhanced; the specific implementation method comprises the following steps: the different kinds of green plants in the image have different green performances, and the RGB three-channel proportion of the corresponding areas is different, for example, the leaf color of the willow-leaf-shaped plant is yellow green, the g/r of the willow-leaf-shaped plant is relatively large, the leaf color of the needle-leaf-shaped plant is blue green, and the g/b of the needle-leaf-shaped plant is relatively small. Therefore, to perform different treatments on different kinds of green plants, different threshold criteria need to be set according to the real green performance of the different kinds of green plants:
such as
Willow leaf shaped plants: 1.2 Thrg/r and 1.3 Thrg/b;
needle leaf plants: 1.3 Thrg/r and 1.2 Thrg/b;
oval or round plants: 1.3 Thrg/r and 1.3 Thrg/b;
wide-egg or narrow-egg plants: thrg/r ═ 1.3 and Thrg/b ═ 1.3.
It should be noted that, the green enhancement in the present text is different from the green enhancement in the conventional sense, the green enhancement in the conventional sense is to enhance the green by adjusting the value of R, G, B three channels, and since the green enhancement is the effect of the whole image, which is global, it will bring certain side effects, in order to weaken the influence of this part, the present invention provides an algorithm for determining whether a plant is the subject of this image, and it is determined whether the plant is the subject of the image mainly by two conditions, and it can be determined that the plant is the subject of the image by satisfying any one of the following conditions:
firstly, calculating the position of the plant in the image;
secondly, calculating the area ratio of the plant in the image;
first, assume that the center coordinates of the image are C (x, y) and the center coordinates of the plant are G (x)1，y1) The length and width of the image are W, H respectively, and the distance between the center point of the plant and the center point of the image is
If the following conditions are met, judging that the plant is the main body of the original image;
secondly, when the central coordinate of the plant and the central coordinate of the image do not meet the condition, but the pixel point ratio of the plant is high, the plant can be judged to be the main body of the original image; because the plant information in the image is more, the image is still attracted by a great number of plants when being observed, and therefore, the color enhancement can be performed on the scenes;
suppose that the number of pixel points in the plant area is n1The number of the integral pixel points of the image is n, if n1Judging that the plant is the main body of the original image if the plant is more than or equal to 0.4 x n;
generally, the plant area occupies more than forty percent of the whole image area, but may also be thirty percent or twenty percent, and the proportion value can be adjusted according to the preference of the user.
It will be appreciated that the above scheme has determined the plants in the original image, as well as the plant species, the direction of color enhancement and whether the plant is the subject of the image, and that in a particular color enhancement, assuming that a green enhancement is required for the plants in the image, the image input pixel value is (R) inin，Gin，Bin) The output value after the differential color enhancement is (R)out，Gout，Bout) Then the green enhancement process can be implemented by the following formula:
Rout＝k1*Rin+b1
Gout＝k2*Gin+b2
Bout＝k3*Bin+b3
wherein k is1，k2，k3Representing the intensity coefficients of the RGB three channels, b1，b2，b3Representing the offset of the RGB three channels.
Empirically, the direction of enhancement is:
for example, willow-shaped plants are enhanced towards the yellow-green color:
k1＝1.1；k2＝1.1；k3＝1.0；b1＝5；b2＝5；b3＝0；
coniform plants are enhanced towards bluish green:
k1＝1.0；k2＝1.1；k3＝1.1；b1＝0；b2＝5；b3＝5；
the elliptical or round plants are enhanced towards brilliant green:
k1＝1.1；k2＝1.1；k3＝1.1；b1＝5；b2＝5；b3＝5；
wide-ovular or narrow-ovular plants are enhanced towards brilliant green:
k1＝1.1；k2＝1.1；k3＝1.1；b1＝5；b2＝5；b3＝5；
of course, the user can perform color enhancement for different kinds of green plants according to the preference of the user.
In summary, the method of the present invention identifies the plants in the image, further determines the plant colors, and determines whether the plants are the main body of the image, and finally performs the differentiated color enhancement on the plants in the image according to the above points. The invention solves the problems that the color of the plant in the image is not bright enough, and the like, and is particularly suitable for enhancing the green of the green plant in the image.
Referring to fig. 3, a second aspect of the present invention provides a system for enhancing colors of plants in an image, comprising: acquisition module 100, SSD model 101, color validation module 102, body judgment module 103, and color enhancement module 104, wherein,
the obtaining module 100 is configured to obtain an original image of a color to be enhanced;
the SSD model 101 is used for detecting and identifying the types of plants in an original image;
the color confirmation module 102 is used for further judging the color of the plant according to the value of G/R, G/B in RGB three channels in the plant area detected in the original image;
the main body judging module 103 is configured to judge whether the plant is a main body of the original image;
if the plant is the main body of the original image, the plant in the original image is subjected to differential color enhancement by the color enhancement module 104 according to the color feature of the plant.
Among them, the ssd (single Shot multi box detector) model is a single-stage object detector. Different from a two-stage detection method, the single-stage target detection does not perform region recommendation, but directly regresses a boundary box and classification probability of a target from a feature map; SSDs exploit this single-phase detection idea and improve it: and detecting the target with the corresponding scale on the feature maps with different scales.
And detecting and classifying the green plants by using an SSD model detection algorithm, and judging whether the class of the green plants in the shot picture is willow-leaf-shaped plants, needle-leaf-shaped plants, round or elliptical-tip green plants, wide-egg-shaped plants or narrow-egg-shaped plants according to the outline shape of the plant leaves.
After the plant species is judged according to the plant leaf contour shape, the color of the plant in the original image is further judged according to the value of G/R, G/B in RGB three channels in the plant region detected in the original image.
Judging whether the plant is a main shooting object in shooting according to the area ratio of the plant in the original image or/and the position relation of the plant in the original image; if so,
performing differential color enhancement according to leaf color characteristics of different leaf-shaped plants, such as willow leaf-shaped plants (such as grassland, bamboo, etc.), namely enhancing the leaf color of the plants to be yellow green; for example, coniform plants (such as Chinese pine, arborvitae, etc.) should have blue-green leaves, while oval and oval leaves (such as glossy privet, scindapsus, etc.) should have brilliant green leaves.
In conclusion, the method solves the problems that the colors of plants in the image are not bright enough and the like, and is particularly suitable for enhancing the green color of green plants in the image.
In a preferred aspect provided by the second aspect of the present invention, before identifying the plant in the original image using the SSD model, training the SSD model is further included; training the SSD model comprises:
collecting a plurality of sample images containing plants, and classifying according to the leaf shapes of the plants;
inputting the sample image into an SSD model, setting prior frames with different sizes, matching plant categories with different shapes, and generating a corresponding feature mapping chart;
the feature mapping graph is converted through a prediction convolution layer to obtain a prediction frame for distinguishing plant categories and corresponding category confidence coefficients;
and updating the SSD model parameters through the classification loss function and the classification loss function to obtain the optimal SSD model.
The preparation work of the data sample collects a plurality of sample images containing common plants, and it can be understood that the larger the collected base number is, the better the plant types are; the plants in these sample images are then labeled and can be classified into four categories: willow leaf shaped plant (such as grass, bamboo leaf, Clivia, etc.); coniform plants (such as Chinese pine, spruce, cedar, arborvitae, etc.); round (including oval and elliptical tip) leaf plants (such as fructus Ligustri Lucidi, fructus Gardeniae, Sophora japonica, and Cinnamomum camphora); oval (including wide and narrow oval) leafy plants (e.g., scindapsus aureus, myrcia, crabapple, tea tree, etc.);
therefore, the type of the plant in the sample image can be preliminarily judged, and the conventional color of the plant can be judged according to the type of the plant; of course, most plants have a meristematic and wilting stage, and some plants have green leaves in the growing stage, but in the wilting stage, the leaves of these plants may have a yellow color, and therefore, further color measurement of the plants in the original image is required.
In a preferred embodiment of the second aspect of the present invention, the detecting and identifying the plant species in the original image comprises:
inputting an original image into a trained SSD model, performing descending order arrangement on the bounding boxes through a class confidence coefficient, and calculating the intersection and parallel ratio of the bounding boxes and a prior box if the confidence coefficient is higher than a preset value;
and obtaining the optimal prediction frame and the category of each category by using a non-maximum suppression method, and outputting an identification result.
Inputting plant pictures to be identified into a trained SSD model, carrying out descending order arrangement on the bounding boxes through class confidence, calculating the intersection ratio of the bounding boxes and the prior boxes, obtaining the optimal prediction box of each classification and the class to which the optimal prediction box belongs by using a non-maximum suppression method, and outputting an identification result, wherein the intersection ratio of the bounding boxes is higher than 80%; namely four categories of plants: willow leaf shaped plants; needle-leaved plants; oval or oval tip plants; wide-egg or narrow-egg plants.
Judging the color of the plant comprises the steps of obtaining the edge of the plant leaf through an SSD model, describing the boundary area of the plant, and counting the G/R of the plant pixel points in the boundary areaaveAnd G/BaveAnd judging the real color of the plant.
It can be understood that, because the intelligent identification algorithm has certain misjudgment, and the growth period or the withering period of the plants, in order to further improve the accuracy of green plant identification, when the plants with various types and special colors, such as red maple leaves or photinia fraseri, exist in the image are excluded, it is necessary to further judge the real color expression of the green plants or other non-green plants, and whether the plants are the green plants of the category according to the value of G/R, G/B in the RGB three channels in the detection area of the green plants in the picture. The edge of green leaf is obtained through the SSD model of the above-mentioned training that advances, draws out the boundary region of the green plant that detects, and statistics green plant pixel's G/Rave and G/Bave in this boundary region need satisfy following condition:
thrg/r and Thrg/b are not suitable to be set too large, otherwise green plants are wrongly judged as non-green plants. Because the green expressions of the leaf colors of different kinds of green plants are different, the real green expressions of different plants can be enhanced; the specific implementation method comprises the following steps: the different kinds of green plants in the image have different green performances, and the RGB three-channel proportion of the corresponding areas is different, for example, the leaf color of the willow-leaf-shaped plant is yellow green, the g/r of the willow-leaf-shaped plant is relatively large, the leaf color of the needle-leaf-shaped plant is blue green, and the g/b of the needle-leaf-shaped plant is relatively small. Therefore, to perform different treatments on different kinds of green plants, different threshold criteria need to be set according to the real green performance of the different kinds of green plants:
such as
Willow leaf shaped plants: 1.2 Thrg/r and 1.3 Thrg/b;
needle leaf plants: 1.3 Thrg/r and 1.2 Thrg/b;
oval or round plants: 1.3 Thrg/r and 1.3 Thrg/b;
wide-egg or narrow-egg plants: thrg/r ═ 1.3 and Thrg/b ═ 1.3.
It should be noted that, the green enhancement in the present text is different from the green enhancement in the conventional sense, the green enhancement in the conventional sense is to enhance the green by adjusting the value of R, G, B three channels, and since the green enhancement is the effect of the whole image, which is global, it will bring certain side effects, in order to weaken the influence of this part, the present invention provides an algorithm for determining whether a plant is the subject of this image, and it is determined whether the plant is the subject of the image mainly by two conditions, and it can be determined that the plant is the subject of the image by satisfying any one of the following conditions:
firstly, calculating the position of the plant in the image;
secondly, calculating the area ratio of the plant in the image;
first, assume that the center coordinates of the image are C (x, y) and the center coordinates of the plant are G (x)1，y1) The length and width of the image are W, H respectively, and the distance between the center point of the plant and the center point of the image is
If the following conditions are met, judging that the plant is the main body of the original image;
secondly, when the central coordinate of the plant and the central coordinate of the image do not meet the condition, but the pixel point ratio of the plant is high, the plant can be judged to be the main body of the original image; because the plant information in the image is more, the image is still attracted by a great number of plants when being observed, and therefore, the color enhancement can be performed on the scenes;
suppose that the number of pixel points in the plant area is n1The number of the integral pixel points of the image is n, if n1Judging that the plant is the main body of the original image if the plant is more than or equal to 0.4 x n;
generally, the plant area occupies more than forty percent of the whole image area, but may also be thirty percent or twenty percent, and the proportion value can be adjusted according to the preference of the user.
It will be appreciated that the above scheme has determined the plants in the original image, as well as the plant species, the direction of color enhancement and whether the plant is the subject of the image, and that in a particular color enhancement, assuming that a green enhancement is required for the plants in the image, the image input pixel value is (R) inin，Gin，Bin) The output value after the differential color enhancement is (R)out，Gout，Bout) Then the green enhancement process can be implemented by the following formula:
Rout＝k1*Rin+b1
Gout＝k2*Gin+b2
Bout＝k3*Bin+b3
wherein k is1，k2，k3Representing the intensity coefficients of the RGB three channels, b1，b2，b3Representing the offset of the RGB three channels.
Empirically, the direction of enhancement is:
for example, willow-shaped plants are enhanced towards the yellow-green color:
k1＝1.1；k2＝1.1；k3＝1.0；b1＝5；b2＝5；b3＝0；
coniform plants are enhanced towards bluish green:
k1＝1.0；k2＝1.1；k3＝1.1；b1＝0；b2＝5；b3＝5；
the elliptical or round plants are enhanced towards brilliant green:
k1＝1.1；k2＝1.1；k3＝1.1；b1＝5；b2＝5；b3＝5；
wide-ovular or narrow-ovular plants are enhanced towards brilliant green:
k1＝1.1；k2＝1.1；k3＝1.1；b1＝5；b2＝5；b3＝5；
of course, the user can perform color enhancement for different kinds of green plants according to the preference of the user.
In summary, the system of the present invention identifies the plants in the image, further determines the plant colors, and determines whether the plants are the main body of the image, and finally performs the differentiated color enhancement on the plants in the image according to the above points. The invention solves the problems that the color of the plant in the image is not bright enough, and the like, and is particularly suitable for enhancing the green of the green plant in the image.
It is understood that all or part of the steps in the methods according to the embodiments may be implemented by a program instructing associated hardware, and the program may be stored in a storage medium readable by a computer device and used for executing all or part of the steps in the methods according to the embodiments. The computer devices, including but not limited to: personal computers, servers, general-purpose computers, special-purpose computers, network devices, embedded devices, programmable devices, intelligent mobile terminals, intelligent home devices, wearable intelligent devices, vehicle-mounted intelligent devices, and the like; the storage medium includes but is not limited to: RAM, ROM, magnetic disk, magnetic tape, optical disk, flash memory, U disk, removable hard disk, memory card, memory stick, network server storage, network cloud storage, etc.
It should be understood that equivalents and modifications of the technical solution and inventive concept thereof may occur to those skilled in the art, and all such modifications and alterations should fall within the scope of the appended claims.
Claims (10)
1. A method of enhancing plant color in an image, the method comprising:
acquiring an original image of a color to be enhanced, and detecting and identifying the type of plants in the original image by using an SSD model;
further judging the color of the plant according to the value of G/R, G/B in RGB three channels in the plant area detected in the original image; judging whether the plant is the main body of the original image;
if the plant is the subject of the original image;
and according to the color characteristics of the plants, performing differentiated color enhancement on the plants in the original image.
2. The method of claim 1, wherein before the detecting and the identifying the plant species in the original image by using the SSD model, the method further comprises training the SSD model, and the training method comprises:
collecting a plurality of sample images containing plants, and classifying according to the leaf shapes of the plants;
inputting the sample image into an SSD model, setting prior frames with different sizes, matching plant categories with different shapes, and generating a corresponding feature mapping chart;
the feature mapping graph is converted through a prediction convolution layer to obtain a prediction frame for distinguishing plant categories and corresponding category confidence coefficients;
and updating the SSD model parameters through the classification loss function and the classification loss function to obtain the optimal SSD model.
3. The method of claim 2, wherein detecting and identifying the type of plant in the original image comprises
Inputting an original image into a trained SSD model, performing descending order arrangement on the bounding boxes through a class confidence coefficient, and calculating the intersection and parallel ratio of the bounding boxes and a prior box if the confidence coefficient is higher than a preset value;
and obtaining the optimal prediction frame and the category of each category by using a non-maximum suppression method, and outputting an identification result.
4. The method of claim 1, wherein the determining the color of the plant comprises obtaining an edge of a leaf of the plant through an SSD model, tracing a boundary region of the plant, and counting G/R of pixels of the plant in the boundary regionaveAnd G/BaveAnd judging the real color of the plant.
5. The method of claim 1, wherein the determining whether the plant is the subject of the original image is determined according to a position of the plant in the original image or/and a pixel ratio of the plant in the original image.
6. The method of claim 5, wherein determining whether the plant is the subject of the original image according to the position of the plant in the original image comprises:
setting a point in the original image as an image center coordinate C (x, y), and setting a point in the plant area as a plant center coordinate G (x)1，y1) The length and width of the original image are respectivelyW, H, the distance between the center point of the plant and the center point of the image is
If the following conditions are met, judging that the plant is the main body of the original image;
7. the method of claim 5, wherein determining whether the plant is the subject of the original image based on the pixel proportion of the plant in the original image comprises:
calculating the number n of pixel points of the original image, and calculating the number n of pixel points of the plant region in the original image1，
If n is1And if the value is more than or equal to 0.4 x n, judging that the plant is the main body of the original image.
8. The method of claim 1, wherein the performing the differential color enhancement on the plant in the original image according to the color feature of the plant comprises:
obtaining input pixel value (R) of plant in original imagein，Gin，Bin) The output value after the differential color enhancement is (R)out，Gout，Bout) Then, the process of enhancing color is:
Rout＝k1*Rin+b1
Gout＝k2*Gin+b2
Bout＝k3*Bin+b3
wherein k is1，k2，k3Representing the intensity coefficients of the RGB three channels, b1，b2，b3Representing the offset of the RGB three channels.
9. A system for enhancing plant color in an image, the system comprising: the device comprises an acquisition module, an SSD model, a color confirmation module, a main body judgment module and a color enhancement module, wherein the acquisition module is used for acquiring an original image of a color to be enhanced;
the SSD model is used for detecting and identifying the types of plants in the original image;
the color confirmation module is used for further judging the color of the plant according to the value of G/R, G/B in RGB three channels in the plant area detected in the original image;
the main body judging module is used for judging whether the plant is a main body of the original image;
if the plant is the main body of the original image, the plant in the original image is subjected to differential color enhancement through the color enhancement module according to the color characteristics of the plant.
10. The system of claim 9, further comprising training the SSD model prior to identifying vegetation in an original image using the SSD model detection; training the SSD model comprises:
collecting a plurality of sample images containing plants, and classifying according to the leaf shapes of the plants;
inputting the sample image into an SSD model, setting prior frames with different sizes, matching plant categories with different shapes, and generating a corresponding feature mapping chart;
the feature mapping graph is converted through a prediction convolution layer to obtain a prediction frame for distinguishing plant categories and corresponding category confidence coefficients;
and updating the SSD model parameters through the classification loss function and the classification loss function to obtain the optimal SSD model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011202232.7A CN112307983B (en) | 2020-11-02 | 2020-11-02 | Method and system for enhancing plant color in image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011202232.7A CN112307983B (en) | 2020-11-02 | 2020-11-02 | Method and system for enhancing plant color in image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112307983A true CN112307983A (en) | 2021-02-02 |
| CN112307983B CN112307983B (en) | 2024-03-26 |
Family
ID=74333337
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011202232.7A Active CN112307983B (en) | 2020-11-02 | 2020-11-02 | Method and system for enhancing plant color in image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112307983B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101193316A (en) * | 2006-11-21 | 2008-06-04 | 安凯（广州）软件技术有限公司 | A self-adapted white balance correction method |
| CN106709922A (en) * | 2016-12-29 | 2017-05-24 | 江苏省无线电科学研究所有限公司 | Image based forage grass coverage and biomass automatic detection method |
| US20170358106A1 (en) * | 2015-01-09 | 2017-12-14 | Hitachi Maxell, Ltd. | Plant information acquisition system, plant information acquisition device, plant information acquisition method, crop management system and crop management method |
| CN107958470A (en) * | 2017-12-18 | 2018-04-24 | 维沃移动通信有限公司 | A kind of color correcting method, mobile terminal |
| US20180330166A1 (en) * | 2017-05-09 | 2018-11-15 | Blue River Technology Inc. | Automated plant detection using image data |
| CN110991454A (en) * | 2019-12-23 | 2020-04-10 | 云南大学 | Blade image recognition method and device, electronic equipment and storage medium |
| CN111476280A (en) * | 2020-03-27 | 2020-07-31 | 海南医学院 | Plant leaf identification method and system |
-
2020
- 2020-11-02 CN CN202011202232.7A patent/CN112307983B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101193316A (en) * | 2006-11-21 | 2008-06-04 | 安凯（广州）软件技术有限公司 | A self-adapted white balance correction method |
| US20170358106A1 (en) * | 2015-01-09 | 2017-12-14 | Hitachi Maxell, Ltd. | Plant information acquisition system, plant information acquisition device, plant information acquisition method, crop management system and crop management method |
| CN106709922A (en) * | 2016-12-29 | 2017-05-24 | 江苏省无线电科学研究所有限公司 | Image based forage grass coverage and biomass automatic detection method |
| US20180330166A1 (en) * | 2017-05-09 | 2018-11-15 | Blue River Technology Inc. | Automated plant detection using image data |
| CN111163628A (en) * | 2017-05-09 | 2020-05-15 | 蓝河技术有限公司 | Automatic plant detection using image data |
| CN107958470A (en) * | 2017-12-18 | 2018-04-24 | 维沃移动通信有限公司 | A kind of color correcting method, mobile terminal |
| CN110991454A (en) * | 2019-12-23 | 2020-04-10 | 云南大学 | Blade image recognition method and device, electronic equipment and storage medium |
| CN111476280A (en) * | 2020-03-27 | 2020-07-31 | 海南医学院 | Plant leaf identification method and system |
Non-Patent Citations (4)
| Title |
|---|
| K.K.THYAGHARAJAN 等: "A Review of Visual Descriptors and Classification Techniques Used in Leaf Species Identification", 《ARCHIVES OF COMPUTATIONAL METHODS IN ENGINEERING》, pages 933 - 960 * |
| NEHA GOYAL 等: "On Solving leaf classification using linear regression", 《MULTIMEDIA TOOLS AND APPLICATIONS》, pages 4533 - 4551 * |
| 朱德利;杨德刚;万辉;杨雨浓;: "用于低照度图像增强的自适应颜色保持算法", 《计算机工程与应用》, vol. 55, no. 24, pages 190 - 195 * |
| 赵振芬;陈远金;张猛蛟;陈东启;王岭雪;蔡毅;: "景深差分提取绿色植物的真彩色夜视图像颜色校正", 《红外技术》, vol. 42, no. 09, pages 886 - 892 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112307983B (en) | 2024-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Aquino et al. | vitisBerry: An Android-smartphone application to early evaluate the number of grapevine berries by means of image analysis | |
| CN113065558A (en) | Lightweight small target detection method combined with attention mechanism | |
| TWI687159B (en) | Fry counting system and fry counting method | |
| CN109101934A (en) | Model recognizing method, device and computer readable storage medium | |
| CN105894655B (en) | Paper currency detection and recognition methods under complex environment based on RGB-D cameras | |
| CN104361357B (en) | Photo album categorizing system and sorting technique based on image content analysis | |
| CN109190456B (en) | Multi-feature fusion overlook pedestrian detection method based on aggregated channel features and gray level co-occurrence matrix | |
| CN109740572A (en) | A kind of human face in-vivo detection method based on partial color textural characteristics | |
| CN102194108A (en) | Smiley face expression recognition method based on clustering linear discriminant analysis of feature selection | |
| Obrador et al. | Towards category-based aesthetic models of photographs | |
| CN106778786A (en) | Apple disease recognition methods based on log-spectral domain laminated gradient direction histogram | |
| CN104008404B (en) | Pedestrian detection method and system based on significant histogram features | |
| CN112434647A (en) | Human face living body detection method | |
| Shuai et al. | An improved YOLOv5-based method for multi-species tea shoot detection and picking point location in complex backgrounds | |
| CN113052234A (en) | Jade classification method based on image features and deep learning technology | |
| JP3962517B2 (en) | Face detection method and apparatus, and computer-readable medium | |
| Dong et al. | Effective and efficient photo quality assessment | |
| CN112307983A (en) | Method and system for enhancing plant colors in image | |
| CN112115824B (en) | Fruit and vegetable detection method, fruit and vegetable detection device, electronic equipment and computer readable medium | |
| CN112183633B (en) | Hyperspectral image salient target detection method based on improved FT algorithm | |
| CN108664921A (en) | Image-recognizing method and system based on bag of words under a kind of Android platform | |
| CN114429662A (en) | Face recognition and retrieval system based on big data | |
| CN110348530B (en) | Method for identifying lipstick number | |
| Nachlieli et al. | Skin-sensitive automatic color correction | |
| CN114549668B (en) | Method for detecting fruit maturity on tree based on visual saliency map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |