EP3529771A1 - Determining the grain weight of an ear - Google Patents
Determining the grain weight of an earInfo
- Publication number
- EP3529771A1 EP3529771A1 EP17780770.8A EP17780770A EP3529771A1 EP 3529771 A1 EP3529771 A1 EP 3529771A1 EP 17780770 A EP17780770 A EP 17780770A EP 3529771 A1 EP3529771 A1 EP 3529771A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- ear
- determining
- grains
- image
- spindle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 claims abstract description 102
- 235000013339 cereals Nutrition 0.000 claims abstract description 100
- 240000008042 Zea mays Species 0.000 claims description 16
- 235000005824 Zea mays ssp. parviglumis Nutrition 0.000 claims description 16
- 235000002017 Zea mays subsp mays Nutrition 0.000 claims description 16
- 235000005822 corn Nutrition 0.000 claims description 16
- 239000004464 cereal grain Substances 0.000 claims description 14
- 230000009466 transformation Effects 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 7
- 238000011161 development Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 241000209140 Triticum Species 0.000 claims description 4
- 235000021307 Triticum Nutrition 0.000 claims description 4
- 238000006073 displacement reaction Methods 0.000 claims description 2
- 230000004720 fertilization Effects 0.000 claims description 2
- 238000005259 measurement Methods 0.000 claims description 2
- 239000010902 straw Substances 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 17
- 238000012545 processing Methods 0.000 description 13
- 238000004458 analytical method Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 9
- 238000001514 detection method Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 241000196324 Embryophyta Species 0.000 description 6
- 238000004590 computer program Methods 0.000 description 6
- 230000000875 corresponding effect Effects 0.000 description 5
- 230000018109 developmental process Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000005314 correlation function Methods 0.000 description 4
- 230000000295 complement effect Effects 0.000 description 3
- 230000003203 everyday effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000012271 agricultural production Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 238000009313 farming Methods 0.000 description 2
- 238000003306 harvesting Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 240000005979 Hordeum vulgare Species 0.000 description 1
- 235000007340 Hordeum vulgare Nutrition 0.000 description 1
- 206010061217 Infestation Diseases 0.000 description 1
- 241000209056 Secale Species 0.000 description 1
- 235000007238 Secale cereale Nutrition 0.000 description 1
- 241000607479 Yersinia pestis Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000002079 cooperative effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000013213 extrapolation Methods 0.000 description 1
- 239000003337 fertilizer Substances 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000000575 pesticide Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000009331 sowing Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000005303 weighing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/02—Agriculture; Fishing; Forestry; Mining
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01G—HORTICULTURE; CULTIVATION OF VEGETABLES, FLOWERS, RICE, FRUIT, VINES, HOPS OR SEAWEED; FORESTRY; WATERING
- A01G22/00—Cultivation of specific crops or plants not otherwise provided for
- A01G22/20—Cereals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/188—Vegetation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
Definitions
- the invention generally relates to a prediction of crop yield, and more particularly to determining a weight of all grains of an ear of a cereal grain in a grain field.
- the invention further relates to a system for determining a weight of all grains of an ear of a cereal grain and to a related computer system.
- a first object of the present invention is thus a method for determining a weight of all grains of an ear of a cereal grain, the method comprising
- Another object of the present invention is a computer program product comprising a computer-readable storage medium in which program elements for determining a weight of all grains of an ear of grain cereal are stored, which, when executed by a processor, cause the processor, the inventive Perform the procedure.
- Another object of the present invention is a system for determining a weight of all grains of an ear of a cereal grain, the system comprising
- a receiving unit for receiving a digital image of the spike in a spindle-end view of the spike, wherein the spike in the digital image is in front of a spike
- Reference card is located as background
- a spindle stage calculating unit adapted to determine a number of spindle stages of the ear by a template matching method
- a grain count determining unit adapted to determine a grain number of the ear by multiplying the determined spindle levels by a factor
- a weight determining unit adapted to determine the weight of all grains of the ear by multiplying the determined grain number by a calibration factor
- a method of determining a weight of all grains of an ear of a cereal grain comprises providing a digital image of the ear in a spindle-stepped view of the ear, wherein the ear of the digital image is shown in front of a reference map as a background.
- the method further comprises determining a length of the ear along the longitudinal axis of the ear by separating image pixels of the digital image of the ear from the background and comparing pixel coordinates at one end of the ear with pixel coordinates of the ear at an opposite end of the ear in the longitudinal direction of the ear with picture marks on the reference card.
- the method comprises determining a number of spindle stages of the ear by a template matching method, determining a grain number of the ear by multiplying the determined spindle stages by a factor, and determining the weight of all grains of the ear of corn by multiplying the determined number of grains by a calibration factor.
- a system for determining a weight of all grains of an ear of a cereal half has a receiving unit for receiving a digital image of the ear in a spindle-end view of the ear, wherein the ear is in the digital image in front of a reference card as a background.
- the system further comprises a surveying unit adapted to determine a length of the ear along the longitudinal axis of the ear by separating image pixels of the digital image of the ear from the background. Furthermore, the measuring unit for comparing pixel coordinates at one end of the ear with pixel coordinates of the ear at an opposite end of the ear in the lengthwise direction of the ear is adapted with picture marks on the reference card.
- the system has:
- a spindle stage calculation unit configured to determine a number of spindle stages of the ear by a template matching method
- a grain count determining unit configured to determine a grain number of the ear by multiplying the determined spindle levels by a factor
- a weight determination unit configured to determine the weight of all grains of the ear of corn by multiplying the determined number of grains by a calibration factor.
- the presented system can be implemented as part of a smartphone.
- the method presented here by a powerful form of a smartphone can be completely or partially performed by this smartphone.
- the determination of the total number of grains of an ear of corn may also be carried out on a dedicated computer specially adapted for this purpose, a server computer or any other computer system.
- embodiments have the form of a corresponding computer program product.
- This may include instructions that, when executed on a computer system, perform steps of the described method.
- a front-end everyday objects in use - such as smartphones - can be used for improved yield prediction.
- Using a mobile device - such as a mobile phone with a camera - is enough to enable farmers to make an improved yield prediction of their cornfields.
- One or two digital images of an ear of corn are already enough to enable the farmer to significantly improve his field yield forecast.
- Another simple everyday item in the form of a reference card does not complicate the handling and acceptance of the method. In this case, the ear can either be picked or cut and placed on the reference card, or the ear can remain on the stalk and the reference card can be easily placed behind the ear.
- the scale of the reference card provides a clear, unadulterated scale with the digital image.
- the required computing power for the automated measurement of the ear and the grain weight of the ear can be made available in a data center.
- This data center can be operated anywhere. It can either be a computer used by a farmer, several farmers can operate the computer together or a service provider takes over the analysis work and offers the required computing power.
- the latter would have the further advantage that the service could be operated in the form of a cloud computing service for a large number of farmers in different regions or even transnationally. It would also be easier to take into account parallels between different regions, global and even local weather conditions or even regionally known pest infestation, use of fertilizers or use of pesticides, etc.
- the digital image can be transmitted via a mobile network to an evaluation computer.
- the analysis can be carried out and the result transmitted wirelessly back to the farmer or the mobile device.
- An extrapolation from the grain weight of an ear to the entire field could be made by conventional methods.
- the analysis could also be carried out directly on site.
- the necessary calculation algorithms could be made available in the form of a smartphone app.
- a dedicated calculator special processor or special hardware
- a smartphone for digital recording.
- the farmer could also use a conventional digital camera and transmit the digital recording of the ear of corn in another form to the computer for analysis: for example, via wired communication technologies or relay stations that have known communication channels such as WLAN, Bluetooth or other comparable means of communication.
- the template matching method used to determine the number of spindle stages provides, by the nature of the digital recordings of the ear - in the form of the spindle stage view - a good basis for the further image processing and determination steps.
- the method also includes determining a 1000 grain weight of the grains of the ear.
- This value also known as Tear Weight (TKG) - is a common calculation value for estimating yields in agricultural environments and indicates the weight of 1000 grains of a grain portion. It can be calculated from the grain weight of an ear and the determined number of grains of the ear.
- the template matching method, a pixel-wise displacement of a selected image template consisting of a central portion of the ear of the entire ear in the longitudinal direction of the ear have.
- the method may comprise respectively determining a respective similarity factor of the image template with a respective covered ear portion in each shift position.
- the x-direction of the representation may be the pixel number or position, and in the y-direction a similarity value of the respective ear-portion with the template may be plotted.
- the selected portion of the ear can occupy about 15-25% of the ear in a central region of the ear.
- this embodiment may include determining the number of spindles from the x-y plot. Since the template matching method is a well-known method in the field of image processing, conventional program library functions and modules can be used. The use of this matching method provides good accuracy and robustness against variations in the illumination geometry in the determination of the spindle stages. This is advantageous because the number of spindle stages has a significant influence on the number of grains of the ear. An additional spindle level of the spike can be equivalent to 4 additional grains, which can increase the total number of grains of honor by up to 10%. Consequently, the most accurate possible detection of the number of spindle stages is synonymous with the accuracy of the proposed method.
- determining the number of spindles from the x-y representation may include determining the number of relative maxima of a similarity value by simple counting. This procedure requires little computing power, but is not the most accurate compared to other methods because the degree of similarity to the ends of the ear decreases and accordingly the maxima are no longer as pronounced as in the central ear area.
- determining the number of spindles from the x-y plot may include determining a mean period length from the distances of the relative maxima of a similarity value to each other and determining the number of spindles by dividing the ear length by the period length.
- this embodiment now described may have a higher degree of accuracy in determining the number of spindle stages. This is because the relative maxima in the x-y representation may be more pronounced than in the previous embodiment. This results in a higher accuracy in determining the number of spindle stages of an ear.
- the method may comprise a transformation of the position of the pixels of the ear opposite the reference card, so that oblique perspectives of the digital image of the ear can be compensated, and so that the longitudinal axis of the ear parallel to a Side of the reference card can lie.
- This allows a more accurate recording of the length of the ear.
- by such a transformation distortions can be compensated in whole or in part by oblique recording of the ear.
- the method may comprise erecting the ear corresponding to a transformation of the pixels of the ear opposite the reference card so that the longitudinal axis of the ear can be parallel to the side of the reference card.
- Under an upright can be a certain "Straight bend" of honor can be understood, so that a central axis of the ear can be aligned substantially parallel to mutually parallel side edges.
- the image section can be taken with the identified spike object and subjected to a morphological image operation. This procedure has the advantage of better conditions for The template matching method, since a curvature of the ear in the matching process need not be considered.
- a developmental stage of the ear may be greater than or equal to 60 according to the BBCH code. At an earlier stage of development of the ear, greater inaccuracies in determining the grain weight of an ear can occur. From a development stage which is greater than 60 according to the BBCH code, experience has shown that good accuracy can be achieved in determining the grain weight.
- a color value of the reference card may be in marked contrast to a mean color value of the ear, e.g. in an RGB color model.
- a different color model can also be selected since color models can generally be converted into one another by simple transformation.
- the contrasting color value proposed here may, for example, be a complementary color of a substantial color component of the ear. A blue or green-blue hue has proven to be advantageous. In this way, a slight differentiation of pixels of the background and pixels of the ear located in front of the background can be possible.
- the spindle step view may be that view of the ear that provides the smallest viewing surface when a longitudinal axis of the ear is vertical.
- the spindle step view is thus a view of the ear, which represents a flower view rotated by 90 ° about the longitudinal axis of the ear. In this way, a reproducibility of the process can be facilitated.
- a further advantageous embodiment of the method may allow the digital image of the ear to be taken on the stem of the ear or separately from the stem. Thus, it is not necessary to cut or tear off the ear before taking a digital image of the ear.
- the digital recording of the spike along with the reference card can be done in any situation. It would be advantageous if a central plane parallel to the longitudinal axis of the ear would run parallel to the surface of the reference card.
- the image marks of the reference card can have at least two image markers.
- the image marks may have at least one scale.
- the two image markers may be formed, for example, by image corner markers. In this way, the corners of the reference card can be elegantly identified by means of image processing methods.
- the calibration factor may have at least one dependence on one of the following factors: variety of the ear, growth stage of the ear, weather (long term and short term), geographical location and fertilization status. Further influencing parameters can be taken into account at any time.
- embodiments may take the form of an associated compute rogramm areas that can be accessed by a computer-usable or computer-readable medium.
- the instructions may cause a computer - such as a smartphone, a server, or a combination of both - to perform processing steps in accordance with the presented method.
- the computer usable or computer readable medium may be any device having elements for storing, communicating, transporting or forwarding the program together with the instruction processing system.
- Fig. 1 shows a block diagram of an embodiment of the inventive method for determining the total grain number of an ear of a cereal grain.
- Fig. 2 shows a first part of a block diagram of an implementation nearer embodiment of the proposed method.
- FIG. 3 shows a second part of the block diagram of the implementation of the proposed method of FIG. 2.
- FIG. 4 shows an ear of wheat shown in abstract and an example of a reference card together with an ear of corn lying thereon.
- Fig. 4a shows a picture of an ear and a view of the spindle stages of an ear.
- Fig. 5 shows an exemplary diagram for determining the ear length.
- FIG. 6 shows an exemplary diagram of a cross-correlation function for determining the number of spindle stages.
- Fig. 7 shows a block diagram of a system for determining the total grain number of an ear of a cereal grain.
- FIG. 8 shows a block diagram of an example of a computer system together with the system according to FIG. 7.
- cereal haulm or "ear of a cereal haulm” need not be interpreted further. It may be an ordinary cereal crop that is on an agricultural field is growing. Typically, the grain may be wheat, rye or barley.
- digital image describes a digital image of a real scene that typically can be captured with a digital camera, and the digital image or image may be composed of pixels of different color values to create a visual overall impression.
- flower view of the ear describes a view of the ear in which the grains are clearly visible.
- the flower view can also be referred to as grain view of the ear, as in this view the grains of the ear are most visible In the view, the awns extend predominantly left and right away from the ear. In this view, the viewing area of the ear is the largest
- spindle stage view describes a 90 ° rotated view of the ear, that is a view of the ear narrow view of the ear of corn. Here you look on the narrower side of the ear or on the awns of the ear when the longitudinal axis of the ear is vertical.
- a "reference map” in the context of this description is a flat object - for example a monochrome map - whose color value is well different from that of the ear.
- a complementary hue - e.g., blue - to a typical hue color value has been found to be advantageous.
- template matching method is known to a person skilled in the art as a method for detecting a structure of a digitally displayed object, for example, a detailed description can be found in S. Kim, J. McNames, Automatic spike detection based on adaptive template matching for extracellular neuronal recordings "', Journal of Neuroscience Methods 165 pp. 165-174, 2007.
- development stage or “developmental stage” describes a stage in the natural life cycle of a plant - here a cereal - from sowing to harvesting It has been found that the use of the "BBCH code” to describe the developmental stage a plant is helpful.
- the abbreviation "BBCH” stands officially for "Biologische Bundesweg, Bundessortenamt and CHemische Industrie” and describes a phenomenological development stage of plants.
- the code starts at 00 and ends at 89.
- a BBCH code between 10 and 19 describes an early stage of development of a leaf. From a BBCH code of 60, the flower of the plant appears (up to 69). The next ten steps each describe the development of the fruit (70-79), the seed maturity (80-89) and the death (90-99 - of annual plants) of the plant.
- Fig. 1 shows a block diagram of an embodiment of the inventive method 100 for determining the total number of grains of an ear of a cereal grain.
- the method initially comprises providing, 102, a digital image of the ear in a spindle-end view of the ear.
- the ear should be in the digital image in front of a reference card as a background when recording.
- the reference card will, for convenience, have the complementary color (e.g., blue) to a typical color of an ear of corn (yellowish).
- the method comprises determining, 104, a length of the ear along the longitudinal axis of the ear of corn by separating image pixels of the digital image of the ear from the background. This separation can advantageously be done by means of a color histogram method. In this way a coherent surface of the ear can be distinguished against the background of the reference card.
- the method involves comparing, 106, pixel coordinates at one end of the ear Pixel coordinates of the ear at an opposite end of the ear in the longitudinal direction of the ear with picture marks on the reference card. By using a scale on the reference card it is easy to determine the length of the ear. For this purpose, only corresponding y-coordinates have to be subtracted from each other.
- a determination 108, a number of spindle stages of the ear can be made by a template matching method, to which a determination of a grain number of the ear (step 110) by a multiplication of the determined spindle stages by a factor , which indicates the number of grains per spindle level and has, for example, a value of 4, follows.
- determining 112 the weight of all grains of the ear is achieved by multiplying the determined number of grains by a calibration factor.
- the calibration factor can take into account a large number of variables.
- the process begins with providing a digital image of the ear.
- a photograph of the spike in the flower view - in the view in which the grains of the spike are clearly visible - is made in front of a reference card.
- determining an area of the flower view of the ear of corn by separating image pixels of the digital image of the ear from the background, e.g. by means of a color histogram method, and comparing the area occupied by an ear of the ear with image marks on the reference map.
- the image marks may be the scale of the reference map or the known distances of other image marks on the reference map.
- the weight of all grains of the ear is determined by multiplying the determined area of the ear with a calibration factor. It has been shown that there is a pronounced direct correlation between the projection area of the spike in flower view and the number of grains of the spike. This phenomenon is used here to simply and elegantly determine the grain weight of the ear. This alternative method can also be used particularly well from a growth stage that is greater than 60 BBHC. However, it works even at lower BBHC values.
- Fig. 2 shows a first part of a block diagram of an implementation closer embodiment of the proposed method.
- a digital image 208 of an ear is received along with a reference card.
- a geometry correction 202 also includes a corner end detection 204 of the corners of a colored area on the reference map. This is followed by a transformation 206 of perspective and image section, so that areas outside the colored background with the ear of corn lying on it are ignored.
- the image section 210 obtained in this way is forwarded to an ear detection function 212.
- the actual ear detection is done by means of an analysis 214 by means of a color histogram method to distinguish pixels of the ear and the colored background from each other (segmentation foreground / background 216).
- the detected spike object is masked, 218.
- detected image pixels of the Background is shown as a logical "0.”
- the result 220 is a representation of the ear separated from the background.
- a spike preprocessing 222 is performed. This may include a step of lighting and contrast optimization 224.
- a transformational straightening of the ear and a further reduction of the image section to be processed can be carried out (step 226).
- an optical awning detection of the view of the ear is possible (step 227).
- the view of the ear is a spindle stage view.
- the further processing of the result 228 of the grain preprocessing is based on FIG. 3.
- Fig. 3 shows a second part of a block diagram of an implementation nearer embodiment of the proposed method. This is where the actual ear analysis 302 takes place.
- a geometry analysis 304 is required, the result of which is a determination of the ear length 309.
- corner marks - allows in conjunction with the distance to the Halmansatz at the lower end of the ear, a determination of the length of the ear in the longitudinal direction, as shown in Fig. 5.
- a middle selected region 310 of the ear 228 in the shape shown at this time is moved pixel-wise in the vertical direction along the vertical longitudinal axis of the ear of 228 on this.
- a similarity factor is determined, which is determined mathematically by the cross-correlation function 308 between template and image function.
- maxima values which lie at a periodic distance from one another result in the course of the cross-correlation function 308.
- a result of such a correlation analysis 308 is shown in Fig. 6, which provides the period length and thus the distance of the spindle stages to each other.
- the ratio of ear length 309 to period length leads to an exact measure for half the number of spindle stages, because the periodicity of the spindle stages is very clear and pronounced with a constant distance.
- the grain analysis 312 is carried out with the determination of the number of grains 316 and the determination of the 1000 grain weight 314.
- a yield calculation 318 of the entire field, or a subarea thereof, can follow via a yield formula 320 ,
- Fig. 4 shows an ear of corn 208 shown in abstract and an example of a colored area 402 of a reference map (which may be larger than the area 402; color not shown) together with the ear of corn 208 thereon.
- the colored area 402 has image marks such as a scale 404 and for example Dolphinmarken 414 on.
- the Jardineckmarken 414 may have different forms.
- awns 406 are symbolically represented, which can be of different lengths depending on the type of grain.
- a piece of the straw 410 is still shown, which has only for the detection of the lower ear field its meaning for the presented method.
- the view shown here is - for reasons of simplicity - the flower view of the spike.
- the spindle stage view in which the actual digital recording of the ear takes place, is a view rotated by 90 ° about the longitudinal axis of the ear.
- the ear 208 should be aligned on the colored surface 402 of the reference card so that the longitudinal axis 412 of the ear 208 is aligned as parallel as possible to a side line of the colored surface 402.
- a typically curved shape of the ear 208 may be formed by a transformation of the Representation of the ear 208 are adapted so that the longitudinal axis of the ear is actually aligned parallel to a side line of the colored area 402 of the reference card.
- the reference map is typically a little larger than the colored area 402 contained on it, whose color is "blue", for example.
- an actual image of an ear 208 is a contiguous region (shown, for example, in FIGS. 2, 220, 228).
- the form of representation of the ear 208 used herein is to be understood as merely representing the orientation of the ear 208 relative to the reference map ,
- FIG. 4a shows an illustration of an ear 208 (left) and a view 426 of the spindle stages of an ear (right).
- the illustration of the ear 208 clearly shows the different grains 416, 418, 420, 422 in the lower region of the ear and the stalk 410. Accordingly, the more abstract shape of the ear on the right side of FIG. 4a shows the different spindle stages 424 of the ear 208th
- FIG. 5 shows an exemplary diagram 500 for determining the length of the ear. It can be seen here that the width of the ear (y-axis) is plotted per pixel row (x-axis) that relates to the ear.
- the individual relative maxima - or a closely related group of relative maxima - refer to one spindle stage each. By simply counting the relative maxima or the groups of the relative maxima, the number of spindle stages can be identified. From the beginning of the pixels of the ear at about line 60 and the end of the pixels of the ear at about line 1710 results using the scale of the reference card or by knowing the width of an individual pixel or a pixel line the ear length 502.
- FIG. 6 shows an exemplary diagram 600 of a cross-correlation function for determining the number of spindle stages on the basis of the template matching method.
- the respective position of the template pattern from the middle of the ear
- a correlation value similarity value
- Match - a correlation score of practically 1.
- the template is exactly in its original place.
- a period length 604 can be determined which corresponds to the distance of the individual spindle stages from one another. From the determined length of the ear and the average determined period length 604, the number of spindle stages can also be calculated by division and rounding.
- Fig. 7 shows a block diagram of a system for determining the total grain number of an ear of a cereal grain.
- the system includes a receiving unit 702 for receiving a digital image of the ear in a spindle-stepped view of the ear.
- the receiving unit is a digital camera.
- a digital recording of the ear of a digital camera is recorded and transmitted to the receiving unit 702 - optionally wirelessly.
- the ear of the digital image is recorded in front of a reference card as a background.
- the system may include a display unit 704.
- the system includes a surveying unit 706.
- the system comprises a spindle stage calculation unit 708 adapted to determine a number of spindle stages of the ear by a template matching method, and a grain number determination unit 710 adapted to determine a grain number of the ear by multiplying the ear determined spindle stages with a factor.
- a weight determining unit 712 adapted to determine the weight of all grains of the ear by multiplying the determined number of grains by a calibration factor.
- the system can be part of a server system which receives the digital image (s) from a digital camera, for example a smartphone.
- a digital camera for example a smartphone.
- Embodiments of the invention may be implemented in conjunction with virtually any type of computer, including, but not limited to, a smartphone, regardless of the platform used to store and execute program code.
- FIG. 8 exemplifies a computer system 800 suitable for executing program code related to the proposed method.
- Computer system 800 is only one example of a suitable computer system and is not intended to be a limitation on the scope of any use or functionality of the invention described herein. On the contrary, the computer system 800 is capable of implementing any feature or functionality of the embodiments described herein.
- the computer system 800 includes components that may cooperate with a variety of other general or dedicated computer system environments and / or configurations.
- Examples of known computer systems, environments, and / or configurations that may be suitable to cooperate with computer system 800 include, but are not limited to, tablet computers, notebook computers, and / or other mobile computing systems, and / or Smartphones as well as multiprocessor systems, microprocessor-based systems, programmable consumer electronics or even digital cameras or PDAs (Personal Digital Assistant).
- the computer system 800 is described herein in a general context of instructions executable by a computer system. These may also be program modules that are executed by the computer system 800. Generally, program modules include program routines, subprograms, objects, components, processing and / or decision logic, data structures, etc. that perform a particular task or represent a particular abstract data type.
- the computer system 800 may be implemented in the form of a "general purpose” computing system, wherein the components of the computer system 800 include, but are not limited to, one or more processing units 802 (CPU), a memory system 804, and a System bus 818 which connects various system components including main memory 804 to processor 802.
- processing units 802 CPU
- memory system 804 main memory 804
- System bus 818 which connects various system components including main memory 804 to processor 802.
- Computer system 800 also includes various computer-readable media. Such media includes all media accessible by the computer system 800. This includes both volatile and non-volatile media, which can be removable as well as permanently installed.
- the main memory 804 may also include computer-readable media in the form of a volatile memory. This can be, for example, a Random Access Memory (RAM) or a cache memory.
- the computer system 800 may further comprise removable and non-removable storage media.
- the storage system 812 may include the ability to store data on a non-removable memory chip.
- the storage media may be connected to the system bus 806 through one or more data interfaces.
- the memory 804 may include at least one computer program product having a plurality of program modules (at least one) configured and configured to configure the computer system to perform the functions of embodiments of the invention.
- a program comprising a plurality of program modules may for example be stored in the memory 804, as well as an operating system, one or more application programs, program modules and / or program data.
- the computer system 800 may further communicate with a plurality of external devices, such as a keyboard 808, a pointer instrument (“mouse") 810, a display (not shown), etc. These devices may be incorporated, for example, in a touch-sensitive screen 812 (touch screen ) may be combined to facilitate intuitive interaction with the computer system 800.
- the computer system 800 may also include acoustic input / output devices 816.
- other connections may be present to communicate with one or more other data processing devices (modem, network ports, etc
- such communication may be via input / output (I / O) interfaces, and computer system 800 may be over one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or.
- the network adapter 814 may communicate with other components of the computer system 800 via the system bus 818. It should also be appreciated that although not shown, other hardware and / or software components may be used in conjunction with the computer system 800. This concerns, for example, micro-code, device drivers, redundant processing units, etc.
- system 700 may be connected to the bus system 818 for determining a weight of all grains of an ear of a cereal grain.
- the computer system 700 for determining a weight of all grains of an ear of corn may receive the digital shot, determine the weight, and transmit the result back to the mobile device.
- the system 700 may also be integrated into a mobile computer system (e.g., high performance smartphone).
- the present invention can be realized as a system, a method and / or a computer program product or a combination thereof.
- the computer program product may comprise a computer readable storage medium (or simply "medium") containing computer readable program instructions for causing a processor to implement aspects of the present invention.
- the medium can be based on electronic, magnetic, electromagnetic waves, infrared light or semiconductor systems which are also suitable for transmission. These include solid state memory, random access memory (RAM), and read only memory (ROM: read-only memory).
- RAM random access memory
- ROM read only memory
- the computer readable program instructions described herein may be downloaded to the appropriate computer system from a potential service provider over a mobile network connection or a stationary network.
- the computer readable program instructions for performing operations of the present invention may be any type of machine dependent or machine independent instructions, microcode, firmware, status code or object code, in any combination written in one or more programming languages.
- the programming languages may be C ++, Java or similar modern programming languages or conventional procedural programming languages such as the "C" programming language or similar programming languages.
- the computer readable program instructions may be executed entirely on the computer system Circuits such as programmable logic devices, field programmable gate array (PGA) or programmable logic arrays (PLA) execute the instructions using status information in the computer readable program instructions to customize the electronic circuit (s) to accommodate aspects of the present invention to carry out the present invention.
- These computer readable program instructions may be provided to a processor of a "general purpose computer” or more specifically to computer hardware or other programmable data processing devices to generate a machine such that the instructions executed by the respective processor will generate means for implementing the functions / actions illustrated in the corresponding flowchart and / or block diagram or blocks thereof.
- These computer readable program instructions may also be stored on a computer-readable storage medium so as to cause a computer or programmable data processing device to execute the instructions stored in the medium by the respective processor, so that aspects or actions described in that document Procedure be performed.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Tourism & Hospitality (AREA)
- Animal Husbandry (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Mining & Mineral Resources (AREA)
- General Business, Economics & Management (AREA)
- Marine Sciences & Fisheries (AREA)
- Human Resources & Organizations (AREA)
- Agronomy & Crop Science (AREA)
- Geometry (AREA)
- Botany (AREA)
- Environmental Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
Abstract
Description
Claims
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP16194687 | 2016-10-19 | ||
EP16194686 | 2016-10-19 | ||
PCT/EP2017/076035 WO2018073093A1 (en) | 2016-10-19 | 2017-10-12 | Determining the grain weight of an ear |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3529771A1 true EP3529771A1 (en) | 2019-08-28 |
Family
ID=60037630
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17780770.8A Withdrawn EP3529771A1 (en) | 2016-10-19 | 2017-10-12 | Determining the grain weight of an ear |
EP17790724.3A Withdrawn EP3528609A1 (en) | 2016-10-19 | 2017-10-16 | Yield prediction for a cornfield |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP17790724.3A Withdrawn EP3528609A1 (en) | 2016-10-19 | 2017-10-16 | Yield prediction for a cornfield |
Country Status (6)
Country | Link |
---|---|
US (3) | US11010913B2 (en) |
EP (2) | EP3529771A1 (en) |
CN (2) | CN109863530B (en) |
BR (2) | BR112019007937A8 (en) |
RU (1) | RU2769288C2 (en) |
WO (2) | WO2018073093A1 (en) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10219449B2 (en) * | 2013-03-07 | 2019-03-05 | Blue River Technology Inc. | System and method for plant dislodgement |
CN113011220A (en) * | 2019-12-19 | 2021-06-22 | 广州极飞科技股份有限公司 | Spike number identification method and device, storage medium and processor |
US11823408B2 (en) * | 2020-03-13 | 2023-11-21 | Oregon State University | Apparatus and method to quantify maize seed phenotypes |
CN112101681B (en) * | 2020-09-25 | 2023-04-28 | 北京师范大学 | Method for monitoring winter wheat dry hot air disasters based on remote sensing NDPI time sequence |
CN113139934B (en) * | 2021-03-26 | 2024-04-30 | 上海师范大学 | Rice grain counting method |
TWI818334B (en) * | 2021-10-19 | 2023-10-11 | 國立中興大學 | Method for estimating grain moisture content and harvest time of grains with mobile phone images |
DE102022110185A1 (en) | 2022-04-27 | 2023-11-02 | Deere & Company | Method and arrangement for measuring a grain-specific size on a harvesting machine |
DE102022129876A1 (en) | 2022-11-11 | 2024-05-16 | Deere & Company | Method and arrangement for determining a mass and/or size-specific size of grain crops |
Family Cites Families (39)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4076633A (en) * | 1974-10-18 | 1978-02-28 | The Procter & Gamble Company | Fabric treating articles with improved conditioning properties |
JP2617014B2 (en) * | 1990-04-26 | 1997-06-04 | 日本リライアンス 株式会社 | Sheet length measurement system |
US8031910B2 (en) * | 2003-09-17 | 2011-10-04 | Syngenta Participations Ag | Method and apparatus for analyzing quality traits of grain or seed |
RU2308679C2 (en) * | 2005-09-08 | 2007-10-20 | ИПБОЮЛ Ващенко Юрий Ефимович | Method of mapping land |
WO2007068056A1 (en) * | 2005-12-14 | 2007-06-21 | Grains Research And Development Corporation | Stain assessment for cereal grains |
US9508007B2 (en) * | 2006-11-07 | 2016-11-29 | The Curators Of The University Of Missouri | Method of predicting crop yield loss due to N-deficiency |
US9195891B2 (en) * | 2006-11-07 | 2015-11-24 | The Curators Of The University Of Missouri | Method of predicting crop yield loss due to n-deficiency |
US8208680B2 (en) * | 2006-11-07 | 2012-06-26 | The Curators Of The University Of Missouri | Method of predicting crop yield loss due to N-deficiency |
JP2011167163A (en) * | 2010-02-22 | 2011-09-01 | Pasuko:Kk | Method of generating paddy rice crop yield forecasting model, and method of forecasting crop yield of paddy rice |
CN101944231B (en) | 2010-08-19 | 2012-01-04 | 北京农业智能装备技术研究中心 | Method for extracting wheatear morphological parameters |
WO2013030964A1 (en) | 2011-08-30 | 2013-03-07 | 富士通株式会社 | Imaging device, work support program, information provision method, and information provision program |
JP5729476B2 (en) | 2011-08-30 | 2015-06-03 | 富士通株式会社 | Imaging device and imaging support program |
CA2861591A1 (en) * | 2011-12-30 | 2013-07-04 | Pioneer Hi-Bred International, Inc. | Immature ear photometry in maize |
CN102612892B (en) * | 2012-03-02 | 2014-05-07 | 北京农业信息技术研究中心 | Identification method for sprouting conditions of wheat ears |
CN103632157B (en) * | 2012-08-24 | 2018-01-05 | 南京农业大学 | Individual plant wheat fringe portion kernal number method of counting |
US10234439B2 (en) * | 2012-11-07 | 2019-03-19 | Airscout Inc. | Methods and systems for analyzing a field |
JP5950166B2 (en) | 2013-03-25 | 2016-07-13 | ソニー株式会社 | Information processing system, information processing method of image processing system, imaging apparatus, imaging method, and program |
CN103190224B (en) * | 2013-03-26 | 2015-02-18 | 中国农业大学 | Computer vision technique-based corn ear species test method, system and device |
KR20140125229A (en) * | 2013-04-18 | 2014-10-28 | 한국전자통신연구원 | Product measuring system of fruit tree and operation method thereof |
FR3013874A1 (en) * | 2013-11-26 | 2015-05-29 | Invivo Agrosolutions | METHOD FOR PROCESSING AGRONOMIC DATA FOR ESTIMATING PERFORMANCE OF A CULTURE OF THE POACEE FAMILY |
CN103810522B (en) * | 2014-01-08 | 2017-02-08 | 中国农业大学 | Counting method and device for corn ear grains |
CN103793686B (en) * | 2014-01-16 | 2017-12-15 | 中国农业大学 | A kind of method of output of the fruit tree early prediction |
WO2015160837A2 (en) * | 2014-04-15 | 2015-10-22 | Raven Industries, Inc. | Reaping based yield monitoring system and method for the same |
CN104021369A (en) * | 2014-04-30 | 2014-09-03 | 南京农业大学 | Grain counting method for spike of single rice based on digital image processing technology |
US9903979B2 (en) * | 2014-09-23 | 2018-02-27 | Deere & Company | Yield estimation |
US10126282B2 (en) * | 2014-09-23 | 2018-11-13 | Deere & Company | Yield estimation |
US10186029B2 (en) * | 2014-09-26 | 2019-01-22 | Wisconsin Alumni Research Foundation | Object characterization |
US9734400B2 (en) * | 2015-01-30 | 2017-08-15 | AgriSight, Inc. | System and method for field variance determination |
US10115187B2 (en) * | 2015-01-30 | 2018-10-30 | Raytheon Company | Apparatus and processes for classifying and counting corn kernels |
US10028426B2 (en) * | 2015-04-17 | 2018-07-24 | 360 Yield Center, Llc | Agronomic systems, methods and apparatuses |
WO2017004074A1 (en) * | 2015-06-30 | 2017-01-05 | Precision Planting Llc | Systems and methods for image capture and analysis of agricultural fields |
US10019791B2 (en) * | 2015-07-24 | 2018-07-10 | Raytheon Company | Apparatus and methods for estimating corn yields |
CN105427275B (en) * | 2015-10-29 | 2017-11-24 | 中国农业大学 | Crop field environment wheat head method of counting and device |
CN105574853B (en) * | 2015-12-07 | 2018-05-15 | 中国科学院合肥物质科学研究院 | The method and system that a kind of wheat head grain number based on image recognition calculates |
CN205607838U (en) * | 2016-05-06 | 2016-09-28 | 北京农业信息技术研究中心 | Wheat spike number measuring device |
WO2018049289A1 (en) * | 2016-09-09 | 2018-03-15 | Cibo Technologies, Inc. | Systems for adjusting agronomic inputs using remote sensing, and related apparatus and methods |
WO2018049288A1 (en) * | 2016-09-09 | 2018-03-15 | Cibo Technologies, Inc. | Systems for learning farmable zones, and related methods and apparatus |
WO2018049290A1 (en) * | 2016-09-09 | 2018-03-15 | Cibo Technologies, Inc. | Systems for determining agronomic outputs for a farmable region, and related methods and apparatus |
US10909367B2 (en) * | 2017-03-02 | 2021-02-02 | Basecamp Networks, LLC | Automated diagnosis and treatment of crop infestations |
-
2017
- 2017-10-12 EP EP17780770.8A patent/EP3529771A1/en not_active Withdrawn
- 2017-10-12 BR BR112019007937A patent/BR112019007937A8/en not_active Application Discontinuation
- 2017-10-12 US US16/342,341 patent/US11010913B2/en active Active
- 2017-10-12 WO PCT/EP2017/076035 patent/WO2018073093A1/en unknown
- 2017-10-12 CN CN201780064463.6A patent/CN109863530B/en active Active
- 2017-10-16 BR BR112019007962A patent/BR112019007962A2/en not_active Application Discontinuation
- 2017-10-16 RU RU2019114132A patent/RU2769288C2/en active
- 2017-10-16 CN CN201780064471.0A patent/CN109843034B/en active Active
- 2017-10-16 US US16/342,363 patent/US10984548B2/en active Active
- 2017-10-16 EP EP17790724.3A patent/EP3528609A1/en not_active Withdrawn
- 2017-10-16 WO PCT/EP2017/076320 patent/WO2018073163A1/en unknown
-
2021
- 2021-04-19 US US17/234,589 patent/US20210241482A1/en not_active Abandoned
Also Published As
Publication number | Publication date |
---|---|
WO2018073163A1 (en) | 2018-04-26 |
US10984548B2 (en) | 2021-04-20 |
CN109843034A (en) | 2019-06-04 |
RU2019114132A3 (en) | 2021-01-27 |
RU2019114132A (en) | 2020-11-20 |
BR112019007937A8 (en) | 2023-04-25 |
WO2018073093A1 (en) | 2018-04-26 |
US11010913B2 (en) | 2021-05-18 |
CN109863530B (en) | 2022-11-15 |
US20210241482A1 (en) | 2021-08-05 |
BR112019007937A2 (en) | 2019-07-02 |
BR112019007962A2 (en) | 2019-07-02 |
RU2769288C2 (en) | 2022-03-30 |
EP3528609A1 (en) | 2019-08-28 |
US20190236800A1 (en) | 2019-08-01 |
CN109863530A (en) | 2019-06-07 |
US20190286904A1 (en) | 2019-09-19 |
CN109843034B (en) | 2022-08-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2018073093A1 (en) | Determining the grain weight of an ear | |
Bai et al. | Field-based scoring of soybean iron deficiency chlorosis using RGB imaging and statistical learning | |
Oliver et al. | Integrating farmer knowledge, precision agriculture tools, and crop simulation modelling to evaluate management options for poor-performing patches in cropping fields | |
EP3701449A1 (en) | Yield estimation in the cultivation of crop plants | |
Dutta et al. | Segmentation of unhealthy leaves in cruciferous crops for early disease detection using vegetative indices and Otsu thresholding of aerial images | |
DE112014001474T5 (en) | Multivariable yield monitoring device and method for same | |
DE112015002189T5 (en) | Image filter based on row identification | |
DE112015000972T5 (en) | Image processing apparatus, image processing method and image processing program | |
WO2017071928A1 (en) | Method and information system for detecting at least one plant planted on a field | |
DE112020002954T5 (en) | Feature quantity extraction device, feature quantity extraction method, identification device, identification method and program | |
Tresch et al. | Easy MPE: extraction of quality microplot images for UAV-based high-throughput field phenotyping | |
CN104751122B (en) | A kind of computational methods and system of crops disease index | |
DE112020003158T5 (en) | Training method for a target acquisition model, method and device for characterizing the data | |
EP3673425A1 (en) | Yield estimation in the cultivation of crop plants | |
WO2013182179A1 (en) | Method for contactless determination of the current nutritional state of a plant population and for processing said information | |
EP3796135B1 (en) | Method for assisting a user involved in an agricultural activity | |
DE202017007534U1 (en) | Multiscale 3D texture synthesis | |
DE102021204256A1 (en) | Mower positioning configuration system | |
DE102015221092A1 (en) | Method and information system for detecting at least one plant parameter data set of a plant growing on a field | |
DE102017217258A1 (en) | Method for classifying plants | |
Olsen | Improving the accuracy of weed species detection for robotic weed control in complex real-time environments | |
DE102014108924B4 (en) | A semi-supervised method for training an auxiliary model for multi-pattern recognition and detection | |
CN115424151A (en) | Agricultural intelligent platform based on image processing | |
DE112020001625T5 (en) | Image search system and method | |
DE112019004975T5 (en) | METHOD, SYSTEM AND DEVICE FOR DETERMINING A SUPPORT STRUCTURE DEPTH |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20190520 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20201202 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20230320 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20230801 |