CN110612813A - Leaf vegetable cutting system based on embedded visual platform - Google Patents
Leaf vegetable cutting system based on embedded visual platform Download PDFInfo
- Publication number
- CN110612813A CN110612813A CN201910995213.5A CN201910995213A CN110612813A CN 110612813 A CN110612813 A CN 110612813A CN 201910995213 A CN201910995213 A CN 201910995213A CN 110612813 A CN110612813 A CN 110612813A
- Authority
- CN
- China
- Prior art keywords
- cutting
- leaf
- module
- image
- leaf vegetable
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01D—HARVESTING; MOWING
- A01D45/00—Harvesting of standing crops
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/155—Segmentation; Edge detection involving morphological operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30188—Vegetation; Agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Environmental Sciences (AREA)
- Harvesting Machines For Specific Crops (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a leaf vegetable cutting system based on an embedded visual platform, which comprises a hardware module and a control module, wherein the hardware module comprises a conveying belt, an infrared positioning mechanism, a slider mechanism, a CMOS visual identification component, a cutter module, an adjusting module and a transverse module, the control module comprises a control unit, a power supply circuit, a data acquisition circuit and a cutting control circuit, the data acquisition circuit is electrically connected with the CMOS visual identification component, and the cutting control circuit is electrically connected with the cutter module. The invention relates to a design of planting vegetables by a groove type facility, adopts a rhizome cutting and harvesting mode, is difficult to process by utilizing a gray characteristic value aiming at a leaf vegetable cutting image, can accurately detect and position a target, has good performance of the whole cutting system, is moderate in length retention of a cutting stem of the vegetables in the harvesting process, is orderly cut, and can assist in orderly storage of the leaf vegetables.
Description
Technical Field
The invention relates to the technical field of vegetable planting, in particular to a leaf vegetable cutting system based on an embedded visual platform.
Background
According to data of agricultural parts, the development trend of facility vegetable industry in China is good, the planting area is increased year by year, and the planting area is estimated to reach about 6158 ten thousand mu in 2020. The variety of green vegetables cultivated in China is various, the shapes of different kinds of leaf vegetables are different, the great difference exists in the aggregate shapes and physical properties such as cultivation modes, planting densities and the like, and the harvest is basically finished manually. However, with the shortage of agricultural labor force, the labor cost is gradually increased, and the research on the full-automatic leaf vegetable production and harvesting equipment is gradually a hotspot at present. In recent years, the research on mechanized leaf vegetable harvesters in China is more, and the mechanized leaf vegetable harvesters comprise small intelligent leaf vegetable harvesters capable of intelligently adjusting cutting width and cutting stubble height, harvesters suitable for greenhouse vegetables (such as burclovers, bean seedlings and small vegetable seedlings) and flexible orderly clamping and collecting vegetable harvesters, wherein leaf vegetable harvesting is an important part for realizing full-automatic general vegetable harvesting.
The organic substrate is adopted to recycle and cultivate vegetables, which is an important mark applied to vegetable production in the prior art, effectively solves the problems of treatment and utilization of agricultural and forestry byproducts and environmental pollution, and is widely applied to planting technology at present. However, the existing vegetable harvester cannot be well applied to the leaf vegetable scene of the harvest groove type facility planting.
Disclosure of Invention
Based on the technical problems in the background art, the invention provides a leaf vegetable cutting system based on an embedded visual platform and applied to planting leaf vegetables in a trough type facility.
The invention provides a leaf vegetable cutting system based on an embedded visual platform, which comprises a hardware module and a control module, wherein the hardware module comprises a conveying belt, an infrared positioning mechanism, a slider mechanism, a CMOS visual identification component, a cutter module, an adjusting module and a transverse module, the control module comprises a control unit, a power supply circuit, a data acquisition circuit and a cutting control circuit, the data acquisition circuit is electrically connected with the CMOS visual identification component, and the cutting control circuit is electrically connected with the cutter module.
Preferably, the control unit is an STM32H743 processor, the core adopts a 32-bit ARM Cortex-M7, and the CMOS visual identification component is an Ov7725 type low-voltage CMOS image sensor.
The invention also provides a using method of the leaf vegetable cutting system based on the embedded visual platform, which comprises the following steps:
s1: the system is initialized, each module is reset, leaf vegetable boxes needing to be harvested are conveyed to a cutting station through a conveying belt, and a sliding block mechanism is ejected out from the oblique lower side of each leaf vegetable box and clamps the two sides of leaf vegetables in the leaf vegetable boxes, so that leaves of the leaf vegetables are gathered towards the middle to expose roots and stems of the leaf vegetables;
s2: the CMOS vision recognition component carries out root and stem position algorithm recognition on the leaf vegetables, and the height of the cutter module is adjusted through the adjusting module according to the recognized position to respectively cut the leaf vegetables in the leaf vegetable box;
s3: after the cutting of the whole box of leaf vegetables is finished, the mechanism resets, the vegetable box flows to the lower station, and the next box of leaf vegetables to be collected is conveyed to the cutting station continuously.
Preferably, during the working process of the CMOS visual recognition component, the camera is first started, calibrated, in order to obtain accurate color restoration of the image, white balance of the camera is closed, a leaf threshold is set, a region where a root of the leaf vegetable is located is detected based on the set threshold, the identified region is framed in the ROI, the framed ROI image is further binarized, all pixels of the threshold image in the threshold function are changed into white by an Otsu algorithm binarization processing method, all pixels outside the threshold are changed into black, the binarized image is then eroded and expanded by an erosion function, namely, a size, and an expansion function, namely, a size, which is used for removing unnecessary points at the adjacent position of the edge, and a size, which is used for setting the number of the adjacent points to be removed, of white miscellaneous points and edge expansion near the edge of the image changes with the size of the threshold function, and calculating the ratio of the image distance to the actual distance of the cutting knife through a distance measurement algorithm, transmitting the cutting position to a lower computer through a serial port, and adjusting the position of the cutting knife to cut.
The invention has the beneficial effects that:
the method is specially designed for planting vegetables in a groove type facility, adopts a rhizome cutting and harvesting mode, aims at the problem that a leaf vegetable cutting image is difficult to process by utilizing a gray characteristic value, adopts a single-channel image synthesized by color difference components of RGB color space to carry out Otsu self-adaptive threshold value cutting, and can quickly and effectively cut the cutting image.
Drawings
Fig. 1 is a schematic diagram of a hardware module structure of a leaf vegetable cutting system based on an embedded visual platform according to the present invention;
FIG. 2 is a hardware module block diagram of a leaf vegetable cutting system based on an embedded visual platform provided by the invention;
FIG. 3 is a block diagram of a CMOS vision recognition component of a leaf vegetable cutting system based on an embedded vision platform according to the present invention;
FIG. 4 is a hardware module block diagram of a leaf vegetable cutting system based on an embedded visual platform according to the present invention;
FIG. 5 is a flow chart of visual identification of a leaf vegetable cutting system based on an embedded visual platform according to the present invention;
FIG. 6 is a preliminary green vegetable identification diagram in a test of the embedded visual platform-based leaf vegetable cutting system provided by the invention;
FIG. 7 is a leaf vegetable binarization graph in the experiment of the leaf vegetable cutting system based on the embedded visual platform provided by the invention;
FIG. 8 is a graph of the corrosion expansion operation of a leaf vegetable binary image in a test of the leaf vegetable cutting system based on the embedded visual platform;
fig. 9 is a diagram of a reference object in an experiment of the leaf vegetable cutting system based on the embedded visual platform.
In the figure: the device comprises a sliding block mechanism 1, a leaf vegetable box 2, a cutter 3, a CMOS (complementary metal oxide semiconductor) visual identification component 4, a cutter module 5, an adjusting module 6, a transverse module 7, a positioning mechanism 8 and a conveying belt 9.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
Example 1, referring to fig. 1 to 9, a sample for measuring the maturity of leaf vegetables is taken from an experimental base of agricultural academy of sciences, the base adopts a matrix reuse technology to plant the leaf vegetables, the width of a planting matrix groove is 15cm, the length of the planting matrix groove is 32cm, the planting distance of the leaf vegetables is 4cm, 8 leaf vegetables are planted in one matrix groove, and the harvested varieties mainly comprise green vegetables, endive, lettuce and the like.
The test is carried out in experiments of agricultural facilities and equipment research institutes of agricultural academy in 2019, 8, 25 and mainly used for testing the performance of a leaf vegetable cutting control system, the leaf vegetables in the experiment are mainly green vegetables, the height of the mature green vegetables is about 3.5cm, and 5 leaves of one plant are scattered all around on average.
The test procedure was as follows:
since the image generates a great amount of noise during the acquisition or transmission, the image is pre-processed during the acquisition and collection, and the resolution of the image is tested at 160px by 120 px.
Firstly, calibrating a camera, closing white balance, setting a rhizome threshold value of the green vegetables, detecting the region where the green vegetables are located, and preliminarily locating the identified green vegetables in the frame in the ROI region, as shown in fig. 6 (b).
And secondly, performing binarization image processing, and performing binarization processing through an Otsu algorithm, wherein the method is also called as a maximum inter-class variance method or an Otsu algorithm, the maximum inter-class variance between the image background and the identification target is subjected to automatic threshold segmentation, the larger the inter-class variance between the image background and the identification target is, the larger the difference between the image background and the identification target is, the smaller the difference between the partial image background and the identification target is, when the partial image background or the partial identification target is mistakenly divided into the background, the smallest probability of the mistaken division of the binarization processing through the Otsu algorithm is, and the target can be accurately identified.
The leaf vegetables identified based on the set threshold value are subjected to binarization processing, and as shown in fig. 7(a), the threshold value is further adjusted to segment the positions of the roots and stems of the vegetables as shown in fig. 7 (b).
Further, the image is subjected to corrosion expansion, the image is expanded and then corroded, the boundary of the target image can be effectively distinguished, the binary image is corroded, noise can be eliminated, independent graphic elements can be effectively segmented, the limit region of the root and stem of the green vegetable in the image is identified, and a root and stem expansion corrosion operation diagram of the green vegetable is shown in fig. 8.
And finally, calculating the cutting distance of the cutting knife, and calculating the cutting execution height of the cutting knife according to the relative position of the cutting knife and the vegetable rhizome, so that the relation between the actual execution height of the cutting knife and the height of the cutting knife to be adjusted on the image needs to be calculated, the test identifies the distance of the vegetable rhizome and the fixed position of the planted vegetable in the leaf vegetable box, so that the distance can be measured in the form of a reference object, and the actual distance of the cutting knife is measured according to the size proportional relation of the reference object.
According to the position of the green vegetable in the leaf vegetable box, a reference object is put at the same position, and the ratio of the fixed distance from the camera to the reference object to the actual distance in the image can be obtained according to the geometric relation of the reference object in the camera, such as the following formula (formula 1)
The ratio of the actual size to the length of the reference object can be obtained from the geometric relationship of the reference object in the real environment, such as the following formula (formula 2)
Substituting equation 2 into equation 1 can obtain that the actual length is inversely proportional to the pixel of the camera, and usingCalculating the actual distance and the mapAnd obtaining the actual distance of the cutting knife by the ratio relation of the upper distances, and arranging a circular reference object as shown in figure 9.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.
Claims (4)
1. Leaf dish cutting system based on embedded vision platform, including hardware module and control module, its characterized in that, the hardware module includes conveyer belt (9), infrared positioning mechanism (8), slider mechanism (1), CMOS vision identification subassembly (4), cutter module (5), adjustment module (6) and horizontal module (7), control module includes the control unit, supply circuit, data acquisition circuit and control cutting circuit, and data acquisition circuit and CMOS vision identification subassembly (4) electric connection, control cutting circuit and cutter module (5) electric connection.
2. The leaf vegetable cutting system based on the embedded vision platform as claimed in claim 1, wherein the control unit is an STM32H743 processor, a core adopts a 32-bit ARM Cortex-M7, and the CMOS vision recognition component (4) is an Ov7725 type low-voltage CMOS image sensor.
3. The use method of the leaf vegetable cutting system based on the embedded visual platform is characterized by comprising the following steps:
s1: the system is initialized, each module is reset, the leaf vegetable box (2) to be harvested is conveyed to a cutting station through a conveying belt (9), and the sliding block mechanism (1) is ejected out from the oblique lower side of the leaf vegetable box (2) and clamps the two sides of leaf vegetables in the leaf vegetable box (2), so that the leaves of the leaf vegetables are gathered towards the middle to expose the roots and stems of the leaf vegetables;
s2: the CMOS vision recognition component (4) performs root and stem position algorithm recognition on the leaf vegetables, adjusts the height of the cutter module (5) through the adjusting module (6) according to the recognized position, and cuts the leaf vegetables in the leaf vegetable box (2) respectively;
s3: after the cutting of the whole leaf vegetables is finished, the mechanism is reset, the vegetable box (2) flows to the next station, and the next leaf vegetable box (2) to be cut is continuously conveyed to the cutting station.
4. The use method of the leaf cutting system based on the embedded visual platform as claimed in claim 3, wherein during the operation of the CMOS visual recognition component (4), the camera is firstly started, calibrated, the white balance of the camera is closed in order to obtain accurate color restoration of the image, the threshold value of the leaf vegetable is set, the region where the root and the stem of the leaf vegetable are located is detected based on the set threshold value, the identified region is framed in the ROI, the framed ROI image is further binarized, all pixels of the threshold value image in the threshold value function are changed into white by the binarization processing method of the Otsu algorithm, all pixels outside the threshold value are changed into black, the binarized image is corroded by the corrosion function, threshold and the expansion function, and the expansion cutter is used for determining the target region, wherein the size is the redundant point at the adjacent position of the removed edge, the threshold function threshold is used for setting the number of adjacent points to be removed, white miscellaneous points near the edge of an image and the edge expansion change along with the threshold function, the ratio of the image distance of the cutting knife (3) to the actual distance is calculated through a distance measurement algorithm, the cutting position is transmitted to a lower computer through a serial port, and the cutting knife (3) adjusts the position to cut.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910995213.5A CN110612813A (en) | 2019-10-18 | 2019-10-18 | Leaf vegetable cutting system based on embedded visual platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910995213.5A CN110612813A (en) | 2019-10-18 | 2019-10-18 | Leaf vegetable cutting system based on embedded visual platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110612813A true CN110612813A (en) | 2019-12-27 |
Family
ID=68926196
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910995213.5A Pending CN110612813A (en) | 2019-10-18 | 2019-10-18 | Leaf vegetable cutting system based on embedded visual platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110612813A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111617983A (en) * | 2020-05-16 | 2020-09-04 | 安徽昭日农业科技有限公司 | Vegetable sorting device |
| CN112425356A (en) * | 2020-11-11 | 2021-03-02 | 沈阳农业大学 | Control circuit of electric leaf vegetable harvester |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH08322408A (en) * | 1995-05-31 | 1996-12-10 | Kyushu Electric Power Co Inc | Treatment of lower leaf of leaf vegetable and apparatus therefor |
| CN104303699A (en) * | 2014-11-14 | 2015-01-28 | 山东农业大学 | Harvesting device and method of potted leaf vegetables |
| CN107135736A (en) * | 2017-05-21 | 2017-09-08 | 西北农林科技大学 | It is a kind of to be colonized the recovering device that plate cultivates vegetables |
| CN108453739A (en) * | 2018-04-04 | 2018-08-28 | 北京航空航天大学 | Stereoscopic vision positioning mechanical arm grasping system and method based on automatic shape fitting |
| CN108575258A (en) * | 2018-03-06 | 2018-09-28 | 王振轩 | A kind of tuberous root crops intelligence cutting method and device |
| CN108848889A (en) * | 2018-06-22 | 2018-11-23 | 江苏农林职业技术学院 | A kind of leaf vegetables harvest machinery device and its control method based on image recognition |
| CN109863874A (en) * | 2019-01-30 | 2019-06-11 | 深圳大学 | A kind of fruit and vegetable picking method, picker and storage medium based on machine vision |
| CN110140513A (en) * | 2019-04-25 | 2019-08-20 | 北京城农科工科技发展有限公司 | A kind of planting groove field planting vegetables autonomous harvesting machine |
| CN209299744U (en) * | 2018-11-21 | 2019-08-27 | 重庆宏双达机械工程有限公司 | A kind of cutting mechanism based on leaf vegetables cutting |
-
2019
- 2019-10-18 CN CN201910995213.5A patent/CN110612813A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH08322408A (en) * | 1995-05-31 | 1996-12-10 | Kyushu Electric Power Co Inc | Treatment of lower leaf of leaf vegetable and apparatus therefor |
| CN104303699A (en) * | 2014-11-14 | 2015-01-28 | 山东农业大学 | Harvesting device and method of potted leaf vegetables |
| CN107135736A (en) * | 2017-05-21 | 2017-09-08 | 西北农林科技大学 | It is a kind of to be colonized the recovering device that plate cultivates vegetables |
| CN108575258A (en) * | 2018-03-06 | 2018-09-28 | 王振轩 | A kind of tuberous root crops intelligence cutting method and device |
| CN108453739A (en) * | 2018-04-04 | 2018-08-28 | 北京航空航天大学 | Stereoscopic vision positioning mechanical arm grasping system and method based on automatic shape fitting |
| CN108848889A (en) * | 2018-06-22 | 2018-11-23 | 江苏农林职业技术学院 | A kind of leaf vegetables harvest machinery device and its control method based on image recognition |
| CN209299744U (en) * | 2018-11-21 | 2019-08-27 | 重庆宏双达机械工程有限公司 | A kind of cutting mechanism based on leaf vegetables cutting |
| CN109863874A (en) * | 2019-01-30 | 2019-06-11 | 深圳大学 | A kind of fruit and vegetable picking method, picker and storage medium based on machine vision |
| CN110140513A (en) * | 2019-04-25 | 2019-08-20 | 北京城农科工科技发展有限公司 | A kind of planting groove field planting vegetables autonomous harvesting machine |
Non-Patent Citations (2)
| Title |
|---|
| 汪懋华等: "《现代精细农业理论与实践》", 31 October 2012 * |
| 王财盛等: "基于机器视觉的采茶机割刀控制方法", 《计算机测量与控制》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111617983A (en) * | 2020-05-16 | 2020-09-04 | 安徽昭日农业科技有限公司 | Vegetable sorting device |
| CN112425356A (en) * | 2020-11-11 | 2021-03-02 | 沈阳农业大学 | Control circuit of electric leaf vegetable harvester |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yu et al. | Automatic image-based detection technology for two critical growth stages of maize: Emergence and three-leaf stage | |
| Stajnko et al. | Estimation of number and diameter of apple fruits in an orchard during the growing season by thermal imaging | |
| McCarthy et al. | Applied machine vision of plants: a review with implications for field deployment in automated farming operations | |
| CN100416590C (en) | Method for automatically identifying field weeds in crop seeding-stage using site and grain characteristic | |
| CN110163138B (en) | Method for measuring and calculating wheat tillering density based on multispectral remote sensing image of unmanned aerial vehicle | |
| CN109447945B (en) | Quick counting method for basic wheat seedlings based on machine vision and graphic processing | |
| CN201600330U (en) | System for recognizing and locating mature pineapples | |
| CN109255757B (en) | Method for segmenting fruit stem region of grape bunch naturally placed by machine vision | |
| BR102014027364A2 (en) | cutting height control and measurement system for a base harvester, method, and harvester | |
| CN102495005A (en) | Method for diagnosing crop water deficit through hyperspectral image technology | |
| Howarth et al. | Measurement of seedling growth rate by machine vision | |
| CN110612813A (en) | Leaf vegetable cutting system based on embedded visual platform | |
| CN103278503A (en) | Multi-sensor technology-based grape water stress diagnosis method and system therefor | |
| CN113469112B (en) | Crop growth condition image identification method and system | |
| CN102663396B (en) | Method for automatically detecting rice milky ripe stage | |
| CN113920106A (en) | Corn growth three-dimensional reconstruction and stem thickness measurement method based on RGB-D camera | |
| CN112734083A (en) | Rice harvester path planning control system based on machine vision | |
| KR20190069648A (en) | Method for measuring growth amount by image analyzing ginseng | |
| CN115950838A (en) | Summer corn drought unmanned aerial vehicle rapid monitoring and distinguishing method based on chlorophyll content | |
| CN110741790B (en) | Multi-claw transplanting-sorting processing method for plug seedlings based on depth camera | |
| He et al. | Visual detection of rice rows based on Bayesian decision theory and robust regression least squares method | |
| CN107993243B (en) | Wheat tillering number automatic detection method based on RGB image | |
| Chen et al. | An automatic plant growth measurement system for plant factory | |
| CN111886982B (en) | Detection method of dry land planting operation quality real-time detection system | |
| CN114937078A (en) | Automatic weeding method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191227 |