CN112528967B - Image recognition-based difference material monitoring method and device and electronic equipment - Google Patents
Image recognition-based difference material monitoring method and device and electronic equipment Download PDFInfo
- Publication number
- CN112528967B CN112528967B CN202110173232.7A CN202110173232A CN112528967B CN 112528967 B CN112528967 B CN 112528967B CN 202110173232 A CN202110173232 A CN 202110173232A CN 112528967 B CN112528967 B CN 112528967B
- Authority
- CN
- China
- Prior art keywords
- image
- incoming material
- cargo
- information
- incoming
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention relates to the technical field of truck loading material conveying, and discloses a method and a device for monitoring different materials based on image recognition and electronic equipment. In the invention, a color image and three-dimensional scanning data of an incoming material are obtained; determining a region image of the incoming material based on the color image and the three-dimensional scanning data of the incoming material; searching a first database based on the area image of the incoming material to obtain a first image set; determining first information of the incoming material based on the three-dimensional scanning data of the incoming material and the first image set; an output location of the incoming material is determined based on the first information. The monitoring method provided by the invention can intelligently identify the differentiated materials on the conveying belt, and reduce the operation amount of incoming material identification while improving the identification speed and accuracy; on the basis of intelligent identification, the output position of differentiated materials is determined in a self-adaptive mode, the loading efficiency is improved, the normal operation of automatic loading is guaranteed, and dependence on manpower is eliminated.
Description
Technical Field
The embodiment of the invention relates to the technical field of loading material conveying, in particular to a method and a device for monitoring different materials based on image recognition and electronic equipment.
Background
The inventor finds that at least the following problems exist in the prior art: in the material loading process, along with the development of industrial automation technology, the condition that different specifications and sizes and different shapes of differential materials are loaded in the same carriage gradually appears, the differential material loading generally uses a conveying belt to convey goods to a loading position, and the automatic loading robot carries out stacking in the carriage. However, when the automatic loading robot is used for loading, the order of incoming materials of different materials is a stacking type planned by an offline mode, the existing automatic loading technology can only directly receive incoming materials and stack the incoming materials to a preset position, and the incoming materials cannot be intelligently judged and adaptively processed, when the incoming materials are different from preset goods information during loading of the different materials, the sizes and the shapes of the materials are different from the sizes and the shape information of goods to be stacked at the preset position, so that the stacking type of the goods actually stacked in a carriage is different from the planning stacking type, the phenomenon of collapse of the goods is easy to occur, the automatic loading fails, on the other hand, the information of the goods is changed and cannot be matched with the planning stacking type, and the goods information in the loading, unloading and management processes cannot correspond. On the basis of offline planning stack type, when the incoming material is foreign matter, the existing automatic loading technology still needs to rely on a manual processing mode to remove the material, so that the normal operation of automatic loading is ensured, and the dependence on manual work cannot be completely eliminated.
Disclosure of Invention
The embodiment of the invention aims to provide a method and a device for monitoring differential materials based on image recognition and electronic equipment, aiming at the technical problems that in the prior art, incoming materials cannot be recognized and processed in a self-adaptive mode on the basis of offline stack type planning, so that automatic loading cannot be separated from manual work and failure is prone to happening.
In order to solve the technical problem, an embodiment of the present invention provides a method for monitoring a different material based on image recognition, including the following steps:
step S1, acquiring color images and three-dimensional scanning data of incoming materials;
step S2, determining the area image of the incoming material based on the color image of the incoming material and the three-dimensional scanning data;
step S3, searching a first database based on the area image of the incoming material to obtain a first image set;
step S4, determining first information of the incoming material based on the three-dimensional scanning data of the incoming material and the first image set;
and step S5, determining the output position of the incoming material based on the first information, wherein the output position comprises a sorting position and a buffering position.
Preferably, the step S2 specifically includes:
step S21, identifying a first area image of the incoming material based on the color image of the incoming material;
step S22, calculating the recognition accuracy of the first region image;
step S23, if the identification accuracy is greater than or equal to a first accuracy threshold, taking the first area image as the area image of the incoming material;
step S24, if the recognition accuracy is smaller than the first accuracy threshold, modifying the first area image according to the three-dimensional scanning data, and obtaining an area image of the incoming material.
Preferably, a first threshold is preset, and the first accuracy threshold is obtained by modifying the first threshold according to the information of the cargo to be loaded by the vehicle.
Preferably, the step S3 specifically includes:
step S31, obtaining the first time according to the pile shape
Individual cargo information, second
The individual cargo information includes the name of the cargo, the size of the cargo, the orientation of the cargo,
，
total number of items in the stack;
step S32, according to the second step
Extracting the information of each cargo from the template library
A cargo image of the individual cargo;
step S33, dividing the first side of the stack according to the shape of the stack
Instantiation of a cargo image of an individual cargo;
in step S34, if
Then completing the creation of the first data subset; if not, then,
returning to step S31;
and matching the area image of the incoming material with each cargo image in the first data subset, and forming the first image set by the cargo images with the matching degree larger than a preset value.
Preferably, the step S4 specifically includes:
determining boundary information of incoming materials based on the three-dimensional scanning data of the incoming materials, calculating the sizes of the incoming materials, searching goods with the same size from the first image set to serve as target images of the incoming materials, and taking second attribute information of the target images as the first information of the incoming materials, wherein the second attribute information at least comprises goods sequence numbers.
Preferably, the step S5 specifically includes:
step S51, determining standard information of the current incoming material from the stack shape, wherein the standard information at least comprises a first number;
step S52, calculating the difference between the first number and the cargo sequence number;
step S53, determining the remaining number of the goods in the arranging unit, wherein the remaining number of the goods refers to the number of the goods which are not received in the current arranging period;
step S54, if the absolute value of the difference is larger than the quantity of the remaining goods, determining that the output position of the incoming material is a cache position;
step S55, if the absolute value of the difference is less than or equal to the residual goods quantity, determining the output position of the incoming material as a sorting position.
Preferably, the method for monitoring the difference materials based on the image recognition further comprises:
step S6, after the output position of the supplied material is determined, determining the standard information of the next supplied material from the stack shape, and updating the residual goods number of the arranging unit;
step S7, judging whether the cache position has materials;
and step S8, if yes, judging whether the materials in the cache position meet the condition of being output to the sorting position or not based on the first number of the standard information of the next incoming material, the sequence number of the materials in the cache position and the updated number of the remaining goods in the sorting unit, if yes, outputting the materials in the cache position, and if not, returning to the step S1 to continue monitoring the latest incoming material.
Preferably, the method for monitoring the difference materials based on the image recognition further comprises:
the output position further comprises a rejection position, and if the matching degree of the area image of the incoming material and each goods image of the first data subset is smaller than or equal to a preset value, the output position of the incoming material is determined to be the rejection position.
The embodiment of the invention also provides a device for monitoring the difference materials based on image recognition, which comprises:
the data acquisition module is used for acquiring a color image and three-dimensional scanning data of an incoming material;
a region determination module for determining a region image of the incoming material based on the color image and the three-dimensional scan data of the incoming material;
the searching module is used for searching a first database based on the regional image of the incoming material to obtain a first image set;
an information determination module to determine first information of the incoming material based on the three-dimensional scan data of the incoming material and the first image set;
and the output module is used for determining the output position of the incoming material based on the first information, and the output position comprises a sorting position and a cache position.
The embodiment of the invention also provides the image recognition-based electronic equipment for monitoring the differential materials, which is characterized by comprising one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the foregoing methods.
Compared with the prior art, the method and the device have the advantages that incoming materials on the conveying belt are intelligently identified and adaptively processed, various incoming material conditions are automatically identified and processed comprehensively, the accuracy of differential material conveying is improved, and the success rate of differential material loading is improved; the method has the advantages that the sequence number information of incoming materials is intelligently identified based on the color images and the three-dimensional scanning data of the incoming materials, the regional images are extracted adaptively and accurately, the interference of the surrounding environment and a conveying belt to goods in the subsequent searching and matching process is avoided, the matching accuracy of the incoming materials is improved, the number of pixel points needing to be subjected to matching operation is reduced, the operation amount of searching and matching is reduced, whether the three-dimensional data are introduced to correct the regional images or not can be determined adaptively through the judgment of the identification accuracy, and the acquisition speed of the regional images and the accuracy of the regional images are considered; when the output position is determined based on the sequence number, the differentiated material monitoring method on the conveying belt improves the loading efficiency by arranging the after-loading mode, does not frequently trigger the link of correcting the output position of the incoming material, improves the loading efficiency and reduces the frequency of adjusting the output position.
The foregoing description is only an overview of the technical solutions of the present invention, and the embodiments of the present invention are described below in order to make the technical means of the present invention more clearly understood and to make the above and other objects, features, and advantages of the present invention more clearly understandable.
Drawings
One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.
Fig. 1 is a schematic diagram of a method for monitoring a dissimilar material based on image recognition according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that numerous technical details are set forth in order to provide a better understanding of the present application in various embodiments of the present invention. However, the technical solution claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments. The following embodiments are divided for convenience of description, and should not constitute any limitation to the specific implementation manner of the present invention, and the embodiments may be mutually incorporated and referred to without contradiction.
The first embodiment of the invention relates to a method for monitoring different materials based on image recognition, the flow of which is shown in fig. 1 and specifically comprises the following steps:
step S1, acquiring color images and three-dimensional scanning data of incoming materials;
the invention is applied to the field of differential material conveying and loading, incoming materials on a conveying belt have different sizes, shapes and types, and a color image camera and a three-dimensional scanning device are arranged on the conveying belt to acquire color images and three-dimensional scanning data of the incoming materials transmitted on the conveying belt in real time.
Step S2, determining the area image of the incoming material based on the color image and the three-dimensional scanning data of the incoming material;
the area image of the incoming material is an area image which only contains the material and does not contain an environmental background, and the area image of the incoming material is determined by comprehensively utilizing the color image and the three-dimensional scanning data. When the goods of conveyer belt are discerned, the regional image of coming material is drawed from the colour image and is carried out intelligent recognition, has avoided surrounding environment, conveyer belt to the search matching process after to produce the interference to the goods, has improved the degree of accuracy that the coming material matches, when the image of gathering in real time and image match search in the database simultaneously, has reduced the pixel quantity that needs carry out the matching operation, has reduced the operand that searches for the matching.
Step S3, searching a first database based on the area image of the incoming material to obtain a first image set;
the first database is pre-stored with a template library of various goods image information, the goods image information in the template library is drawn by a user in advance or imported from the outside, the goods image can be extracted from pictures shot by the user and pictures in a network as a template, the image information of each goods is a template, the image information of each goods carries first attribute information, the first attribute information comprises goods names and goods sizes for system calling, the existing or the goods image drawn by the user is used as the template library, when the goods image corresponding to the current vehicle is determined, the user does not need to make image information for each kind of goods in real time, the image information is called from the template library only according to the goods names and sizes, the workload of workers is reduced, after the goods to be loaded of the vehicle are determined, the goods images of all goods can be automatically determined, the automatic loading method has the advantages that manual drawing is not needed, professional system operation skills such as drawing, photographing and intercepting are not needed, workers do not need to be involved in the intermediate process, images of all goods corresponding to the current vehicle can be directly obtained, and the dependence of the automatic loading method on professional technicians is reduced.
When the offline stack type planning of the current vehicle is completed, determining all goods and the sequence of the goods corresponding to the current vehicle, taking the goods images of all the goods corresponding to the vehicle as a first data subset in a first database, calling the goods images from a template library to form the first data subset, wherein each goods image in the first data subset has second attribute information, the second attribute information comprises a goods name, a goods size and a goods sequence number, matching the region image obtained in the step S2 with the goods images in the first data subset to obtain a first image set, wherein the first image set comprises the goods images with the matching degree larger than a first preset value, and each goods image has second attribute information. After the offline stack type planning of the vehicle is completed, the information and the sequence of the goods are determined, the vehicle is taken as a unit, a first data subset is constructed, when the images are matched and searched, the images only need to be compared with partial images in a database, the matched data range is primarily screened, the operation amount in the matching process is reduced, meanwhile, the attribute information of the goods sequence number is added when the first data subset is constructed, the goods images in a template library are instantiated according to the offline stack type planning, and an information basis is provided for the following incoming material self-adaptive processing step.
Step S4, determining first information of the incoming material based on the three-dimensional scanning data of the incoming material and the first image set;
determining boundary information of the incoming material based on the three-dimensional scanning data of the incoming material, calculating the size of the incoming material, searching goods with the same size from the first image set to be used as a target image of the incoming material, and using second attribute information of the target image as first information of the incoming material. When the images are matched, goods with the same shape may have different size information, the method calculates the real size of the incoming material according to the accurate three-dimensional scanning data, further searches and determines a target image from the first image set based on the size, obtains the actual size of the incoming material through the three-dimensional data, can avoid obtaining the inaccurate size from the color image by adopting complex segmentation methods such as a foreground and a background in the prior art, simultaneously avoids registering and comparing the incoming material in the color image with the size of the goods image in the database, simplifies the operation of size matching, and improves the accuracy of size information.
And step S5, determining the output position of the incoming material based on the first information.
The first information comprises a cargo sequence number, the cargo sequence number is compared with cargo information corresponding to the current loading step, and the output position of the incoming material is determined and comprises a sorting position and a cache position. Compared with the method for directly stacking the incoming materials in the prior art, the method improves the fault tolerance of the automatic loading method, and can adaptively process the situation of improper incoming material sequence.
In summary, the present embodiment provides a method for monitoring a different material based on image recognition, which utilizes a color image and three-dimensional scanning data to recognize information of incoming materials, then adaptively determines an output position of the incoming materials according to the information of the incoming materials, intelligently recognizes the incoming materials according to the information acquired in real time, and performs adaptive processing on the incoming materials, so as to solve the technical problem in the prior art that loading of the differentiated material is prone to failure due to receiving and stacking of the incoming materials only, integrate the color image and the three-dimensional scanning data during incoming material recognition, compared with object recognition based on the color image in the prior art, the color image is used for image matching calculation, the accurate three-dimensional scanning data is used for extracting parameter information of goods, and avoid using a complex image processing technology to process the color image so as to obtain inaccurate parameter information affected by factors such as background, color image quality, and the like, the calculation amount for obtaining the incoming material parameter information is reduced, and meanwhile, the accuracy of the incoming material parameter information is improved.
The second embodiment of the present invention relates to a method for monitoring a different material based on image recognition, which is the same as the first embodiment, and is not repeated herein.
The embodiment two provides a method for monitoring the difference materials based on image recognition, which comprises the following steps:
step S1, acquiring color images and three-dimensional scanning data of incoming materials;
the method is characterized in that the vehicle is used for mixing and loading various different cargos to achieve the purposes of maximum space utilization rate and maximum quantity of loaded cargos, the conveyor belt correspondingly conveys differentiated materials, the sizes, the shapes and the types of the different materials are different, if the sequence of actual incoming materials does not accord with the sequence of the stack type planning, the stack type is directly unstable, automatic loading fails, in order to improve the fault tolerance of the automatic loading method, the method comprehensively utilizes color image information and three-dimensional scanning data to identify images, and when the cargos exist on the conveyor belt, the color image and the three-dimensional scanning data of the incoming materials on the conveyor belt are obtained. Furthermore, a first parameter of the color image is obtained, the first parameter comprises brightness, contrast and saturation, a first quality score of the color image is obtained based on the first parameter, if the first quality score is smaller than a quality score threshold value, a second parameter of the color image is calculated by taking the quality score threshold value as a target quality score, the second parameter is a parameter value combination which enables the quality score of the color image to be equal to the target quality score after each parameter in the first parameter is changed, and the color image is corrected according to the second parameter. In order to ensure the accuracy of incoming material identification, the color image is corrected by taking the mass fraction threshold as a reference line, so that the influence of light, environment and camera parameters on the incoming material identification result is reduced, and the identification accuracy is improved.
For calculating the quality parameters, firstly obtaining a blank image without material passing, and comparing the blank image with the blank image without material to obtain a brightness difference value when the material passes
Color difference of different value
Saturation difference value
Contrast difference value
。
The quality parameter
The calculation formula of (2) is as follows:
step S2, determining the area image of the incoming material based on the color image and the three-dimensional scanning data of the incoming material;
determining an area image of an incoming material from a color image, specifically comprising the following steps:
step S21, identifying a first area image of the incoming material based on the color image of the incoming material;
step S22, calculating the recognition accuracy of the first region image;
step S23, if the identification accuracy is more than or equal to the first accuracy threshold, the first area image is taken as the area image of the incoming material;
and step S24, if the identification accuracy is smaller than a first accuracy threshold, correcting the first area image according to the three-dimensional scanning data to obtain an area image of the incoming material. The method comprises the steps of firstly registering three-dimensional scanning data with color image data, determining the corresponding relation between the three-dimensional scanning data and the color image data, correcting boundary information of a first area in the color image based on the corresponding relation and the three-dimensional scanning data, and acquiring an area image of an incoming material from the color image based on the corrected boundary information.
According to the precision requirement of image matching search, whether the accurate incoming material area image needs to be obtained or not is selected in a self-adaptive mode, compared with a method for obtaining the incoming material area image only through a color image, the accurate area image of the material can be obtained for a scene needing high-precision material identification through introduction of three-dimensional scanning data, other environment information or lack of partial material image information is avoided being introduced into the area image due to pixel point error identification, the integrity of the area image and the precision of the area image are guaranteed, in addition, for the scene with low material image precision requirement, the three-dimensional scanning data does not need to be introduced, through judgment of the identification precision, whether the three-dimensional data is introduced to correct the area or not can be determined in a self-adaptive mode, and the speed of obtaining the area image and the precision of the area image.
Furthermore, the first accuracy threshold may be determined according to the type of the cargo to be loaded by the current vehicle, the first threshold is preset, the first accuracy threshold is obtained by modifying the first threshold according to the information of the cargo to be loaded by the vehicle, and if the number of the types of the cargo to be loaded by the vehicle is greater than the number of the types of the cargo to be loaded by the vehicle
And size of the goodsIf the shapes are different, reducing the preset proportion on the basis of the first threshold value to be used as a first accuracy threshold value; if the number of the types of the cargos to be loaded by the vehicle is less than or equal to
And the size and shape of the cargo are repeated, the preset proportion is increased on the basis of the first threshold value as a first accuracy threshold value, the first accuracy threshold value is a reference for determining whether to introduce three-dimensional scanning data to correct the regional image, different first accuracy threshold values are set in a self-adaptive manner according to the requirements of different application scenes on the regional image accuracy, compared with the judgment mode of the fixed threshold value in the prior art, the changed threshold value can improve the adaptability of the regional image acquisition method, the operation amount of regional image acquisition is reduced in scenes with larger cargo information difference, a lower threshold value is set so that a small amount of images need to introduce three-dimensional scanning data to obtain regional images of incoming materials, in scenes with smaller cargo information difference, a higher threshold value is set so as to obtain accurate regional images in order to identify the incoming materials in similar images, the accuracy of recognition is improved.
Step S3, searching a first database based on the area image of the incoming material to obtain a first image set;
the method comprises the following steps of obtaining order information corresponding to a current vehicle, finishing off-line stack type planning based on the order information and the vehicle information, and creating a first data subset corresponding to the current vehicle according to the planned stack type, and specifically comprises the following steps:
step S31, obtaining the first time according to the pile shape
Individual cargo information, second
The individual cargo information includes the name of the cargo, the size of the cargo, the orientation of the cargo, and the like.
When the off-line stack type planning is finished, the total number of the current cargos to be loaded on the vehicle and the name, the size, the orientation and the stacking position of each cargo are determined,
，
the total number of items in the stack.
Step S32, according to
Extracting the information of each cargo from the template library
A cargo image of the individual cargo. The images in the template library have first attribute information according to
Searching and matching the information of each cargo from the template library, and extracting
A cargo image of the individual cargo.
Step S33, according to the shape of the pile
A cargo image of the individual cargo is instantiated. Specifically, the first one is extracted from the template library
And after the goods image of each goods, adding goods sequence number information items in the first attribute information according to the stack shape, and obtaining second attribute information after instantiation of the goods image.
In step S34, if
Then completing the creation of the first data subset; if not, then,
returning to step S31.
According to the method, the first data subset can be automatically generated according to the planned stack shape, the cargo image corresponding to the current vehicle does not need to be manually created, manual intervention is not needed in the whole process, the dependence on professional operation technology is reduced, and the cost of the monitoring method is reduced. In addition, goods are matched based on the first data, compared with the prior art that objects are identified by searching in the whole database, the goods matching method and the goods matching system can reduce the data searching range according to the planned stack shape, avoid interference of similar goods images which are irrelevant to the current vehicle to incoming material identification, reduce the data quantity of matching search and improve the accuracy of incoming material identification.
On the basis of the first data subset, the area image of the incoming material is matched with each cargo image in the first data subset in an intelligent fuzzy comparison mode, and the cargo images with the matching degree larger than a preset value form a first image set. According to the invention, only a primary identification range is selected in image matching, and a secondary matching link is added on the basis of image matching.
Step S4, determining first information of the incoming material based on the three-dimensional scanning data of the incoming material and the first image set;
and calculating the actual size of the incoming material based on the three-dimensional scanning data, searching size information which is the same as the actual size information based on the second attribute information of the images in the first image set, taking the matched cargo image as a target image, and taking the second attribute information of the target image as the first information of the incoming material to finish the identification of the incoming material. Compared with the method for realizing object identification by fusing color images and three-dimensional data in the prior art, the method has the advantages that the three-dimensional scanning data can be used as the correction information of the color images on one hand and can be used as the acquisition information of the incoming material parameters on the other hand, and the three-dimensional scanning data is one of matching factors during goods identification, so that the three-dimensional scanning data is fully utilized under double actions, the data operation amount in the incoming material identification process is reduced, and the accuracy of the incoming material identification is improved.
And step S5, determining the output position of the incoming material based on the first information.
And determining an output position of the incoming material based on the planned stack shape and first information, wherein the output position comprises a sorting position and a cache position, and the first information comprises a cargo sequence number.
The output position determination process specifically includes the steps of:
and step S51, determining standard information of the current incoming material from the planned stack shape, wherein the standard information at least comprises a first number.
Step S52, calculating a difference between the first number and the cargo sequence number.
Step S53, determining the remaining quantity of the goods in the sorting unit, where the remaining quantity of the goods refers to the quantity of the goods that have not been received in the current sorting cycle, for example, the sorting unit can receive the goods maximally
The goods are arranged and stacked, and the goods are arranged in the current arranging unit
The remaining quantity of the goods is
，
，
，
、
Is a positive integer. In order to improve the loading efficiencyAccording to the invention, the goods with the preset quantity are arranged together and stacked to the designated position in the carriage, and the quantity of the remaining goods is determined according to the quantity of the goods required to be received in each arrangement period.
Step S54, if the absolute value of the difference is larger than the quantity of the remaining goods, determining the output position of the incoming material as a cache position;
and step S55, if the absolute value of the difference is less than or equal to the number of the remaining goods, determining the output position of the incoming material as a finishing position.
According to the intelligent incoming material identification result of the step S4, the invention judges whether the current incoming material sequence and type are correct, thereby realizing the self-adaptive determination of the output position of the incoming material, improving the scientificity and fault tolerance of differential material conveying, ensuring that the loading robot receives goods according to the planned preset sequence and improving the success rate of loading. Compared with the scheme of only comparing whether the two serial numbers are the same or not, the loading robot collects, arranges and then stacks the incoming materials in a certain number after the incoming materials on the conveying belt are output, so that the loading efficiency of the loading robot is improved, correspondingly, a fault-tolerant mechanism that the output position is corrected when the serial numbers are not completely the same as the first serial numbers is not adopted in the invention, the allowable range of the difference between the serial numbers and the first serial numbers is increased, the link of correcting the incoming material output position cannot be frequently triggered by a differentiated material monitoring method on the conveying belt, the loading efficiency is improved, and the frequency of adjusting the output position is reduced; compared with the mode that the alarm prompt depends on manual processing in the foreign matter identification in the prior art, the goods with the improper incoming material sequence are received by the buffer position, the error of the improper incoming material sequence is automatically processed, and the dependence of the whole automatic loading method on manual work is reduced.
Further, after step S5, the method further includes:
step S6, after the input material output position is determined, determining the standard information of the next input material from the planned stack shape, and updating the residual goods number of the arranging unit;
step S7, judging whether materials exist in the cache position;
and step S8, if yes, judging whether the materials in the cache position meet the condition of being output to the arranging position or not based on the first number of the standard information of the next incoming material, the sequence number of the materials in the cache position and the updated number of the remaining goods in the arranging unit, if yes, outputting the materials in the cache position, and if not, returning to the step S1 to continue monitoring the latest incoming material. If the loading robot outputs the goods in the arranging unit to the carriage, the arranging unit starts to receive and arrange the goods in the next round at the moment, and the updated quantity of the remaining goods is the maximum capacity of the arranging unit
At this time, the material sequence number of the cache position is compared with the first number, and if the absolute value of the difference between the two numbers is less than or equal to
And if so, indicating that the material belongs to the receiving turn of the current sorting unit, and outputting the material to the sorting unit. Compared with the mode of foreign matter monitoring alarm or foreign matter removal in the prior art, the method can judge that the incoming materials are only in improper sequence but still belong to the range of target goods, so that the incoming materials are output according to the correct incoming material sequence, the situation that complete stack shapes cannot be obtained during loading due to the fact that the goods are directly discarded is avoided, all preset goods cannot be loaded, the stack shapes are unstable, an automatic loading and transporting task fails is avoided, and the intelligence of the self-adaptive processing method is improved.
And step S8, if no material exists in the cache position, returning to step S1 to continue monitoring the latest incoming material.
Further, step S3 includes that the output position further includes a rejection position, and if the matching degree between the area image of the incoming material and each cargo image of the first data subset is smaller than or equal to a preset value, it indicates that the current incoming material does not belong to the cargo to be loaded of the vehicle, and the output position of the incoming material is determined to be the rejection position. According to the invention, foreign matters can be further directly removed on the basis of intelligent processing of the materials coming from the wrong sequence, so that the fault tolerance of differential material conveying is improved. And calculating the continuous times of the output positions as the elimination positions, if the continuous times are greater than a continuous threshold value, indicating that the batch of goods is possibly conveyed wrongly, sending prompt information to the monitoring host computer, and prompting monitoring personnel to check the goods condition. The invention adopts a full-automatic method to complete the conveying and loading of goods, avoids the situation that the goods at an entrance are mistakenly input in a whole batch, monitors and alarms the continuous times, timely reminds a monitoring host to check the source of the goods, prevents the cache output position of the differentiated material monitoring method from being fully loaded in a short time, prevents all goods from being rejected, repeatedly judges uselessly for a long time and improves the operation efficiency of the monitoring method.
Step S4 further includes that the output position further includes a rejection position, and if there is no image in the first image set that is the same as the actual size information, the output position of the incoming material is determined based on the first data subset, the incoming material actual size information, and the incoming material region image correction matching result.
Specifically, goods images with the same size are searched in a first data subset based on the actual size information of incoming materials to serve as a second image set, the second image set is matched with the incoming material area images, the goods images with the matching degree higher than a second preset value and the matching degree maximum are used as target images, and identification of the incoming materials is completed; and if the matching degrees are all lower than a second preset value, determining the output position of the incoming material as a rejection position. Compared with the mode of primary discrimination or repeated discrimination in the prior art, the secondary correction discrimination takes the size as a primary screening condition, takes the image matching degree as a target image determining condition, adopts different modes to correct the discrimination result, avoids the matching error of the repeated original discrimination link, can still find the target image through the secondary correction discrimination under the condition of the matching error of the original primary screening link, and improves the accuracy of incoming material discrimination.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
The third embodiment of the present invention relates to a device for monitoring different materials based on image recognition, comprising:
the data acquisition module is used for acquiring a color image and three-dimensional scanning data of an incoming material;
the area determining module is used for determining an area image of the incoming material based on the color image and the three-dimensional scanning data of the incoming material;
the searching module is used for searching a first database based on the area image of the incoming material to obtain a first image set;
the information determining module is used for determining first information of the incoming material based on the three-dimensional scanning data of the incoming material and the first image set;
and the output module is used for determining the output position of the incoming material based on the first information.
The invention provides instructions for executing the method of any one of the first embodiment and the second embodiment.
It should be understood that this embodiment is a system example corresponding to the first embodiment, and may be implemented in cooperation with the first and second embodiments. The related technical details mentioned in the first and second embodiments are still valid in this embodiment, and are not described herein again to reduce repetition. Accordingly, the related-art details mentioned in the present embodiment can also be applied to the first embodiment.
It should be noted that each module referred to in this embodiment is a logical module, and in practical applications, one logical unit may be one physical unit, may be a part of one physical unit, and may be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, elements that are not so closely related to solving the technical problems proposed by the present invention are not introduced in the present embodiment, but this does not indicate that other elements are not present in the present embodiment.
A fourth embodiment of the invention relates to an image recognition-based differential material monitoring electronic device, comprising one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any of embodiments one, two.
Where the memory and processor are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting together one or more of the various circuits of the processor and the memory. The bus may also connect various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor.
The processor is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And the memory may be used to store data used by the processor in performing operations.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples for carrying out the invention, and that various changes in form and details may be made therein without departing from the spirit and scope of the invention in practice.
Claims (8)
1. The method for monitoring the different materials based on the image recognition is characterized by comprising the following steps of:
step S1, acquiring color images and three-dimensional scanning data of incoming materials;
step S2, determining the area image of the incoming material based on the color image of the incoming material and the three-dimensional scanning data;
step S3, searching a first database based on the area image of the incoming material to obtain a first image set;
step S4, determining first information of the incoming material based on the three-dimensional scanning data of the incoming material and the first image set;
step S5, determining the output position of the incoming material based on the first information, wherein the output position comprises a sorting position and a cache position;
step S6, after the output position of the supplied material is determined, determining the standard information of the next supplied material from the stack shape, and updating the residual goods number of the arranging unit;
step S7, judging whether the cache position has materials;
step S8, if yes, judging whether the materials in the cache position meet the condition of being output to the sorting position or not based on the first number of the standard information of the next incoming material, the sequence number of the materials in the cache position and the updated remaining goods number of the sorting unit, if yes, outputting the materials in the cache position, and if not, returning to the step S1 to continue monitoring the latest incoming material;
wherein, step S3 specifically includes:
step S31, obtaining the first time according to the pile shape
Individual cargo information, second
The individual cargo information includes the name of the cargo, the size of the cargo, the orientation of the cargo,
，
total number of items in the stack;
step S32, according to the second step
Extracting the information of each cargo from the template library
A cargo image of the individual cargo;
step S33, dividing the first side of the stack according to the shape of the stack
Instantiation of a cargo image of an individual cargo;
in step S34, if
Then completing the creation of the first data subset; if not, then,
returning to step S31;
and matching the area image of the incoming material with each cargo image in the first data subset, and forming the first image set by the cargo images with the matching degree larger than a preset value.
2. The method for monitoring differential materials based on image recognition according to claim 1, wherein the step S2 specifically includes:
step S21, identifying a first area image of the incoming material based on the color image of the incoming material;
step S22, calculating the recognition accuracy of the first region image;
step S23, if the identification accuracy is greater than or equal to a first accuracy threshold, taking the first area image as the area image of the incoming material;
step S24, if the recognition accuracy is smaller than the first accuracy threshold, modifying the first area image according to the three-dimensional scanning data, and obtaining an area image of the incoming material.
3. The differential material monitoring method based on image recognition according to claim 2,
a first threshold value is preset, and the first threshold value is corrected according to cargo information needing to be loaded by a vehicle to obtain a first accuracy threshold value.
4. The method for monitoring differential materials based on image recognition according to claim 1, wherein the step S4 specifically includes:
determining boundary information of incoming materials based on the three-dimensional scanning data of the incoming materials, calculating the sizes of the incoming materials, searching goods with the same size from the first image set to serve as target images of the incoming materials, and taking second attribute information of the target images as the first information of the incoming materials, wherein the second attribute information at least comprises goods sequence numbers.
5. The method for monitoring different materials based on image recognition according to claim 4, wherein the step S5 specifically comprises:
step S51, determining standard information of the current incoming material from the stack shape, wherein the standard information at least comprises a first number;
step S52, calculating the difference between the first number and the cargo sequence number;
step S53, determining the remaining number of the goods in the arranging unit, wherein the remaining number of the goods refers to the number of the goods which are not received in the current arranging period;
step S54, if the absolute value of the difference is larger than the quantity of the remaining goods, determining that the output position of the incoming material is a cache position;
step S55, if the absolute value of the difference is less than or equal to the residual goods quantity, determining the output position of the incoming material as a sorting position.
6. The image recognition-based differential material monitoring method according to claim 1, further comprising:
the output position further comprises a rejection position, and if the matching degree of the area image of the incoming material and each goods image of the first data subset is smaller than or equal to a preset value, the output position of the incoming material is determined to be the rejection position.
7. A difference material monitoring device based on image recognition is characterized in that the difference material monitoring device based on image recognition comprises:
the data acquisition module is used for acquiring a color image and three-dimensional scanning data of an incoming material;
a region determination module for determining a region image of the incoming material based on the color image and the three-dimensional scan data of the incoming material;
the searching module is used for searching a first database based on the regional image of the incoming material to obtain a first image set;
an information determination module to determine first information of the incoming material based on the three-dimensional scan data of the incoming material and the first image set;
the output module is used for determining the output position of the incoming material based on the first information, and the output position comprises a sorting position and a cache position;
the difference material monitoring device based on image recognition is further used for determining standard information of the next incoming material from the stack shape after determining the output position of the incoming material, and updating the number of the remaining goods in the sorting unit; judging whether materials exist in the cache position or not; if yes, judging whether the materials in the cache position meet the condition of being output to the sorting position or not based on the first number of the standard information of the next incoming material, the sequence number of the materials in the cache position and the updated number of the remaining goods of the sorting unit, if yes, outputting the materials in the cache position, and if not, continuing to monitor the latest incoming material by the data acquisition module;
the searching module is used for executing the following steps:
step S31, obtaining the first time according to the pile shape
Individual cargo information, second
The individual cargo information includes the name of the cargo, the size of the cargo, the orientation of the cargo,
，
total number of items in the stack;
step S32, according to the second step
Extracting the information of each cargo from the template library
A cargo image of the individual cargo;
step S33, dividing the first side of the stack according to the shape of the stack
Instantiation of a cargo image of an individual cargo;
in step S34, if
Then completing the creation of the first data subset; if not, then,
returning to step S31;
and matching the area image of the incoming material with each cargo image in the first data subset, and forming the first image set by the cargo images with the matching degree larger than a preset value.
8. An image recognition based differential material monitoring electronic device, comprising one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing the method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110173232.7A CN112528967B (en) | 2021-02-09 | 2021-02-09 | Image recognition-based difference material monitoring method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110173232.7A CN112528967B (en) | 2021-02-09 | 2021-02-09 | Image recognition-based difference material monitoring method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112528967A CN112528967A (en) | 2021-03-19 |
| CN112528967B true CN112528967B (en) | 2021-04-20 |
Family
ID=74975555
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110173232.7A Active CN112528967B (en) | 2021-02-09 | 2021-02-09 | Image recognition-based difference material monitoring method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112528967B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111626665A (en) * | 2020-05-09 | 2020-09-04 | 武汉中岩科技股份有限公司 | Intelligent logistics system and method based on binocular vision |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI596941B (en) * | 2016-09-19 | 2017-08-21 | Vivotek Inc | Image capturing method and monitoring apparatus with supplemental lighting modulation |
| CN112207047A (en) * | 2019-07-11 | 2021-01-12 | 昆山华复精密金属有限公司 | Automatic detection line |
| CN111798528A (en) * | 2020-07-01 | 2020-10-20 | 西交利物浦大学 | Fabric color difference detection method, device, storage medium and equipment |
-
2021
- 2021-02-09 CN CN202110173232.7A patent/CN112528967B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111626665A (en) * | 2020-05-09 | 2020-09-04 | 武汉中岩科技股份有限公司 | Intelligent logistics system and method based on binocular vision |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112528967A (en) | 2021-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220129701A1 (en) | System and method for detecting objects in a digital image, and system and method for rescoring object detections | |
| CN102119343B (en) | Target detection device and target detection method | |
| CN109754089B (en) | Model training system and method | |
| CN113947688B (en) | Prefabricated part data management system and method based on BIM cloud platform | |
| CN114826885B (en) | Equipment fault monitoring system and method based on data analysis | |
| EP4055454A1 (en) | Systems, methods, and media for manufacturing processes | |
| WO2021017796A1 (en) | Object recognition apparatus, system and method | |
| CN114130673A (en) | Control method, device and equipment of sorting equipment and storage medium | |
| CN112528967B (en) | Image recognition-based difference material monitoring method and device and electronic equipment | |
| CN112418568B (en) | Semi-online stack type planning method and robot | |
| US11704516B2 (en) | Electronic label management apparatus and method | |
| US11922680B2 (en) | Image determination device, image determination method, and non-transitory computer readable medium storing program | |
| CN113298885A (en) | Binocular calibration method and device, equipment and storage medium | |
| CN115963114B (en) | Lithium battery production quality inspection system | |
| CN114800533B (en) | Sorting control method and system for industrial robot | |
| US20190325606A1 (en) | Inspection apparatus | |
| EP4024360A1 (en) | Infrared remote control code matching method and apparatus, computer device, and readable storage medium | |
| CN108957384A (en) | A kind of turnover box and electric energy measuring equipment discriminating direction method and apparatus | |
| CN113643265A (en) | Quality grade determination method, device, equipment and medium based on artificial intelligence | |
| WO2021183468A1 (en) | Systems, methods, and media for manufacturing processes | |
| CN116308065B (en) | Intelligent operation and maintenance management method and system for logistics storage equipment | |
| CN112241975A (en) | Matching positioning method and matching positioning device for feature template | |
| EP3886428A1 (en) | System and edge device | |
| US20230316718A1 (en) | Learning model generating method and inspection device | |
| CN117566173A (en) | Cigarette appearance detection method, controller, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |