WO2023280080A1 - 阴影消除装置和方法、空盘识别装置和方法 - Google Patents
阴影消除装置和方法、空盘识别装置和方法 Download PDFInfo
- Publication number
- WO2023280080A1 WO2023280080A1 PCT/CN2022/103447 CN2022103447W WO2023280080A1 WO 2023280080 A1 WO2023280080 A1 WO 2023280080A1 CN 2022103447 W CN2022103447 W CN 2022103447W WO 2023280080 A1 WO2023280080 A1 WO 2023280080A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- edge
- area
- template
- empty
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 81
- 230000008030 elimination Effects 0.000 title claims abstract description 49
- 238000003379 elimination reaction Methods 0.000 title claims abstract description 49
- 238000003708 edge detection Methods 0.000 claims abstract description 91
- 238000005286 illumination Methods 0.000 claims abstract description 26
- 230000002452 interceptive effect Effects 0.000 claims description 78
- 238000012545 processing Methods 0.000 claims description 41
- 230000008569 process Effects 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 28
- 238000000605 extraction Methods 0.000 description 16
- 238000004422 calculation algorithm Methods 0.000 description 11
- 230000012447 hatching Effects 0.000 description 9
- 238000012546 transfer Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 6
- 238000007689 inspection Methods 0.000 description 6
- 230000003044 adaptive effect Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000007781 pre-processing Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 239000000470 constituent Substances 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 230000000153 supplemental effect Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000001678 irradiating effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000004064 recycling Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000036561 sun exposure Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/94—Dynamic range modification of images or parts thereof based on local image properties, e.g. for local contrast enhancement
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
- G06T2207/20192—Edge enhancement; Edge preservation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30112—Baggage; Luggage; Suitcase
Definitions
- the present application relates to the field of luggage security inspection, in particular, to a shadow eliminating device and method, and an empty disk identifying device and method.
- the inventors of the present application found through investigation and research that at present, there are mainly two technologies that can realize the automatic identification of empty trays.
- One is based on laser ranging technology, which is used to judge whether the pallet is empty or not. In this case, it needs to rely on a dense laser ranging array, which is very expensive and thus has poor practicability.
- the other is a technology based on image recognition to determine whether the tray is empty. In this case, it is easy to mistakenly recognize the distracting media such as advertisements, stickers, pictures, etc., and shadows caused by lighting placed on the tray as luggage. , so that the recognition accuracy is low, and sometimes manual intervention is required to achieve effective recognition, resulting in low automation and low recognition efficiency.
- the present application proposes a shadow-adaptive empty disk identification device and method capable of identifying illuminated shadow areas on a tray, thereby improving the accuracy of empty disk identification.
- an embodiment of the present application provides a shadow elimination device, including: an image acquisition module, which acquires an image with supplementary light and an image without supplementary light; The difference between them is used to obtain the illuminated area image; the edge detection module performs edge detection on the image without supplementary light to obtain the edge image; and the shadow line elimination module removes the part corresponding to the illuminated area image from the edge image to obtain the actual edge image.
- the image with supplementary light is captured by an infrared light emitting device in a state of supplementary light.
- the illuminated area image acquisition module judges that the pixel belongs to the illuminated area image when the ratio of the pixel values of the image with supplementary light and the image without supplementary light is within a predetermined threshold range.
- the edge detection module performs edge detection after performing Gaussian blur processing on the image without supplementary light.
- the shadow line elimination module determines that the area ratio of the illuminated area in the neighborhood of any pixel point of the edge image is within a predetermined area threshold range, the pixel point is removed.
- the shadow line elimination module includes: a neighborhood determination unit, for each pixel in the edge image, determines the neighborhood of the pixel; The area corresponding to the neighborhood; the illuminated area ratio judging part, judging whether the area ratio of the illuminated area in the area is within the predetermined area threshold range; the edge point removal part, eliminating the pixel points whose area ratio is within the predetermined area threshold range; and an actual edge image output unit that generates and outputs an actual edge image based on the first remaining edge points.
- another embodiment of the present application provides a device for identifying an empty disc, including: the above-mentioned shadow eliminating device; Images without fill light were taken with empty trays.
- it also includes: a template matching module, which matches the actual edge image with the interference medium image template; and a corresponding area elimination module, which eliminates edge points corresponding to the interference medium image template in the actual edge image, The second remaining edge point is obtained, and the empty disk judging module uses the second remaining edge point to replace the actual edge image to judge whether it is an empty tray.
- another embodiment of the present application provides an empty disk identification system, including: a light emitting device; When the device is turned on, the empty tray is photographed to obtain an image with supplementary light; the above-mentioned empty disk identification device.
- An embodiment of the present application provides an empty disk identification system, including: a light emitting device; an image acquisition device, which photographs an empty tray when the light emitting device is turned off to obtain an image without supplementary light, and captures the image when the light emitting device is turned on The empty tray is photographed to obtain an image with supplementary light; the above-mentioned empty disk identification device; and the interference medium image template generation device generates an interference medium image template including edge points of the interference medium by performing edge detection on a plurality of chassis images .
- another embodiment of the present application provides a method for eliminating shadows, including: an image acquisition step, acquiring an image with supplementary light and an image without supplementary light; an illumination difference step, based on the difference between the image with supplementary light and the image without supplementary light The difference is to obtain the illuminated area image; the edge detection step is to perform edge detection on the image without supplementary light to obtain the edge image; and the shadow line elimination step is to remove the part corresponding to the illuminated area image from the edge image to obtain the actual edge image.
- the image with supplementary light is obtained by shooting in a state of supplementary light by an infrared light emitting method.
- the illumination difference step when the ratio of the pixel values of the image with supplementary light and the image without supplementary light is within a predetermined threshold range, it is determined that the pixel belongs to the illuminated area image.
- the edge detection is performed after Gaussian blur processing is performed on the image without supplementary light.
- the shadow line elimination module determines that the area ratio of the illuminated area in the neighborhood of any pixel point of the edge image is within a predetermined area threshold range, the pixel point is removed.
- the step of eliminating shadow lines includes: a neighborhood determination step, for each pixel in the edge image, determine the neighborhood of the pixel; The area corresponding to the neighborhood; the illuminated area proportion judgment step, judging whether the area proportion of the illuminated area in the area is within the predetermined area threshold; the edge point removal step, eliminating the pixel points whose area proportion is within the predetermined area threshold; and an actual edge image output step, generating and outputting an actual edge image according to the first remaining edge points.
- another embodiment of the present application provides a method for identifying an empty disk, including: the above-mentioned shadow elimination method; and an empty disk judging step, based on the actual edge image, judging whether it is an empty tray, with a supplementary light image and Images without fill light were taken with empty trays.
- it also includes: a template matching step, matching the actual edge image with the interference medium image template; and a corresponding area elimination step, eliminating edge points corresponding to the interference medium image template in the actual edge image, The second remaining edge point is obtained, and the empty disk judging step replaces the actual edge image, and judges whether it is an empty tray based on the second remaining edge point.
- the method further includes: a generation step of an interference medium image template, which generates an interference medium image template including edge points of the interference medium by performing edge detection on a plurality of chassis images.
- another embodiment of the present application provides a computer-readable storage medium, which stores a program, and the program causes the computer to perform the following processing: the image acquisition step, acquiring an image with supplementary light and an image without supplementary light; the illumination difference step, according to The difference between the image with fill light and the image without fill light is obtained to obtain an image of the illuminated area; the edge detection step is to perform edge detection on the image without fill light to obtain an edge image; and the step of removing shadow lines is to remove from the edge image. part, to obtain the actual edge image.
- the actual edge image that eliminates the edge line produced by the illumination is generated through the image with supplementary light and the image without supplementary light.
- the actual edge image can be used in the identification of empty disks, thereby reducing the impact of illumination on the identification of empty disks .
- FIG. 1 is a schematic diagram showing the configuration of an empty disc recognition system according to the first embodiment
- FIG. 2 is a functional block diagram showing Example 1 of the empty disk identification device 30 according to the first embodiment
- FIG. 3 is a flowchart showing Example 1 of the empty tray identification method according to the first embodiment
- FIG. 4 is a flow chart showing an example of the corresponding area elimination step S34 and the empty tray determination step S35 of the empty tray identification method according to the first embodiment
- FIG. 5 is a flowchart schematically showing an example of Example 1 of the method for identifying an empty pallet according to the first embodiment
- FIG. 6 is a flowchart showing an example of edge detection step S32 of the empty tray recognition method according to the first embodiment
- FIG. 7 is a flowchart showing an example of the template matching step S33 of the empty pallet identification method according to the first embodiment
- FIG. 8 is a functional block diagram showing Example 2 of the empty disk identification device 30 according to the first embodiment
- FIG. 9 is a flowchart showing Example 2 of the empty tray identification method according to the first embodiment.
- FIG. 10 is a flowchart showing an example of the chassis outline extraction step S36 of the empty pallet recognition method according to the first embodiment
- FIG. 11 is a flowchart showing an example of the correcting step S37 of the empty pallet recognition method according to the first embodiment
- FIG. 12 is a functional block diagram showing Example 1 of the interference medium image template generating device 40 according to the second embodiment
- Fig. 13 shows a flow chart of an example of Embodiment 1 of the method for generating an interference medium image template according to the second embodiment
- FIG. 14 is a functional block diagram showing Example 2 of the interference medium image template generating device 40 according to the second embodiment
- Fig. 15 is a flow chart showing Example 2 of the interference medium image template generation method related to the second embodiment
- FIG. 16 is a flow chart showing an example of the final template generation step S42 of the interference medium image template generation method according to the second embodiment
- FIG. 17 shows an initial template
- (b) of FIG. 17 shows a final template
- FIG. 18 is a schematic diagram showing the configuration of an empty disc recognition system according to the second embodiment.
- FIG. 19 is a functional block diagram showing an example of Example 3 of the interference medium image template generation device 40 according to the second embodiment
- FIG. 20 is a flow chart showing Example 3 of the interference medium image template generation method related to the second embodiment
- FIG. 21 is a flow chart showing an example of the running phase updating step S43 of the interference medium image template generation method according to the second embodiment
- 22 is a schematic diagram showing the configuration of an empty disk recognition system according to the third embodiment.
- FIG. 23 is a functional block diagram showing Example 1 of the shadow canceling device 70 according to the third embodiment.
- Example 1 of the shadow removal method according to the third embodiment is a flowchart showing Example 1 of the shadow removal method according to the third embodiment.
- FIG. 25 show a flowchart of an example of the illumination difference step S72 of the shadow removal method according to the third embodiment
- FIG. 26 is a flowchart showing an example of the hatching removing step S74 of the shadow removing method according to the third embodiment
- FIG. 27 show schematic diagrams of edge images before and after the processing of the hatching removal step
- FIG. 28 is a functional block diagram showing Example 2 of the empty disk identification device 30" according to the third embodiment.
- FIG. 29 is a flow chart showing Example 2 of the empty tray identification method according to the third embodiment.
- FIG. 30 is a functional block diagram schematically showing Example 3 of the empty disc recognition system according to the third embodiment.
- the first embodiment of the present application provides an empty disc that can adaptively detect trays with interfering media Identify devices, systems and methods.
- interfering medium refers to, for example, advertisements, stickers, printed pictures on the tray, etc., which can interfere with the identification of empty discs and are different from the tray structure itself.
- Setting the interfering medium means that the interfering medium can be attached to the tray by means of pasting, printing, or the like.
- FIG. 1 is a schematic diagram showing the configuration of an empty disk recognition system according to this embodiment.
- the empty disk identification system 100 can be used in a luggage security inspection scenario.
- the empty disk identification system 100 may include a tray transfer device 10 , an image acquisition device 20 , and an empty disk identification device 30 .
- the tray transfer device 10 may be configured to transfer trays along a first direction X (horizontal direction).
- the tray can be used to hold luggage.
- the image capture device 20 captures images of pallets passing through the image capture area 21 via the pallet transfer device 10. In other words, an image is taken of the pallet passing through the image capture area.
- the image acquisition device 20 may be a camera, more specifically, may be a camera for taking still pictures, or may be a camera for taking videos.
- the image can be either a static image or a moving image.
- FIG. 1 Although only one image acquisition module is shown in FIG. 1 , those skilled in the art should understand that two or more image acquisition modules can be provided.
- the empty disk identification device 30 may be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof.
- the device 30 for identifying an empty disk can be realized by combining software and hardware with any suitable electronic device such as a desktop computer, a tablet computer, a smart phone, or a server equipped with a processor.
- the processor of the empty disk identification device 30 can execute the empty disk identification method described later.
- the empty disk identifying device 30 may also include a memory (not shown), a communication module (not shown), and the like.
- the memory of the empty disc identification device 30 can store the steps for performing the method for identifying an empty disc described later, as well as data related to the identification of an empty tray.
- the memory may be, for example, ROM (Read Only Memory image, read-only memory), RAM (Random Access Memory, random access memory), etc.
- the memory has a storage space for program codes for executing any steps in the following blank disk identification method. When these program codes are read and executed by the processor, the following empty disk identification method is executed.
- These program codes can be read from or written into one or more computer program products.
- These computer program products comprise program code carriers such as hard disks, compact discs (CDs), memory cards or floppy disks. Such computer program products are typically portable or fixed storage units.
- Program codes for performing any of the steps in the following methods can also be downloaded via the network.
- the program code can eg be compressed in a suitable form.
- the communication module in the empty disk identification device 30 may support establishing a direct (eg, wired) communication channel or a wireless communication channel between the empty disk identification device 30 and an external electronic device, and perform communication via the established communication channel.
- the communication module sends instructions to the image capture device 20 via the network, and receives data from the image capture device 20 via the network.
- the empty disk identification device 30 may also include output units such as a display, a microphone, and a speaker, for displaying or broadcasting the status of the tray (that is, whether it is an empty tray or a non-empty tray), or an alarm.
- output units such as a display, a microphone, and a speaker, for displaying or broadcasting the status of the tray (that is, whether it is an empty tray or a non-empty tray), or an alarm.
- the empty disc identification device 30 of the first embodiment can accurately identify an empty tray even when a disturbing medium is installed on the bottom of the tray.
- the empty disc identification system of the present application may not include the tray conveying device 10, for example, the tray may be artificially placed on the image of the image acquisition device 20 Collection area 21.
- FIG. 2 is a functional block diagram schematically showing Example 1 of the empty disk identification device 30 according to the first embodiment.
- These functional modules can be realized by the processor of the empty disk identification device 30 executing the program stored in the memory of the empty disk identification device 30 .
- the empty disc identification device 30 includes: an image acquisition module 31, which acquires the image of the tray from the image acquisition device 20, and the tray is provided with an interfering medium at the bottom; Edge detection, obtaining an edge image; template matching module 33, matching the edge image with the interference medium image template; corresponding area elimination module 34, eliminating edge points corresponding to the interference medium image template in the edge image to obtain remaining edge points; and The empty tray judging module 35 judges whether it is an empty tray based on the remaining edge points.
- FIG. 3 is a flowchart schematically showing Example 1 of the empty tray identification method according to the first embodiment.
- step S31 the image acquisition module 31 acquires an image of the tray provided with an interfering medium at the bottom from the image acquisition device 20 .
- the image may be acquired directly from the image acquisition device 20 , or may be acquired via other external devices communicating with the image acquisition device 20 and the empty disc identification device 30 .
- the captured image is uploaded to the server by the image capture device 20, and the empty disc identification device 30 downloads it from the server.
- step S32 the edge detection module 32 performs edge detection on the chassis area in the image to obtain an edge image.
- the chassis area can be determined from the image captured with the chassis by the existing technology in the art, and the edge detection can be performed on the chassis area using the existing edge detection technology to obtain the edge image.
- step S33 the template matching module 33 matches the edge image with the interference medium image template to find the best matching template.
- the best matching template is the template with the best matching degree to the edge image among all the interference medium image templates.
- the edge image obtained in step S32 may be matched with the interfering medium image template stored in the storage unit (not shown) of the empty disk identification device 30 .
- the interference medium image template includes edge points of the interference medium image, and may also include edge lines composed of some edge points.
- step S34 the corresponding area elimination module 34 eliminates the edge points corresponding to the interference medium image template in the edge image to obtain the remaining edge points num.
- step S35 the empty disc judging module 35 judges whether it is an empty tray based on the remaining edge point num.
- FIG. 4 shows an example of the corresponding area elimination step S34 and the empty disk judgment step S35.
- step S341 the edge points corresponding to the interference medium image template in the edge image are eliminated.
- the edge points are represented by white (pixel value 255), therefore, in step S341, it can be checked whether there is a point with pixel value 255.
- the neighborhood may be a square area with a side length of 2K centered on the pixel point.
- the edge points corresponding to the best matching template can be removed from the edge image.
- step S342 calculate the number of remaining edge points in the edge image after step S341, which is recorded as num.
- step S34 may further include a cropping step, in which the edge image may be cropped to the same size as the interference medium image template.
- step S351 set the threshold of remaining edge points as Tnum, and judge whether the number of remaining edge points is greater than the threshold Tnum. If num>Tnum, then enter step S352, judge that the tray is not empty, if num ⁇ Tnum, then enter step S353, judge that the tray is empty.
- the concentration degree of the distribution of the remaining edge points can also be set as S, and the threshold of the concentration degree is set as Sth, if num>Tnum and S>Sth, it is judged that the tray is not empty, otherwise the tray is empty . In the case that the number of remaining edge points is large but these remaining edge points are not concentrated, it may not mean that there is luggage in the tray, so the recognition result may be wrong.
- the degree of concentration in addition to the number of remaining edge points , to more accurately identify non-empty pallets.
- FIG. 5 is a flowchart schematically showing an example of this Embodiment 1.
- edge detection step S32 and the template matching step S33 can also be implemented in the following manner.
- step S32 for the captured image of the pallet, the chassis area of the pallet is identified through image recognition.
- the edge detection module 32 first performs Gaussian blur processing on the chassis area for the image of the chassis area, then selects two thresholds, and obtains an edge image by double-threshold canny edge detection.
- FIG. 6 shows an example of the edge detection step S32.
- step S321 Gaussian blur processing is performed on the input image of the chassis area.
- step S322 two thresholds T 1 , T 2 are selected.
- the two thresholds can be adaptively selected by learning live images through machine learning.
- the images collected by the image acquisition device 102 will have obvious differences in brightness.
- the thresholds T 1 and T 2 it is possible to reduce non-item edges caused by weak light, etc., from being detected as edge images. In some cases, it can also adapt to the difference in the overall brightness of the image.
- step S323 using the two thresholds T 1 and T 2 selected in step S322, a double-threshold canny edge detection algorithm is used to perform edge detection and obtain an edge image.
- step S33 it is judged whether the interference medium image template needs to be reversed.
- the interference medium image template is reversed.
- the interference medium image template can be flipped 180 degrees. This is because the angle of the tray passing through the image acquisition area can be limited, or the chassis image or edge image at different angles can be rotated.
- the edge image has the same shape and angle as the interference medium image template, However, there may be a left-right or upside-down situation.
- flipping the disturbance medium image template 180 degrees comparing the disturbance medium image template before flipping and the disturbance medium image template after flipping with the edge image, it can be more effective to find the best matching template.
- the flipping process may not be performed, that is, steps S331-S332 may not be required.
- step S333 the edge image is matched with each interference medium image template, and the template matching value is calculated and saved.
- the template matching value represents the matching degree between the edge image and the interference medium image template, specifically the matching degree of the edge lines and edge points of the two.
- the interference medium image template There are two storage methods for the interference medium image template. One is to pre-store a corresponding interfering medium image template in the storage unit for each interfering medium; the other is to pre-store two interfering media before and after inversion in the storage unit for each interfering medium Image template.
- the sticking direction of the interference medium in the acquired image may not match the direction of the interference medium in the interference medium image template.
- step S333 the flipped interference medium image template is matched with the edge image.
- step S331 it is also judged that inversion is not necessary. That is, when there are N interfering media and 2N interfering media image templates are stored, flipping is considered unnecessary, and the edge image is matched with the stored 2N interfering media image templates in the subsequent step S333.
- step S334 after calculating the template matching values for all the interference medium image templates, determine the maximum value of the template matching values and the best matching position.
- step S335 the interference medium image template corresponding to the maximum value of the template matching value is output as the best matching template.
- the edge points corresponding to the interference medium image template are removed from the edge image, and based on the remaining edge points
- FIG. 8 is a functional block diagram schematically showing Example 2 of the empty disk recognition device 30 according to the first embodiment.
- the empty disk identification device 30 also includes: a chassis contour extraction module 36, which extracts the chassis area contour from the image acquired by the image acquisition module 31; The smallest circumscribed rectangle of the contour of the chassis area is rotated, and the edge detection area is cut out as the chassis area.
- FIG. 9 is a flowchart schematically showing Example 2 of the empty tray recognition method according to the first embodiment. Steps S31 to S35 in FIG. 9 are the same as those in Embodiment 1 (FIG. 3). It will not be described in detail here. The difference between FIG. 9 and FIG. 3 is that, between step S31 and step S32, there are steps S36 and S37.
- the chassis outline extraction module 36 extracts the chassis area outline from the image acquired by the image acquisition module 31 .
- the position and range of the chassis area of the pallet are determined, and the chassis area outline as the range of the chassis area is extracted.
- FIG. 10 shows an example of the chassis contour extraction step S36.
- step S361 the chassis contour extraction module 36 performs adaptive binarization on the input image to obtain a binarized image.
- adaptive binarization can use any existing algorithm.
- adaptive binarization it is possible to adapt to environmental changes. For example, an image becomes brighter or darker depending on the environment.
- step S362 an opening operation or a closing operation is performed on the binarized image.
- the edge of the chassis in the binarized image obtained in step S361 may appear discontinuous.
- the appropriate opening and closing kernel operation kernel
- step S363 an edge image is obtained by using an edge detection algorithm.
- the edge detection algorithm may use any known edge detection algorithm.
- step S364 a contour extraction algorithm is used to extract edge contour lines as chassis area contours.
- the contour extraction algorithm can adopt any known contour extraction algorithm.
- the contour line is extracted from the image after these processes, and the contour line with better continuity and integrity can be extracted.
- chassis area outline extracted in step S364 can be directly output as the input of subsequent step S37 or step S32. However, it can also be used as shown in FIG. As shown, the process of step S365 is also performed.
- the chassis contour extraction module 36 judges whether the chassis region contour extracted in the S364 step exceeds the predetermined threshold range, if exceeded, then enters the S366 step, judges that the tray is not empty, and ends the processing, if not exceeded, then Enter step S367, and output the contour of the chassis area.
- the predetermined threshold range may be represented by an upper limit value and a lower limit value. Exceeding the predetermined threshold range means less than the lower limit or greater than the upper limit.
- the upper limit value can be preset according to the size of the chassis of the pallet.
- the lower limit value can be set in advance according to the size of the chassis of the disk, the size of the interfering medium usually pasted, and the like.
- step S365 it is further judged whether the edge contour extracted in step S364 is a contour line representing the chassis area according to the predetermined threshold range. If the contour of the chassis area is greater than the upper limit of the predetermined threshold range, the edge contour extracted in step S364 may not represent the contour line of the chassis area, but the contour line whose size is larger than the bottom of the tray, for example, it may indicate that there is luggage in the tray and The case where the baggage straddles the outside of the pallet, etc.
- the edge profile extracted in the step S364 may not be a contour line representing the chassis area, but a contour line whose size is very small relative to the chassis area, for example, the size is smaller than Usually the size of the interfering medium pasted, etc.
- step S365 By performing the processing in step S365, the extraction accuracy of the chassis outline can be further improved.
- the chassis area contour with the smallest area is selected for output.
- multiple contour lines related to the chassis area may be detected, and by selecting the contour line with the smallest area among them, the interference of the redundant contour lines to the subsequent operation can be avoided, and the recognition result is more accurate.
- the upper limit value and the lower limit value can also be set to relatively close values according to the size of the chassis of the pallet.
- the contour line representing the chassis area can be quickly, accurately and uniquely identified.
- the rotation module 37 performs rotation based on the minimum circumscribed rectangle of the chassis area contour output by the chassis contour extraction module 36 , and cuts out the edge detection area as the chassis area.
- FIG. 11 shows an example of the up-rotation step S37.
- step S371 the minimum circumscribed rectangle, rotation angle ⁇ , length and width values, and center point coordinates (x, y) are obtained for the chassis area outline extracted in step S36.
- the algorithm for obtaining the minimum circumscribed rectangle may use any existing algorithm.
- step S372 use affine transformation to rotate ⁇ ° around the center point of the smallest circumscribed rectangle to obtain the image of the rotated bottom area.
- step S373 the clipping area information is obtained by calculating the length and width of the minimum circumscribed rectangle and the coordinates (x, y) of the center point, and the edge detection area is clipped out.
- the edge detection area means an area to be the object of edge detection.
- the edge detection area in step S373 may be directly output as the chassis image to the edge detection module 32 for edge detection processing in step S32.
- the processing of step S374 is also performed as shown in FIG. 11 .
- step S374 the edge detection area is scaled to a specified size as the chassis area.
- the specified size is determined according to the size of the interference medium image template.
- step S373 Due to the influence of factors such as differences between the trays themselves and changes in the surrounding environment during imaging, there will be slight differences in the size of the edge detection area cut out in step S373, and such differences can be reduced by performing scaling in step S374, so that The image consistency of the chassis region as the edge detection object in step S32 is higher, and the influence of the size of the input image on the recognition result can be reduced.
- the edge detection module 32 may perform edge detection on the chassis area output from the rotation module 37 via step S37 as an object.
- the empty disk recognition device 30 can also be On the basis of Embodiment 1, only the chassis contour extraction module 36 is included, and only the processing in step S36 is performed, and the chassis area contour output by the chassis contour extraction module 36 is directly used as the detection object of the edge detection module 32 . At this time, for subsequent template matching, etc., it is necessary to restrict the placement direction and angle of the tray on the tray transfer device 10, for example, place the tray on the tray transfer device 10 according to the specified direction and angle.
- the area that needs to be detected can be quickly and accurately determined, thereby reducing the workload of edge detection, and making the orientation of the edge image consistent with
- the interference medium image template is easier to achieve consistency, which makes template matching more efficient and more accurate, thereby effectively improving the efficiency and accuracy of empty disc identification.
- a device and method for identifying an empty disk capable of identifying interfering media are shown.
- the stored interfering media image template and edge image are used to perform match.
- the interference medium image template needs to be stored in advance, or the interference medium image template can be obtained from an external device. Therefore, as a second embodiment, an apparatus and method for generating an interference medium image template are provided.
- the application scenario of the second embodiment is not limited to the scenario of the first embodiment.
- an interfering medium image template needs to be generated in advance.
- the interfering medium image template generation system for generating the interfering medium image template may have a structure similar to that of the blank disk identification system shown in FIG.
- the medium image template generation device 40 in other words, the interference medium image template generation system includes the tray conveying device 10 , the image acquisition device 20 , and the interference medium image template generation device 40 .
- the interfering medium image template generation device 40 can also be the same as the empty disk identification device 30 described in the first embodiment, and can be provided with any suitable electronic device such as a desktop computer, a tablet computer, a smart phone, a server, etc. It is realized by combining software and hardware.
- the interfering medium image template generating device 40 can communicate with the blank disc identifying device 30 of the first embodiment, and send the generated interfering medium image template to the empty disc identifying device 30 for use.
- the generated interfering medium image templates may also be uploaded to a server, and the empty disk identification device 30 may download these interfering medium image templates from the server for use.
- the interfering medium image template generating device 40 can generate an interfering medium image template by performing edge detection on multiple chassis images.
- the interfering medium image template includes the edge points of the interfering medium and is used to identify empty pallets, and the multiple chassis images are respectively obtained by photographing a plurality of empty pallets on which the interfering medium is disposed.
- the specific configuration of the interference medium image template generation device 40 and the generation method of the interference medium image template executed in the interference medium image template generation device 40 will be described in detail.
- FIG. 12 is a functional block diagram schematically showing Example 1 of the interference medium image template generation device 40 according to the second embodiment.
- the interfering medium image template generation device 40 includes: an initial template generation module 41, which performs edge detection on multiple chassis images, counts the number of edge points at the same position, and generates initial template.
- the interference medium image template generation device 40 stores and/or outputs the initial template as an interference medium image template.
- the initial template generation module 41 may include: an edge detection part, which performs edge detection on a plurality of chassis images; a counting part, which counts the number of edge points at the same position; and, a template generation part, according to the counting result to generate the initial template.
- the initial template generation module 41 performs edge detection on multiple chassis images, counts edge points at the same position, and generates an initial template according to the counting result.
- FIG. 13 shows an example of the generation method of the interference medium image template in Embodiment 1.
- step S411 acquire N images of empty trays from the image acquisition device 20 .
- the process of acquiring images may be the same as step S31 in the first embodiment.
- step S412 the interfering medium area at the bottom of the tray is obtained through a pre-processing operation, and an edge detection is performed on the interfering medium area at the bottom of the tray to obtain an edge image.
- the pre-processing can use the existing operations, for example, the chassis contour extraction step S36 and the rotation step S37 in the first embodiment can be used.
- edge detection can be performed in the same manner as in step S32 of the first embodiment.
- step S413 generate a two-dimensional count variable that is consistent with the size of the image and has a value of zero, and use the two-dimensional count variable to count the sum of the number of edge points at each position in the N images. Specifically, if the image I is an edge point at position (x, y), then add 1 to the value v at (x, y) of the two-dimensional count variable, otherwise it remains unchanged. That is, the sum of the number of edge points is the count value of the edge points.
- the two-dimensional count variable may be a two-dimensional vector V, or a pixel value of a two-dimensional image, or a two-dimensional storage space, etc.
- the size of the image here is the image of the chassis area after the pre-processing.
- step S414 the ratio v/N of the number of edge points at any position (x, y) in the two-dimensional count variable is calculated.
- step S415 it is judged whether the ratio v/N calculated in step S414 is greater than the threshold T. If v/N>T, go to step S416 and save the point in the template. For example, set the pixel value of the corresponding position in the template to 255 (white). Otherwise, go to step S417 and not save the point to the template. For example, set the pixel value of the corresponding position in the template to 0 (black).
- step S4108 it is judged whether the processing of steps S414-S417 has been completed for all positions in the two-dimensional counting variable, if yes, then end the processing, and enter step S419 to output the initial template, if not, then return to step S414 and continue Processing of the next position. In this way, the entire two-dimensional count variable is traversed to obtain an initial template.
- the threshold T is a preset value for the purpose of obtaining the edge of the interfering medium.
- the optimal value can be obtained based on multiple trials.
- the value of the threshold is preferably such that the lines of the edges in the obtained initial template are relatively thin.
- the threshold T can be appropriately set according to the usage state of the tray and the like. Specifically, when the tray is relatively new and less stained or worn, there are fewer edges other than the shape of the tray itself and the shape of the interfering medium. Therefore, T can take a larger value, such as a value close to 1 , the edge points of the chassis area can be effectively identified.
- T In the case of a lot of stains or wear on the tray, there are many edges other than the shape of the tray itself and the shape of the interfering medium caused by the stain or wear. Therefore, T needs to take a smaller value. In the actual use process, the stains on the tray will gradually increase, so T usually uses a value less than 1. For the tray in daily use, according to the test results, it is optional that T is 0.3-0.4. , an edge of appropriate thickness can be obtained.
- steps S413 to S4108 although it is shown that the proportion v/N of the number of edge points is compared with the threshold T to determine whether to keep the edge points. However, it is also possible to directly use the count value of the edge points obtained in step S414 to compare with the threshold to determine whether to keep the edge points.
- an initial template is generated based on the count value. Accordingly, it is possible to generate an initial template including edge points that effectively represent actual edges of the interference medium image.
- FIG. 14 is a functional block diagram schematically showing Example 2 of the interference medium image template generation device 40 according to the second embodiment.
- the interference medium image template generation device 40 includes a final template generation module 42 in addition to the initial template generation module 41 . That is, the interfering medium image template generation device 40 includes: an initial template generation module 41, which performs edge detection on multiple chassis images, and counts the number of edge points at the same position, and generates an initial template according to the counting result; and , the final template generation module 42 obtains the remaining edge points according to the difference between the edge points of each of the multiple chassis images and the edge points of the initial template, and updates the initial template according to the remaining edge points, thereby generating the final template .
- the interference medium image template generation device 40 stores and/or outputs the final template as the interference medium image template instead of the initial template.
- the final template generation module 42 may include: an edge image acquisition unit, which acquires edge images of N pallet bottom regions; a corresponding area elimination unit, which eliminates edge points corresponding to the initial template in the edge images to obtain remaining edge points; and a template update unit, which updates the initial template according to the remaining edge points.
- the edge image acquisition unit may further include an image acquisition unit that acquires N images of the tray; an edge detection unit that performs edge detection on the chassis area in the images to obtain an edge image.
- the image acquisition unit, the edge detection unit, and the corresponding area elimination unit in the final template generation module 42 correspond to the image acquisition module 31, the edge detection module 32, and the empty disk identification device 30 in the first embodiment.
- the region elimination modules 34 are respectively similar. The difference between the two is that, firstly, the final template generation module 42 acquires multiple images with trays, and performs edge detection and corresponding area elimination on the multiple images, while the empty disc recognition device 30 only targets the An image for empty disk identification is processed; the second is that the corresponding area is eliminated in the empty disk identification device 30 after the best matching template after template matching, and the corresponding area is eliminated in the interference medium image template generation device 40 is the initial template.
- FIG. 15 is a flowchart schematically showing Example 2 of the interference medium image template generation method according to the second embodiment.
- the processing of the initial template generation step S41 is the same as that described in Embodiment 1, and thus repeated description is omitted.
- step S42 the final template generation module 42 obtains the remaining edge points according to the difference between the edge points in each chassis image and the edge points of the initial template, and updates the initial template according to the remaining edge points, thereby generating the final template .
- FIG. 16 shows an example of the final template generating step S42.
- step S421 N edge images of the bottom area of the tray are acquired.
- the step S42 is executed after the step S41 is performed, the edge image output by the step S412 can be obtained directly. In this case, the calculation amount can be reduced and the calculation speed can be accelerated.
- N edge images of the bottom area of the tray can be obtained in the same manner as step S411 and step S412 shown in FIG. 13 .
- N edge images of the bottom area of the tray can also be obtained in the same manner as the image acquisition step S31 and the edge detection step S32 in FIG. 3 (or FIG. 5 ) of the first embodiment.
- the edge image acquisition unit of the final template generation module 42 includes: an image acquisition unit that acquires N images of the pallet; an edge detection unit that performs edge detection on the chassis area in the image to obtain an edge image.
- step S422 for each edge image obtained in step S421 , the edge corresponding to the initial template is eliminated to obtain the remaining edge image edge_left.
- the edge points corresponding to the edge points of the initial template among the edge points of the edge image of the chassis image can be found by neighborhood processing.
- the neighborhood P l (x E , y E ) of the pixel point P(x E , y E ) in the edge image determines the relationship with the pixel point P(x E , y E ) in the initial template
- the area P l (x M , y M ) corresponding to the neighborhood if there is a point in P l (x M , y M ) whose pixel value represents an edge point (for example, a point with a pixel value of 255), then the The pixel point P(x E , y E ) is set as a non-edge point, otherwise, the edge point is reserved, and in this way, all the edge points in the multiple edge images are traversed.
- the neighborhood may be a square area whose center is the pixel point and whose side length is l.
- step S422 has the same function as the processing in step S341 in FIG. 4 in the first embodiment, and both are to eliminate corresponding edge points in the edge image and the template. Therefore, although the specific examples in step S422 and step S341 are different, they can be used interchangeably.
- step S422 the problem that the edge of the template becomes thicker due to simply superimposing the edge points on the template can be effectively avoided, which affects the judgment.
- step S423 the number num of edge points of the remaining edge image edge_left of the edge image is calculated.
- the processing at step S423 is the same as the processing at step S343 in FIG. 6 in the first embodiment.
- step S424 it is judged whether the number num of edge points is greater than the threshold Tnum. If not, enter step S425, and in step S425, do not update the template.
- step S424 When it is judged in step S424 that the number num of edge points is greater than the threshold Tnum (num>Tnum), go to step S426.
- step S426 a counting image img_count having the same size as the edge image is received, and the remaining edge points of the edge image are counted at corresponding positions (corresponding pixels) in the counting image img_count.
- the corresponding position is a remaining edge point
- the pixel value of the pixel point is used as the count value and one is added.
- step S427 it is judged whether the processing of steps S422-S426 has been completed for all edge images (for example, N images). If yes, enter step S428; otherwise, return to step S422, and continue to perform the processing of steps S422-S426 on the next edge image.
- the pixel value of each pixel in the count image img_count after the step S427 represents: the count value after counting the remaining edge points of all the edge images according to each pixel.
- step S428 for each pixel in the counting image img_count, it is judged whether the pixel value of the pixel is greater than the threshold Tcount. That is, it is judged whether the count value at each pixel in the count image img_count is greater than the threshold Tcount. If the pixel value is greater than the threshold Tcount, go to step S429; otherwise, go to step S425.
- the threshold Tcount may be set according to the number of chassis images, or may be set as a constant value. From the viewpoint of both calculation accuracy and calculation efficiency, optionally, the range of constant integers may be 3-4.
- step S429 if the pixel value of a certain pixel point is greater than the threshold Tcount, update the corresponding pixel point in the template as an edge point. For example, the pixel value of the corresponding pixel in the template is set to 255, so that the pixel is white, indicating an edge point.
- step S4210 it is judged whether the processing of steps S428-S429 has been completed for all pixels in the count image img_count. If yes, enter step S4211 to output the updated final template, otherwise, return to step S428 to judge the next pixel in the counting image img_count.
- the corresponding pixels in the template are only updated when the count value exceeds a certain threshold, so that it is possible to avoid occasional noise in a few edge images.
- the remaining edge points that occur are updated into the template. Therefore, the accuracy of the interference medium image template can be improved, and the efficiency of generating the interference medium image template can be improved.
- the remaining edge image edge_left may also be expanded.
- the edge line in the remaining edge image edge_left can be made thicker through expansion processing, which can avoid the situation that the edge line in the remaining edge image edge_left is too thin or the edge points in multiple edge images cannot be aligned, so the count value of the count image img_count is too scattered . In other words, it is possible to avoid the problem that multiple inputs cannot be aligned, so that when multiple images are input, even in the case of stains or scratches, the update is still unsuccessful.
- Fig. 17 shows the interfering medium image template generated by the interfering medium image template generating device, in order to compare the effects before and after the final template generation step S42, (a) of Fig. 17 shows the initial template, and (b) of Fig. 17 shows the final template .
- the final template has more edge points than the initial template, so that the edge points that were not effectively recognized in the initial template can be added to the interference medium image template after the final template generation process.
- the extra edge point at the upper right of (b) of FIG. 17 may be due to stains on the tray or the like.
- the template is further updated through the above-mentioned final template generation step to generate the final template, so that the previously unrecognized edge can be added to the interference medium image template point, can further improve the accuracy of the interference medium image template.
- the second embodiment can also be implemented in the form of an empty disk recognition system. That is, Example 1 and Example 2 of the second embodiment described above are combined with the first embodiment.
- the empty disk identification system includes a tray conveying device 10 , an image acquisition device 20 , an empty disk identification device 30 , and an interference medium image template generation device 40 .
- the empty disk identification device 30 performs template matching based on the interference medium image template generated by the interference medium image template generation device 40 .
- Fig. 18 shows the situation that the interfering medium image template generating device 40 and the empty disk identifying device 30 are physically arranged as one, however, the interfering medium image template generating device 40 and the empty disc identifying device 30 can also be separated physically set up.
- the device 30 for identifying an empty disk may be set in a terminal at a security inspection site, and the device 40 for generating an image template for interference medium may be set in a server or the like.
- the empty disc recognition device 30 has the same configuration as the first embodiment and executes the same processing, detailed description thereof will be omitted.
- the interfering medium image template generating device 40 further includes: a running phase updating module 43 on the basis of the above-mentioned embodiment 1 or embodiment 2.
- FIG. 19 is a functional block diagram schematically showing an example of Example 3 of the interference medium image template generation device 40 according to the second embodiment.
- FIG. 19 shows the case of simultaneously having the initial template generating module 41 , the final template generating module 42 , and the running phase updating module 43 , it is also possible to have only the template generating module 41 and the running phase updating module 43 .
- the processing performed by the running stage updating module 43 is similar to that of the final template generation module 42 in Embodiment 2, the only difference is that the running stage updating module 43 acts in the running stage of the empty disk identification device, and only when the update is input When the signal is signaled, the existing template is updated based on the remaining edge points obtained by eliminating the edge points of the existing template from the edge points of the real-time pallet image.
- the real-time tray image is obtained by photographing an empty tray conveyed on the tray conveying device 10 by the image acquisition device 20 during the operation stage of the empty tray identifying device 30 , and the chassis of the tray is provided with an interfering medium.
- the so-called existing template is the initial template output by the above-mentioned initial template generation module 41 or the final template output by the final template generation module 42 .
- the running phase update module 43 may include: an image acquisition unit, an edge detection unit, a corresponding area elimination unit, and a template update unit. Among them, the difference is that, firstly, what is acquired from the image acquisition device in the image acquisition unit is the image collected in real time during the operation phase of the empty disk identification device 30; secondly, when the corresponding area is eliminated, not only
- the initial template output by the initial template generation module 41 may also be the final template output by the final template generation module 42 .
- the running stage updating module 43 includes: an image acquisition part, which acquires a real-time pallet image from an image acquisition device; an edge detection part, which performs edge detection on the chassis area in the real-time pallet image, and obtains an edge image; a corresponding area elimination part, which converts the edge image The edge points corresponding to the existing template are eliminated to obtain the remaining edge points; and the template updating part updates the existing template according to the remaining edge points.
- An existing template is either an initial template or a final template.
- the running stage update module 43 further includes an update signal judging unit, and the update signal judging unit judges whether an update signal is received.
- the template update unit of the running stage update module 43 updates the template only when the update signal determination unit determines that an update signal has been received.
- the running stage update module 43 also includes a flip judging unit, which judges whether the direction of the interfering medium in the real-time tray image is consistent with the direction of the interfering medium in the existing template, and if not, it is judged that the edge image needs to be corrected. Inversion is performed, and after inversion, it is input to the above-mentioned corresponding area elimination unit. That is, subsequent corresponding region elimination processing and template updating processing are performed based on the flipped edge image.
- FIG. 20 is a flowchart schematically showing Example 3 of the interference medium image template generation method according to the second embodiment.
- Embodiment 3 shown in FIG. 20 The difference between Embodiment 3 shown in FIG. 20 and Embodiment 2 shown in FIG. 15 is that the method for generating an image template of an interference medium further includes a template update step S43 in the running phase.
- step S43 the operation of the empty disk identification device is performed only when an update signal is input, based on the remaining edge obtained by eliminating the edge points corresponding to the edge points of the existing template from the edge points of the real-time tray image Click to update the existing template.
- FIG. 21 shows an example of the running phase updating step S43.
- step S430 N real-time tray images of empty trays are acquired from the image acquisition device 20 .
- step S431 edge detection is performed on the interfering medium area at the bottom of the tray, and an edge image is acquired.
- step S430 and step S431 may be the same as step S411 and step S412 in Embodiment 1 respectively, or the same as step S31 and step S32 in the first embodiment.
- Steps S432 to S435 in FIG. 21 are the same as steps S422 to S425 in FIG. 16 , and thus description thereof will be omitted.
- steps S432 to S433 are also the same as the steps S342 to S343 (refer to FIG. 4 and FIG. 5 ) in the first embodiment respectively, and the above step S434 is also the same as the step S351 in the first embodiment (refer to FIGS. 4 and 5 ). 5) Same.
- the running stage update module 43 may not be additionally provided with the above-mentioned image acquisition unit, edge detection unit, and corresponding area elimination unit, but can be shared with the empty disk identification device 30 . That is, the image acquisition module 31 , edge detection module 32 , and corresponding area elimination module 34 of the empty disk identification device 30 of the first embodiment are used as the image acquisition unit, edge detection unit, and corresponding area elimination unit of the running phase update module 43 . Therefore, the configuration of the running stage update module 43 can be simplified, the calculation amount can be reduced, and the calculation speed can be increased.
- step S436 it is judged whether an update signal is received.
- the update signal may be entered by an operator of the empty disk identification system. For example, when it is found that the tray being conveyed by the tray transfer device 10 and entering the image capture device 20 is an empty tray, an update signal is input.
- step S426 is performed after steps S430 to S434 , however, step S426 may be performed before step S430 or before step S432 .
- the above-mentioned steps S430-S434 are respectively the same as the steps S31-S351 in FIG. 5 . Therefore, in the case of sharing the image acquisition unit, the edge detection unit, and the corresponding area elimination unit with the empty disk identification device 30, after the processing of the image acquisition unit, the edge detection unit, and the corresponding area elimination unit is completed, it is judged whether an update is received. Signal, therefore, without affecting the normal operation of the panel identification device 30, the update of the template can be realized.
- step S437 it is judged whether flipping is required.
- step S438 it is judged whether the direction of the interfering medium in the collected real-time tray image is consistent with the direction of the interfering medium in the existing template, and if not, it is judged that the edge image needs to be reversed, and step S438 is entered.
- step S4308 flip the edge image.
- the subsequent step S439 is performed using the flipped edge image.
- Steps S439 to S4314 in FIG. 21 are the same as steps S426 to S4211 in FIG. 16 , and thus description thereof will be omitted.
- the interfering medium image template can also be updated during the operation phase of the empty disk identification system, the interfering medium image template can be updated in real time according to the wear and tear of the interfering medium on the tray, and the interfering medium image template can be further improved. The accuracy of the template, thereby further improving the accuracy of empty disk recognition.
- the third embodiment of the present application provides a method that can adaptively detect pallets with shadows caused by lighting. Empty disk identification device, system and method.
- FIG. 22 is a schematic diagram showing the configuration of the empty disc recognition system according to this embodiment.
- the empty disk identification system of this embodiment may include a tray transfer device 10, an image acquisition device 20, an empty disk identification device 30', and a light emitting device 50.
- the empty disk identifying device 30' includes: an empty disk judging module 60 and a shadow eliminating device 70.
- the tray conveying device 10 and the image capturing device 20 are the same as those in the first and second embodiments, so the description thereof will be omitted.
- the light emitting device 50 is a device capable of irradiating light to the image capture area 21 of the image capture device 20 .
- the light emitting device 50 can function as a supplementary light of the image capture device 20 .
- the light only needs to be able to supplement the image captured by the image acquisition device, so that the image with supplementary light is different from the image without supplementary light.
- the light emitting device 50 may be an infrared light emitting device.
- the change of the pixel value in the image with supplementary light is more obvious than that in the image without supplementary light, which is beneficial for shadow elimination.
- the image capture device 20 and the light emitting device 50 are physically separated in FIG. 22 , the image capture device 20 and the light emitting device 50 may also be physically integrated.
- the empty disk identification device 30' is the same as the empty disk identification device 30 of the first and second embodiments, and can be provided with any suitable electronic equipment such as a desktop computer, a tablet computer, a smart phone, a server, etc., with software and hardware. realized in a combined manner.
- the difference between the empty disk identification device 30' of this embodiment and the empty disk identification device 30 of the first and second embodiments is that it includes a shadow eliminating device 70.
- FIG. 23 is a functional block diagram schematically showing Example 1 of the shadow canceling device 70 according to the third embodiment.
- the shadow elimination device 70 includes: an image acquisition module 71, which acquires images with and without supplementary light; an image acquisition module 72 of the illuminated area, according to the difference between the image with supplementary light and the image without supplementary light, to obtain the illumination The area image; the edge detection module 73, which detects the edge of the image without supplementary light to obtain the edge image; and the shadow line elimination module 74, removes the part corresponding to the illuminated area image from the edge image, and obtains the actual edge image.
- having fill light refers to light compensation.
- the image without supplementary light refers to the image directly captured by the image capture device 20 .
- the image with supplementary light refers to the image acquired by the image acquisition device 20 with light compensation.
- the image without supplementary light refers to the image captured by the image acquisition device 20 in the state where the light emitting device 50 is not used to supplement the light to the image acquisition area 21 of the image acquisition device 20 .
- the image with supplementary light is an image captured by the image capture device 20 in a state where the image capture region 21 of the image capture device 20 is supplemented with light by the light emitting device 50 .
- FIG. 24 is a flowchart schematically showing Example 1 of the shadow removal method according to this embodiment.
- step S71 the image acquisition module 71 acquires an image with supplementary light and an image without supplementary light.
- step S72 the illumination area image acquisition module 72 obtains an illumination area image according to the difference between the image with supplementary light and the image without supplementary light.
- the illuminated area image acquisition module 72 may determine that the pixel belongs to the illuminated area image when the ratio of the pixel values of the image with supplementary light and the image without supplementary light is within a predetermined threshold range.
- the predetermined threshold range is a range around 1, for example, 0.9-1.1 is optional.
- a ratio of the pixel values of the image with and without light is close to 1, indicating that the difference between the image with light and the image without light is not obvious.
- the illuminated area refers to the area in the image of the empty pallet that is illuminated by sunlight. Sun exposure is the main cause of shadows. Therefore, by judging the illuminated area, it is possible to effectively eliminate the influence of shadows caused by sunlight on the identification of empty disks.
- the intensity of sunlight is much greater than that of infrared supplementary light and ambient light.
- the pixel values differ significantly. Therefore, in the present application, the illuminated area can be identified through the change of the pixel value of the same pixel in the image with supplementary light and the image without supplementary light.
- the predetermined threshold range is also a range around 1, for example, 0.9-1.1 is optional.
- step S73 the edge detection module 73 performs edge detection on the image without supplementary light NL to obtain its edge image NLE.
- step S73 Since the edge detection in step S73 can be performed in the same manner as in step S32 of the first embodiment, detailed description will be omitted.
- step S73 the edge detection module 73 performs edge detection after performing Gaussian blur processing on the image without supplemental light, similarly to the first embodiment.
- step S74 the shadow line elimination module 74 removes the portion corresponding to the illuminated area image R from the edge image NLE to obtain an actual edge image.
- the shadow line elimination module 74 may determine that the area ratio of the illuminated area within the neighborhood of any pixel point in the edge image is within a predetermined area threshold range, and remove the pixel point.
- the shadow line elimination module 74 may include: a neighborhood determination unit, for each pixel in the edge image, determine the neighborhood of the pixel; a corresponding area determination unit, in the illuminated area image The area corresponding to the neighborhood; the illuminated area ratio judging section, judging whether the area ratio of the illuminated area in the area is within the preset area threshold range; the edge point removal section, eliminating the area ratio within the threshold pixel points within the range; and an actual edge image output unit, which generates and outputs an actual edge image according to the remaining edge points.
- step S741 the neighborhood determination unit determines the neighborhood of any pixel in the edge image NLE.
- step S742 the corresponding area determination unit determines an area corresponding to the above-mentioned neighborhood of the pixel point in the illuminated area image RL.
- the area corresponding to the neighborhood P l (x NLE , y NLE ) is determined and denoted as P l (x RL , y RL ).
- step S743 the illuminated area ratio judging unit judges whether the area ratio of the illuminated area in the area P l (x RL , y RL ) is within a preset area threshold range. If yes, go to step S745, if not, go to step S744.
- the area ratio is the ratio of the area of the illuminated area to the entire area in the area P l (x RL , y RL ).
- the area ratio of the illuminated area is within the preset area threshold range, it means that there are both illuminated areas and shadowed areas in the area P l (x RL , y RL ) in the illuminated area image corresponding to the neighborhood of the pixel , in other words, the edge in this area is a shadow edge, thus, the pixel point is likely to be a shadow edge, rather than the edge of an object, stain, scratch, etc. Therefore, this pixel point should not be used as an edge point.
- step S744 the edge point removal unit judges the pixel point as an edge point and keeps the edge point. That is, the pixel value of the pixel point P(x NLE , y NLE ) is kept as the original value (for example, kept as 255).
- the edge point removal unit removes the pixel point P(x NLE , y NLE ) from the edge image NLE.
- the pixel value of the pixel point P(x NLE , y NLE ) is set to 0 (black).
- step S746 it is judged whether the above-mentioned processing in steps S741-745 has been performed on all pixels in the edge image NLE.
- step S747 the actual edge image output unit generates and outputs an actual edge image based on the remaining edge points after the above operations.
- FIG. 27 shows the edge image NLE before the hatching removal processing of the S74 step
- (b) of Fig. 27 shows the actual edge image after removing the hatching from the edge image NLE by the hatching removal processing of S74 .
- the hatching caused by the shadow is effectively eliminated through the above-mentioned hatching removal processing.
- the empty disk judging module 60 of the empty disk identifying device 30' can judge whether the tray is an empty tray based on the actual edge image output by the shadow removing device 70.
- the empty disc judging module 60 for judging whether or not the tray is empty based on the actual edge image may employ any existing configuration.
- the same configuration as that of the empty disk judging module 35 of the first embodiment may be employed.
- the empty tray judging module 35 can use the edge points of the actual edge image output by the shadow removing device 70 as the remaining edge points, and use, for example, steps S351 to S353 to judge whether the tray is empty.
- the actual edge image that eliminates the edge line produced by the illumination is generated through the image with supplementary light and the image without supplementary light, and the actual edge image can be used in the identification of the empty disc, thereby reducing the impact of the illumination on the identification of the empty disc. influences.
- Embodiment 1 shows an empty disk identification device 30' capable of shadow elimination and empty disk identification based on the remaining edge points after shadow elimination. However, it is also possible to eliminate the edge points corresponding to the interfering medium while eliminating the edge points caused by the illumination, and then perform empty disk identification. That is, Example 1 of the third embodiment described above can be combined with the first embodiment, and will be described as Example 2 here.
- FIG. 28 is a functional block diagram schematically showing Example 2 of the empty disc recognition device 30" according to the third embodiment.
- the empty disk identifying device 30" of this embodiment also includes the same template matching module 33, corresponding area eliminating module 34, empty disk Judgment module 35.
- the empty disc identification device 30" includes: a shadow eliminating device 70, which eliminates edge points corresponding to the shadow area from the image without supplemental light according to the image with supplementary light and the image without supplementary light, to obtain the actual edge image; the template matching module 33, The actual edge image is matched with the interference medium image template; the corresponding area elimination module 34 eliminates the edge points corresponding to the interference medium image template in the actual edge image to obtain the remaining edge points; and the empty disk judgment module 35, based on the remaining edge points, Determine whether it is an empty pallet.
- a shadow eliminating device 70 which eliminates edge points corresponding to the shadow area from the image without supplemental light according to the image with supplementary light and the image without supplementary light, to obtain the actual edge image
- the template matching module 33 The actual edge image is matched with the interference medium image template
- the corresponding area elimination module 34 eliminates the edge points corresponding to the interference medium image template in the actual edge image to obtain the remaining edge points
- the empty disk judgment module 35 based on the remaining edge points, Determine whether it is
- FIG. 29 is a flowchart schematically showing Example 2 of the empty tray recognition method according to the third embodiment.
- the method for identifying an empty disk in this embodiment includes the same steps S33 to S35 as in FIG. 3 of the first embodiment in addition to steps S71 to S74 that are the same as those in FIG. 23 in Embodiment 1.
- step S33 the actual edge image output in step S74 is matched with the interference medium image template.
- step S34 the edge points corresponding to the interference medium image template in the actual edge image are eliminated to obtain remaining edge points.
- step S35 based on the remaining edge points, it is judged whether it is an empty tray.
- Example 2 of the present embodiment shows an example in which Example 1 is combined with the first embodiment.
- this embodiment mode can also be implemented in combination with the first and second embodiment modes at the same time. Therefore, here, as Example 3, a case where Example 1 is combined with both the first and second embodiments will be described.
- FIG. 30 is a functional block diagram schematically showing Example 3 of the empty disk recognition system according to the third embodiment.
- the empty disk identification system of this embodiment includes: the same empty disk identification device 30" and light emitting device 50, image acquisition device 20, tray transfer device 10, and the same as the second embodiment The device 40 for generating an interference medium image template.
- processors may be, but are not limited to, general purpose processors, special purpose processors, application specific processors, or field programmable logic circuits. It can also be understood that each block in the block diagrams and/or flowcharts and combinations of blocks in the block diagrams and/or flowcharts can also be realized by dedicated hardware for performing specified functions or actions, or can be implemented by dedicated hardware and Combination of computer instructions to achieve.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
本申请涉及阴影消除装置和方法、空盘识别装置和方法。该阴影消除装置包括:图像获取模块,获取有补光图像和无补光图像;光照区域图像获取模块,根据所述有补光图像和所述无补光图像之间的差异,获得光照区域图像;边缘检测模块,对所述无补光图像进行边缘检测,获得边缘图像;以及阴影线消除模块,从所述边缘图像中去除与所述光照区域图像对应的部分,获得实际边缘图像。
Description
相关申请的交叉引用
本申请要求享有于2021年07月07日提交的名称为“阴影消除装置和方法、空盘识别装置和方法”的中国专利申请202110774907.3的优先权,该申请的全部内容通过引用并入本文中。
本申请涉及行李安检领域,尤其是,涉及阴影消除装置和方法、空盘识别装置和方法。
在传统的行李安检领域中,安检员需要引导旅客将行李放入托盘中,将托盘放上传送带,使得托盘通过传送带进入成像装置,由成像装置对行李以及托盘进行成像,通过图像识别来判断行李中是否包含禁止携带的物品。在旅客取走行李之后,安检员需要人工回收空托盘,以便重新利用。因此,存在安检员的工作负担变重,旅客通过安检的通过效率低,排队时间长等问题。另外,如果对于空托盘也进行安检,则也会加长安检时间。
从而,如果能实现空托盘的自动识别,有利于空托盘的自动回收,或者空托盘的快速排除,能够大大减轻安检员的工作负担,加快安检速度,改善旅客的安检体验。
发明内容
本申请的发明人经过调查研究发现,目前,能够实现空托盘的自动识别的技术主要有两种。一种是,基于激光测距的技术,通过激光测距来判断托盘是否为空,在这种情况下,需要依赖于密集的激光测距阵列,其成本非常高,从而实用性差。另一种是,基于图像识别来判断托盘是否为空 的技术,在这种情况下,容易错误地将设置在托盘中的广告、贴纸、图画等的干扰介质、光照导致的阴影等识别为行李,从而识别准确率低,有时还需要人工介入才能做到有效识别,从而导致自动化程度低,识别效率低。
本申请提出了一种能够识别出托盘上的光照阴影区域、从而能够提高空盘识别准确率的阴影自适应的空盘识别装置和方法。
第一方面,本申请的一个实施例提供了一种阴影消除装置,包括:图像获取模块,获取有补光图像和无补光图像;光照区域图像获取模块,根据有补光图像和无补光图像之间的差异,获得光照区域图像;边缘检测模块,对无补光图像进行边缘检测,获得边缘图像;以及阴影线消除模块,从边缘图像中去除与光照区域图像对应的部分,获得实际边缘图像。
根据本申请实施例的一个方面,有补光图像是通过红外线发光装置进行补光的状态下拍摄而得的。
根据本申请实施例的一个方面,光照区域图像获取模块在有补光图像和无补光图像的像素值的比值在预定阈值范围内时,判断该像素点属于光照区域图像。
根据本申请实施例的一个方面,边缘检测模块对无补光图像进行高斯模糊处理之后,进行边缘检测。
根据本申请实施例的一个方面,阴影线消除模块判断在边缘图像的任意像素点的邻域内的光照区域面积占比在预定面积阈值范围内时,去掉该像素点。
根据本申请实施例的一个方面,阴影线消除模块包括:邻域确定部,针对边缘图像中的每个像素点,确定该像素点的邻域;对应区域确定部,在光照区域图像中确定与邻域对应的区域;光照区域占比判断部,判断区域中的光照区域的面积占比是否在预定面积阈值范围内;边缘点去除部,消除面积占比位于预定面积阈值范围内的像素点;以及实际边缘图像输出部,根据第一剩余边缘点生成实际边缘图像并输出。
第二方面,本申请的另一个实施例提供一种空盘识别装置,包括:上述的阴影消除装置;以及空盘判断模块,基于实际边缘图像,判断是否为空的托盘,有补光图像和无补光图像是拍摄空的托盘来获得的。
根据本申请实施例的一个方面,还包括:模板匹配模块,将实际边缘图像与干扰介质图像模板进行匹配;以及对应区域消除模块,将实际边缘图像中与干扰介质图像模板对应的边缘点消除,得到第二剩余边缘点,空盘判断模块使用第二剩余边缘点来代替实际边缘图像,判断是否为空的托盘。
第三方面,本申请的另一个实施例提供一种空盘识别***,包括:发光装置;图像采集装置,在发光装置关闭的状态下对空的托盘进行拍摄,获得无补光图像,并在发光装置开启的状态下对空的托盘进行拍摄,获得有补光图像;上述的空盘识别装置。
本申请的一个实施例提供一种空盘识别***,包括:发光装置;图像采集装置,在发光装置关闭的状态下对空的托盘进行拍摄,获得无补光图像,并在发光装置开启的状态下对空的托盘进行拍摄,获得有补光图像;上述的空盘识别装置;以及干扰介质图像模板生成装置,通过对多张底盘图像进行边缘检测,生成包含干扰介质的边缘点的干扰介质图像模板。
第四方面,本申请的另一个实施例提供一种阴影消除方法,包括:图像获取步骤,获取有补光图像和无补光图像;光照差异步骤,根据有补光图像和无补光图像之间的差异,获得光照区域图像;边缘检测步骤,对无补光图像进行边缘检测,获得边缘图像;以及阴影线消除步骤,从边缘图像中去除与光照区域图像对应的部分,获得实际边缘图像。
根据本申请实施例的一个方面,有补光图像是通过红外线发光方法进行补光的状态下拍摄而得的。
根据本申请实施例的一个方面,在光照差异步骤中,在有补光图像和无补光图像的像素值的比值在预定阈值范围内时,判断该像素点属于光照区域图像。
根据本申请实施例的一个方面,在边缘检测步骤中,对无补光图像进行高斯模糊处理之后,进行边缘检测。
根据本申请实施例的一个方面,阴影线消除模块判断在边缘图像的任意像素点的邻域内的光照区域面积占比在预定面积阈值范围内时,去掉该像素点。
根据本申请实施例的一个方面,阴影线消除步骤包括:邻域确定步骤,针对边缘图像中的每个像素点,确定该像素点的邻域;对应区域确定步骤,在光照区域图像中确定与邻域对应的区域;光照区域占比判断步骤,判断区域中的光照区域的面积占比是否在预定面积阈值范围内;边缘点去除步骤,消除面积占比位于预定面积阈值范围内的像素点;以及实际边缘图像输出步骤,根据第一剩余边缘点生成实际边缘图像并输出。
第五方面,本申请的另一个实施例提供一种空盘识别方法,包括:上述的阴影消除方法;以及空盘判断步骤,基于实际边缘图像,判断是否为空的托盘,有补光图像和无补光图像是拍摄空的托盘来获得的。
根据本申请实施例的一个方面,还包括:模板匹配步骤,将实际边缘图像与干扰介质图像模板进行匹配;以及对应区域消除步骤,将实际边缘图像中与干扰介质图像模板对应的边缘点消除,得到第二剩余边缘点,空盘判断步骤代替实际边缘图像,而基于第二剩余边缘点,判断是否为空的托盘。
根据本申请实施例的一个方面,还包括:干扰介质图像模板生成步骤,通过对多张底盘图像进行边缘检测,生成包含干扰介质的边缘点的干扰介质图像模板。
第六方面,本申请的另一个实施例提供一种计算机可读存储介质,存储有程序,程序使得计算机执行以下处理:图像获取步骤,获取有补光图像和无补光图像;光照差异步骤,根据有补光图像和无补光图像之间的差异,获得光照区域图像;边缘检测步骤,对无补光图像进行边缘检测,获得边缘图像;以及阴影线消除步骤,从边缘图像中去除与光照区域图像对应的部分,获得实际边缘图像。
在本申请中,通过有补光图像和无补光图像,生成消除了光照所产生的边缘线的实际边缘图像,该实际边缘图像可以用于空盘识别中,从而减少光照对空盘识别的影响。
为了更清楚地说明本申请实施例的技术方案,下面将对本申请实施例中所需要使用的附图作简单地介绍,显而易见地,下面所描述的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据附图获得其他的附图。
图1是示出第一实施方式所涉及的空盘识别***的构成的示意图;
图2是示出第一实施方式所涉及的空盘识别装置30的实施例1的功能模块图;
图3是示出第一实施方式所涉及的空托盘识别方法的实施例1的流程图;
图4是示出第一实施方式所涉及的空托盘识别方法的对应区域消除步骤S34和空盘判断步骤S35的一个示例的流程图;
图5是示意性示出第一实施方式所涉及的空托盘识别方法的实施例1的一个示例的流程图;
图6是示出第一实施方式所涉及的空托盘识别方法的边缘检测步骤S32的一个示例的流程图;
图7是示出第一实施方式所涉及的空托盘识别方法的模板匹配步骤S33的一个示例的流程图;
图8是示出第一实施方式所涉及的空盘识别装置30的实施例2的功能模块图;
图9是示出第一实施方式所涉及的空托盘识别方法的实施例2的流程图;
图10是示出第一实施方式所涉及的空托盘识别方法的底盘轮廓提取步骤S36的一个示例的流程图;
图11是示出第一实施方式所涉及的空托盘识别方法的旋正步骤S37的一个示例的流程图;
图12是示出第二实施方式所涉及的干扰介质图像模板生成装置40的实施例1的功能模块图;
图13示出第二实施方式所涉及的干扰介质图像模板生成方法的实施例1的一个示例的流程图;
图14是示出第二实施方式所涉及的干扰介质图像模板生成装置40的实施例2的功能模块图;
图15是示出第二实施方式所涉及的干扰介质图像模板生成方法的实施例2的流程图;
图16是示出第二实施方式所涉及的干扰介质图像模板生成方法的最终模板生成步骤S42步骤的一个示例的流程图;
图17的(a)示出初始模板,图17的(b)示出最终模板;
图18是示出第二实施方式所涉及的空盘识别***的构成的示意图;
图19是示出第二实施方式所涉及的干扰介质图像模板生成装置40的实施例3的一个示例的功能模块图;
图20是示出第二实施方式所涉及的干扰介质图像模板生成方法的实施例3的流程图;
图21是示出第二实施方式所涉及的干扰介质图像模板生成方法的运行阶段更新步骤S43的一个示例的流程图;
图22是示出第三实施方式所涉及的空盘识别***的构成的示意图;
图23是示出第三实施方式所涉及的阴影消除装置70的实施例1的功能模块图;
图24是示出第三实施方式所涉及的阴影消除方法的实施例1的流程图;
图25的(a)~(d)示出第三实施方式所涉及的阴影消除方法的光照差异步骤S72的一个示例的流程图;
图26是示出第三实施方式所涉及的阴影消除方法的阴影线消除步骤S74的一个示例的流程图;
图27的(a)和(b)示出在进行阴影线消除步骤的处理前后的边缘图像的示意图;
图28是示出第三实施方式所涉及的空盘识别装置30”的实施例2的功能模块图;
图29是示出第三实施方式所涉及的空托盘识别方法的实施例2的流程图;
图30是地示出第三实施方式所涉及的空盘识别***的实施例3的功能模块图。
下面将参照附图更详细地描述本申请的示例性的实施方式或实施例。虽然附图中显示了本申请的示例性的实施例,然而应当理解,可以以各种形式实现本申请而不应被这里阐述的实施方式或实施例所限制。相反,提供这些实施方式或实施例是为了能够更清楚地理解本申请。
本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象,而不是用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换,以便这里描述的本申请的实施方式或实施例能够以除了图示或描述的以外的顺序实施。此外,术语“包括”和“具有”以及他们的任何变形,意图在于覆盖不排他的包含,例如,包含了一系列步骤或单元的过程、方法、***、产品或设备,不限于清楚地列出的步骤或单元,而是可以包括没有清楚地列出的其它步骤或单元。文中的相同或相似的标号表示具有相同或相似的功能的构成要素。
<第一实施方式>
为了解决现有的空盘识别技术对于设置有干扰介质的托盘的识别准确率较低的问题,本申请的第一实施方式提供一种能够对设置有干扰介质的托盘进行自适应检测的空盘识别装置、***和方法。在这里,“干扰介质”是指,例如,托盘上的广告、贴纸、印刷画等,能够对空盘识别带来干扰的、有别于托盘结构本身的介质。“设置有干扰介质”是指,可以通过粘贴、印刷等方式使得干扰介质附着在托盘上。
图1示出本实施方式所涉及的空盘识别***的构成的示意图。该空盘识别***100可以用于行李安检场景中。该空盘识别***100可以包括托盘传送装置10、图像采集装置20、以及空盘识别装置30。
如图1所述,托盘传送装置10可以被配置为沿着第一方向X(水平方向)传送托盘。在这里,托盘可以用于容纳行李。
图像采集装置20对经由托盘传送装置10通过图像采集区域21的托盘 进行图像采集。换言之,拍摄通过图像采集区域的托盘的影像。
在示例性实施例中,图像采集装置20可以是摄像头,更为具体来说,可以是拍摄静态图片的相机,也可以是拍摄视频的摄像机。换言之,在这里,图像既可以是静态图像,也可以是运动图像。
尽管图1中仅示出了一个图像采集模块,但是本领域技术人员应当理解,可以设置两个或更多个图像采集模块。
空盘识别装置30可以以硬件方式实现,或者以在一个或者多个处理器上运行的软件模块实现,或者以它们的组合实现。例如,空盘识别装置30可以由设置有处理器的台式计算机、平板电脑、智能电话、服务器等任何合适的电子设备,以软硬件相结合的方式实现。
空盘识别装置30的处理器可以执行后述的空盘识别方法。
空盘识别装置30还可以包括存储器(未示出)和通信模块(未示出)等。
空盘识别装置30的存储器可以存储执行后述的空盘识别方法的步骤,以及与用于进行空托盘识别相关的数据等。存储器可以是例如ROM(Read Only Memory image,只读存储器)、RAM(Random Access Memory,随机存取存储器)等。存储器具有用于执行以下空盘识别方法中的任何步骤的程序代码的存储空间。这些程序代码被处理器读取并执行时执行下述的空盘识别方法。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。这些计算机程序产品包括诸如硬盘,光盘(CD)、存储卡或者软盘之类的程序代码载体。这样的计算机程序产品通常为便携式或者固定存储单元。用于执行以下方法中的任何步骤的程序代码也可以通过网络进行下载。程序代码可以例如以适当形式进行压缩。
空盘识别装置30中的通信模块可以支持在空盘识别装置30与外部电子装置之间建立直接(例如,有线)通信信道或无线通信信道,并经由建立的通信信道执行通信。例如,通信模块经由网络向图像采集装置20发送指令,并且经由网络从图像采集装置20接收数据。
此外,空盘识别装置30还可以包括显示器、麦克风、扬声器等输出 部,以用于显示或播报托盘的状态(即,是空托盘还是非空托盘)、或者告警等。
第一实施方式的空盘识别装置30在托盘的底部设置有干扰介质的情况下,也能够准确地识别出空托盘。
另外,虽然本申请中示出了存在托盘传送装置10的示例,然而,本申请的空盘识别***也可以不包含托盘传送装置10,例如,可以人为地将托盘放到图像采集装置20的图像采集区域21。
(实施例1)
图2是示意性地示出第一实施方式所涉及的空盘识别装置30的实施例1的功能模块图。这些功能模块可以通过空盘识别装置30的处理器执行存储在空盘识别装置30的存储器中的程序来实现。
如图2所示,空盘识别装置30包括:图像获取模块31,从图像采集装置20获取托盘的图像,所述托盘在底部设置有干扰介质;边缘检测模块32,对图像中的底盘区域进行边缘检测,获得边缘图像;模板匹配模块33,将边缘图像与干扰介质图像模板进行匹配;对应区域消除模块34,将边缘图像中与干扰介质图像模板对应的边缘点消除,得到剩余边缘点;以及空盘判断模块35,基于剩余边缘点,判断是否为空的托盘。
以下,对在实施例1中空盘识别装置30所执行的空托盘识别方法进行说明。
图3是示意性示出第一实施方式所涉及的空托盘识别方法的实施例1的流程图。
在S31步骤中,图像获取模块31从图像采集装置20获取托盘的图像,该托盘在底部设置有干扰介质。
在这里,可以从图像采集装置20直接获取图像,也可以经由与图像采集装置20和空盘识别装置30通信的其他外部设备获取。例如,由图像采集装置20将所采集的图像上传至服务器,空盘识别装置30再从服务器下载等。
在S32步骤中,边缘检测模块32对图像中的底盘区域进行边缘检测,获得边缘图像。
在这里,可以通过本领域已有的技术对拍摄有底盘的图像确定底盘区域,并对该底盘区域使用已有的边缘检测技术进行边缘检测,获得边缘图像。
在S33步骤中,模板匹配模块33将边缘图像与干扰介质图像模板进行匹配,寻找最佳匹配模板。在这里,最佳匹配模板是所有干扰介质图像模板中与边缘图像的匹配度最好的模板。
具体来说,可以将在S32步骤中获得的边缘图像与存储在空盘识别装置30的存储部(未图示)中的干扰介质图像模板进行匹配。在这里,干扰介质图像模板包含干扰介质图像的边缘点,还可以包含由一些边缘点构成的边缘线。
在S34步骤中,对应区域消除模块34将边缘图像中与干扰介质图像模板对应的边缘点消除,得到剩余边缘点num。
在S35步骤中,空盘判断模块35基于剩余边缘点num,判断是否为空的托盘。
图4示出对应区域消除步骤S34和空盘判断步骤S35的一个示例。
在S341步骤中,消除边缘图像中与干扰介质图像模板对应的边缘点。
作为一个示例,假设与边缘图像中的每个边缘点(x
i,y
i)对应的最佳匹配模板上的像素点为(X
j,Y
j),在最佳匹配模板中查找该像素点(X
j,Y
j)的邻域内是否存在像素值表示是边缘点的点,若存在,则将像素点为(x
i,y
i)设为非边缘点,若不存在,则保持不变,如此,对边缘图像中的所有的边缘点进行遍历。在本申请中,边缘点用白色(像素值255)表示,因此,在S341步骤中,可以查找是否存在像素值255的点。另外,邻域可以是以该像素点为中心,边长为2K的正方形区域。
通过上述S341步骤的处理,能够从边缘图像中去除与最佳匹配模板相对应的边缘点。
在S342步骤中,计算边缘图像中经过S341步骤之后的剩余边缘点的数量,记为num。
另外,S34步骤在S341步骤之前,还可以包括裁剪步骤,在该裁剪步骤中可以将边缘图像裁剪为与干扰介质图像模板相同的尺寸。通过该处理, 可以排除由于图像大小导致的偏差,对判断结果的影响。
在S351步骤中,将剩余边缘点的阈值为Tnum,判断剩余边缘点的数量是否大于阈值Tnum。如果num>Tnum,则进入S352步骤,判断为托盘是非空的,如果num≤Tnum,则进入S353步骤,判断为托盘是空的。
另外,还可以将剩余边缘点的分布的集中程度设为S,并将集中程度的阈值设为Sth,如果num>Tnum且S>Sth,则判断为托盘是非空的,否则为托盘是空的。在剩余边缘点的个数多但这些剩余边缘点不集中的情况下,有可能并不是表示托盘中有行李,从而可能识别结果有误,通过在剩余边缘点的数量之外还考虑其集中程度,能够更加准确地识别出非空的托盘。
将上述的图3和图4结合,则可以获得图5。即,图5是示意性示出该实施例1的一个示例的流程图。
另外,虽然以上说明了通过现有技术来实现边缘检测步骤S32和模板匹配步骤S33的示例,然而,上述的边缘检测步骤S32和模板匹配步骤S33还可以通过以下方式实现。
在S32步骤中,针对拍摄有托盘的图像,通过图像识别来识别出托盘的底盘区域。边缘检测模块32针对该底盘区域的图像,先对底盘区域进行高斯模糊处理之后,选定两个阈值,通过双阈值canny边缘检测来获得边缘图像。
具体来说,图6示出边缘检测步骤S32的一个示例。
在S321步骤中,对输入的底盘区域的图像进行高斯模糊处理。
在S322步骤中,选择两个阈值T
1,T
2。
在这里,两个阈值可以通过机器学习对现场图像进行学习来自适应地选取。
在不同环境下,图像采集装置102所采集的图像在亮度上会存在明显差异,通过自适应地调整阈值T
1,T
2,能够减少弱光照等造成的非物品边缘被作为边缘图像来检测出来的情况,还可以适应图像在整体亮度上的差异。
另外,通过设定合理的阈值,能够在保持主要边缘不变的情况下,尽 可能的去除图像噪声的影响。
在S323步骤中,利用S322步骤中选择的两个阈值T
1,T
2,采用双阈值canny边缘检测算法,进行边缘检测,获得边缘图像。
通过上述的自适应双阈值的canny边缘检测算法,能够在保证准确率的情况下,尽可能的去除由于弱光照、噪声响应造成的非物品边缘的影响。
接下来,利用图7对模板匹配步骤S33的一个示例进行说明。
在S331步骤中,判断是否需要对干扰介质图像模板进行翻转。
在S332步骤中,对干扰介质图像模板进行翻转。在这里,可以对干扰介质图像模板进行180度翻转。这是由于,通过图像采集区域的托盘的角度可以进行限制,或者对于不同角度的底盘图像或边缘图像可以进行旋正,经过这些处理后,边缘图像是与干扰介质图像模板一样的形状和角度,然而,有可能出现左右颠倒或上下颠倒的情况,通过对干扰介质图像模板进行180度翻转,将翻转前的干扰介质图像模板和翻转后的干扰介质图像模板都与边缘图像进行比较,能够更有效地寻找最佳匹配模板。
在这里,如果所粘贴的干扰介质是中心对称图形,则可以不进行翻转处理,即可以没有S331~S332步骤。
在S333步骤中,将边缘图像分别与每一个干扰介质图像模板进行匹配,计算并保存模板匹配值。该模板匹配值表示边缘图像和干扰介质图像模板之间的匹配程度,具体而言是两者的边缘线、边缘点的匹配程度。
针对干扰介质图像模板,可以有两种存储方式。一种是,针对每一个干扰介质在存储部中预先存储一个对应的干扰介质图像模板;另一种是,针对每一个干扰介质在存储部中预先存储有翻转前和翻转后的两个干扰介质图像模板。
①在针对每一个干扰介质在存储部中预先存储一个对应的干扰介质图像模板的情况下,所获取的图像中的干扰介质的粘贴方向和干扰介质图像模板中的干扰介质的方向有可能不一致。
此时,可以通过比较所存储的干扰介质图像模板中的干扰介质的方向和边缘图像中的干扰介质的方向来判断两者是否一致,若不一致,则认为需要翻转。
在这种情况下,后述的S333步骤中,将翻转后的干扰介质图像模板与边缘图像进行匹配。
换言之,当存在N个干扰介质时,如果存储部中存储有N个干扰介质图像模板,判断是否需要对干扰介质图像模板进行翻转,若需要,对干扰介质图像模板进行翻转,将边缘图像分别与不进行翻转或进行翻转之后的共计N个干扰介质图像模板进行匹配。
②在针对每一个干扰介质在存储部中预先存储有翻转前和翻转后的两个干扰介质图像模板的情况下,在S331步骤中,也被判断为不需要进行翻转。即,当存在N个干扰介质时,存储有2N个干扰介质图像模板,则认为不需要进行翻转,而在后续的S333步骤中将边缘图像与所存储的2N个干扰介质图像模板进行匹配。
接下来,在S334步骤中,在对所有干扰介质图像模板计算模板匹配值之后,确定模板匹配值的最大值和最佳匹配位置。
在S335步骤中,将与模板匹配值的最大值对应的干扰介质图像模板作为最佳匹配模板输出。
在本实施例中,通过对托盘的底盘区域的图像进行边缘检测,并将边缘图像与干扰介质图像模板进行比较,从边缘图像中去掉与干扰介质图像模板对应的边缘点,基于剩余的边缘点判断是否为空盘,从而能够消除干扰介质的边缘线或边缘点对空盘识别的影响,在托盘上设置有干扰介质的情况下,也能够准确地识别托盘是否为空盘。
(实施例2)
图8是示意性地示出第一实施方式所涉及的空盘识别装置30的实施例2的功能模块图。
如图8所示,空盘识别装置30还包括:底盘轮廓提取模块36,从图像获取模块31所获取的图像中提取底盘区域轮廓;以及,旋正模块37,基于底盘轮廓提取模块36所输出的底盘区域轮廓的最小外接矩形,进行旋正,并剪切出边缘检测区域,作为底盘区域。
以下,对在实施例2中空盘识别装置30所执行的空托盘识别方法进行说明。
图9是示意性示出第一实施方式所涉及的空托盘识别方法的实施例2的流程图。图9中的S31步骤~S35步骤与实施例1(图3)相同。在这里不再详细描述。图9与图3的区别在于,在S31步骤和S32步骤之间,还具有S36步骤和S37步骤。
具体来说,在S36步骤中,底盘轮廓提取模块36从图像获取模块31所获取的图像中提取底盘区域轮廓。换言之,在S36步骤中,确定托盘的底盘区域的位置和范围,提取出作为底盘区域的范围的底盘区域轮廓。
图10示出底盘轮廓提取步骤S36的一个示例。
在S361步骤中,底盘轮廓提取模块36对所输入的图像进行自适应二值化,得到二值化图像。在这里,自适应二值化可以使用已有的任意算法。通过采用自适应二值化,可以适应环境变化。例如,图像根据环境的变化而变得或明或暗。
在S362步骤中,对二值化图像进行开操作或闭操作。
不同场景下,由于光照、托盘之间存在差异等因素影响,在S361步骤中获得的二值化图像的底盘边缘会出现不连续的情况。通过根据所获取的图像,选择适当的开闭核(操作核)的大小,判断应当对二值化图像进行开操作还是闭操作,从而进行适当的开操作或闭操作,由此能够使得轮廓线更加连续且完整,可选的是,形成闭合的曲线。
在S363步骤中,采用边缘检测算法获取边缘图像。
在这里,边缘检测算法可以采用已知的任意的边缘检测算法。
在S364步骤中,采用轮廓提取算法,提取边缘轮廓线,作为底盘区域轮廓。在这里,轮廓提取算法可以采用已知的任意的轮廓提取算法。
通过对输入的图像进行自适应二值化,再进行开操作或闭操作,从进行了这些处理之后的图像中提取轮廓线,能够提取连续性和完整性更好的轮廓线。
另外,在图10所示的例子中虽然未示出,但是可以将在S364步骤中提取的底盘区域轮廓直接输出,作为后续的S37步骤、或者S32步骤的输入,然而,也可以如图10所示的那样,还进行S365步骤的处理。
在S365步骤中,底盘轮廓提取模块36判断在S364步骤中提取出的 底盘区域轮廓是否超出预定阈值范围,若超出,则进入S366步骤,判断该托盘为非空,结束处理,若不超出,则进入S367步骤,输出该底盘区域轮廓。
在这里,预定阈值范围可以由上限值和下限值来表示。超出预定阈值范围表示小于下限值或者大于上限值。其中,上限值可以根据托盘的底盘的大小来预先设定。下限值可以根据盘的底盘的大小、通常粘贴的干扰介质的大小等来预先设定。
从而,S365步骤中根据该预定阈值范围来进一步判断在S364步骤中提取的边缘轮廓是否是表示底盘区域的轮廓线。如果底盘区域轮廓大于预定阈值范围的上限值,则S364步骤中提取的边缘轮廓可能不是表示底盘区域的轮廓线,而是尺寸大于托盘的底部的轮廓线,例如,可能表示托盘中有行李且该行李横跨在托盘外部等的情况。如果底盘区域轮廓小于预定阈值范围的下限值,则S364步骤中提取的边缘轮廓也可能不是表示底盘区域的轮廓线,而是尺寸相对于底盘区域来说非常小的轮廓线,例如,尺寸小于通常所粘贴的干扰介质的尺寸等情况。
通过进行S365步骤的处理,能够进一步提高底盘轮廓线的提取准确度。
另外,如果有多个位于预定阈值范围内的底盘区域轮廓,则选择面积最小的底盘区域轮廓输出。此时,可能检测出了多个与底盘区域相关的轮廓线,通过选择其中面积最小的轮廓线,能够避免多余轮廓线对后续操作的干扰,识别结果更为准确。
另外,上限值和下限值也可以根据托盘的底盘的大小来设定成比较接近的值,此时,能够快速、准确、唯一地识别出表示底盘区域的轮廓线。
通过进行上述S365~S367步骤的处理,能够快速排除非空的托盘,对于无法快速排除的托盘才进行后续的处理,从而能够整体上提高空盘识别效率。
返回图9,在37步骤中,旋正模块37基于底盘轮廓提取模块36所输出的底盘区域轮廓的最小外接矩形,进行旋正,并剪切出边缘检测区域,作为底盘区域。
图11示出旋正步骤S37的一个示例。
在S371步骤中,对在S36步骤中提取出的底盘区域轮廓获得最小外接矩形、旋转角度θ、长宽值以及中心点坐标(x,y)。
在这里,获得最小外接矩形的算法可以采用已有的任意的算法。
在S372步骤中,运用仿射变换绕最小外接矩形的中心点旋转θ°得到旋转后的底部区域的图像。
在S373步骤中,根据最小外接矩形长宽值和中心点坐标(x,y)计算得到裁剪区域信息,并裁剪出边缘检测区域。在这里,边缘检测区域表示要作为边缘检测对象的区域。虽然未图示,但是在S37步骤中,可以将S373步骤的边缘检测区域作为底盘图像直接输出给边缘检测模块32进行S32步骤的边缘检测处理。然而,可选的是,如图11所示那样还进行S374步骤的处理。
在S374步骤中,将边缘检测区域缩放至指定大小来作为底盘区域。在这里,指定大小根据干扰介质图像模板的大小来确定。
因受托盘自身之间差异、成像时的周围环境变化等因素的影响,在S373步骤中裁剪出的边缘检测区域的大小会存在微小的差异,通过进行S374步骤的缩放能够减少这样的差异,使得作为S32步骤的边缘检测对象的底盘区域的图像的一致性更高,能够减少所输入的图像的尺寸对识别结果的影响。
如图9所示,在S32步骤中,边缘检测模块32可以对经由S37步骤从旋正模块37输出的底盘区域为对象进行边缘检测。
另外,在实施例2中,虽然示出了同时具有底盘轮廓提取模块36和旋正模块37、并进行S36步骤和S37步骤的情况,然而,在本申请中,空盘识别装置30也可以在实施例1的基础上仅包含底盘轮廓提取模块36、且仅进行S36步骤的处理,并将底盘轮廓提取模块36输出的底盘区域轮廓直接作为边缘检测模块32的检测对象。此时,为了后续的模板匹配等,需要对托盘在托盘传送装置10上的放置方向、角度等进行一定限制,例如,按照指定的方向和角度将托盘放置在托盘传送装置10上等。
在本实施例中,通过对输入的图像进行底盘轮廓提取和旋正,能够快 速、准确地确定需要进行边缘检测的区域,从而能够减少边缘检测的工作量,而且能够使得该边缘图像的朝向与干扰介质图像模板更容易达到一致,使得模板匹配的效率更高且更准确,从而能够有效地提高空盘识别的效率和准确率。
<第二实施方式>
在上述的第一实施方式中,示出了一种能够识别干扰介质的空盘识别装置和方法,在第一实施方式的模板匹配步骤中,利用所存储的干扰介质图像模板与边缘图像进行了匹配。为了进行这样的模板匹配,需要预先存储有干扰介质图像模板,或者能够从外部设备获得干扰介质图像模板。因此,作为第二实施方式,提供一种干扰介质图像模板生成装置和方法。
然而,第二实施方式的应用场景不限于第一实施方式的场景,在需要识别托盘上干扰介质的场景中,通常,均需要预先生成干扰介质图像模板。
在第二实施方式中,进行干扰介质图像模板生成的干扰介质图像模板生成***可以具有如图1所示的空盘识别***相似的结构,其区别在于,代替空盘识别装置30,而包含干扰介质图像模板生成装置40,换言之,干扰介质图像模板生成***包括托盘传送装置10、图像采集装置20、以及干扰介质图像模板生成装置40。
该干扰介质图像模板生成装置40也可以与第一实施方式所述的空盘识别装置30同样地,可以由设置有处理器的台式计算机、平板电脑、智能电话、服务器等任何合适的电子设备,以软硬件相结合的方式实现。
该干扰介质图像模板生成装置40可以与第一实施方式的空盘识别装置30进行通信,将生成的干扰介质图像模板发送给空盘识别装置30使用。也可以将生成的干扰介质图像模板上传到服务器,空盘识别装置30可以从该服务器中下载这些干扰介质图像模板来使用。
干扰介质图像模板生成装置40能够通过对多张底盘图像进行边缘检测,来生成干扰介质图像模板。在这里,干扰介质图像模板包含干扰介质的边缘点,并且用于识别空的托盘,多张底盘图像是分别对多个空的托盘进行拍摄得到的,该托盘的底盘上设置有干扰介质。
以下,对干扰介质图像模板生成装置40的具体构成以及在该干扰介 质图像模板生成装置40中执行的干扰介质图像模板生成方法进行详细说明。
(实施例1)
图12是示意性地示出第二实施方式所涉及的干扰介质图像模板生成装置40的实施例1的功能模块图。
如图12所示,干扰介质图像模板生成装置40包括:初始模板生成模块41,通过对多张底盘图像进行边缘检测,并对同一位置处的边缘点的个数进行计数,根据计数结果来生成初始模板。干扰介质图像模板生成装置40将该初始模板作为干扰介质图像模板存储和/或输出。
具体来说,初始模板生成模块41可以包括:边缘检测部,对多张底盘图像进行边缘检测;计数部,对同一位置处的边缘点的个数进行计数;以及,模板生成部,根据计数结果来生成初始模板。
以下,对在实施例1中干扰介质图像模板生成装置40所执行的干扰介质图像模板生成方法进行说明。
在实施例1的干扰介质图像模板生成方法中,初始模板生成模块41通过对多张底盘图像进行边缘检测,并对同一位置处的边缘点进行计数,根据计数结果来生成初始模板。
图13示出实施例1的干扰介质图像模板生成方法的一个示例。
在S411步骤中,从图像采集装置20获取N张拍摄有空托盘的图像。在这里,获取图像的过程可以与第一实施方式的S31步骤相同。然而,在后续不进行图像的翻转而生成模板时,在S411步骤中,需要保持所获取的图像中的干扰介质的朝向是一致的。
在这里,由于考虑到托盘在不同姿态下,图像质量会存在差异,因此,针对每个托盘采集多张图像。
在S412步骤中,通过前处理操作获取托盘底部干扰介质区域,并对托盘底部干扰介质区域进行边缘检测,获取边缘图像。
在这里,前处理可以使用已有的操作,例如可以采用第一实施方式中的底盘轮廓提取步骤S36、旋正步骤S37等。
在这里,边缘检测可以采用与第一实施方式的S32步骤相同的方式。
在S413步骤中,生成一个与图像大小一致、且值为零的二维计数变量,利用该二维计数变量统计N张图像中的每个位置处的边缘点数量之和。具体来说,若图像I在位置(x,y)处为边缘点,则在二维计数变量的(x,y)处的值v加1,否则保持不变。即,该边缘点数量之和即为边缘点的计数值。该二维计数变量可以是二维向量V、或者二维图像的像素值、或者二维存储空间等。在这里,在对图像进行前处理的情况下,这里的图像大小为经过前处理之后的底盘区域的图像。
在S414步骤中,计算二维计数变量中的任意位置(x,y)处的边缘点数量的占比v/N。
在S415步骤中,判断在S414步骤中计算的占比v/N是否大于阈值T。若v/N>T,则进入S416步骤中,将该点保留至模板。例如,将模板中的对应位置的像素值设为255(白色)。否则,进入S417步骤,不将该点保留至模板。例如,将模板中的对应位置的像素值设为0(黑色)。
在S418步骤中,判断是否对二维计数变量中的所有位置均完成S414~S417步骤的处理,若是,则结束处理,并进入S419步骤,输出初始模板,若否,则返回S414步骤,继续进行下一个位置的处理。如此,遍历整个二维计数变量,获取初始模板。
其中,阈值T是为了得到干扰介质的边缘为目的而预先设定的值。可以根据多次试验来获得的最优值。该阈值的取值以获取的初始模板中的边缘的线条相对较细为宜。阈值T,可以根据托盘的使用状态等适当设定。具体来说,在托盘比较新,污渍或磨损等较少的情况下,除了托盘本身形状、干扰介质形状以外的其他的边缘较少,因此,T可以取较大的值,例如接近1的值,就可以有效地识别出底盘区域的边缘点。而在托盘上有较多污渍或磨损等情况下,污渍或磨损等导致的除了托盘本身形状、干扰介质形状以外的其他的边缘较多,因此,T需要取较小的值。在实际的使用过程中,托盘上的污渍会逐渐增加,因此,T通常使用小于1的值,对于日常使用中的托盘来说,根据测试结果,可选的是,T为0.3~0.4,此时,能够获得粗细程度适当的边缘。
在S413步骤~S418步骤中,虽然示出了将边缘点数量的占比v/N与阈 值T进行比较来判断是否保留边缘点。然而,也可以直接使用S414步骤中获得的边缘点的计数值与阈值进行比较来判断是否保留边缘点。
如上所述,通过对边缘图像中的各位置处的边缘点进行计数,基于该计数值生成初始模板。由此,能够生成包含有效地表示干扰介质图像的实际边缘的边缘点的初始模板。
因此,在本实施例中,通过对多张底盘图像进行边缘检测,并对同一位置处的边缘点进行计数,根据计数结果来生成初始模板,能够生成有效地表示干扰介质图像的边缘点的分布情况的干扰介质图像模板,通过这样的干扰介质图像模板可以实现干扰介质自适应的空盘识别。
(实施例2)
图14是示意性地示出第二实施方式所涉及的干扰介质图像模板生成装置40的实施例2的功能模块图。
如图14所示,干扰介质图像模板生成装置40除了初始模板生成模块41之外,还包括最终模板生成模块42。即,干扰介质图像模板生成装置40包括:初始模板生成模块41,通过对多张底盘图像进行边缘检测,并对同一位置处的边缘点的个数进行计数,根据计数结果来生成初始模板;以及,最终模板生成模块42,根据多张底盘图像中的每一张的边缘点和初始模板的边缘点之间的差异,获得剩余边缘点,根据剩余边缘点来更新初始模板,由此生成最终模板。在这里,干扰介质图像模板生成装置40代替所述初始模板而将所述最终模板作为所述干扰介质图像模板存储和/或输出。
具体来说,最终模板生成模块42可以包括:边缘图像获取部,获取N张托盘底部区域的边缘图像;对应区域消除部,将边缘图像中与初始模板对应的边缘点消除,得到剩余边缘点;以及模板更新部,根据剩余边缘点来更新初始模板。其中,边缘图像获取部还可以包括图像获取部,从获取N张拍摄了托盘的图像;边缘检测部,对该图像中的底盘区域进行边缘检测,获得边缘图像。
由此,可以看出最终模板生成模块42中的图像获取部、边缘检测部、对应区域消除部与第一实施方式中的空盘识别装置30中的图像获取模块 31、边缘检测模块32、对应区域消除模块34分别类似。两者的区别在于,一是,最终模板生成模块42获取拍摄有托盘的多张图像,并对多张图像分别进行边缘检测、对应区域消除等处理,而在空盘识别装置30中仅针对要进行空盘识别的一张图像进行处理;二是,在空盘识别装置30中进行对应区域消除的是经过模板匹配之后的最佳匹配模板,而在干扰介质图像模板生成装置40进行对应区域消除的是初始模板。
以下,对在实施例2中干扰介质图像模板生成装置40所执行的干扰介质图像模板生成方法进行说明。
图15是示意性示出第二实施方式所涉及的干扰介质图像模板生成方法的实施例2的流程图。
初始模板生成步骤S41的处理与在实施例1中说明的相同,因此省略重复说明。
在S42步骤中,最终模板生成模块42根据每一张底盘图像中的边缘点和初始模板的边缘点之间的差异,获得剩余边缘点,根据剩余边缘点来更新初始模板,由此生成最终模板。
具体来说,图16示出最终模板生成步骤S42的一个示例。
在S421步骤中,获取N张托盘底部区域的边缘图像。
在这里,由于在进行S41步骤之后再执行S42步骤,因此,可以直接获取S412步骤输出的边缘图像。这种情况下,能够减少计算量,加快运算速度。
另外,可以与图13所示的S411步骤和S412步骤相同的方式获得N张托盘底部区域的边缘图像。也可以与第一实施方式的图3(或图5)的图像获取步骤S31、边缘检测步骤S32步骤相同的方式分别获得N张托盘底部区域的边缘图像。此时,最终模板生成模块42的边缘图像获取部包括:图像获取部,从获取N张拍摄了托盘的图像;边缘检测部,对该图像中的底盘区域进行边缘检测,获得边缘图像。
返回图16,在S422步骤中,针对每一张从在S421步骤所获取的边缘图像,消除与初始模板对应的边缘,获取剩余边缘图像edge_left。
作为示例,可以通过邻域处理来寻找在底盘图像的边缘图像的边缘点 中与初始模板的边缘点对应的边缘点。
作为一个示例,在边缘图像中确定像素点P(x
E,y
E)的邻域P
l(x
E,y
E),并在初始模板中确定与像素点P(x
E,y
E)的邻域对应的区域P
l(x
M,y
M),若在P
l(x
M,y
M)中存在像素值表示是边缘点的点(例如,像素值为255的点),则将该像素点P(x
E,y
E)设为非边缘点,否则,保留边缘点,如此,对多张边缘图像中的所有的边缘点进行遍历。另外,邻域可以是以该像素点为中心,边长为l的正方形区域。
上述的S422步骤的处理与第一实施方式中的图4的S341步骤的处理的功能是相同的,均是消除边缘图像和模板中的对应的边缘点。因此,虽然在S422步骤和S341步骤中的具体示例不同,然而两者是可以互换使用的。
通过S422步骤的操作,可以有效避免因在模板上简单叠加边缘点,导致模板边缘***,影响判断的问题。
在S423步骤中,计算该边缘图像的剩余边缘图像edge_left的边缘点的数量num。该S423步骤的处理与第一实施方式中的图6的S343步骤的处理是相同的。
在S424步骤中,判断边缘点的数量num是否大于阈值Tnum。若否,则进入S425步骤,在S425步骤中,不更新模板。
通过在剩余边缘图像edge_left的边缘点的数量num小于或等于Tnum时,不更新模板,能够避免边缘图像和初始模板的重合程度较大时,还进行模板的更新,而仅在于边缘图像和初始模板的差异较大时进行更新,从而提高更新效率。
在S424步骤中判断为边缘点的数量num大于阈值Tnum(num>Tnum)时,进入S426步骤。
在S426步骤中,接收与边缘图像相同大小的一个计数图像img_count,并在该计数图像img_count中的对应的位置(对应的像素),对该张边缘图像的剩余边缘点进行计数。在这里,作为示例,若该对应的位置是剩余边缘点,则将该像素点的像素值作为计数值加一。
在S427步骤中,判断是否对所有边缘图像(例如,N张)完成了 S422~S426步骤的处理。若是,则进入S428步骤,否则,返回S422步骤,继续对下一个边缘图像进行S422~S426步骤的处理。
其中,除了对第一张边缘图像进行计数时在S426步骤中从外部接收计数图像img_count之外,从第二张边缘图像开始在已有的计数图像img_count上继续进行计数。因此,经过S427步骤之后的计数图像img_count的各像素点的像素值表示:对所有边缘图像的剩余边缘点,按照每个像素点进行计数后的计数值。
在S428步骤中,针对计数图像img_count中的每一个像素点,判断该像素点的像素值是否大于阈值Tcount。即,判断计数图像img_count中的各像素点处的计数值是否大于阈值Tcount。若像素值大于阈值Tcount,则进入S429步骤,否则,进入S425步骤。阈值Tcount可以根据底盘图像的张数来设定,也可以设定为恒定的值。从兼顾计算精度和计算效率的观点来说,可选的是,恒定的整数范围可以为3~4。
在S429步骤中,若某一像素点的像素值大于阈值Tcount,则将模板中的对应的像素点更新为边缘点。例如,将模板中的对应的像素点的像素值设为255,从而使得该像素点为白色,表示边缘点。
在S4210步骤中,判断是否已经对计数图像img_count中的所有的像素点完成了S428~S429步骤的处理。若是,则进入S4211步骤,输出更新后的最终模板,否则,返回S428步骤中,对计数图像img_count中的下一个像素点进行判断。
通过利用计数图像对多张边缘图像中均作为剩余边缘点的像素点进行计数,在该计数值超过一定阈值时才更新模板中的对应像素点,从而能够避免少数几张边缘图像中作为噪声偶尔出现的剩余边缘点被更新到模板中的情况。从而能够提高干扰介质图像模板的准确性,提高生成干扰介质图像模板的效率。
在S424步骤中判断为边缘点的数量num大于阈值Tnum(num>Tnum)时,还可以对剩余边缘图像edge_left进行膨胀处理。通过膨胀处理能够使得剩余边缘图像edge_left中的边缘线更粗,能够避免剩余边缘图像edge_left中的边缘线太细或多个边缘图像中的边缘点无法对齐从而计数 图像img_count的计数值过于分散的情况。换言之,能够避免多次输入无法对齐,从而在导致输入多张图像时,在污渍或划痕的情况下,也依然更新不成功的问题。
图17示出干扰介质图像模板生成装置所生成的干扰介质图像模板,为了比较最终模板生成步骤S42前后的效果,图17的(a)示出初始模板,图17的(b)示出最终模板。从图17可以看出,最终模板与初始模板相比多了一些边缘点,从而经过最终模板生成处理能够在干扰介质图像模板中补入在初始模板中未有效地识别出的边缘点。例如,在图17的(b)的右上方的多出来的边缘点可能是由于托盘上的污渍等造成的。
由此可知,在本实施例中,通过在生成初始模板的基础上,经由上述的最终模板生成步骤进一步更新模板来生成最终模板,从而能够在干扰介质图像模板中补入之前未识别出的边缘点,能够进一步提高干扰介质图像模板的准确度。
(实施例3)
第二实施方式也可以以空盘识别***的方式来实现。即,将上述第二实施方式的实施例1和实施例2与第一实施方式相结合。
如图18所示,空盘识别***包括托盘传送装置10、图像采集装置20、空盘识别装置30、以及干扰介质图像模板生成装置40。空盘识别装置30基于干扰介质图像模板生成装置40所生成的干扰介质图像模板进行模板匹配。
在图18中虽然示出干扰介质图像模板生成装置40和空盘识别装置30在物理上作为一体设置的情况,然而,干扰介质图像模板生成装置40和空盘识别装置30在物理上也可以分开设置。例如,可以将空盘识别装置30设置在安检现场的终端中,而将干扰介质图像模板生成装置40设置在服务器端等。
在这里,空盘识别装置30具有与第一实施方式相同的构成,并执行相同的处理,因此省略详细说明。
在这种情况下,在本实施例3中,干扰介质图像模板生成装置40在上述的实施例1或实施例2的基础上,还包括:运行阶段更新模块43。
图19是示意性地示出第二实施方式所涉及的干扰介质图像模板生成装置40的实施例3的一个示例的功能模块图。虽然图19中示出了同时具有初始模板生成模块41、最终模板生成模块42、运行阶段更新模块43的情况,但是也可以仅具有模板生成模块41和运行阶段更新模块43。
该运行阶段更新模块43所执行的处理与实施例2中的最终模板生成模块42类似,区别仅在于,运行阶段更新模块43在空盘识别装置的运行阶段动作,并且,仅在被输入了更新信号时,基于从实时托盘图像的边缘点中消除已有模板的边缘点得到的剩余边缘点,更新已有模板。
在这里,实时托盘图像是在空盘识别装置30的运行阶段,对在托盘传送装置10上传送的空的托盘通过图像采集装置20进行拍摄得到的,该托盘的底盘上设置有干扰介质。所谓的已有模板是上述的初始模板生成模块41输出的初始模板或最终模板生成模块42输出的最终模板。
具体来说,运行阶段更新模块43与最终模板生成模块42同样地可以包括:图像获取部、边缘检测部、对应区域消除部、模板更新部。其中,区别在于,一是,在图像获取部中从图像采集装置获取的是在空盘识别装置30的运行阶段实时采集的图像;二是,在进行对应区域消除时,所使用的不仅可以是初始模板生成模块41输出的初始模板,也可以是最终模板生成模块42输出的最终模板。
即,运行阶段更新模块43包括:图像获取部,从图像采集装置获取实时托盘图像;边缘检测部,对实时托盘图像中的底盘区域进行边缘检测,获得边缘图像;对应区域消除部,将边缘图像中与已有模板对应的边缘点消除,得到剩余边缘点;以及模板更新部,根据剩余边缘点来更新已有模板。已有模板是初始模板或最终模板。
另外,运行阶段更新模块43还包括更新信号判定部,该更新信号判定部,判断是否接收到更新信号。运行阶段更新模块43的模板更新部仅在更新信号判定部判断为接收到更新信号时,进行模板更新。
另外,运行阶段更新模块43还包括翻转判断部,该翻转判断部判断实时托盘图像中的干扰介质的方向是否与已有模板中的干扰介质的方向一致,当不一致时,判断为需要对边缘图像进行翻转,在进行翻转后,输入 到上述的对应区域消除部。即,后续的对应区域消除处理和模板更新处理基于翻转后的边缘图像来进行。
图20是示意性示出第二实施方式所涉及的干扰介质图像模板生成方法的实施例3的流程图。
图20所示的实施例3与图15所示的实施例2的区别在于,干扰介质图像模板生成方法还包括运行阶段模板更新步骤S43。
在S43步骤中,在空盘识别装置的运行阶段动作,仅在被输入了更新信号时,基于从实时托盘图像的边缘点中消除与已有模板的边缘点对应的边缘点来得到的剩余边缘点,更新已有模板。
具体来说,图21示出运行阶段更新步骤S43的一个示例。
在S430步骤中,从图像采集装置20获取N张拍摄有空托盘的实时托盘图像。
在S431步骤中,对托盘底部干扰介质区域进行边缘检测,获取边缘图像。
上述的S430步骤和S431步骤分别可以与实施例1的S411步骤和S412步骤相同,或者与第一实施方式中的S31步骤、S32步骤相同。
图21中的S432步骤~S435步骤与图16的S422步骤~S425步骤分别相同,因此省略说明。
上述的S432步骤~S433步骤也分别与第一实施方式中的S342步骤~S343(参考图4、图5)相同,上述的S434步骤也与第一实施方式中的S351步骤(参考图4、图5)相同。
即,上述的S430~S434步骤与图5中的S31~S351步骤分别相同。因此,运行阶段更新模块43可以不另外设置上述的图像获取部、边缘检测部、对应区域消除部,而与空盘识别装置30共用。即,将第一实施方式的空盘识别装置30的图像获取模块31、边缘检测模块32、对应区域消除模块34用作运行阶段更新模块43的图像获取部、边缘检测部、对应区域消除部。从而,能够简化运行阶段更新模块43的构成,减少计算量,加快运算速度。
接着,在S436步骤中,判断是否接收到更新信号。该更新信号可以 由空盘识别***的操作员来输入。例如,当发现由托盘传送装置10正在运送并且要进入图像采集装置20的托盘为空托盘时,输入更新信号。
在图20所示的示例中,在S430步骤~S434步骤之后执行S426步骤,然而,也可以在S430步骤之前、或者在S432步骤之前执行S426步骤。然而,由于如上所述的那样,上述的S430~S434步骤与图5中的S31~S351步骤分别相同。因此,在与空盘识别装置30共用图像获取部、边缘检测部、对应区域消除部的情况下,在图像获取部、边缘检测部、对应区域消除部的处理结束之后,再判断是否收到更新信号,因此,不影响控盘识别装置30的正常工作,就能够实现模板的更新。
在S437步骤中,判断是否需要翻转。
具体来说,判断所采集的实时托盘图像中的干扰介质的方向是否与已有模板中的干扰介质的方向一致,当不一致时,判断为需要对边缘图像进行翻转,进入S438步骤。
在S438步骤中,对边缘图像进行翻转。利用翻转后的边缘图像进行后续的S439步骤。
图21中的S439步骤~S4314步骤与图16的S426步骤~S4211步骤分别相同,因此省略说明。
在本实施例中,由于在空盘识别***的运行阶段也能够对干扰介质图像模板进行更新,从而能够根据托盘上的干扰介质的磨损等来实时更新干扰介质图像模板,能够进一步提高干扰介质图像模板的准确度,从而进一步提高空盘识别的准确度。
<第三实施方式>
为了解决现有的空盘识别技术对于存在光照导致的阴影的托盘的识别准确率较低的问题,本申请的第三实施方式提供一种能够对存在光照导致的阴影的托盘进行自适应检测的空盘识别装置、***和方法。
图22示出本实施方式所涉及的空盘识别***的构成的示意图。如图22所示,本实施方式的空盘识别***可以包括托盘传送装置10、图像采集装置20、以及空盘识别装置30’、发光装置50。其中,空盘识别装置30’包括:空盘判断模块60和阴影消除装置70。
其中,托盘传送装置10和图像采集装置20与第一、第二实施方式相同,因此省略说明。
发光装置50是能够向图像采集装置20的图像采集区域21照射光的装置。发光装置50可以作为图像采集装置20的补光灯来发挥作用。该光只要能够对图像采集装置的图像采集进行补光,使得有补光图像与无补光图像有区别即可。
可选的是,发光装置50可以是红外线发光装置。在发光装置50为红外线发光装置的情况下,有补光图像中的像素值的变化相对于无补光图像更加明显,有利于进行阴影消除。
虽然在图22中图像采集装置20和发光装置50在物理上是分开的,然而,图像采集装置20和发光装置50在物理上也可以是一体的。
空盘识别装置30’与第一、第二实施方式的空盘识别装置30同样地,可以由设置有处理器的台式计算机、平板电脑、智能电话、服务器等任何合适的电子设备,以软硬件相结合的方式实现。
本实施方式的空盘识别装置30’与第一、第二实施方式的空盘识别装置30的区别在于,包括阴影消除装置70。
以下,对空盘识别装置30’所具有的阴影消除装置70进行详细说明。
(实施例1)
图23是示意性地示出第三实施方式所涉及的阴影消除装置70的实施例1的功能模块图。
如图23所示,阴影消除装置70包括:图像获取模块71,获取有补光图像和无补光图像;光照区域图像获取模块72,根据有补光图像和无补光图像之间的差异,获得光照区域图像;边缘检测模块73,对无补光图像进行边缘检测,获得边缘图像;以及阴影线消除模块74,从边缘图像中去除与光照区域图像对应的部分,获得实际边缘图像。
在这里,有补光是指进行了灯光补偿。具体来说,无补光图像是指图像采集装置20直接采集的图像。有补光图像是指对图像采集装置20的图像采集进行了灯光补偿的图像。
例如,在如图1所示的空盘识别***具中,无补光图像是指未利用发 光装置50对图像采集装置20的图像采集区域21进行补光的状态下由图像采集装置20采集的图像。有补光图像是在利用发光装置50对图像采集装置20的图像采集区域21进行了补光的状态下由图像采集装置20采集的图像。
(实施例1)
图24是示意性示出本实施方式所涉及的阴影消除方法的实施例1的流程图。
在S71步骤中,图像获取模块71从获取有补光图像和无补光图像。
在S72步骤中,光照区域图像获取模块72根据有补光图像和无补光图像之间的差异,获得光照区域图像。
具体来说,光照区域图像获取模块72可以在有补光图像和无补光图像的像素值的比值在预定阈值范围内时,判断该像素点属于所述光照区域图像。
在这里,该预定阈值范围为1附近的范围,例如,可选的是0.9~1.1。有补光图像和无补光图像的像素值的比值在1附近表示有补光图像和无补光图像之间的差异不明显。
在这里,光照区域是指空托盘图像中受到阳光照射的区域。阳光照射是形成阴影的主要原因。因此,通过判断光照区域,能够有效地消除由于阳光照射而产生的阴影对空盘识别的影响。
在采用红外线相机作为发光装置50进行补光的情况下,阳光光强远远大于红外补光和环境光的光照,在补光前后,被阳光照射的区域,像素值差异不明显,而未被阳光照射的区域,像素值差异明显。因此,在本申请中,通过有补光图像和无补光图像中的相同像素的像素值的变化,能够识别出光照区域。
以下,参考图25的(a)~(d)来详细说明光照差异步骤S72的一个示例。
将在S71步骤中获取的有补光图像设为L(参考图25的(a)),将无补光图像设为NL(参考图25的(b))。
在S72步骤中,计算有补光图像L和无补光图像NL中的每个相同像 素点的像素值的比值r=NL/L,由此得到比值图像R(参考图25的(c))。如果该像素点的比值r在预定阈值范围内,则认为该像素点位于光照区域,通过将该像素点的像素值设为255(白色)。否则,认为该像素点位于阴影区域,将该像素点的像素值设为0(黑色),由此获得光照区域图像RL(参考图25的(d))。在这里,该预定阈值范围也为1附近的范围,例如,可选的是0.9~1.1。
在S73步骤中,边缘检测模块73对无补光图像NL进行边缘检测,获得其边缘图像NLE。
由于S73步骤中的边缘检测可以采用与第一实施方式的S32步骤相同的方式,因此,省略详细说明。
在S73步骤中,边缘检测模块73也与第一实施方式同样地,对无补光图像进行高斯模糊处理之后,进行边缘检测。
在S74步骤中,阴影线消除模块74从边缘图像NLE中去除与光照区域图像R对应的部分,获得实际边缘图像。
在S74步骤中,阴影线消除模块74可以判断在边缘图像的任意像素点的邻域内的光照区域的面积占比在预定面积阈值范围内时,去掉该像素点。
作为一个示例,阴影线消除模块74可以包括:邻域确定部,针对边缘图像中的每个像素点,确定该像素点的邻域;对应区域确定部,在所述光照区域图像中确定与所述邻域对应的区域;光照区域占比判断部,判断所述区域中的光照区域的面积占比是否在预先设定的面积阈值范围内;边缘点去除部,消除面积占比位于所述阈值范围内的像素点;以及实际边缘图像输出部,根据剩余边缘点生成实际边缘图像并输出。
利用图26,对该阴影线消除模块74所执行的阴影线消除步骤S74的具体示例进行说明。
在S741步骤中,邻域确定部针对边缘图像NLE中的任意像素点,确定该像素点的邻域。
具体来说,将边缘图像NLE的像素点设为P(x
NLE,y
NLE),确定以该像素点P(x
NLE,y
NLE)为中心、且边长为l的矩形区域的邻域 P
l(x
NLE,y
NLE)。
在S742步骤中,对应区域确定部在光照区域图像RL中确定与上述的该像素点的邻域对应的区域。
具体来说,在光照区域图像RL中,确定与邻域P
l(x
NLE,y
NLE)对应的区域,并表示为P
l(x
RL,y
RL)。
在S743步骤中,光照区域占比判断部判断区域P
l(x
RL,y
RL)中的光照区域的面积占比是否在预先设定的面积阈值范围内。若是,则进入S745步骤,若否,则进入S744步骤。该面积占比是在区域P
l(x
RL,y
RL)中,光照区域的面积在整个区域中所占的比例。
光照区域的面积占比在预先设定的面积阈值范围内表示与该像素点的邻域对应的光照区域图像中的区域P
l(x
RL,y
RL)内既有光照区域,又有阴影区域,换言之,该区域内的边缘是阴影边缘,从而,该像素点很可能是阴影边缘,而非物品、污渍、划痕等的边缘,因此,该像素点不应当作为边缘点。
在S744步骤中,边缘点去除部将该像素点判断为边缘点,保留该边缘点。即,将该像素点P(x
NLE,y
NLE)的像素值保持原来的值(例如,保持为255)。
在S745步骤中,边缘点去除部从边缘图像NLE中去掉该像素点P(x
NLE,y
NLE)。例如,将该像素点P(x
NLE,y
NLE)的像素值设为0(黑色)。
在S746步骤中,判断是否对边缘图像NLE中的所有像素点执行了上述S741~745步骤的处理。
在S747步骤,实际边缘图像输出部根据进行上述操作之后的剩余边缘点生成实际边缘图像并输出。
图27的(a)示出进行S74步骤的阴影线消除处理之前的边缘图像NLE,图27的(b)示出从边缘图像NLE中通过S74的阴影线消除处理去掉阴影线之后的实际边缘图像。通过图27的(a)和(b),经由上述阴影线消除处理,有效地消除了阴影所造成的阴影线。
空盘识别装置30’的空盘判断模块60可以基于阴影消除装置70所输出的实际边缘图像,判断该托盘是否为空托盘。
在这里,基于实际边缘图像来判断是否为空托盘的空盘判断模块60可以采用已有的任意的构成。例如,可以采用与第一实施方式的空盘判断模块35相同的构成。该空盘判断模块35可以将阴影消除装置70所输出的实际边缘图像的各边缘点作为剩余边缘点,采用例如S351~S353步骤等来判断是否为空托盘。
在实施例1中,通过有补光图像和无补光图像,生成消除了光照所产生的边缘线的实际边缘图像,该实际边缘图像可以用于空盘识别中,从而减少光照对空盘识别的影响。
(实施例2)
实施例1示出了能够进行阴影消除,并基于阴影消除之后的剩余边缘点进行空盘识别的空盘识别装置30’。然而,也可以在消除光照所造成的边缘点的同时,再消除与干扰介质对应的边缘点,之后再进行空盘识别。即,可以将上述第三实施方式的实施例1与第一实施方式相结合,在这里,作为实施例2进行说明。
图28是示意性地示出第三实施方式所涉及的空盘识别装置30”的实施例2的功能模块图。
如图28所示,本实施例的空盘识别装置30”除了实施例1的阴影消除装置70之外,还包括与第一实施方式相同的模板匹配模块33、对应区域消除模块34、空盘判断模块35。
换言之,空盘识别装置30”包括:阴影消除装置70,根据有补光图像和无补光图像,从无补光图像中消除与阴影区域对应的边缘点,获得实际边缘图像;模板匹配模块33,将实际边缘图像与干扰介质图像模板进行匹配;对应区域消除模块34,将实际边缘图像中与干扰介质图像模板对应的边缘点消除,得到剩余边缘点;以及空盘判断模块35,基于剩余边缘点,判断是否为空的托盘。
图29是示意性示出第三实施方式所涉及的空托盘识别方法的实施例2的流程图。
如图29所示,本实施例的空盘识别方法除了与实施例1的图23相同的步骤S71~S74步骤之外,还包括与第一实施方式的图3相同的步骤 S33~S35步骤。
在S33步骤中,将S74步骤输出的实际边缘图像与干扰介质图像模板进行匹配。
在S34步骤中,将该实际边缘图像中与干扰介质图像模板对应的边缘点消除,得到剩余边缘点。
在S35步骤中,基于该剩余边缘点,判断是否为空的托盘。
通过该实施例2,不仅能够消除光照的影响,还能够消除干扰介质的影响,从而能够进一步提高空盘识别的准确性。
(实施例3)
本实施方式的实施例2示出了将实施例1与第一实施方式相结合的实施例。然而,本实施方式还可以同时与第一、第二实施方式相结合来实施。因此,在此,作为实施例3,说明将实施例1同时与第一、第二实施方式相结合的情况。
图30是示意性地示出第三实施方式所涉及的空盘识别***的实施例3的功能模块图。
如图30所示,本实施例的空盘识别***包括:与实施例2相同的空盘识别装置30”和发光装置50、图像采集装置20、托盘传送装置10、以及与第二实施方式相同的干扰介质图像模板生成装置40。
由于其各构成部分在上述的实施方式和实施例中已经进行了详细说明,因此不再重复说明。
通过该实施例3,不仅能够消除光照和干扰介质的影响,还能够生成干扰介质图像模板,从而能够进一步提高空盘识别的准确性。
上面参考根据本申请的实施例的方法、装置(***)和计算机程序产品的流程图和/或框图描述了本申请的各方面。应当理解,流程图和/或框图中的每个方框以及流程图和/或框图中各方框的组合可以由计算机程序指令实现。这些计算机程序指令可被提供给通用计算机、专用计算机、或其它可编程数据处理装置的处理器,以产生一种机器,使得经由计算机或其它可编程数据处理装置的处理器执行的这些指令使能对流程图和/或框图的一个或多个方框中指定的功能/动作的实现。这种处理器可以是但不限于是 通用处理器、专用处理器、特殊应用处理器或者现场可编程逻辑电路。还可理解,框图和/或流程图中的每个方框以及框图和/或流程图中的方框的组合,也可以由执行指定的功能或动作的专用硬件来实现,或可由专用硬件和计算机指令的组合来实现。
以上,虽然结合附图描述了本申请的实施方式和具体实施例,但是本领域技术人员可以在不脱离本申请的精神和范围的情况下做出各种修改和变形,这样的修改和变形均落入由所述权利要求所限定的范围之内。
Claims (20)
- 一种阴影消除装置,包括:图像获取模块,获取有补光图像和无补光图像;光照区域图像获取模块,根据所述有补光图像和所述无补光图像之间的差异,获得光照区域图像;边缘检测模块,对所述无补光图像进行边缘检测,获得边缘图像;以及阴影线消除模块,从所述边缘图像中去除与所述光照区域图像对应的部分,获得实际边缘图像。
- 如权利要求1所述的阴影消除装置,其中,所述有补光图像是通过红外线发光装置进行补光的状态下拍摄而得的。
- 如权利要求1所述的阴影消除装置,其中,所述光照区域图像获取模块在所述有补光图像和所述无补光图像的像素值的比值在预定阈值范围内时,判断该像素点属于所述光照区域图像。
- 如权利要求1所述的阴影消除装置,其中,所述边缘检测模块对所述无补光图像进行高斯模糊处理之后,进行边缘检测。
- 如权利要求1所述的阴影消除装置,其中,所述阴影线消除模块判断在所述边缘图像的任意像素点的邻域内的所述光照区域面积占比在预定面积阈值范围内时,去掉该像素点。
- 如权利要求5所述的阴影消除装置,其中,所述阴影线消除模块包括:邻域确定部,针对边缘图像中的每个像素点,确定该像素点的邻域;对应区域确定部,在所述光照区域图像中确定与所述邻域对应的区域;光照区域占比判断部,判断所述区域中的光照区域的面积占比是否在预定面积阈值范围内;边缘点去除部,消除面积占比位于所述预定面积阈值范围内的像素点;以及实际边缘图像输出部,根据第一剩余边缘点生成实际边缘图像并输出。
- 一种空盘识别装置,包括:所述权利要求1至6中的任一项所述的阴影消除装置;以及空盘判断模块,基于所述实际边缘图像,判断是否为空的托盘,所述有补光图像和所述无补光图像是拍摄空的托盘来获得的。
- 如权利要求7所述的空盘识别装置,还包括:模板匹配模块,将所述实际边缘图像与干扰介质图像模板进行匹配;以及对应区域消除模块,将所述实际边缘图像中与所述干扰介质图像模板对应的边缘点消除,得到第二剩余边缘点,所述空盘判断模块使用所述第二剩余边缘点来代替实际边缘图像,判断是否为空的托盘。
- 一种空盘识别***,包括:所述权利要求7或8所述的空盘识别装置;发光装置;以及图像采集装置,在所述发光装置关闭的状态下对空的托盘进行拍摄,获得所述无补光图像,并在所述发光装置开启的状态下对空的托盘进行拍摄,获得所述有补光图像。
- 一种空盘识别***,包括:所述权利要求8所述的空盘识别装置;发光装置;图像采集装置,在所述发光装置关闭的状态下对空的托盘进行拍摄,获得所述无补光图像,并在所述发光装置开启的状态下对空的托盘进行拍摄,获得所述有补光图像;以及干扰介质图像模板生成装置,通过对多张底盘图像进行边缘检测,生成包含干扰介质的边缘点的所述干扰介质图像模板。
- 一种阴影消除方法,包括:图像获取步骤,获取有补光图像和无补光图像;光照差异步骤,根据所述有补光图像和所述无补光图像之间的差异,获得光照区域图像;边缘检测步骤,对所述无补光图像进行边缘检测,获得边缘图像;以及阴影线消除步骤,从所述边缘图像中去除与所述光照区域图像对应的部分,获得实际边缘图像。
- 如权利要求11所述的阴影消除方法,其中,所述有补光图像是通过红外线发光方法进行补光的状态下拍摄而得的。
- 如权利要求11所述的阴影消除方法,其中,在所述光照差异步骤中,在所述有补光图像和所述无补光图像的像素值的比值在预定阈值范围内时,判断该像素点属于所述光照区域图像。
- 如权利要求11所述的阴影消除方法,其中,在所述边缘检测步骤中,对所述无补光图像进行高斯模糊处理之后,进行边缘检测。
- 如权利要求11所述的阴影消除方法,其中,所述阴影线消除模块判断在所述边缘图像的任意像素点的邻域内的所述光照区域面积占比在预定面积阈值范围内时,去掉该像素点。
- 如权利要求15所述的阴影消除方法,其中,所述阴影线消除步骤包括:邻域确定步骤,针对边缘图像中的每个像素点,确定该像素点的邻域;对应区域确定步骤,在所述光照区域图像中确定与所述邻域对应的区域;光照区域占比判断步骤,判断所述区域中的光照区域的面积占比是否在预定面积阈值范围内;边缘点去除步骤,消除面积占比位于所述预定面积阈值范围内的像素点;以及实际边缘图像输出步骤,根据第一剩余边缘点生成实际边缘图像并输出。
- 一种空盘识别方法,包括:所述权利要求1至16中的任一项所述的阴影消除方法;以及空盘判断步骤,基于所述实际边缘图像来判断是否为空的托盘,所述有补光图像和所述无补光图像是拍摄空的托盘来获得的。
- 如权利要求17所述的空盘识别方法,还包括:模板匹配步骤,将所述实际边缘图像与干扰介质图像模板进行匹配;以及对应区域消除步骤,将所述实际边缘图像中与所述干扰介质图像模板对应的边缘点消除,得到第二剩余边缘点,所述空盘判断步骤基于所述第二剩余边缘点,判断是否为空的托盘,以代替基于所述实际边缘图像来判断是否为空的托盘。
- 如权利要求17所述的空盘识别方法,还包括:干扰介质图像模板生成步骤,通过对多张底盘图像进行边缘检测,生成包含干扰介质的边缘点的所述干扰介质图像模板。
- 一种计算机可读存储介质,存储有程序,所述程序使得计算机执行以下处理:图像获取步骤,获取有补光图像和无补光图像;光照差异步骤,根据所述有补光图像和所述无补光图像之间的差异,获得光照区域图像;边缘检测步骤,对所述无补光图像进行边缘检测,获得边缘图像;以及阴影线消除步骤,从所述边缘图像中去除与所述光照区域图像对应的部分,获得实际边缘图像。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP22836831.2A EP4369286A1 (en) | 2021-07-07 | 2022-07-01 | Shadow elimination device and method, empty disk recognition device and method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202110774907.3 | 2021-07-07 | ||
CN202110774907.3A CN115601245B (zh) | 2021-07-07 | 2021-07-07 | 阴影消除装置和方法、空盘识别装置和方法 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023280080A1 true WO2023280080A1 (zh) | 2023-01-12 |
Family
ID=84801090
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/103447 WO2023280080A1 (zh) | 2021-07-07 | 2022-07-01 | 阴影消除装置和方法、空盘识别装置和方法 |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP4369286A1 (zh) |
CN (1) | CN115601245B (zh) |
WO (1) | WO2023280080A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116664413A (zh) * | 2023-03-27 | 2023-08-29 | 北京拙河科技有限公司 | 一种基于阿贝尔收敛算子的图像体积雾消除方法及装置 |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116055900B (zh) * | 2023-03-30 | 2023-06-09 | 北京城建智控科技股份有限公司 | 一种基于摄像装置的图像质量修正方法 |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2017123069A (ja) * | 2016-01-07 | 2017-07-13 | キヤノン株式会社 | 画像処理装置および方法、および撮像装置 |
CN107146210A (zh) * | 2017-05-05 | 2017-09-08 | 南京大学 | 一种基于图像处理的检测去除阴影方法 |
CN107451546A (zh) * | 2017-07-14 | 2017-12-08 | 广东欧珀移动通信有限公司 | 虹膜识别方法及相关产品 |
JP2018098583A (ja) * | 2016-12-09 | 2018-06-21 | キヤノン株式会社 | 画像処理装置及び方法、及び撮像装置 |
US20200126202A1 (en) * | 2018-10-23 | 2020-04-23 | International Business Machines Corporation | Detection for abnormal connectivity on a product |
CN113033545A (zh) * | 2019-12-24 | 2021-06-25 | 同方威视技术股份有限公司 | 空托盘识别方法及其装置 |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8610670B2 (en) * | 2006-08-31 | 2013-12-17 | Japan Display West Inc. | Imaging and display apparatus, information input apparatus, object detection medium, and object detection method |
CN101236606B (zh) * | 2008-03-07 | 2010-12-08 | 北京中星微电子有限公司 | 视频监控中的阴影消除方法及*** |
CN101763513A (zh) * | 2010-02-26 | 2010-06-30 | 成都三泰电子实业股份有限公司 | 一种消除光照影响的前景提取方法 |
CN101860664A (zh) * | 2010-06-13 | 2010-10-13 | 华中科技大学 | 一种对光照变化鲁棒的运动目标检测方法 |
TWI451342B (zh) * | 2010-10-29 | 2014-09-01 | Univ Nat Chiao Tung | Shadow Removal Method in Mobile Light Source Environment |
CN103363410B (zh) * | 2012-04-06 | 2016-04-20 | 重庆稼砷光电科技有限公司 | 一种远近光一体的车灯 |
CN103942812B (zh) * | 2014-03-12 | 2017-02-08 | 华南理工大学 | 基于混合高斯与边缘检测的运动目标检测方法 |
CN106203305B (zh) * | 2016-06-30 | 2020-02-04 | 北京旷视科技有限公司 | 人脸活体检测方法和装置 |
CN107067423A (zh) * | 2016-12-16 | 2017-08-18 | 天津大学 | 一种适用于开放赋存环境的文物本体微变监测的方法 |
CN111496255B (zh) * | 2020-06-16 | 2022-06-10 | 中国航空制造技术研究院 | 一种激光选区熔化图像数据采集装置 |
-
2021
- 2021-07-07 CN CN202110774907.3A patent/CN115601245B/zh active Active
-
2022
- 2022-07-01 EP EP22836831.2A patent/EP4369286A1/en active Pending
- 2022-07-01 WO PCT/CN2022/103447 patent/WO2023280080A1/zh active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2017123069A (ja) * | 2016-01-07 | 2017-07-13 | キヤノン株式会社 | 画像処理装置および方法、および撮像装置 |
JP2018098583A (ja) * | 2016-12-09 | 2018-06-21 | キヤノン株式会社 | 画像処理装置及び方法、及び撮像装置 |
CN107146210A (zh) * | 2017-05-05 | 2017-09-08 | 南京大学 | 一种基于图像处理的检测去除阴影方法 |
CN107451546A (zh) * | 2017-07-14 | 2017-12-08 | 广东欧珀移动通信有限公司 | 虹膜识别方法及相关产品 |
US20200126202A1 (en) * | 2018-10-23 | 2020-04-23 | International Business Machines Corporation | Detection for abnormal connectivity on a product |
CN113033545A (zh) * | 2019-12-24 | 2021-06-25 | 同方威视技术股份有限公司 | 空托盘识别方法及其装置 |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116664413A (zh) * | 2023-03-27 | 2023-08-29 | 北京拙河科技有限公司 | 一种基于阿贝尔收敛算子的图像体积雾消除方法及装置 |
CN116664413B (zh) * | 2023-03-27 | 2024-02-02 | 北京拙河科技有限公司 | 一种基于阿贝尔收敛算子的图像体积雾消除方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
CN115601245A (zh) | 2023-01-13 |
CN115601245B (zh) | 2023-12-12 |
EP4369286A1 (en) | 2024-05-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2023280080A1 (zh) | 阴影消除装置和方法、空盘识别装置和方法 | |
CN111369545B (zh) | 边缘缺陷检测方法、装置、模型、设备及可读存储介质 | |
CN111582085B (zh) | 单据拍摄图像识别方法及装置 | |
US7894689B2 (en) | Image stitching | |
US8983200B2 (en) | Object segmentation at a self-checkout | |
CN101464418B (zh) | 缺陷检测方法以及缺陷检测装置 | |
US20200240927A1 (en) | Dirt detection on screen | |
US20170011272A1 (en) | Realtime object measurement | |
CN111767908B (zh) | 字符检测方法、装置、检测设备及存储介质 | |
CN109829510A (zh) | 一种产品品质分级的方法、装置和设备 | |
WO2023280081A1 (zh) | 空盘识别装置和方法 | |
CN111932462B (zh) | 图像降质模型的训练方法、装置和电子设备、存储介质 | |
CN115100104A (zh) | 玻璃油墨区的缺陷检测方法、装置、设备及可读存储介质 | |
CN109975307A (zh) | 基于统计投影训练的轴承表面缺陷检测***及检测方法 | |
CN113033558A (zh) | 一种用于自然场景的文本检测方法及装置、存储介质 | |
CN113920434A (zh) | 一种基于目标的图像翻拍检测方法、装置及介质 | |
CN115830604A (zh) | 面单图像矫正方法、装置、电子设备及可读存储介质 | |
CN113361657A (zh) | 用于识别烟箱箱号的装置、方法、烟丝装箱设备和工控机 | |
US20170352170A1 (en) | Nearsighted camera object detection | |
CN115619783B (zh) | 产品加工缺陷的检测方法及装置、存储介质、终端 | |
CN116681677A (zh) | 一种锂电池缺陷检测方法、装置及*** | |
CN116645351A (zh) | 一种复杂场景的缺陷在线检测方法及*** | |
WO2022056875A1 (zh) | 一种铭牌图像的分割方法、装置和计算机可读存储介质 | |
CN111935480B (zh) | 一种用于图像获取装置的检测方法及相关装置 | |
WO2021135676A1 (zh) | 一种拍照背景虚化方法、移动终端及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22836831 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2022836831 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2022836831 Country of ref document: EP Effective date: 20240207 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |