CN113468918A

CN113468918A - Package detection method, device, equipment and storage medium

Info

Publication number: CN113468918A
Application number: CN202010241771.5A
Authority: CN
Inventors: 尹东富; 刘庆龙; 刘俊杰; 白童垚; 程啸; 刘嫚玉
Original assignee: SF Technology Co Ltd
Current assignee: SF Technology Co Ltd; SF Tech Co Ltd
Priority date: 2020-03-31
Filing date: 2020-03-31
Publication date: 2021-10-01

Abstract

The embodiment of the application discloses a parcel detection method, a device, equipment and a storage medium, wherein the parcel detection method comprises the following steps: acquiring a current frame image in a monitoring video stream corresponding to a monitoring area of a transmission package; acquiring a first reference area image of a first preset area when no package exists in the monitoring area and a second reference area image of a second preset area when no package exists in the monitoring area, and judging whether the image of the first preset area in the current frame image is matched with the first reference area image or not; if the image of the first preset area in the current frame image is not matched with the image of the first reference area, judging whether the image of the second preset area in the current frame image is matched with the image of the second reference area; and if the image of the second preset area in the current frame image is not matched with the image of the second reference area, wrapping and cutting the current frame image to obtain at least one wrapping image to be stored. The parcel can be guaranteed to be cut, and the condition that the parcel is missed to be detected is avoided simultaneously.

Description

Package detection method, device, equipment and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a package detection method, device, apparatus, and storage medium.

Background

In public places such as airports, subways, stations and the like, security check machines with X-rays are generally used for carrying out security detection on packages of users so as to ensure the life and property security of people. Generally, a security inspection machine comprises a motor, a conveying system, a machine body enclosed in the middle of the conveying system and an X-ray machine. The user's parcel is placed in conveying system's one end, and conveying system will transmit the parcel to the other end from the one end of placing, and at the in-process of transmission, the X-ray machine will scan the formation X-ray formation of image to the parcel at the security installations passageway to can detect whether there are contraband in the parcel.

However, when the speed of the conventional security inspection machine is fast or the number of packages on the conveying system is large, the phenomenon that the packages cannot be cut due to too close continuous distances occurs, and the packages are missed to be inspected.

That is, the package detection method in the prior art may cause package omission.

Disclosure of Invention

The embodiment of the application provides a parcel detection method, an image processing method and an image processing device, and aims to solve the problem of how to avoid missing detection of parcels.

In a first aspect, the present application provides a package detection method, comprising:

acquiring a current frame image in a monitoring video stream corresponding to a monitoring area of a transmission package;

acquiring a first reference area image of a first preset area when no package exists in the monitoring area and a second reference area image of a second preset area when no package exists in the monitoring area, wherein the first preset area and the second preset area are sequentially arranged along a preset direction, and the preset direction is the moving direction of images in the monitoring video stream;

judging whether the image of a first preset area in the current frame image is matched with the image of a first reference area;

if the image of the first preset area in the current frame image is not matched with the image of the first reference area, judging whether the image of the second preset area in the current frame image is matched with the image of the second reference area;

and if the image of the second preset area in the current frame image is not matched with the image of the second reference area, wrapping and cutting the current frame image to obtain at least one wrapping image to be stored.

Wherein the package detection method further comprises:

if the image of the first preset area in the current frame image is matched with the first reference area image, acquiring a previous frame image of the current frame image;

judging whether an image of a first preset area of a previous frame image of a current frame image is matched with a first reference area;

and if the image of the first preset area of the previous frame image of the current frame image is not matched with the first reference area, wrapping and cutting the current frame image to obtain the at least one to-be-stored wrapping image.

Wherein the package detection method further comprises:

acquiring at least one stored parcel image which is stored in an image set latest;

removing the duplicate of the parcel image to be stored based on the at least one stored parcel image to obtain a duplicate-removed parcel image to be stored;

and storing the package image to be stored after the duplication removal into the image set.

Wherein the storing the package image to be stored after the duplication removal into the image set includes:

respectively acquiring a first external rectangle corresponding to each duplicate-removed parcel to be stored so as to obtain at least one first external rectangle;

judging whether a first external rectangle overlapped exists in the at least one first external rectangle;

if the overlapped first external rectangle exists in the at least one external rectangle, acquiring a second external rectangle corresponding to a union set of the overlapped first external rectangles;

storing the second bounding rectangle and the non-overlapping first bounding rectangle graph into the image set.

Wherein, judging whether the at least one first external rectangle has an overlapped first external rectangle comprises:

judging whether the first external rectangle meets a preset image condition or not;

if yes, judging whether the first external rectangles meeting the preset conditions exist overlapped first external rectangles or not.

Wherein the preset image condition includes:

the length of the first external rectangle is greater than a preset length value, and the width of the first external rectangle is greater than a preset width value;

and/or the ratio of the length of the first circumscribed rectangle to the width of the first circumscribed rectangle is within a first preset ratio range;

and/or the ratio of the number of the preset pixel points in the first external rectangle to the total number of the pixel points is within a second preset ratio range.

Wherein, the judging whether the image of the first preset area in the current frame image is matched with the image of the first reference area comprises:

judging whether the image similarity between the image of the first preset area in the current frame image and the image of the first reference area is greater than a preset threshold value or not;

if yes, judging that the image of a first preset area in the current frame image is matched with the image of a first reference area; if not, judging that the image of the first preset area in the current frame image is not matched with the image of the first reference area.

In a second aspect, the present application provides a package detection device comprising:

the first acquisition unit is used for acquiring a current frame image in a monitoring video stream corresponding to a monitoring area for transmitting the package;

a second obtaining unit, configured to obtain a first reference area image of a first preset area when no package exists in the monitoring area, and a second reference area image of a second preset area when no package exists in the monitoring area, where the first preset area and the second preset area are sequentially arranged along a preset direction, and the preset direction is a moving direction of images in the monitoring video stream;

the first judging unit is used for judging whether the image of the first preset area in the current frame image is matched with the image of the first reference area;

a second judging unit, configured to judge whether an image of a second preset region in the current frame image matches a second reference region image if the image of the first preset region in the current frame image does not match the first reference region image, where the first preset region and the second preset region are sequentially arranged along a preset direction, and the preset direction is a moving direction of an image in the monitoring video stream;

and the package cutting unit is used for performing package cutting on the current frame image to obtain at least one package image to be stored if the image of the second preset area in the current frame image is not matched with the second reference area image.

In a third aspect, the present application provides a package detection apparatus comprising:

one or more processors;

a memory; and

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the package detection method of any of the first aspects.

In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program to be loaded by a processor for performing the steps of the package detection method according to any one of the first aspect.

The application provides a parcel detection method, which comprises the steps of judging whether an image of a first preset area in a current frame image is matched with a first reference area image or not after the current frame image of a monitoring video stream is obtained, and if the image of the first preset area in the current frame image is not matched with the first reference area image, indicating that the first preset area in the current frame image has a parcel characteristic and a parcel does not completely enter a monitoring picture; and then judging whether the image of the second preset area in the current frame image is matched with the image of the second reference area, if the image of the second preset area in the current frame image is not matched with the image of the second reference area, indicating that the package is about to move out of the monitoring picture, and at the moment, cutting the package, so that the package can be cut, and the condition that the package is missed to be detected is avoided.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic flow chart diagram illustrating an embodiment of a package inspection method provided in an embodiment of the present application;

FIG. 2 is a schematic diagram illustrating a moving direction of an image in a surveillance video stream according to an embodiment of a parcel detection method provided by the present application;

FIG. 3 is a schematic diagram of a first predetermined area and a second predetermined area in an embodiment of a package inspection method provided in the embodiments of the present application;

FIG. 4 is a schematic diagram of a parcel just entering a first predetermined area in one embodiment of a parcel detection method as provided in embodiments of the present application;

FIG. 5 is a schematic diagram of an embodiment of a package inspection method according to the present application, wherein a package is about to cross a first predetermined area;

FIG. 6 is a schematic diagram of a new parcel arriving at a second predetermined area in an embodiment of a parcel detection method according to the present application;

FIG. 7 is a schematic view of a first predetermined area at intervals between packages in an embodiment of a method of package inspection provided in embodiments of the present application;

FIG. 8 is a diagram illustrating an image of a previous frame of a current frame in an embodiment of a parcel detection method according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a first circumscribed rectangle of a package A in an embodiment of a package inspection method provided in embodiments of the present application;

FIG. 10 is a schematic view of a second circumscribed rectangle of a plurality of parcels in one embodiment of a parcel detection method provided in embodiments of the present application;

FIG. 11 is a schematic view of an embodiment of a package inspection device provided in embodiments of the present application;

fig. 12 is a schematic structural diagram of an embodiment of a package detection device provided in the embodiments of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In the description of the present application, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed in a particular orientation, and be operated, and thus should not be considered as limiting the present application. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.

In this application, the word "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes are not set forth in detail in order to avoid obscuring the description of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

The embodiment of the application provides a parcel detection method, a device, equipment and a storage medium. The following are detailed below.

First, an embodiment of the present application provides a package detection method, where the package detection method includes:

acquiring a current frame image in a monitoring video stream corresponding to a monitoring area of a transmission package; acquiring a first reference area image of a first preset area when no package exists in a monitoring area and a second reference area image of a second preset area when no package exists in the monitoring area, wherein the first preset area and the second preset area are sequentially arranged along a preset direction, and the preset direction is the moving direction of images in a monitoring video stream; judging whether the image of a first preset area in the current frame image is matched with the image of a first reference area; if the image of the first preset area in the current frame image is not matched with the image of the first reference area, judging whether the image of the second preset area in the current frame image is matched with the image of the second reference area; and if the image of the second preset area in the current frame image is not matched with the image of the second reference area, wrapping and cutting the current frame image to obtain at least one wrapping image to be stored.

Referring to fig. 1, fig. 1 is a schematic flow chart of an embodiment of a package detection method provided in the embodiment of the present application. As shown in fig. 1, the package detection method includes:

s101, obtaining a current frame image in a monitoring video stream corresponding to a monitoring area of a transmission package.

In the embodiment of the application, the original monitoring video stream corresponding to the monitoring area of the package to be transmitted is obtained through the security check machine, and the current frame image is extracted from the original monitoring video stream. The security inspection machine, also called a security inspection instrument, comprises a security inspection X-ray machine, a luggage security inspection machine, a channel X-ray machine, a physical inspection X-ray machine, an X-ray security inspection instrument, an X-ray luggage security inspection machine, an X-ray detector, an X-ray foreign matter detector, an X-ray security inspection machine, an X-ray luggage detector, a three-class detector and a hazard detector. The security inspection machine is widely applied to airports, railway stations, subway stations, bus stations, government buildings, embassy, conference centers, exhibition centers, hotels, shopping malls, large-scale activities, post offices, schools, logistics industries, industrial detection and the like.

In a specific embodiment, the original monitoring video stream of the security check machine is acquired through the interface according to the model and the interface mode of the host machine or the display of the security check machine. The Interface may be a VGA (Video Graphics Array), a DVI (Digital Video Interface), an HDMI (High Definition Multimedia Interface), etc., wherein the VGA Interface employs an analog signal, the DVI and the HDMI use a Digital signal, and the HDMI supports audio output compared to the DVI Interface. When the security check machine supports multiple interface modes, the interface modes are preferably selected according to the sequence of HDMI, DVI and VGA so as to reduce the loss of video signals.

The image reading interface of the existing security check machine is not opened basically, and meanwhile, due to the differences of the model number, the interface, the factory time and the like of the security check machine, the interface modes of the security check machine with some opened interfaces are different, and the interfaces are required to be adapted according to different security check machines. Therefore, it is difficult to obtain information such as a dynamic picture storage path, and it is also inconvenient to analyze and count the number of packages in real time, identify objects in the packages, detect whether the packages contain contraband, give an alarm, and the like. Even in an off-line picture storage state, pictures cut and stored by a picture cutting tool carried by part of the conventional security inspection machine manufacturers are poor in quality, and if the problems that packages are over-cut, the packages close to the packages cannot be cut off and the like exist, high-quality package images cannot be reserved. And the display screen of the security check machine can display the package video stream information in real time, so that the video stream information is processed, and each package picture can be obtained by cutting.

In another specific embodiment, the original monitoring video stream of the security check machine is acquired by recording the screen of the display screen of the security check machine. The security check machine manufacturers are numerous, interfaces are different, the bottom layer information or the interface form of the cut images of one or two security check machines can be obtained, but all the interfaces cannot be obtained, the cut images of all the manufacturers and all the brands of security check machines cannot be obtained, and the limitation is large. The monitoring video stream of the security check machine is acquired by adopting a screen recording mode, the condition that the monitoring video stream cannot be acquired due to the fact that an interface of the security check machine is not opened can be avoided, and the monitoring video stream can be acquired by aiming at any security check machine. At present, a security check machine manufacturer cuts packages in a screen recording mode. And the screen recording mode is adopted, so that the method is irrelevant to security check machine manufacturers, and any security check machine can be packaged and segmented by using an algorithm.

It should be noted that, when the original monitoring video stream of the security check machine is acquired through the interface, the monitoring area is a shooting interface of the security check machine; when the original monitoring video stream of the security check machine is obtained by recording the screen of the display screen of the security check machine, the monitoring area is the shooting interface of the screen recording device.

After the original monitoring video stream is obtained, judging whether a popup window exists at the front end of a monitoring window of the original monitoring video stream, if the popup window does not exist at the front end of the monitoring window of the original monitoring video stream, indicating that the monitoring window is not blocked, obtaining a current frame image of the original monitoring video stream, and forming a normal monitoring video stream; and if the pop-up window exists at the front end of the monitoring window of the monitoring video stream, which indicates that the monitoring window is blocked, and the image at the moment is an abnormal frame, suspending the acquisition of the current frame image of the monitoring video stream. Further, when a popup exists at the front end of a monitoring window for monitoring the video stream, prompt information is sent out so that a worker can process the popup.

In actual operation, firstly, the process of the security check machine is obtained through a program, and the position of each display window in the process display screen of the security check machine is displayed. And when the new display window is positioned at the forefront end of the display screen, namely the default monitoring window is shielded, judging the new display window to be an abnormal frame, stopping data transmission and giving an alarm. And when the new display window does not shield the default monitoring window or disappears, judging the new display window to be a normal frame, and recovering the image data transmission.

Because the original monitoring video stream of the security check machine is output by the industrial control computer through the display card, the output security check video picture will include: in addition, the security check video may also include contents of a non-security check image interface, for example, the industrial control computer minimizes the security check image interface, or switches to another system interface, or displays other contents on the security check image interface during the process of displaying the security video, so that the output original surveillance video stream will carry the contents including the non-security check image interface, and the X-ray image display area is an area where a package appears. The control panel area is a functional area for controlling the playing of the security check video.

In a specific embodiment, images in the normal surveillance video stream are cut based on a preset effective detection area to obtain effective video frame images, and an effective surveillance video stream is formed. When no package exists in the security check machine, gray level inspection is carried out on images in an original monitoring video stream of the security check machine, areas with equal gray values are determined, inscribed rectangles of the areas with the equal gray values are determined to be preset effective detection areas, the images in the preset effective detection areas in a normal monitoring video stream are extracted, and effective video frame images are obtained. When no package exists in the security inspection machine, the image in the monitoring video stream of the security inspection machine is approximately white, the color difference of a menu bar, a boundary and the like is large, after the image is subjected to binarization processing, a middle blank area can be clearly obtained, and the maximum area rectangular frame is taken, so that the preset effective detection area can be obtained. Of course, the preset effective detection area may also be determined manually, for example, a certain rectangular frame area may be set manually, for example, the preset effective detection area is set to 1000 × 1000, and the area includes complete package information and does not include other information such as a software menu or a background.

In a specific embodiment, after the effective surveillance video stream is obtained, the effective video frame images in the effective surveillance video stream are scaled to obtain the scaled effective surveillance video stream. By cutting the image and carrying out scale scaling processing on the cut image, the small-scale scaling has small influence on the package edge detection, but the image processing speed can be greatly accelerated.

In a preferred embodiment, a current frame image of the scaled active surveillance video stream is obtained. Of course, in other embodiments, the current frame image may be obtained from any one of the original surveillance video stream, the normal surveillance video stream, and the scaled effective surveillance video stream.

S102: and acquiring a first reference area image of the first preset area when no package exists in the monitoring area and a second reference area image of the second preset area when no package exists in the monitoring area.

Referring to fig. 2 and fig. 3, fig. 2 is a schematic diagram illustrating a moving direction of an image in a surveillance video stream according to an embodiment of a parcel detection method provided in the present application; fig. 3 is a schematic diagram of a first preset area and a second preset area in an embodiment of a parcel detection method provided in the embodiment of the present application.

In this embodiment, the images in the surveillance video stream include a first preset region 1000 and a second preset region 2000, which are sequentially arranged. The first preset area 1000 and the second preset area 2000 are sequentially arranged along a preset direction, where the preset direction is a moving direction of images in the surveillance video stream.

Specifically, before the current frame image of the surveillance video stream is obtained, the moving direction of the image in the surveillance video stream is obtained, and the first preset area 1000 and the second preset area 2000 are determined according to the moving direction of the image in the surveillance video stream. Specifically, at least two frames of images in the monitoring video stream are obtained, the motion trail of the preset image is obtained according to the position of the same preset image in the at least two frames of images, and the moving direction of the images in the monitoring video stream is determined according to the motion trail of the preset image. Two image areas of an image in a monitoring video stream in a preset direction are determined as a first preset area 1000 and a second preset area 2000.

In a specific embodiment, the preset image is an n-column pixel area with a wrapping feature, n is not less than 1, two front and back frames of images in the surveillance video stream are acquired, and the position of the n-column pixel area in the two front and back frames of images is acquired, for example, the position of the n-column pixel area on the front frame of image is P1, and the position of the n-column pixel area on the front frame of image is P2, then it is determined that the object moving direction is F1, and the object moves from left to right. If P1 is on the right side of P2, it is determined that the object moving direction is opposite to the F1 direction, and is moving from right to left.

In a specific embodiment, the width of the image in the surveillance video stream is L, the distance between the first preset area 1000 and the boundary of the entry end of the image in the surveillance video stream is d, and d is a distance, which is set according to specific situations. The second predetermined area 2000 is 0.75L of the width from the entrance end boundary. The first preset area 1000 and the second preset area 2000 may be rectangular areas.

In the embodiment of the present application, the first reference area image is an image on the first preset area 1000 of an image in a monitoring video stream, which is acquired when a package is not entered in a security inspection machine. The second reference area image is an image on a second preset area 2000 of an image in the surveillance video stream acquired when the package is not entered in the security check machine. Generally, images in the surveillance video stream acquired when no package enters the security check machine are white background images, and therefore, the first reference area image and the second reference area image may both be white background images. Of course, the first reference area image and the second reference area image are different according to different security inspection machines, and the first reference area image and the second reference area image are determined according to specific situations.

S103, judging whether the image of the first preset area in the current frame image is matched with the image of the first reference area.

In the embodiment of the present application, it is determined whether the image similarity between the image in the first preset region 1000 in the current frame image and the image in the first reference region is greater than a preset threshold. The preset threshold may be set according to a specific type of the security inspection machine, for example, 80%, 90%, and the like. The image matching is a commonly used algorithm in image processing, and includes a squared difference matching method, a normalized squared difference matching method, a correlation matching method normalized correlation matching method, a correlation coefficient matching method, a normalized correlation coefficient matching method, and the like.

For example, when there is no parcel in the security check machine, the current frame image is a white background image, the similarity between the image on the first preset area 1000 in the current frame image and the first reference area image is 100%, and when a new parcel gradually enters the security check machine, the similarity between the image on the first preset area 1000 in the current frame image and the first reference area image gradually decreases, for example, when the similarity between a new object and the previous template decreases to less than 80%, it indicates that a new object enters. This threshold is based on the fact that each security check machine will have a small change.

If the image similarity between the image in the first preset area 1000 in the current frame image and the first reference area image is greater than the preset threshold, judging that the image in the first preset area 1000 in the current frame image is matched with the first reference area image, and indicating that the first preset area 1000 in the current frame image has no wrapping feature; if the image similarity between the image in the first preset area 1000 in the current frame image and the image in the first reference area is not greater than the preset threshold, it is determined that the image in the first preset area 1000 in the current frame image is not matched with the image in the first reference area, which indicates that the first preset area 1000 in the current frame image has a wrapping feature.

And S104, if the image of the first preset area in the current frame image is not matched with the image of the first reference area, judging whether the image of the second preset area in the current frame image is matched with the image of the second reference area.

Referring to fig. 4 and 5, fig. 4 is a schematic diagram illustrating that a parcel has just entered a first predetermined area in an embodiment of a parcel detection method provided in the embodiments of the present application; fig. 5 is a schematic diagram of an embodiment of a parcel about to cross a first predetermined area in a parcel detection method provided by an embodiment of the present application.

In the embodiment of the present application, if the image on the first preset area 1000 in the current frame image is not matched with the first reference area image, it indicates that the first preset area 1000 in the current frame image has a wrapping feature. At this point, it may be that the package has just entered the first predetermined area 1000, as shown in FIG. 4. Of course, it is also possible that the package will leave the first predetermined area 1000, as shown in FIG. 5. At this time, if further determination is needed, it is determined whether the image of the second preset region 2000 in the current frame image matches the second reference region image. Specifically, the step of determining whether the image of the second preset region 2000 in the current frame image matches the image of the second reference region may refer to S103, which is not described herein again.

And S105, if the image of the second preset area in the current frame image is not matched with the image of the second reference area, wrapping and cutting the current frame image to obtain at least one to-be-stored wrapping image.

With reference to fig. 5, in the embodiment of the present disclosure, the image of the second preset area 2000 in the current frame image is not matched with the image of the second reference area, which indicates that the second preset area 2000 has a parcel feature, and at this time, the parcel is located between the first preset area 1000 and the second preset area 2000 and is about to move out of the monitoring screen, and a forced cutting is required. And performing package cutting on the current frame image to obtain at least one package image to be stored. That is, the forced cutting procedure is performed for, i.e., when the consecutive parcels occupy a large portion of the screen. If the parcel is moving from left to right, the forced cutting procedure is started when the parcel is above the screen 3/4 in the horizontal direction.

Specifically, the package cutting process mainly comprises extracting the outermost outline of the package. And generating a boundary contour by adopting expansion, corrosion and boundary extraction modes. Erosion shrinks the borders of the parcel in the image and can be used to eliminate small and meaningless objects. The expansion can enlarge the wrapping range, and background points in contact with the wrapping are combined into the wrapping, so that the wrapping boundary is expanded outwards to fill some holes in the wrapping and eliminate small particle noise in the wrapping. In the image cutting process of the actual security inspection machine, in order to ensure the integrity of the package and remove the tiny holes in the package, closed operation of expansion first and corrosion later is carried out. Because the horizontal direction parcel interval is less, more inflation can lead to unconnected parcel to be in the same place, in order to cut out different parcels conveniently, carries out little inflation, and the operation of corroding more in the horizontal direction. While in the vertical direction, normal erosion and expansion operations are performed.

Image edges are reactions of discontinuities in local characteristics of the image, such as gray scale, color, texture discontinuities, etc. The image has gentle pixel change along the edge direction and severe pixel change perpendicular to the edge direction. Such changes in the edge can be detected by differential operators such as Prewitt, Sobel, Lapacian, etc., using first or second order derivatives to detect the edge.

And extracting the boundary contour by using a Sobel operator for the X-ray machine image with gradually changed gray scale and little noise. The Sobel operator has the characteristics of high detection speed, high efficiency and the like. When edge extraction is carried out based on a Sobel operator, firstly, brightness difference approximate values in the vertical direction and the vertical direction are respectively solved, and the horizontal gray value and the longitudinal gray value of each pixel are combined to calculate the gradient of the point; and then defining a threshold value, setting points with gradients larger than the threshold value as edge points, and calculating to obtain a binary image of the gradient image, namely the edge image.

When the image of the X-ray machine is noisy, a Canny algorithm is used for extracting the boundary contour. In the process of extracting the contour, firstly, a Gaussian filter is used for filtering noise, the gradient size and the gradient direction of each pixel point in the image are calculated, the non-maximum value suppression is carried out on the gradient size, and a double-threshold detection method is used for detecting and connecting edges. When detecting the edge, the strong edge and the weak edge are respectively detected by using two different thresholds, and when the weak edge is connected with the strong edge, the weak edge is simultaneously output to the stored image.

Preferably, the image between the boundary of the entering end of the current frame image and the second preset region 2000 is cut forcibly. That is, the image from the left boundary to the second preset area 2000 is subjected to the parcel cutting process. Therefore, the upper processed image and the lower processed image do not have overlapped areas, and repeated cutting cannot be caused.

Further, before the image between the boundary of the entrance end of the current frame image and the second preset area 2000 is forcedly cut, acquiring the moving distance of the image in the monitoring video stream in the time period from the last forced cutting time to the current time, and when the moving distance of the image in the monitoring video stream in the time period from the last forced cutting time to the current time exceeds the distance between the boundary of the entrance end of the current frame image and the second preset area 2000, performing forced cutting; if the moving distance of the image in the monitoring video stream in the time period from the last forced cutting time to the current time does not exceed the distance from the boundary of the entering end of the current frame image to the second preset area 2000, the forced cutting is not carried out.

Referring to fig. 6, fig. 6 is a schematic view illustrating a new parcel arriving at the second predetermined area again in an embodiment of the parcel detection method according to the embodiment of the present application.

For example, an image composed of fixed n columns of pixels is taken at a position d2 from the left boundary at the same time as the template. And when the package continuously enters, matching operation is carried out in the image by using the template, and the moving distance of the image in the monitoring video stream in the time period from the last forced cutting time to the current time is obtained. The image of the last forced cut is the image including parcel a, parcel B and parcel C. When the moving distance of the image in the surveillance video stream is 3/4L in the time period from the last forced cutting to the current time at the position D2 on the right side of the second preset area 2000 of the template, it indicates that the new parcel, that is, the image including parcel D, parcel E and parcel F, has completely moved to the maximum boundary, that is, the image of the frame is processed, the position from the left boundary to 3/4 is taken as a parcel image, and the image is subjected to parcel cutting processing.

Further, the parcel detection method further comprises:

(1) if the image on the first preset area 1000 in the current frame image is matched with the first reference area image, a previous frame image of the current frame image is obtained.

Referring to fig. 7, fig. 7 is a schematic diagram illustrating a first predetermined area located at an interval between parcels in an embodiment of a parcel detection method according to the present application; fig. 8 is a schematic diagram of a previous frame image of a current frame in an embodiment of a parcel detection method according to the present application.

If the image on the first preset area 1000 in the current frame image is matched with the first reference area image, it indicates that there is no parcel feature in the first preset area 1000, and the parcel state needs to be determined by combining the previous frame image.

(2) It is determined whether the image on the first preset area 1000 of the previous frame image of the current frame image matches the first reference area.

The step of determining whether the image on the first preset area 1000 of the previous frame image of the current frame image matches the first reference area may refer to S102, and is not repeated herein.

(3) And if the image on the first preset area 1000 of the previous frame image of the current frame image is not matched with the first reference area, wrapping and cutting the current frame image to obtain at least one to-be-stored wrapped image.

If the image on the first preset area 1000 of the previous frame image of the current frame image does not match the first reference area, it indicates that the wrapping feature exists on the first preset area 1000 of the previous frame image of the current frame image. That is, the image of the first preset area 1000 is changed from a white background image to a non-white background image, so that it can be determined that the tail of the parcel begins to leave the first preset area 1000, indicating that the parcel completely enters the monitoring interface, and at this time, the parcel of the current frame image is cut,

further, the parcel detection method further comprises:

(1) at least one stored package image most recently deposited in the image set is acquired.

Wherein, the image set is used for storing the package images after the package cutting. The at least one stored package image may be 1, 2, 3, 4 or more images, as the case may be.

(2) And removing the duplicate of the parcel image to be stored based on at least one stored parcel image to obtain the parcel image to be stored after the duplicate is removed.

In the embodiment of the application, each to-be-stored package image can be subjected to image matching with at least one stored package image one by one, if the to-be-stored package image is matched with any one of the at least one stored package image, it is determined that the to-be-stored package image is already stored, and after the repeated to-be-stored package images are removed, the to-be-stored package image after the duplication is removed is obtained.

(3) And storing the package image to be stored after the duplication removal into an image set.

Further, the method for storing the package image to be stored after the duplication removal into the image set comprises the following steps:

(1) respectively obtaining a first external rectangle 3000 corresponding to each duplicate-removed parcel to be stored so as to obtain at least one first external rectangle 3000;

referring to fig. 9, fig. 9 is a schematic view of a first circumscribed rectangle of a parcel a in an embodiment of a parcel detection method according to the embodiment of the present application.

(2) It is determined whether there is an overlapping first circumscribed rectangle 3000 in the at least one first circumscribed rectangle 3000.

Specifically, whether the first circumscribed rectangle 3000 meets a preset image condition is judged; if yes, it is determined whether there is an overlapping first circumscribed rectangle 3000 in the first circumscribed rectangle 3000 that satisfies the preset condition.

In a specific embodiment, the preset image condition is that the length of the first circumscribed rectangle 3000 is greater than a preset length value and the width of the first circumscribed rectangle 3000 is greater than a preset width value. The preset length value and the preset width value are set according to specific conditions. For smaller sized parcels, these abnormal parcels are filtered by setting the minimum width and height of the rectangular bounding box, and parcels smaller than the threshold are not saved.

In another embodiment, the ratio of the length of the first circumscribed rectangle 3000 to the width of the first circumscribed rectangle 3000 is within a first predetermined ratio range. The first preset ratio is set according to specific conditions. And (3) aiming at the packages with abnormal shapes, the packages are filtered by setting the width-to-height ratio and the aspect ratio smaller than a set threshold value, and the packages exceeding the threshold value are not stored.

In another specific embodiment, the ratio of the number of the predetermined pixels in the first circumscribed rectangle 3000 to the total number of the pixels is within a second predetermined ratio range. The preset pixel point can be a yellow pixel, and the second preset ratio is set according to specific conditions. The general ineffective low resolution parcel is an image close to a yellow background, the color difference in the image is small, and no effective article exists. The method is realized by detecting the proportion of yellow pixels in the gray image to pixels in the whole image, and when the proportion is smaller than a set threshold value, the image is judged to be a low-resolution image, and the storage processing is not carried out.

(3) If there is an overlapping first circumscribed rectangle 3000 in at least one circumscribed rectangle, a second circumscribed rectangle 4000 corresponding to the union set of the overlapping first circumscribed rectangles 3000 is obtained.

Referring to fig. 10, fig. 10 is a schematic diagram of a second external rectangle of a plurality of parcels in an embodiment of the parcel detection method provided in the embodiment of the present application.

Segmenting the plurality of consecutive parcels results in a plurality of circumscribed rectangular frames. Taking two similar parcels as an example, when the two parcels are close to each other, the generated rectangular frames have a certain overlap. Cutting based on single rectangle frame can lead to the parcel cutting incomplete, for avoiding this phenomenon, will close on two rectangle frames and merge to the single biggest rectangle frame that two rectangle frames are constituteed cuts as the benchmark, can guarantee the integrality of parcel. Similarly, when a plurality of continuous wrapping rectangular frames are overlapped, the wrapping is cut by forming a new rectangular frame by the upper, lower, left and right pole positions of the plurality of rectangular frames.

(4) The second circumscribed rectangle 4000 and the non-overlapping first circumscribed rectangle 3000 are graphically stored into the image set.

The method comprises the steps of judging whether an image of a first preset area in a current frame image is matched with an image of a first reference area or not after the current frame image of a monitoring video stream is obtained, and if the image of the first preset area in the current frame image is not matched with the image of the first reference area, indicating that the first preset area in the current frame image has a package characteristic and the package does not completely enter a monitoring picture; and then judging whether the image of the second preset area in the current frame image is matched with the image of the second reference area, if the image of the second preset area in the current frame image is not matched with the image of the second reference area, indicating that the package is about to move out of the monitoring picture, and at the moment, cutting the package, so that the package can be cut, and the condition that the package is missed to be detected is avoided.

In order to better implement the package detection method in the embodiment of the present application, based on the package detection method, an embodiment of the present application further provides a package detection device, as shown in fig. 11, fig. 11 is a schematic structural diagram of an embodiment of the package detection device provided in the embodiment of the present application, where the package detection device includes a first obtaining unit 301, a second obtaining unit 302, a first determining unit 303, a second determining unit 304, and a package cutting unit 305:

a first obtaining unit 301, configured to obtain a current frame image in a surveillance video stream corresponding to a surveillance area where a package is transmitted;

a second obtaining unit 302, configured to obtain a first reference area image of a first preset area when there is no package in the monitored area, and a second reference area image of a second preset area when there is no package in the monitored area, where the first preset area and the second preset area are sequentially arranged along a preset direction, and the preset direction is a moving direction of images in the monitored video stream;

a first judging unit 303, configured to judge whether an image of a first preset region in the current frame image matches a first reference region image;

a second determining unit 304, configured to determine whether an image of a second preset region in the current frame image matches the image of the second reference region if the image of the first preset region in the current frame image does not match the image of the first reference region, where the first preset region and the second preset region are sequentially arranged along a preset direction, and the preset direction is a moving direction of an image in the monitoring video stream;

and a package cutting unit 305, configured to perform package cutting on the current frame image to obtain at least one package image to be stored if the image of the second preset region in the current frame image is not matched with the second reference region image.

The first determining unit 303 is further configured to obtain a previous frame image of the current frame image if the image of the first preset region in the current frame image matches the first reference region image;

The parcel cutting unit 305 is further configured to obtain at least one stored parcel image that is stored in the image set most recently;

The parcel cutting unit 305 is further configured to obtain a first external rectangle corresponding to each duplicate-removed parcel to be stored, so as to obtain at least one first external rectangle;

The first determining unit 303 is further configured to determine whether the first circumscribed rectangle meets a preset image condition;

Wherein the preset image condition includes:

The first determining unit 303 is further configured to determine whether an image similarity between an image of a first preset region in the current frame image and an image of a first reference region is greater than a preset threshold;

The embodiment of the application also provides a package detection device. As shown in fig. 12, fig. 12 is a schematic structural diagram of an embodiment of the package detection device provided in the embodiment of the present application, specifically:

the package detection device may include components such as a processor 501 of one or more processing cores, memory 502 of one or more computer-readable storage media, a power source 503, and an input unit 504. Those skilled in the art will appreciate that the package inspection device configurations shown in the figures do not constitute limitations of package inspection devices, and may include more or fewer components than shown, or some components in combination, or a different arrangement of components. Wherein:

the processor 501 is a control center of the package inspection apparatus, connects various parts of the entire package inspection apparatus by using various interfaces and lines, and performs various functions of the package inspection apparatus and processes data by running or executing software programs and/or modules stored in the memory 502 and calling data stored in the memory 502, thereby performing overall monitoring of the package inspection apparatus. Optionally, processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 501.

The memory 502 may be used to store software programs and modules, and the processor 501 executes various functional applications and data processing by operating the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created from use of the package detection device, and the like. Further, the memory 502 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

The package detection device further comprises a power source 503 for supplying power to each component, and preferably, the power source 503 may be logically connected to the processor 501 through a power management system, so that functions of managing charging, discharging, power consumption management and the like are realized through the power management system. The power supply 503 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.

The package detection apparatus may also include an input unit 504, where the input unit 504 may be used to receive entered numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

Although not shown, the package detection device may further include a display unit or the like, which will not be described in detail herein. Specifically, in this embodiment, the processor 501 in the package detection device loads the executable file corresponding to the process of one or more application programs into the memory 502 according to the following instructions, and the processor 501 runs the application program stored in the memory 502, so as to implement various functions as follows:

acquiring a first reference area image of a first preset area when no package exists in a monitoring area and a second reference area image of a second preset area when no package exists in the monitoring area, wherein the first preset area and the second preset area are sequentially arranged along a preset direction, and the preset direction is the moving direction of images in a monitoring video stream;

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like. The computer program is loaded by a processor to execute the steps of any one of the parcel detection methods provided by the embodiments of the present application. For example, the computer program may be loaded by a processor to perform the steps of:

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and parts that are not described in detail in a certain embodiment may refer to the above detailed descriptions of other embodiments, and are not described herein again.

In a specific implementation, each unit or structure may be implemented as an independent entity, or may be combined arbitrarily to be implemented as one or several entities, and the specific implementation of each unit or structure may refer to the foregoing method embodiment, which is not described herein again.

The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.

The above detailed description is provided for a package detection method, apparatus, device and storage medium provided in the embodiments of the present application, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiments is only used to help understanding the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A package inspection method, characterized in that the package inspection method comprises:

2. The package inspection method according to claim 1, further comprising:

3. The package inspection method according to claim 1 or 2, further comprising:

4. The package inspection method according to claim 3, wherein said storing the deduplicated package image to be stored into the image set comprises:

5. The package inspection method according to claim 4, wherein determining whether there is an overlapping first circumscribed rectangle in the at least one first circumscribed rectangle comprises:

6. The package inspection method according to claim 5, wherein the preset image condition includes:

7. The parcel detection method according to claim 1, wherein said determining whether the image of the first preset area in the current frame image matches the first reference area image comprises:

8. A package inspection device, characterized in that the package inspection device comprises:

9. A package inspection apparatus, characterized in that the package inspection apparatus comprises:

one or more processors;

a memory; and

one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the package detection method of any of claims 1-7.

10. A computer-readable storage medium, having stored thereon a computer program which is loaded by a processor to perform the steps of the package detection method of any one of claims 1 to 7.