CN111223260A - Method and system for intelligently monitoring goods theft prevention in warehousing management - Google Patents
Method and system for intelligently monitoring goods theft prevention in warehousing management Download PDFInfo
- Publication number
- CN111223260A CN111223260A CN202010061581.5A CN202010061581A CN111223260A CN 111223260 A CN111223260 A CN 111223260A CN 202010061581 A CN202010061581 A CN 202010061581A CN 111223260 A CN111223260 A CN 111223260A
- Authority
- CN
- China
- Prior art keywords
- goods
- monitoring
- frame picture
- module
- warehouse
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
- G08B13/19608—Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
- G08B13/19613—Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Economics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Development Economics (AREA)
- Health & Medical Sciences (AREA)
- Entrepreneurship & Innovation (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Image Analysis (AREA)
Abstract
the invention discloses a method for intelligently monitoring goods anti-theft damage in warehousing management, which is characterized in that on the basis of video monitoring, an SSD algorithm is adopted to calculate images and interaction conditions of recognized goods and people in a monitoring area, so that early warning is carried out when the stored goods are possibly stolen.
Description
Technical Field
The invention relates to the technical field of intelligent warehousing management, in particular to a method and a system for intelligently monitoring goods theft prevention in warehousing management.
Background
At present, the warehouse storage intelligent management technology is continuously improved, and the warehouse goods management method is more and more emphasized. A series of processes such as warehousing, transportation, metering, stacking and storage of goods need an efficient and economic treatment method so as to reduce the loss rate of the goods in each link. Especially, in the processes, for the stolen situation of goods, reliable measures are needed to reduce the occurrence rate of the goods, so that the goods loss of the warehouse is reduced, the overall benefit of the warehouse is improved, the further trust of a demand side is won, further more services are brought, and the like, and the method has an extremely effective and long-term promoting effect.
The existing storage anti-theft monitoring mode on the market is based on two ideas, one is to manage the identity recognition of the personnel entering the warehouse by adopting a certain means so as to prevent the personnel with suspected identity from entering; the other is to monitor the movement of the goods in the warehouse to prevent the goods from moving out of the planning area. The two main solutions for anti-theft monitoring are as follows:
1) and identifying the identity of the personnel. Currently, technologies in this aspect are implemented in many ways, and the accuracy or precision is higher and higher, and the corresponding system deployment cost is continuously reduced. For example, the fingerprint identification and face identification are performed by storing a fingerprint or a face of a person in advance, and when the identity needs to be verified, the fingerprint or the face extracted from the person on site is compared with the stored fingerprint or face, so that the real identity of the person can be determined. The fingerprint or face biological characteristics of each person are different and unique, and have stability, and the key characteristics are always unchanged throughout the life, so that the personnel identification realized by the technology can be realized by means of the uniqueness and the stability. After the personnel identification is completed, the personnel entering and exiting the target area can be effectively managed by combining the personnel management system.
2) And monitoring the displacement of the goods. The most mature developed and used today is the RFID (radio frequency identification) technology to monitor the goods entering and exiting the warehouse. The complete RFID system consists of a Reader-writer (Reader), an electronic Tag (Tag) and a data management system. The reader reads the information in the labels on the goods and transmits the information to the data management system, the data management system continuously receives and processes the real-time data of each label, if an abnormity is found, for example, the position information appears in an unplanned area at a certain time point, an alarm or other notification mechanisms can be triggered, and managers are notified to check or solve the situation.
The two monitoring modes are single identification of people entering the warehouse or goods placed in the warehouse, and contact identification between people and goods, and contact identification between vehicles and goods anti-theft early warning are not carried out. The goods are often stolen due to a motion relation among people, vehicles and goods, so that the existing two monitoring modes can not realize good theft prevention.
Besides, the two existing monitoring methods also have the following disadvantages:
1) the fingerprint and face recognition system can only distinguish out the personnel in the system, and the system can be combined with an entrance guard attendance system to recognize the external personnel and prevent the external personnel from entering a warehouse area. But the goods are stolen and damaged to a great extent by internal personnel (or internal and external combination), and the fingerprint and face recognition system cannot solve the problem. Moreover, many times the thief can not enter the warehouse from the occasion with the identity recognition system, which makes the management of the theft and the damage of the goods in the warehouse difficult to be prevented.
2) The RFID method requires adding RFID tags to the goods and deploying an RFID signal acquisition network at a suitable location in the warehouse area, which is limited by warehouse construction, wiring, security, etc. In addition, the identification distance of the RFID is limited, and the tag is too troublesome to install and detach for some goods (metal materials and the like) which do not need to be packaged, and the metal has a shielding effect on information and is not suitable for being used in a warehouse scene.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a method and a system for intelligently monitoring goods theft prevention in warehousing management.
In order to achieve the above object, in a first aspect, an embodiment of the present invention provides a method for intelligently monitoring theft and damage of goods in warehousing management, including the following steps:
step S1: according to the actual area division condition of the warehouse, carrying out area division on the monitoring video pictures of the warehouse, and dividing the video pictures into different monitoring areas;
step S2, extracting a current frame image α and a previous frame picture β in the monitoring video, calculating the difference proportion between the frame image α and the frame picture β, and if the difference proportion exceeds a preset threshold value a, performing the next step;
step S3, carrying out goods identification and human shape identification on the frame picture α through an SSD target detection algorithm, and extracting meta information in the frame picture α, wherein the meta information comprises the containing box information of the object;
and step S4, according to the identified box-containing information and the box-containing information of the shape of the goods, calibrating the external rectangle A and the external rectangle B of the shape of the goods in the frame picture alpha, and alarming if the external rectangle A and the external rectangle B are overlapped in the monitoring area in the frame picture alpha.
Optionally, in step S3, the method further includes: and tracking the identified human figure through a KLT tracking algorithm.
Optionally, in step S4, the alarm process includes:
s41, judging whether the acquisition time of the frame picture is in the early warning time period, if so, carrying out the next step, otherwise, jumping to the step S2;
s42, judging whether the monitoring area in the frame picture is an early warning area, if so, carrying out the next step, otherwise, jumping to the step S2;
and S43, judging the overlapping times of the circumscribed rectangle A and the circumscribed rectangle B in the monitoring area in the limited time, alarming if the times exceed a preset threshold B, otherwise, jumping to the step S2.
In a second aspect, an embodiment of the present invention provides a system for intelligently monitoring goods theft and damage prevention in warehousing management, including:
the system comprises a camera array, a storage and a processing unit, wherein the camera array is used for collecting monitoring video stream data in the warehouse and comprises a plurality of cameras;
the video source module is used for receiving and processing monitoring video stream data, comparing a currently received frame picture α with a last received frame picture β, and calculating a difference proportion between the frame picture α and the frame picture β;
the area planning module is used for carrying out area division on the monitoring video pictures of the warehouse according to the actual area division condition of the warehouse and dividing the video pictures into different monitoring areas;
the goods identification module is used for judging the difference proportion between the frame picture α and the frame picture beta, and when the difference proportion exceeds a preset threshold value a, the goods identification module identifies the goods of the frame picture α through an SSD target detection algorithm and extracts the meta information in the frame picture α, wherein the meta information comprises the containing box information of the object;
the human shape recognition module is used for carrying out human shape recognition on the frame picture α through an SSD target detection algorithm and extracting meta information in the frame image alpha;
the overlap calculation module is used for calibrating a circumscribed rectangle A and a circumscribed rectangle B of the goods in the frame picture α according to the identified box-containing information and the box-containing information of the humanoid of the goods, judging whether the circumscribed rectangle A and the circumscribed rectangle B are overlapped in the monitoring area in the frame image alpha or not, and sending out an early warning signal if the circumscribed rectangle A and the circumscribed rectangle B are overlapped;
and the monitoring alarm module is used for receiving the early warning signal, giving an alarm according to a preset alarm strategy, outputting the overlapping condition of the goods and the human shape to a visual terminal, and allowing monitoring personnel to observe and judge. Optionally, the camera is a 1080P high-definition camera.
Optionally, the human shape recognition module is further configured to track the recognized human shape through a KLT tracking algorithm.
According to the technical system and the method, video stream data are processed by adopting an SSD target detection algorithm based on a convolutional neural network on the basis of video monitoring, images and interaction conditions of goods and people are identified in a monitoring area, and whether the storage goods are stolen or not is estimated, so that early warning can be performed before the storage goods are stolen, storage managers can take countermeasures in advance, the theft rate of the goods is reduced, and the overall economic benefit of storage is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention.
FIG. 1 is a frame structure diagram of a system for intelligently monitoring goods theft prevention in warehousing management by applying the method of the invention;
FIG. 2 is a flow chart of the processing logic of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The embodiments of the present invention, and all other embodiments obtained by a person of ordinary skill in the art without any inventive work, belong to the scope of protection of the present invention.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
As used in this disclosure, "module," "device," "system," and the like are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, or software in execution. In particular, for example, an element may be, but is not limited to being, a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. Also, an application or script running on a server, or a server, may be an element. One or more elements may be in a process and/or thread of execution and an element may be localized on one computer and/or distributed between two or more computers and may be operated by various computer-readable media. The elements may also communicate by way of local and/or remote processes based on a signal having one or more data packets, e.g., from a data packet interacting with another element in a local system, distributed system, and/or across a network in the internet with other systems by way of the signal.
Fig. 1 is a schematic diagram showing a frame structure of a system for intelligently monitoring goods theft prevention in warehousing management applying the method of the present invention, as shown in fig. 1, the system includes: the system comprises a camera array consisting of a plurality of cameras, a video source module, an area planning module, a cargo identification module, a human shape identification module, an overlapping calculation module and a monitoring alarm module.
In order to achieve the effect of real-time detection, the system needs to calculate a large amount of data in real time, and the traditional CPU architecture system cannot meet the requirement. Therefore, in this embodiment, the functional module in the system needs to use the GPU for operation acceleration, specifically, the hardware is supported by using a NVIDIA 2080ti graphics card, the graphics card uses a turing TU102 core, and is manufactured by using a station-integrated circuit 12nmFFN process, so that the performance is strong, and eight paths of 1080p camera data can be processed in parallel at the same time.
The camera is used for collecting monitoring video stream data in the warehouse. Generally, the camera can directly adopt a monitoring camera which is built and debugged in a warehouse and is used. If the definition of camera is not enough, then need more change the camera into 1080P high definition digtal camera to in the frame picture of assurance surveillance video, the definition of goods and anthropomorphic, so that the identification of follow-up module is handled. Meanwhile, if the monitoring range of the camera array overlaps or does not cover each area of the warehouse, the visible range, shooting intersection and relative position of the camera are also required to be adjusted, so that the camera can acquire enough video stream data for subsequent calculation and analysis. Before adjusting the camera, a storage plane needs to be set first, the origin of coordinates of the storage plane is determined, the position of the camera is adjusted according to the origin of coordinates of the storage plane, and the position information, the shooting range and the like of the camera are uploaded to a database to be stored, so that the camera array information can be obtained subsequently.
The video source module is used for receiving and processing monitoring video stream data, and directly reads the video stream data collected by the camera through an RTSP protocol. RTSP (Real Time Streaming Protocol), an application layer Protocol in the TCP/IP Protocol system, defines how a one-to-many application can effectively transmit multimedia data over an IP network. And the video source module receives and unpacks the RTSP, and after the RTSP is locally cached, the RTSP is completely transmitted to the next module.
In this embodiment, the video source module reads video stream data acquired by the camera in real time in a frame picture manner, each frame picture carries meta information, the meta information includes a camera number and a frame number, the camera number refers to a number of the camera acquiring the frame picture in an organized and accurate camera array, and a position, a shooting range and the like of the corresponding camera can be obtained through the number. The frame number refers to the sequential number of the frame picture in the corresponding video stream. Because the frame of the picture is collected by different cameras, the frame numbers of the frame pictures collected by different cameras are accumulated in respective unique digital spaces, and disorder is avoided.
in this embodiment, the video source module is further configured to compare a currently received frame α with a frame β received last time, and calculate a difference ratio between the frame α and the frame β.
Because the cargo identification module and the human shape identification module consume a large amount of hardware, and video pictures do not change greatly for a long time in the cargo monitoring process, the consumption generated by judging the change of two frames of pictures before and after a monitoring video by the video source module is far less than the consumption generated by the cargo identification module and the human shape identification module in object identification. By writing the difference proportion of the two frame pictures before and after the monitoring video into the meta-information, the method is beneficial to the following module to judge whether object identification is needed or not, and the calculation amount is reduced.
The area planning module is used for carrying out area division on the picture of the monitoring video according to the actual area division condition of the warehouse, and determining the boundary, the number and the name of the monitoring area contained in the picture of the monitoring video. In the embodiment, the position and shooting angle of each camera are fixed, so the monitoring area collected by the camera is relatively fixed. Therefore, according to the division condition of the warehouse monitoring area, the video pictures collected by the camera can be divided into different monitoring areas, and the divided monitoring areas are relatively fixed. The monitoring area is defined as a series of continuous points, from the starting point, every two adjacent points are connected until the starting point is reached, the lines are not overlapped, and a closed loop is formed to frame a specific plane space.
The video pictures of the camera are divided into areas through the division of the monitoring areas of the warehouse, so that the subsequent monitoring alarm module can perform alarm logic judgment, and the alarm efficiency is improved.
In this embodiment, the area planning module further has a management function of editing the monitoring area, so that a manager can add, search, modify and delete the monitoring area.
the goods identification module is used for judging whether goods identification is carried out on the frame picture according to the difference proportion carried by the frame picture, when the difference proportion carried by the frame picture α currently received by the goods identification module is larger than or equal to a preset threshold value a, the goods identification module detects and identifies the goods in the frame picture through a built-in SSD target detection algorithm based on a convolutional neural network, otherwise, if the difference proportion between the frame picture α and the previous frame picture β is smaller than the preset threshold value a, object identification operation is not carried out, and the threshold value of the difference proportion can be adjusted according to the environment of monitoring the goods.
When the goods identification module detects and identifies the object in the frame picture, the type of the object and the containing box information of the object can be identified, and the type information and the containing box information of the object are written into the meta information carried by the frame picture. The types of objects are classified into two categories, cargo and human-shaped. Humanoid forms include, in addition to humans, other objects that generally have a mobile nature. The containing box information includes coordinates (X, Y) of the upper left corner of the target object being framed, and the Width (Width) and Height (Height) of the containing box framed in the target object, for a total of four two-dimensional indices.
The human shape recognition module functions similarly to the cargo recognition module, and is mainly used for recognizing and tracking human shapes (and other objects generally having a moving property). Unlike the goods, people (and other objects with moving properties in general) tend to move continuously, and the goods cannot move actively, and are usually fixed in position for a certain period after being stacked in a certain area of the warehouse until being moved out of the warehouse. Thus, the human form recognition module, upon recognizing the human form (and/or other generally moving object), also tracks the human form (and/or other generally moving object) to better determine its absolute position and relative position to the cargo.
In this embodiment, the human shape recognition module also uses the SSD object detection algorithm to detect and recognize human shapes (and/or other objects with moving properties in general) of the objects in the frame. Meanwhile, the human shape recognition module is also internally provided with a KLT tracking algorithm, and target track tracking can be carried out on multi-frame pictures collected by the same camera through the KLT tracking algorithm.
For ease of understanding, the concept principles of the SSD auto-detection algorithm and the KLT tracking algorithm are further explained below.
An SSD (Single Shot Multi Box Detector) algorithm belongs to multi-frame prediction of a one-stage method, adopts a CNN network for detection, uses a multi-scale characteristic diagram, and has the following core design concept:
i) a multiscale profile is employed for detection. I.e. a relatively large signature and a relatively small signature, which are both used for detection. This has the advantage that a larger signature is used to detect relatively small objects, whereas a smaller signature is responsible for detecting large objects.
ii) detection by convolution. And the SSD directly adopts convolution to extract detection results from different feature maps. For a profile with a shape of m x n x p, only a relatively small convolution kernel of 3 x p is needed to obtain the detection values.
iii) setting a prior box. The SSD sets prior frames with different scales or aspect ratios in each unit, and the predicted bounding boxes are based on the prior frames, so that the training difficulty is reduced to a certain extent.
Compared with the Faster-CNN algorithm, the SSD algorithm is more suitable for scenes with higher precision requirements.
The CNN network, namely, the Convolutional Neural network (Convolutional Neural Networks), is a kind of feed-forward Neural network that includes convolution calculation and has a deep structure, is one of the representative algorithms for deep learning, has a characterization learning capability, can perform translation invariant classification on input information according to its hierarchical structure, and has convolution kernel parameter sharing in an implicit layer and sparsity of interlayer connection, so that the Convolutional Neural network can learn lattice characteristics, such as pixels and audio, with a small amount of calculation, has a stable effect, and has no additional feature engineering requirements on data.
In this embodiment, before the cargo identification module system runs formally, the SSD object detection algorithm needs to be identified and trained through the picture data (such as a field picture, a cargo picture, etc.) in the warehouse, so as to ensure that the cargo identification module can subsequently and accurately identify the image of the cargo in each frame of the video stream. For each different warehouse space and different goods, special testing and training are required to improve the adaptability of the goods identification module. The image data required by training is collected through the cameras, specifically, each camera collects 10 hours of videos approximately, the collection time needs to cover various illumination in the morning, at noon and at night and various angles and placement of goods, and frames in the monitoring videos collected by each camera are output to the goods identification module for training after face changing. After the SSD target detection algorithm is supervised, learned and trained, the SSD target detection algorithm can be applied to a system after a satisfactory effect is achieved in a test set (the recall rate and the accuracy rate both reach over 90 percent). The human shape recognition module and the cargo recognition module are the same in principle, and all need to be debugged to different sites to ensure the success rate of recognition.
The KTL Tracking algorithm is named as a Kanade-Lucas-Tomasi Tracking Method, is a classic object Tracking algorithm, can draw a Tracking track and a running direction of a moving object, and is a simple, real-time and efficient Tracking algorithm. The KLT tracking algorithm works with three assumptions:
i) the brightness is constant, and the gray values of corresponding points observed in the previous frame and the next frame are the same;
ii) continuous in time or motion is "small motion"
iii) the spaces are consistent, the adjacent points have similar movement and are kept adjacent.
In one processing flow, input data of the human shape recognition module is a plurality of frames of continuous frame images output by a single camera, the human shape recognition module can track the human shapes in the frames of the continuous early warning and calculate two-dimensional coordinates of the human shapes in the frames to form a coordinate set of the human shapes in a single camera video image until the human shapes leave the video image of the camera, and then the track tracking condition of the forklift at the next camera is calculated by another processing flow.
Specifically, in this embodiment, for the human figure in the frame received by the human figure recognition module, feature points in the human figure image are first found by using feature point extraction algorithms such as SIFT, SURF, FAST, SUSAN, and HARRIS, and these feature points are used as tracking points to perform tracking by using KLT tracking algorithm, and two-dimensional coordinates of these tracking points in the target picture are calculated to form a coordinate set thereof.
The overlapping calculation module is used for calculating the contact conditions between the goods and the people and further analyzing and judging the possibility of various conditions. Specifically, the overlap calculation module calibrates the circumscribed rectangle a and the circumscribed rectangle B of the cargo in the frame picture according to the identified box-containing information of the cargo and the box-containing information of the human shape (and/or other objects generally having moving properties), and determines whether the circumscribed rectangle a and the circumscribed rectangle B overlap in the monitoring area in the frame picture, and if so, sends out an early warning signal.
The overlapping calculation module can convert the overlapping calculation of goods and human figures into a polygonal intersection problem under two-dimensional coordinates by combining the condition that the area planning module divides the monitoring area of the video picture. The division of the surveillance zone itself is divided by a series of points, the surveillance zone also forming a multi-sided row. When the goods are stacked in the goods storage area, the external rectangle of the goods is overlapped with the two-dimensional data point set of the polygon corresponding to the goods storage area, namely, the overlapping occurs.
And the monitoring alarm module is used for receiving the early warning signal, giving an alarm according to a preset alarm strategy, outputting the overlapping condition of the goods and the human shape to a visual terminal, and allowing monitoring personnel to observe and judge. The visual terminal is a terminal with a graphical display interface, such as a computer, a tablet, a smart phone and the like.
Specifically, in order to improve the accuracy of the alarm, the alarm strategy takes the following factors into consideration:
i) time: what time period to alarm and what time period not to alarm can be set according to actual needs;
ii) location which monitoring area alarms and which monitoring area does not alarm; for differences in the warehouse
And iii) the number of occurrences, which is the number of times of overlapping within a limited time of a monitoring area, is counted as an alarm. In the case of a large single cargo (which is a common situation in warehouses), the theft situation often causes the cargo and the figures to overlap continuously for a period of time, and therefore, the number of overlapping occurrences in a limited time can be used as an important characteristic of an alarm.
When the monitoring alarm module needs to send out an alarm, the monitoring alarm module can inform monitoring personnel in various modes of different degrees of strength through abnormal display, short message reminding, buzzer alarm and the like of the monitoring terminal, remind the monitoring personnel to take necessary treatment measures, and manage the whole warehousing system more intelligently. The monitoring alarm module can also perform higher-priority monitoring on related areas (such as warehouse entrances, goods weighing areas, goods storage areas and the like) of goods circulation paths in the warehouse so as to pay important attention. In addition, the monitoring alarm module stores the acquired original data and the final processing data into a database, and the data can be reloaded when needed next time so as to check the current running condition.
Based on the above system, as shown in fig. 2, the method for intelligently monitoring the theft and damage prevention of goods in warehousing management provided by the invention comprises the following steps:
step 1: according to the actual area division condition of the warehouse, the video picture of each camera is divided into areas, and the video picture is divided into different monitoring areas.
Step 2: and monitoring video stream data in each area of the warehouse are acquired through the camera array and transmitted to a server of the video source module through an RTSP (real time streaming protocol) to be processed in the next step in a quasi-real-time manner.
and 3, β a video source module extracts β a current frame picture α and β a previous frame picture beta from β a extracted monitoring video, calculates β a difference ratio between β a frame picture α and β a frame picture beta and writes β a difference ratio into β a meta-information carried by β a frame picture α.
and 4, judging the difference between the frame image alpha and the frame picture β by the goods identification module according to the difference proportion carried by the frame picture α, if the difference proportion is larger than a preset threshold value a, carrying out the next step, otherwise, jumping to the step 3, providing a new frame image gamma by the video source module, and calculating the difference proportion between the frame image gamma and the frame picture α, wherein if the difference proportion is larger than the preset threshold value a, the goods identification module indicates that a large number of new objects possibly enter the video image and the objects need to be identified.
and 5, the goods identification module identifies the goods in the frame picture α through an SSD target detection algorithm, if the goods in the frame picture are identified, the identified goods and box-containing information of the goods are written into the meta information carried by the frame picture α, and the next step is carried out, otherwise, the step 3 is returned.
and 6, the human shape recognition module recognizes the human shape in the frame picture α through an SSD target detection algorithm, if the human shape in the frame picture is recognized, the recognized human shape and human shape containing box information are written into the meta information carried by the frame picture α, and the next step is carried out, otherwise, the step 3 is returned.
and 7, calibrating the external rectangle A and the external rectangle B of the goods in the frame picture α by the overlap calculation module according to the identified box-containing information and the box-containing information of the human shape of the goods, and judging whether the external rectangle A and the external rectangle B are overlapped in the monitoring area in the frame image alpha, if so, generating an early warning signal, otherwise, returning to the step 3.
And 8: and the early warning module receives the early warning signal and gives an alarm according to a preset alarm strategy. Simultaneously, the overlapped condition of the goods and the human shape is output to a visual terminal for monitoring personnel to observe and judge
The process of alarming according to the preset alarming strategy is as follows:
and step 81, judging whether the acquisition time of the frame picture α is in the early warning time period, if so, carrying out the next step, and if not, skipping to the step 3.
and step 82, judging whether the monitoring area in the frame α is an early warning area, if so, carrying out the next step, and if not, skipping to the step 3.
And 83, judging the overlapping times of the external rectangle A and the external rectangle B in the monitoring area in the limited time, alarming if the times exceed a preset threshold value B, and otherwise, skipping to the step 3.
Optionally, in step 6, the human shape recognition module also tracks the recognized human shape through the KLT tracking algorithm. The monitoring alarm module can also display the tracking track of the human shape recognition module on a visual terminal in real time, so that monitoring personnel can observe and judge whether the goods are stolen or not.
According to the technical system and the method, video stream data are processed by adopting an SSD target detection algorithm based on a convolutional neural network on the basis of video monitoring, images and interaction conditions of goods and people are identified in a monitoring area, and whether the stored goods are stolen or not is estimated, so that early warning can be performed on the condition that the stored goods are possibly stolen or not, storage management personnel can take countermeasures in advance, the theft rate of the goods is reduced, and the overall economic benefit of storage is improved.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.
Claims (6)
1. A method for intelligently monitoring goods theft prevention in warehousing management is characterized by comprising the following steps:
step S1: according to the actual area division condition of the warehouse, carrying out area division on the monitoring video pictures of the warehouse, and dividing the video pictures into different monitoring areas;
step S2, extracting a current frame image α and a previous frame picture β in the monitoring video, calculating the difference proportion between the frame image α and the frame picture β, and if the difference proportion exceeds a preset threshold value a, performing the next step;
step S3, carrying out goods identification and human shape identification on the frame picture α through an SSD target detection algorithm, and extracting meta information in the frame picture α, wherein the meta information comprises the containing box information of the object;
and step S4, according to the identified box-containing information and the box-containing information of the shape of the goods, calibrating the external rectangle A and the external rectangle B of the shape of the goods in the frame picture alpha, and alarming if the external rectangle A and the external rectangle B are overlapped in the monitoring area in the frame picture alpha.
2. The method for intelligently monitoring the theft and damage prevention of goods in warehouse management as claimed in claim 1, wherein the step S3 further comprises: and tracking the identified human figure through a KLT tracking algorithm.
3. The method for intelligently monitoring the theft and damage prevention of goods in warehousing management as claimed in claim 1, wherein in the step S4, the alarm process is:
s41, judging whether the acquisition time of the frame picture is in the early warning time period, if so, carrying out the next step, otherwise, jumping to the step S2;
s42, judging whether the monitoring area in the frame picture is an early warning area, if so, carrying out the next step, otherwise, jumping to the step S2;
and S43, judging the overlapping times of the circumscribed rectangle A and the circumscribed rectangle B in the monitoring area in the limited time, alarming if the times exceed a preset threshold B, otherwise, jumping to the step S2.
4. A system for intelligently monitoring goods theft prevention in warehousing management is characterized by comprising:
the system comprises a camera array, a storage and a processing unit, wherein the camera array is used for collecting monitoring video stream data in the warehouse and comprises a plurality of cameras;
the video source module is used for receiving and processing monitoring video stream data, comparing a currently received frame picture α with a last received frame picture β, and calculating a difference proportion between the frame picture α and the frame picture β;
the area planning module is used for carrying out area division on the monitoring video pictures of the warehouse according to the actual area division condition of the warehouse and dividing the video pictures into different monitoring areas;
the goods identification module is used for judging the difference proportion between the frame picture α and the frame picture beta, and when the difference proportion exceeds a preset threshold value a, the goods identification module identifies the goods of the frame picture α through an SSD target detection algorithm and extracts the meta information in the frame picture α, wherein the meta information comprises the containing box information of the object;
the human shape recognition module is used for carrying out human shape recognition on the frame picture α through an SSD target detection algorithm and extracting meta information in the frame image alpha;
the overlap calculation module is used for calibrating a circumscribed rectangle A and a circumscribed rectangle B of the goods in the frame picture α according to the identified box-containing information and the box-containing information of the humanoid of the goods, judging whether the circumscribed rectangle A and the circumscribed rectangle B are overlapped in the monitoring area in the frame image alpha or not, and sending out an early warning signal if the circumscribed rectangle A and the circumscribed rectangle B are overlapped;
and the monitoring alarm module is used for receiving the early warning signal, giving an alarm according to a preset alarm strategy, outputting the overlapping condition of the goods and the human shape to a visual terminal, and allowing monitoring personnel to observe and judge.
5. The system for intelligently monitoring the theft of goods in warehousing management as claimed in claim 4 wherein the camera is a 1080P high definition camera.
6. The system for intelligently monitoring the theft of cargo in warehousing management as recited in claim 4, wherein the human form identification module is further configured to track the identified human form through a KLT tracking algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010061581.5A CN111223260A (en) | 2020-01-19 | 2020-01-19 | Method and system for intelligently monitoring goods theft prevention in warehousing management |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010061581.5A CN111223260A (en) | 2020-01-19 | 2020-01-19 | Method and system for intelligently monitoring goods theft prevention in warehousing management |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111223260A true CN111223260A (en) | 2020-06-02 |
Family
ID=70828444
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010061581.5A Pending CN111223260A (en) | 2020-01-19 | 2020-01-19 | Method and system for intelligently monitoring goods theft prevention in warehousing management |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111223260A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112532950A (en) * | 2020-12-14 | 2021-03-19 | 南通一零八家纺科技有限公司 | Video processing method of home textile visual management system |
| CN112529959A (en) * | 2020-12-14 | 2021-03-19 | 北京爱笔科技有限公司 | Article movement detection method and device |
| CN112598745A (en) * | 2020-12-14 | 2021-04-02 | 北京爱笔科技有限公司 | Method and device for determining person-goods associated events |
| CN114005068A (en) * | 2021-11-08 | 2022-02-01 | 支付宝(杭州)信息技术有限公司 | Method and device for monitoring movement of goods |
| CN114359836A (en) * | 2022-01-10 | 2022-04-15 | 长春师范大学 | Automatic-identification monitoring system and monitoring method based on computer |
| CN114743326A (en) * | 2022-04-07 | 2022-07-12 | 武汉东湖学院 | Intelligent manufacturing workshop anti-theft early warning system with intelligent identification function |
| CN117557968A (en) * | 2024-01-11 | 2024-02-13 | 深圳市明心数智科技有限公司 | Monitoring method, monitoring device, storage medium and computer equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2378598A (en) * | 2001-06-13 | 2003-02-12 | Integrated Design Ltd | Security imaging using illumination. |
| CN101770648A (en) * | 2009-01-06 | 2010-07-07 | 北京中星微电子有限公司 | Video monitoring based loitering system and method thereof |
| CN102881100A (en) * | 2012-08-24 | 2013-01-16 | 济南纳维信息技术有限公司 | Video-analysis-based antitheft monitoring method for physical store |
| CN103686086A (en) * | 2013-12-13 | 2014-03-26 | 广州中国科学院先进技术研究所 | Method for carrying out video monitoring on specific area |
| CN104392573A (en) * | 2014-10-11 | 2015-03-04 | 天津艾思科尔科技有限公司 | Video-based intelligent theft detection method |
| CN107277448A (en) * | 2017-07-11 | 2017-10-20 | 合肥弘恩机电科技有限公司 | A kind of intelligent storage video monitoring system |
| CN109903503A (en) * | 2019-04-16 | 2019-06-18 | 上海天诚比集科技有限公司 | A kind of detection method in video monitoring object intrusion detection region |
| CN110379108A (en) * | 2019-08-19 | 2019-10-25 | 铂纳思(东莞)高新科技投资有限公司 | A kind of method and its system of unmanned shop anti-thefting monitoring |
-
2020
- 2020-01-19 CN CN202010061581.5A patent/CN111223260A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2378598A (en) * | 2001-06-13 | 2003-02-12 | Integrated Design Ltd | Security imaging using illumination. |
| CN101770648A (en) * | 2009-01-06 | 2010-07-07 | 北京中星微电子有限公司 | Video monitoring based loitering system and method thereof |
| CN102881100A (en) * | 2012-08-24 | 2013-01-16 | 济南纳维信息技术有限公司 | Video-analysis-based antitheft monitoring method for physical store |
| CN103686086A (en) * | 2013-12-13 | 2014-03-26 | 广州中国科学院先进技术研究所 | Method for carrying out video monitoring on specific area |
| CN104392573A (en) * | 2014-10-11 | 2015-03-04 | 天津艾思科尔科技有限公司 | Video-based intelligent theft detection method |
| CN107277448A (en) * | 2017-07-11 | 2017-10-20 | 合肥弘恩机电科技有限公司 | A kind of intelligent storage video monitoring system |
| CN109903503A (en) * | 2019-04-16 | 2019-06-18 | 上海天诚比集科技有限公司 | A kind of detection method in video monitoring object intrusion detection region |
| CN110379108A (en) * | 2019-08-19 | 2019-10-25 | 铂纳思(东莞)高新科技投资有限公司 | A kind of method and its system of unmanned shop anti-thefting monitoring |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112532950A (en) * | 2020-12-14 | 2021-03-19 | 南通一零八家纺科技有限公司 | Video processing method of home textile visual management system |
| CN112529959A (en) * | 2020-12-14 | 2021-03-19 | 北京爱笔科技有限公司 | Article movement detection method and device |
| CN112598745A (en) * | 2020-12-14 | 2021-04-02 | 北京爱笔科技有限公司 | Method and device for determining person-goods associated events |
| CN112532950B (en) * | 2020-12-14 | 2022-08-16 | 南通一零八家纺科技有限公司 | Video processing method of home textile visual management system |
| CN112598745B (en) * | 2020-12-14 | 2024-05-28 | 北京爱笔科技有限公司 | Method and device for determining person-goods association event |
| CN114005068A (en) * | 2021-11-08 | 2022-02-01 | 支付宝(杭州)信息技术有限公司 | Method and device for monitoring movement of goods |
| CN114359836A (en) * | 2022-01-10 | 2022-04-15 | 长春师范大学 | Automatic-identification monitoring system and monitoring method based on computer |
| CN114359836B (en) * | 2022-01-10 | 2022-09-13 | 长春师范大学 | Automatic-identification monitoring system and monitoring method based on computer |
| CN114743326A (en) * | 2022-04-07 | 2022-07-12 | 武汉东湖学院 | Intelligent manufacturing workshop anti-theft early warning system with intelligent identification function |
| CN117557968A (en) * | 2024-01-11 | 2024-02-13 | 深圳市明心数智科技有限公司 | Monitoring method, monitoring device, storage medium and computer equipment |
| CN117557968B (en) * | 2024-01-11 | 2024-04-30 | 深圳市明心数智科技有限公司 | Monitoring method, monitoring device, storage medium and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111223260A (en) | Method and system for intelligently monitoring goods theft prevention in warehousing management | |
| Benezeth et al. | Abnormal events detection based on spatio-temporal co-occurences | |
| Tian et al. | IBM smart surveillance system (S3): event based video surveillance system with an open and extensible framework | |
| US7558404B2 (en) | Detection of abnormal crowd behavior | |
| CN111311630B (en) | Method and system for intelligently counting cargo quantity through video in warehouse management | |
| Swathi et al. | Crowd behavior analysis: A survey | |
| CN102346855B (en) | A system and method for classification of moving object during video surveillance | |
| Sabri et al. | Low-cost intelligent surveillance system based on fast CNN | |
| CN110674703A (en) | Video tripwire alarm counting method and flow in intelligent monitoring | |
| Adrian et al. | An overview of intelligent surveillance systems development | |
| Nam | Loitering detection using an associating pedestrian tracker in crowded scenes | |
| Park et al. | A track-based human movement analysis and privacy protection system adaptive to environmental contexts | |
| Elhamod et al. | Real-time semantics-based detection of suspicious activities in public spaces | |
| Singh et al. | An intelligent video surveillance system using edge computing based deep learning model | |
| Yan et al. | Mutually coupled detection and tracking of trucks for monitoring construction material arrival delays | |
| US20230360402A1 (en) | Video-based public safety incident prediction system and method therefor | |
| CN115690683A (en) | Monitoring method and device for state of barrier, computer equipment and storage medium | |
| Micheloni et al. | Exploiting temporal statistics for events analysis and understanding | |
| Vaishnavi et al. | Implementation of Abnormal Event Detection using Automated Surveillance System | |
| Mahin et al. | A simple approach for abandoned object detection | |
| Russel et al. | Ownership of abandoned object detection by integrating carried object recognition and context sensing | |
| Deniziak et al. | An autonomous monitoring system based on object shape detection | |
| CN114241397B (en) | Frontier defense video intelligent analysis method and system | |
| Srilaya et al. | Surveillance using video analytics | |
| Eliazer et al. | Smart CCTV camera surveillance system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200602 |
|
| RJ01 | Rejection of invention patent application after publication |