CN115359416A - Intelligent early warning system for railway freight yard sky eye - Google Patents
Intelligent early warning system for railway freight yard sky eye Download PDFInfo
- Publication number
- CN115359416A CN115359416A CN202210845459.6A CN202210845459A CN115359416A CN 115359416 A CN115359416 A CN 115359416A CN 202210845459 A CN202210845459 A CN 202210845459A CN 115359416 A CN115359416 A CN 115359416A
- Authority
- CN
- China
- Prior art keywords
- image
- target
- detection
- result
- management unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B19/00—Alarms responsive to two or more different undesired or abnormal conditions, e.g. burglary and fire, abnormal temperature and abnormal rate of flow
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B31/00—Predictive alarm systems characterised by extrapolation or other computation using updated historic data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Human Computer Interaction (AREA)
- Artificial Intelligence (AREA)
- Business, Economics & Management (AREA)
- Databases & Information Systems (AREA)
- Emergency Management (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Signal Processing (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Image Analysis (AREA)
Abstract
The utility model provides a railway goods yard sky eye intelligence early warning system, is a system that can real-time intelligent recognition and further standardize the loading and unloading operation, has realized the unified management of many algorithms, and camera data acquisition in the cooperation goods yard adopts the intelligent analysis unit of modularization overall formula, can carry out real-time dynamic's supervision and early warning analysis to operations such as goods yard warehouse, warehouse car side, warehouse train side, goods yard red tablet monitoring department, realizes goods yard operation intelligence inspection and supervision. The invention has the advantages that: the method can be used for analyzing according to a specific recognizer algorithm as required, and has high analysis efficiency and high accuracy of an analysis result. The accident scene can be quickly and accurately positioned, the abnormal condition in the monitoring picture is judged, and an alarm is sent out or other actions are triggered in a fastest and optimal mode; the method can analyze according to a specific identifier algorithm as required, has high analysis efficiency and high accuracy of an analysis result, improves the utilization rate of video monitoring, further improves the operation efficiency of the goods yard, and improves the safety management level of the goods yard.
Description
Technical Field
The invention relates to the technical field of railway freight yard safety, in particular to a railway freight yard sky-eye intelligent early warning system.
Background
The railway freight yard is a production workshop for handling goods carrying, storing, loading, unloading and delivering operations at a railway freight station, and is also a place for connecting railway freight with other freight transport vehicles. The goods yard is that there is certain safe risk in the actual production and the operation in-process of goods yard for the railway, if the railway goods yard takes place in recent years and does not operate according to train switch door operation standard and appear the injure incident of pounding that the door drops and cause, personnel stand in the yellow line of warning and fall the platform, the goods is lost and can not discover in time, loading and unloading operation in-process is not placed the red tablet, this has exposed the railway goods yard relatively backward in the aspect of the safety precaution control to a certain extent, can't carry out timely effectual management and control to above-mentioned risk. At present, a comprehensive video monitoring system is generally implemented in a domestic goods yard, so that a user is difficult to quickly obtain valuable information from numerous monitoring information, the valuable information cannot be matched and linked with a service scene, the value of a video is not really exerted, and the waste of video resources is caused. In such a large number of video streams, human supervision is burdensome and inefficient, and monitoring is still only available for subsequent forensics and retrieval.
Disclosure of Invention
The invention discloses an intelligent early warning system for a railway freight yard sky eye, which aims to solve the problems of untimely and inaccurate data acquisition, low efficiency, poor safety early warning capability of the freight yard and low disposal efficiency caused by manual acquisition of operation information in the traditional freight production organization process.
In order to solve the problems, the invention adopts the following technical scheme: the utility model provides a railway goods yard sky eye intelligence early warning system which characterized in that includes: the image information management unit is connected with the data terminal and used for processing and classifying the acquired image information, the image information management unit is also used for connecting a client and outputting data, the image information management unit transmits the image information to the intelligent recognition management unit and calls an item point recognition sub-algorithm of the intelligent recognition management unit to perform intelligent analysis processing on the image information, and the image information management unit is also used for connecting a server and storing the data. The intelligent recognition management unit is internally provided with an intelligent item point recognition module, the intelligent item point recognition module is used for managing a self-learning sample training model of each item point and executing a sub-algorithm for recognizing each item point, and the sub-algorithm for recognizing the item point comprises the following steps: the method comprises a personnel inspection identification algorithm, a non-through tooling identification algorithm, a cargo space occupation identification algorithm, a key cargo space identification algorithm, a loading and unloading vehicle identification algorithm, a red board identification algorithm, a platform boundary crossing identification algorithm, a smoking identification algorithm, a non-standard door opening and closing identification algorithm and a forklift overspeed identification algorithm, wherein at least one algorithm is used in the item point identification process, and for example, the item points without red boards during operation are identified by combining the loading and unloading vehicle identification algorithm and the red board identification algorithm.
In the invention, the data terminal is a camera in a goods yard and is used for shooting the regional image in the warehouse range, and the shot image data is input to the image information management unit through the data terminal.
After the image information management unit acquires the image data, the regions where the data terminals are located are classified into a warehouse, a train side, an automobile side, a red board shooting place and the like, then the corresponding item point identification sub-algorithm is selected from the intelligent identification management unit according to needs to carry out intelligent identification on the image data, if the intelligent identification result is early warning data, the result and the image data are fed back to the connection client side in real time to carry out alarm and are simultaneously fed back to the server side to store the result and the picture data, and if the intelligent identification result is operation data, the result is fed back to the connection server side to store the result and the picture data.
The server side is a data server and is mainly used for providing strong calculation support and storing picture data, early warning data and goods yard operation data sent by the image information management unit.
The client is system software and is installed on a user computer, receives the early warning information and the early warning pictures sent by the image information management unit in real time, sends sound early warning to managers at the same time, and can also be used for data communication with the server.
The personnel patrol identification algorithm specifically comprises the following steps: s11: analyzing an image acquired by a data terminal, and performing target detection on personnel in the image by adopting a yolov4 network to obtain a circumscribed rectangular frame, a top left vertex coordinate and a width and a height of a personnel target; s12: tracking a personnel target to obtain a personnel target id; s13: and extracting a person position image according to the person positioning result frame, performing target detection on the dressing type (tooling, id =0, uniform, id = 1), filtering out the result of non-uniform (id! = 1), and taking the result with the highest confidence level in the results larger than a confidence level threshold (confidence > 0.8) as the recognition result of the current frame. If the uniform target is identified, setting the target result to be 1, otherwise, setting the target result to be 0; s14: and storing uniform detection results with the same target id in the same result queue. The result queue will store the latest detection result with fixed frame number. Accumulating the result values and judging whether the result values exceed a threshold value; s15: if the threshold value is exceeded, an alarm value is returned to the image information management unit.
The identification algorithm of the tool which is not worn specifically comprises the following steps: s21: analyzing an image acquired by a data terminal, and performing target detection on personnel in the image by adopting a yolov4 network to obtain a circumscribed rectangular frame, a top left vertex coordinate and a width and a height of a personnel target; s22: tracking a personnel target to obtain a personnel target id; s23: and extracting a person position image according to the person positioning result frame, carrying out target detection of dressing types (tooling, id =0, uniform, id = 1), and filtering a result lower than a confidence threshold (confidence < 0.8). If no uniform or tool is identified, setting the target result to be 1, otherwise, setting the target result to be 0; s24: and storing uniform detection results with the same target id in the same result queue. The result queue will store the latest detection result with fixed frame number. Accumulating the result values and judging whether the result values exceed a threshold value; s25: if the threshold value is exceeded, an alarm value is returned to the image information management unit.
The goods space occupation identification algorithm specifically comprises the following steps: s31: analyzing an image acquired by a data terminal, and performing masking operation on the image and a preset detection area to generate a detection image which is totally black except the detection area; s32: carrying out example segmentation on the goods on the detection graph by using a Mask-Rcnn network, and judging whether the percentage of the goods in the goods space exceeds a threshold value or not according to the calculation; s33: carrying out perspective transformation on the example segmentation image of the detection area part according to the four coordinate points of the detection area and the actual length and width of the goods space; s34: calculating the percentage of the cargo example in the perspective transformation image in the cargo space area; s35: the result of the detection is fed back to the image information management unit.
The key cargo space identification algorithm specifically comprises the following steps: s41: analyzing an image acquired by a data terminal, and performing masking operation on the image and a preset detection area to generate a detection image which is totally black except the detection area; s42: detecting the targets of the personnel and the forklift in the image, returning a detection result if the personnel or the forklift targets appear, and executing alarm; s43: if the region to be detected does not have personnel and a target of a forklift, segmenting the foreground and the background of the image by using a Gaussian mixture model; s44: performing morphological opening and closing operations on the foreground image, removing isolated noise points, and extracting a foreground contour; s45: filtering the foreground contour according to the contour area, and deleting the contours which are lower than the area low threshold and higher than the area high threshold; s46: the remaining contour is regarded as the position of the change in the detection area, and the detection result is fed back to the image information management unit.
The loading and unloading vehicle identification algorithm specifically comprises the following steps: s51: analyzing an image acquired by a data terminal, and carrying out target positioning on a train, a single vehicle door, a cavity between vehicle doors, a vehicle number and the like in the image by using a yolov4 network; s52: judging whether the detected target result is in a preset detection area or not according to the occupation ratio of the target in the detection area; s53: extracting the image of the vehicle number part, performing digital identification, and filtering the upper left position of the image and an overlapped identification result; s54: judging whether the number of the reserved car numbers is 7, if the number meets the condition, sequencing the numbers from left to right, combining the numbers into a car number character string in sequence and returning the car number character string to the upper layer; s55: integrating the single car door and the car door cavity target in the detection area, extracting a car door position image, and judging the opening and closing state of the car door; s56: if the combined vehicle door contains a vehicle door cavity, the vehicle door is in an open state, otherwise, the vehicle door is in a closed state; s57: if the door is opened, carrying out target detection on the forklift and the goods; s58: if the vehicle door is in a closed state, carrying out target detection of sealing lock; s59: the result of the detection is fed back to the image information management unit.
The red card identification algorithm specifically comprises the following steps: s61: extracting a detection area image according to the coordinates of the plurality of red card detection areas; s62: inputting the target to a yolov4 network model for target detection of the red card; s63: and feeding back the result of the presence or absence of the red card in the detection areas to the image information management unit.
The platform border crossing identification algorithm specifically comprises the following steps: s71: carrying out target detection on the personnel in the image by using a yolov4 network; s72: judging whether an area of a person in a platform yellow line exists according to whether the percentage of the part of the bottom edge of the person target frame in the detection area to the whole bottom edge exceeds a threshold value; s73: and feeding back the status value result of the boundary crossing of the person to the image information management unit.
The smoking identification algorithm specifically comprises the following steps: s81: analyzing an image acquired by a data terminal, and performing target detection on personnel in the image by adopting a yolov4 network to obtain an external rectangular frame, a top left vertex coordinate and a width and a height of the personnel target; s82: extracting a human face position image from the human target image, and carrying out target detection on the cigarettes; s83: filtering the recognition result of the cigarette with low confidence; s84: the detection result is fed back to the image information management unit.
The nonstandard door opening and closing identification algorithm specifically comprises the following steps: s91: positioning the single car door of the train and the cavity between the car doors by adopting a yolov4 network, and combining the single car door and the cavity belonging to the same carriage together according to the position information to generate an external rectangular frame of the complete car door; s92: extracting a car door image, and identifying the door opening and closing action, a door pulling rope and the vertical edge of a single car door which are currently carried out by the car door; s93: determining whether door opening or closing behaviors and door opening or closing directions exist according to the judged door opening and closing actions and the positions of the door pulling ropes; s94: determining a single-side vehicle door which is opening (closing) the door, taking a fitting edge straight line of upper and lower middle points of left and right edge target frames of the vehicle door, calculating an intersection point with the left edge of the platform, making a vertical line which passes through the intersection point and is vertical to the edge of the platform, and intersecting the right edge of the platform at two points. Fitting a horizontal projection outline of the vehicle door area on the platform by using the four intersection points; s95: setting the vehicle door area and the projection area as detection areas, detecting a target of a person in the detection areas, and determining that the detected target of the person is an illegal behavior; s96: the result of the detection is fed back to the image information management unit.
The forklift overspeed algorithm specifically comprises the following steps: s101: analyzing the images acquired by the data terminal, tracking the forklift by adopting a depersort algorithm, and recording the positions of the forklift with the same ID in the front frame image and the rear frame image; s102: projecting a speed measuring area in the image onto a plane of a real scene to obtain a perspective transformation matrix W; s103, projecting the position of the forklift onto a real scene plane by using a perspective transformation matrix W; s104: calculating the Euclidean distance s of the forklift with the same ID on the real scene plane in the front frame image and the rear frame image; s105: combining the time difference t of the two frames of images before and after the image is combined with a formula v = s/t, and then calculating the speed v (km/h); and S106, feeding back the velocity value result v (km/h) to the image information management unit.
In the invention, in the item recognition sub-algorithm, one or more of a YOLOv4 algorithm, a Gaussian mixture model algorithm, a DeepsORT target tracking algorithm, a Mask-RCNN algorithm, an image morphology algorithm, a Gaussian filter algorithm and a perspective transformation algorithm are adopted for processing the image.
The invention has the advantages that:
the railway freight yard sky-eye intelligent early warning system is matched with camera data acquisition in the freight yard warehouse range, adopts modular intelligent analysis, can dynamically supervise and early warn analyze operation areas such as warehouse goods, a warehouse platform, red card placement and the like in real time, and can effectively prevent personnel from falling off the platform, smoking to cause fire, train doors falling off to smash people, hazards caused by overspeed of a forklift, and dangers of intrusion of other trains and loss of goods after the red card is not placed in the operation process; the accident scene can be quickly and accurately positioned, the abnormal condition in the monitoring picture is judged, and an alarm is sent out or other actions are triggered in a fastest and optimal mode; the method can analyze according to a specific identifier algorithm, has high analysis efficiency and high accuracy of an analysis result, improves the utilization rate of video monitoring, further improves the operation efficiency of the goods yard, and improves the safety management level of the goods yard.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts;
FIG. 1 is a schematic diagram of the system of the present invention;
FIG. 2 is a diagram of the relationship of the elements of the system of the present invention;
FIG. 3 is a flowchart of the inspection process for inspecting personnel according to the present invention;
FIG. 4 is a flow chart of the detection process of the unworn tooling in the present invention;
FIG. 5 is a flow chart of a cargo space occupancy detection process in accordance with the present invention;
FIG. 6 is a flow chart of the key cargo space detection process of the present invention;
FIG. 7 is a flowchart of a lift truck detection process of the present invention;
FIG. 8 is a flow chart of the red card detection process of the present invention;
FIG. 9 is a flowchart illustrating the station crossing detection process according to the present invention;
FIG. 10 is a flow chart of a smoking detection process in the present invention;
FIG. 11 is a flow chart of an open/close door detection process according to the present invention;
fig. 12 is a flowchart of the forklift overspeed detection processing in the present invention.
Detailed Description
In order to make the technical solutions and advantages of the present invention clearer, the following describes the technical solutions in the embodiments of the present invention clearly and completely with reference to the drawings in the embodiments of the present invention.
Railway goods yard sky eye intelligence early warning system includes: the image information management unit is connected with the data terminal and used for processing and classifying the acquired image information, the image information management unit is also used for connecting a client and outputting data, the image information management unit transmits the image information to the intelligent recognition management unit and calls an item point recognition sub-algorithm of the intelligent recognition management unit to perform intelligent analysis processing on the image information, and the image information management unit is also used for connecting a server and storing the data. The intelligent recognition management unit is internally provided with an intelligent item point recognition module, the intelligent item point recognition module is used for managing a self-learning sample training model of each item point and executing a sub-algorithm for recognizing each item point, and the sub-algorithm for recognizing the item point comprises the following steps: the method comprises a personnel inspection identification algorithm, a non-through tooling identification algorithm, a goods space occupation identification algorithm, a key goods space identification algorithm, a loading and unloading vehicle identification algorithm, a red plate identification algorithm, a platform boundary crossing identification algorithm, a smoking identification algorithm, a non-standard door opening and closing identification algorithm and a forklift overspeed identification algorithm, wherein at least one algorithm is used in the item point identification process, and for example, the item points which are not placed with red plates in the operation are identified by combining the loading and unloading vehicle identification algorithm and the red plate identification algorithm.
In the invention, the data terminal is a camera in a goods yard and is used for shooting the regional image in the warehouse range, and the shot image data is input to the image information management unit through the data terminal.
After the image information management unit acquires the image data, the regions where the data terminals are located are classified into a warehouse, a train side, an automobile side, a red board shooting place and the like, then the corresponding item point identification sub-algorithm is selected from the intelligent identification management unit according to needs to carry out intelligent identification on the image data, if the intelligent identification result is early warning data, the result and the image data are fed back to the connection client side in real time to carry out alarm and are simultaneously fed back to the server side to store the result and the picture data, and if the intelligent identification result is operation data, the result is fed back to the connection server side to store the result and the picture data.
The server side is a data server and is mainly used for providing strong calculation support and storing picture data, early warning data and goods yard operation data sent by the image information management unit.
The client is system software and is installed on a user computer, receives early warning information and early warning pictures sent by the image information management unit in real time, sends sound early warning to managers at the same time, and can also be used for data communication with the server.
The intelligent early warning system for the sky eye of the railway freight yard has the core of customized intelligent image analysis technology based on customer requirements, which can be seen by combining the attached drawings 1 and 2. The system comprises a picture information management unit, a client system, a server system, a goods yard service scene application demand management unit, a picture information management unit, a connection client side and a video picture analysis algorithm, wherein the picture information management unit finishes picture collection of a camera and picture area classification, then calls a goods yard identification sub algorithm of the intelligent identification management unit according to the area where the camera is located to respectively carry out intelligent item identification, the early warning information and corresponding picture data are directly fed back to the connection client side to carry out real-time alarm prompt if early warning information exists in an identification result, the operation information in the identification result is stored in the server side, the intelligent identification management unit finishes the demand collection and picture data processing according to the goods yard service scene application demand, trains and generates the video picture analysis algorithm, optimizes and improves the accuracy rate of the item identification according to a self-learning mode, the image information management unit in the system is connected with the client system and the server side to carry out data interaction, and calls goods yard operation information stored in the server side to form a dynamic goods yard and a statistical analysis chart to be displayed to the client system.
The recognizer algorithm of the present invention will be specifically described below.
(1) And (3) a personnel patrol identification algorithm: the personnel patrol identification algorithm specifically comprises the following steps: s11: analyzing an image acquired by a data terminal, and performing target detection on personnel in the image by adopting a yolov4 network to obtain a circumscribed rectangular frame, a top left vertex coordinate and a width and a height of a personnel target; s12: tracking a personnel target to obtain a personnel target id; s13: and extracting a person position image according to the person positioning result frame, carrying out target detection on the dressing type (tooling, id =0, uniform, id = 1), filtering out the result of non-uniform (id! = 1), and taking the result with the highest confidence level larger than a confidence level threshold (confidence > 0.8) as the recognition result of the current frame. If a uniform target is identified, setting the target result to be 1, otherwise, setting the target result to be 0; s14: and storing the uniform detection results with the same target id in the same result queue. The result queue will store the latest detection result with fixed frame number. Accumulating the result values and judging whether the result values exceed a threshold value; s15: if the threshold value is exceeded, an alarm value is returned to the image information management unit.
Specifically, as can be seen from fig. 3, the system receives a service request sent by a user, the intelligent identification management unit invokes a staff patrol identification algorithm to analyze an image, performs target location on staff in the image through the algorithm, performs staff target tracking and dress type detection at the same time, filters out non-uniform results, and outputs the detection result if the rest are uniform targets.
(2) And (3) identifying algorithm of the non-penetrated tool: the identification algorithm of the tool which is not worn specifically comprises the following steps: s21: analyzing an image acquired by a data terminal, and performing target detection on personnel in the image by adopting a yolov4 network to obtain an external rectangular frame, a top left vertex coordinate and a width and a height of the personnel target; s22: tracking a personnel target to obtain a personnel target id; s23: and extracting a person position image according to the person positioning result frame, carrying out target detection of dressing types (tooling, id =0, uniform, id = 1), and filtering a result lower than a confidence threshold (confidence < 0.8). If no uniform or tool is identified, setting the target result to be 1, otherwise, setting the target result to be 0; s24: and storing uniform detection results with the same target id in the same result queue. The result queue will store the latest detection result with fixed frame number. Accumulating the result values and judging whether the result values exceed a threshold value; s25: and if the threshold value is exceeded, returning an alarm value to the image information management unit.
Specifically, as can be seen in fig. 4, the system receives a service request sent by a user, the intelligent identification management unit calls an unworn tooling identification algorithm, analyzes the image, performs target positioning on personnel in the image through the algorithm, performs personnel target tracking and clothing type detection at the same time, filters out uniform results, and outputs the detection result when the remaining non-uniform targets exist.
(3) A goods space occupation identification algorithm: the goods space occupation identification algorithm specifically comprises the following steps: s31: analyzing an image acquired by a data terminal, and carrying out masking operation on the image and a preset detection area to generate a detection image which is totally black except the detection area; s32: carrying out example segmentation on the goods on the detection graph by using a Mask-Rcnn network, and judging whether the percentage of the goods in the goods space exceeds a threshold value or not according to the calculation; s33: carrying out perspective transformation on the example segmentation image of the detection area part according to the four coordinate points of the detection area and the actual length and width of the goods space; s34: calculating the percentage of the cargo example in the perspective transformation image in the cargo space area; s35: the result of the detection is fed back to the image information management unit.
Specifically, as can be seen in fig. 5, the system receives a service request sent by a user, the intelligent identification management unit calls a cargo space occupation identification algorithm, masks the image and blackens all non-detection areas, the algorithm performs example segmentation and perspective transformation on the cargo in the image, the occupation area is calculated according to the cargo example, and the detection result is output.
(4) The key cargo space identification algorithm: the key cargo space identification algorithm specifically comprises the following steps: s41: analyzing an image acquired by a data terminal, and performing masking operation on the image and a preset detection area to generate a detection image which is totally black except the detection area; s42: detecting targets of the personnel and the forklift in the image, returning a detection result if the personnel or the forklift targets appear, and executing alarm; s43: if the area to be detected does not have personnel and a target of a forklift, segmenting the foreground and the background of the image by using a Gaussian mixture model; s44: performing morphological opening and closing operations on the foreground image, removing isolated noise points, and extracting a foreground outline; s45: filtering the foreground contour according to the contour area, and deleting the contours which are lower than the area low threshold and higher than the area high threshold; s46: the remaining contour is regarded as the position where the change occurs in the detection area, and the detection result is fed back to the image information management unit.
Specifically, as can be seen from fig. 6, the system receives a service request sent by a user, the intelligent identification management unit calls a key cargo space identification algorithm, performs masking operation on an image and blackens all non-detection areas, performs target detection on personnel and a forklift in the image through the algorithm, returns a detection result if a target is detected, segments the front background of the model of the graphic if the target is not detected, performs morphological opening and comparing operation on a foreground graph, removes noise points and extracts contours, filters the contours and deletes invalid contours, and outputs the detection result if the remaining contours are regarded as changed areas.
(5) A loading and unloading vehicle identification algorithm: the loading and unloading vehicle identification algorithm specifically comprises the following steps: s51: analyzing an image acquired by a data terminal, and carrying out target positioning on a train, a single vehicle door, a cavity between vehicle doors, a vehicle number and the like in the image by using a yolov4 network; s52: judging whether the detected target result is in a preset detection area or not according to the occupation ratio of the target in the detection area; s53: extracting the image of the vehicle number part, performing digital identification, and filtering the upper left position of the image and the overlapped identification result; s54: judging whether the number of the reserved car numbers is 7, if the number meets the condition, sequencing the numbers from left to right, combining the numbers into a car number character string in sequence and returning the car number character string to the upper layer; s55: integrating the single car door and the car door cavity target in the detection area, extracting a car door position image, and judging the opening and closing state of the car door; s56: if the combined vehicle door contains a vehicle door cavity, the vehicle door is in an open state, otherwise, the vehicle door is in a closed state; s57: if the door is opened, carrying out target detection on the forklift and the goods; s58: if the vehicle door is in a closed state, carrying out target detection of sealing lock; s59: the result of the detection is fed back to the image information management unit.
Specifically, as can be seen from fig. 7, the system receives a service request sent by a user, the intelligent identification management unit calls a loading and unloading vehicle identification algorithm, performs target positioning on a train, a single vehicle door, a cavity between vehicle doors, a vehicle number and the like in an image through the algorithm, returns a result if no train is detected, determines whether a vehicle door exists if a train is detected, determines that a current state is open if a vehicle door is detected and detects a vehicle door cavity, determines that a current state is closed if only a vehicle door is detected, performs sealing lock detection if the vehicle door is closed, performs unloading operation if a target is detected, detects that a train is unloading operation if the vehicle door is open, detects that a cargo exists in the train if a cargo exists, performs unloading operation if a cargo does not exist, and outputs a detection result if a cargo does not exist.
(6) The red card recognition algorithm: the red card identification algorithm specifically comprises the following steps: s61: extracting a detection area image according to the coordinates of the plurality of red card detection areas; s62: inputting the target to a yolov4 network model for target detection of the red card; s63: and feeding back the result of the presence or absence of the red cards in the detection areas to the image information management unit.
Specifically, as can be seen in fig. 8, the system receives a service request sent by a user, the intelligent recognition management unit invokes a red card recognition algorithm, performs target detection on red cards in the image through the algorithm, and returns multiple results if there are multiple red cards in the image, and outputs a detection result.
(7) Platform boundary-crossing identification algorithm: the platform border crossing identification algorithm specifically comprises the following steps: s71: carrying out target detection on the personnel in the image by using a yolov4 network; s72: judging whether an area of a person in a platform yellow line exists according to whether the percentage of the part of the bottom edge of the person target frame in the detection area to the whole bottom edge exceeds a threshold value; s73: and feeding back the status value result of the boundary crossing of the person to the image information management unit.
Specifically, as can be seen from fig. 9, the system receives a service request sent from a user, the intelligent identification management unit invokes a platform boundary crossing identification algorithm, performs target detection on the person target in the image through the algorithm, and outputs a detection result.
(8) Smoking identification algorithm: the smoking identification algorithm specifically comprises the following steps: s81: analyzing an image acquired by a data terminal, and performing target detection on personnel in the image by adopting a yolov4 network to obtain a circumscribed rectangular frame, a top left vertex coordinate and a width and a height of a personnel target; s82: extracting a human face position image from the human target image, and carrying out target detection on the cigarettes; s83: filtering the recognition result of the cigarette with low confidence; s84: and feeding back the detection result to the image information management unit.
Specifically, as can be seen from fig. 10, the system receives a service request sent from a user, the intelligent recognition management unit calls a smoking recognition algorithm, performs target detection on the person in the image through the algorithm, locates the face image of the person, performs cigarette detection, and outputs a detection result.
(9) Non-standard door opening and closing identification algorithm: the nonstandard door opening and closing identification algorithm specifically comprises the following steps: s91: positioning the single car door of the train and the cavity between the car doors by adopting a yolov4 network, and combining the single car door and the cavity belonging to the same carriage together according to the position information to generate an external rectangular frame of the complete car door; s92: extracting a car door image, and identifying the door opening and closing action, a door pulling rope and the vertical edge of a single car door which are currently carried out by the car door; s93: determining whether door opening or closing behaviors and door opening or closing directions exist according to the judged door opening and closing actions and the positions of the door pulling ropes; s94: determining a single-side vehicle door which is opening (closing) the door, taking a fitting edge straight line of upper and lower middle points of left and right edge target frames of the vehicle door, calculating an intersection point with the left edge of the platform, making a vertical line which passes through the intersection point and is vertical to the edge of the platform, and intersecting the right edge of the platform at two points. Fitting a horizontal projection profile of the vehicle door area on the platform by using the four intersection points; s95: setting the vehicle door area and the projection area as detection areas, detecting the target of a person in the detection areas, and determining that the detected target of the person is an illegal behavior; s96: the result of the detection is fed back to the image information management unit.
Specifically, as can be seen in fig. 11, the system receives a service request sent by a user, the intelligent identification management unit invokes an nonstandard door opening and closing identification algorithm, performs target positioning on a single door and a cavity between doors in an image through the algorithm, determines a single-side door which is being opened or closed, detects whether a person exists in a door range, determines that the door is opened or closed in a nonstandard manner if a person target is detected, and outputs a detection result if the door is normally operated.
(10) And (3) identifying the overspeed of the forklift: the forklift overspeed identification algorithm specifically comprises the following steps: s101: analyzing the images acquired by the data terminal, tracking the forklift by adopting a depersort algorithm, and recording the positions of the forklift with the same ID in the front frame image and the rear frame image; s102: projecting a speed measuring area in the image onto a plane of a real scene to obtain a perspective transformation matrix W; s103, projecting the position of the forklift onto a real scene plane by using a perspective transformation matrix W; s104: calculating the Euclidean distance s of a forklift with the same ID on a real scene plane in the front frame image and the rear frame image; s105: combining the time difference t of the two frames of images before and after the image is combined with the formula v = s/t, and then calculating the speed v (km/h); and S106, feeding back the velocity value result v (km/h) to the image information management unit.
Specifically, as can be seen from fig. 12, the system receives a service request sent by a user, the intelligent identification management unit calls a forklift overspeed identification algorithm, the forklift in the image is tracked through the algorithm, the speed measurement area is projected onto a plane of a real scene, the euclidean distance between the front frame image and the rear frame image and the forklift is calculated, the speed of the forklift is calculated according to the time difference between the front frame image and the rear frame image, and if the speed exceeds a limited value, the detection result is output.
In the invention, in the item point recognition sub-algorithm, one or more of a YOLOv4 algorithm, a Gaussian mixture model algorithm, a DeepsORT target tracking algorithm, a Mask-RCNN algorithm, an image morphology algorithm, a Gaussian filter algorithm and a perspective transformation algorithm are adopted for processing the image.
The invention discloses an intelligent early warning system for a railway freight yard sky eye, which is a system capable of intelligently identifying and further standardizing loading and unloading operations in real time, realizes unified management of multiple algorithms, is matched with camera data acquisition in the freight yard, adopts a modular overall intelligent analysis unit, and can dynamically supervise and early warn and analyze operations in real time in a freight yard warehouse, a warehouse automobile side, a warehouse train side, a freight yard red board monitoring position and the like, so that intelligent inspection and supervision of freight yard operations are realized; the method can be used for analyzing according to a specific recognizer algorithm as required, and has high analysis efficiency and high accuracy of an analysis result.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art should be considered to be within the technical scope of the present invention, and the technical solutions and the inventive concepts thereof according to the present invention should be equivalent or changed within the scope of the present invention.
Claims (10)
1. The utility model provides a railway goods yard sky eye intelligence early warning system which characterized in that, railway goods yard sky eye intelligence early warning system, include:
the image information management unit is connected with the data terminal and used for processing and classifying the acquired image information, the image information management unit is also used for connecting a client and outputting data, the image information management unit transmits the image information to the intelligent recognition management unit and calls an item point recognition sub-algorithm of the intelligent recognition management unit to perform intelligent analysis processing on the image information, and the image information management unit is also used for connecting a server and storing the data;
the intelligent recognition management unit is internally provided with an intelligent item point recognition module, the intelligent item point recognition module is used for managing a self-learning sample training model of each item point and executing a sub-algorithm for recognizing each item point, and the sub-algorithm for recognizing the item point comprises the following steps: the system comprises a personnel inspection identification algorithm, a non-through tooling identification algorithm, a cargo space occupation identification algorithm, a key cargo space identification algorithm, a loading and unloading vehicle identification algorithm, a red board identification algorithm, a platform boundary crossing identification algorithm, a smoking identification algorithm, a non-standard door opening and closing identification algorithm and a forklift overspeed algorithm, wherein at least one algorithm is used in the item point identification process.
2. The intelligent railway goods yard sky eye early warning system of claim 1, characterized in that: the personnel patrol identification algorithm specifically comprises the following steps:
s11: analyzing an image acquired by a data terminal, and performing target detection on personnel in the image by adopting a yolov4 network to obtain an external rectangular frame, a top left vertex coordinate and a width and a height of the personnel target;
s12: tracking a personnel target to obtain a personnel target id;
s13: extracting a personnel position image according to a personnel positioning result frame, carrying out target detection on dressing types (tooling, id =0, uniform, id = 1), filtering out results of non-uniform (id! = 1), and taking the result with the highest confidence level in the results larger than a confidence level threshold (confidence > 0.8) as an identification result of the current frame; if the uniform target is identified, setting the target result to be 1, otherwise, setting the target result to be 0;
s14: storing uniform detection results with the same target id in the same result queue; the latest detection result with fixed frame number can be stored in the result queue; accumulating the result values and judging whether the result values exceed a threshold value;
s15: if the threshold value is exceeded, returning an alarm value to the image information management unit;
the identification algorithm of the tool which is not worn specifically comprises the following steps:
s21: analyzing an image acquired by a data terminal, and performing target detection on personnel in the image by adopting a yolov4 network to obtain an external rectangular frame, a top left vertex coordinate and a width and a height of the personnel target;
s22: tracking a personnel target to obtain a personnel target id;
s23: extracting a person position image according to the person positioning result frame, carrying out target detection on dressing types (tooling, id =0, uniform, id = 1), and filtering a result lower than a confidence threshold (confidence < 0.8); if no uniform or tool is identified, setting the target result to be 1, otherwise, setting the target result to be 0;
s24: storing uniform detection results with the same target id in the same result queue; the latest detection result with fixed frame number is stored in the result queue; accumulating the result values and judging whether the result values exceed a threshold value;
s25: if the threshold value is exceeded, an alarm value is returned to the image information management unit.
3. The intelligent railway goods yard sky eye early warning system of claim 1, characterized in that: the goods space occupation identification algorithm specifically comprises the following steps:
s31: analyzing an image acquired by a data terminal, and performing masking operation on the image and a preset detection area to generate a detection image which is totally black except the detection area;
s32: carrying out example segmentation on the goods on the detection graph by using a Mask-Rcnn network, and judging whether the percentage of the goods in the goods space exceeds a threshold value or not according to calculation;
s33: carrying out perspective transformation on the example segmentation image of the detection area part according to the four coordinate points of the detection area and the actual length and width of the goods space;
s34: calculating the percentage of the cargo example in the perspective transformation image in the cargo space area;
s35: the result of the detection is fed back to the image information management unit.
4. The intelligent railway goods yard sky eye early warning system of claim 1, characterized in that: the key cargo space identification algorithm specifically comprises the following steps:
s41: analyzing an image acquired by a data terminal, and performing masking operation on the image and a preset detection area to generate a detection image which is totally black except the detection area;
s42: detecting the targets of the personnel and the forklift in the image, returning a detection result if the personnel or the forklift targets appear, and executing alarm;
s43: if the area to be detected does not have personnel and a target of a forklift, segmenting the foreground and the background of the image by using a Gaussian mixture model;
s44: performing morphological opening and closing operations on the foreground image, removing isolated noise points, and extracting a foreground contour;
s45: filtering the foreground contour according to the contour area, and deleting the contours which are lower than the area low threshold and higher than the area high threshold;
s46: the remaining contour is regarded as the position of the change in the detection area, and the detection result is fed back to the image information management unit.
5. The intelligent railway goods yard sky eye early warning system of claim 1, characterized in that: the loading and unloading vehicle identification algorithm specifically comprises the following steps:
s51: analyzing an image acquired by a data terminal, and carrying out target positioning on a train, a single vehicle door, a cavity between vehicle doors, a vehicle number and the like in the image by using a yolov4 network;
s52: judging whether the detected target result is in a preset detection area or not according to the occupation ratio of the target in the detection area;
s53: extracting the image of the vehicle number part, performing digital identification, and filtering the upper left position of the image and an overlapped identification result;
s54: judging whether the number of the reserved car numbers is 7, if the number meets the condition, sequencing the numbers from left to right, combining the numbers into a car number character string in sequence and returning the car number character string to the upper layer;
s55: integrating the single car door and the car door cavity target in the detection area, extracting a car door position image, and judging the opening and closing state of the car door;
s56: if the combined vehicle door contains a vehicle door cavity, the vehicle door is in an open state, otherwise, the vehicle door is in a closed state;
s57: if the car door is in an open state, carrying out target detection on the forklift and the goods;
s58: if the vehicle door is in a closed state, carrying out target detection of sealing lock on the vehicle door;
s59: the result of the detection is fed back to the image information management unit.
6. The intelligent railway goods yard sky eye early warning system of claim 1, characterized in that: the red card identification algorithm specifically comprises the following steps:
s61: extracting a detection area image according to the coordinates of the plurality of red card detection areas;
s62: inputting the target to a yolov4 network model for target detection of the red card;
s63: and feeding back the result of the presence or absence of the red cards in the detection areas to the image information management unit.
7. The intelligent railway goods yard sky eye early warning system of claim 1, characterized in that: the platform border crossing identification algorithm specifically comprises the following steps:
s71: carrying out target detection on the personnel in the image by using a yolov4 network;
s72: judging whether an area of a person in a platform yellow line exists according to whether the percentage of the part of the bottom edge of the person target frame in the detection area to the whole bottom edge exceeds a threshold value;
s73: and feeding back the result of the state value of the boundary crossing of the existence or nonexistence of the person to the image information management unit.
8. The railway freight yard sky eye intelligent early warning system of claim 1, characterized in that: the smoking identification algorithm specifically comprises the following steps:
s81: analyzing an image acquired by a data terminal, and performing target detection on personnel in the image by adopting a yolov4 network to obtain a circumscribed rectangular frame, a top left vertex coordinate and a width and a height of a personnel target;
s82: extracting a human face position image from the human target image, and carrying out target detection on the cigarettes;
s83: filtering the recognition result of the cigarette with low confidence;
s84: the detection result is fed back to the image information management unit.
9. The intelligent railway goods yard sky eye early warning system of claim 1, characterized in that: the nonstandard door opening and closing identification algorithm specifically comprises the following steps:
s91: positioning the cavities between the single doors and the doors of the train by adopting a yolov4 network, and combining the single doors and the cavities belonging to the same carriage together according to the position information to generate a circumscribed rectangular frame of the complete door;
s92: extracting a car door image, and identifying the door opening and closing action, a door pulling rope and the vertical edge of a single car door which are currently carried out by the car door;
s93: determining whether door opening or closing behaviors and the door opening or closing direction exist according to the judged door opening and closing actions and the positions of the door pulling ropes;
s94: determining a single-side vehicle door which is opening (closing) the door, taking a fitting edge straight line of upper and lower middle points of a left edge target frame and a right edge target frame of the vehicle door, calculating an intersection point with a left side edge of a platform, making a vertical line which passes through the intersection point and is vertical to the edge of the platform, and intersecting the right side edge of the platform at two points; fitting a horizontal projection profile of the vehicle door area on the platform by using the four intersection points;
s95: setting the vehicle door area and the projection area as detection areas, detecting the target of a person in the detection areas, and determining that the detected target of the person is an illegal behavior;
s96: the result of the detection is fed back to the image information management unit.
10. The intelligent railway goods yard sky eye early warning system of claim 1, characterized in that: the forklift overspeed identification algorithm specifically comprises the following steps:
s101: analyzing an image acquired by a data terminal, tracking the forklift by adopting a deepsort algorithm, and recording the position of the forklift with the same ID in the front frame image and the rear frame image;
s102: projecting a speed measurement area in the image onto a plane of a real scene to obtain a perspective transformation matrix W;
s103, projecting the position of the forklift onto a real scene plane by using a perspective transformation matrix W;
s104: calculating the Euclidean distance s of the forklift with the same ID on the real scene plane in the front frame image and the rear frame image;
s105: combining the time difference t of the two frames of images before and after the image is combined with the formula v = s/t, and then calculating the speed v (km/h);
and S106, feeding back the velocity value result v (km/h) to the image information management unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210845459.6A CN115359416A (en) | 2022-07-19 | 2022-07-19 | Intelligent early warning system for railway freight yard sky eye |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210845459.6A CN115359416A (en) | 2022-07-19 | 2022-07-19 | Intelligent early warning system for railway freight yard sky eye |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115359416A true CN115359416A (en) | 2022-11-18 |
Family
ID=84032560
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210845459.6A Pending CN115359416A (en) | 2022-07-19 | 2022-07-19 | Intelligent early warning system for railway freight yard sky eye |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115359416A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116101679A (en) * | 2023-02-01 | 2023-05-12 | 锦同缘建设有限公司 | Intelligent warehouse monitoring method and device, electronic equipment and readable storage medium |
-
2022
- 2022-07-19 CN CN202210845459.6A patent/CN115359416A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116101679A (en) * | 2023-02-01 | 2023-05-12 | 锦同缘建设有限公司 | Intelligent warehouse monitoring method and device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110826538B (en) | Abnormal off-duty identification system for electric power business hall | |
| CN110689054B (en) | Worker violation monitoring method | |
| KR102122859B1 (en) | Method for tracking multi target in traffic image-monitoring-system | |
| CN111629181B (en) | Fire-fighting life passage monitoring system and method | |
| CN110619277A (en) | Multi-community intelligent deployment and control method and system | |
| CN107911663A (en) | A kind of elevator passenger hazardous act intelligent recognition early warning system based on Computer Vision Detection | |
| CN111783744A (en) | Operation site safety protection detection method and device | |
| US20070292029A1 (en) | Cascade plate recognition system | |
| CN104966304A (en) | Kalman filtering and nonparametric background model-based multi-target detection tracking method | |
| CN101727672A (en) | Method for detecting, tracking and identifying object abandoning/stealing event | |
| KR102122850B1 (en) | Solution for analysis road and recognition vehicle license plate employing deep-learning | |
| CN113411542A (en) | Intelligent working condition monitoring equipment | |
| CN112686090A (en) | Intelligent monitoring system for abnormal behaviors in bus | |
| CN111353338B (en) | Energy efficiency improvement method based on business hall video monitoring | |
| CN108583592A (en) | A kind of subway service on buses or trains job information acquisition intelligent detecting method | |
| CN112633057A (en) | Intelligent monitoring method for abnormal behaviors in bus | |
| CN112184773A (en) | Helmet wearing detection method and system based on deep learning | |
| CN210666820U (en) | Pedestrian abnormal behavior detection system based on DSP edge calculation | |
| CN110781735A (en) | Alarm method and system for identifying on-duty state of personnel | |
| CN115359416A (en) | Intelligent early warning system for railway freight yard sky eye | |
| CN113191273A (en) | Oil field well site video target detection and identification method and system based on neural network | |
| CN115660297A (en) | Automatic AI early warning system and method for construction site safety | |
| CN116416281A (en) | Grain depot AI video supervision and analysis method and system | |
| CN111695545A (en) | Single-lane reverse driving detection method based on multi-target tracking | |
| CN111062238A (en) | Escalator flow monitoring method and system based on human skeleton information and multi-target tracking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |