CN115994953A - Power field security monitoring and tracking method and system - Google Patents
Power field security monitoring and tracking method and system Download PDFInfo
- Publication number
- CN115994953A CN115994953A CN202310150307.9A CN202310150307A CN115994953A CN 115994953 A CN115994953 A CN 115994953A CN 202310150307 A CN202310150307 A CN 202310150307A CN 115994953 A CN115994953 A CN 115994953A
- Authority
- CN
- China
- Prior art keywords
- camera
- target
- tracking
- video
- security monitoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Closed-Circuit Television Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a power field safety supervision tracking method, which relates to the technical field of relay protection operation and maintenance and is used for solving the problems of inaccurate existing manual supervision and lack of positioning data, and the method comprises the following steps: calculating internal parameter and external parameter of multiple view cameras; performing personnel positioning according to the internal parameter and the external parameter; receiving a video sent by the camera, extracting skeleton characteristics according to the video, and obtaining a behavior recognition result according to the skeleton characteristics; identifying a preset target in the camera and tracking a track to obtain a target detection result; and outputting the behavior identification result, the target detection result and the operator positioning result as security monitoring information. The invention also discloses a power field security monitoring and tracking system. According to the invention, the multi-view camera is used for positioning and behavior recognition, so that three-dimensional positioning and tracking of personnel and the like on an electric power operation site are realized.
Description
Technical Field
The invention relates to the technical field of relay protection operation and maintenance, in particular to a power field security monitoring and tracking method and system in a power distribution operation environment.
Background
In the construction, communication, power and other engineering industries, an operator needs to frequently perform outdoor operations. Because the outdoor environment is complex, factors such as high pressure, high altitude, pit and the like can bring potential safety hazards to operators; once a security incident occurs, huge personnel and property losses are caused. Therefore, it is necessary to identify and track the on-site behaviors of the operator during the operation process, so as to ensure that the operator uses a compliance operation flow, and identify, early warn and locate dangerous behaviors.
The current supervision mode is usually finished by manual checking, and the monitoring video also depends on manual watching, so that the real-time identification of the behaviors of operators and the early warning of dangerous behaviors cannot be realized. In order to ensure real-time identification and tracking of the behaviors of operators in an outdoor operation scene, research based on human body image identification is currently available so as to conveniently obtain the behavior identification result of the current operator. However, due to the influence of factors such as shielding, detection distance, detection angle, etc., there is a problem that a human body image is easily lost or the image size is easily reduced during recognition.
Therefore, in order to ensure the safety of personnel and equipment in an operation scene, the technical problems that the existing behavior identification is inaccurate and the three-dimensional coordinates of the operators cannot be obtained are solved, and a method for monitoring and tracking the electric power field with the operation requirement guidance is needed to be constructed.
Disclosure of Invention
In order to overcome the defects of the prior art, one of the purposes of the invention is to provide a power field safety monitoring tracking method which realizes the power field safety monitoring tracking by carrying out three-dimensional positioning tracking on operators.
One of the purposes of the invention is realized by adopting the following technical scheme:
a power field security monitoring and tracking method comprises the following steps:
calculating internal parameter and external parameter of multiple view cameras; performing personnel positioning according to the internal parameter and the external parameter;
receiving a video sent by the camera, extracting skeleton characteristics according to the video, and obtaining a behavior recognition result according to the skeleton characteristics;
identifying a preset target in the camera and tracking a track to obtain a target detection result;
and outputting the behavior recognition result, the target detection result and the operator positioning result as security monitoring information.
Further, calculating internal parameters of the plurality of view cameras includes:
receiving a video which is recorded by the camera and contains a checkerboard;
and inputting the video into a camera imaging model to obtain internal parameters and distortion parameters.
Further, calculating parameters of the external parameters of the multiple view cameras includes:
receiving an image sequence sent by the camera;
outputting the checkerboard coordinate pose of the camera through a pose algorithm according to the image sequence;
and selecting one camera as a world coordinate system, and carrying out three-dimensional coordinate transformation on the checkerboard coordinate poses of other cameras to obtain world coordinate system poses serving as the external parameter.
Further, the extracting of the skeleton feature includes:
preprocessing the video;
inputting the preprocessed video into an openpost human body posture estimation network to obtain a skeleton feature map;
inputting the skeleton feature map into an ST-GCN map convolution neural network to obtain 256-dimensional features.
Further, according to the skeleton feature matching, a behavior recognition result is obtained, and the method comprises the following steps:
the 256-dimensional characteristics of the multiple view cameras are connected in series, and downsampling is carried out for 2 times;
inputting the 256-dimensional characteristics subjected to downsampling into a SoftMax classifier;
and outputting the behavior category.
Further, the behavior categories include standing, walking, running, climbing, and climbing.
Further, a preset target in the camera is identified and track tracking is performed to obtain a target detection result, including the following steps:
preprocessing video in the camera;
inputting the preprocessed video into a detection neural network model VSSA-Net to obtain a target class;
and inputting the target category into a target tracking model ByteTrack to track the motion trail, and obtaining a target motion trail detection result.
Further, the target detection result further includes target positioning, and the detection of the target positioning includes:
and calculating world coordinate system coordinates of the same target through a multi-view geometric algorithm according to pixel coordinates of the same target in a plurality of cameras.
Further, the target category includes: personnel's head, safety helmet, birds, small-size inspection robot.
The second objective of the present invention is to provide a power on-site security monitoring tracking system, which identifies operators and other targets through cameras with multiple angles of view, so as to realize on-site security monitoring.
The second purpose of the invention is realized by adopting the following technical scheme:
the electric power field safety monitoring and tracking system is characterized by comprising a multi-camera internal and external parameter calibration unit, a behavior recognition unit based on skeleton characteristics and a target tracking unit; the multi-camera internal and external parameter calibration unit is used for recording the relative pose transformation among the multiple cameras and calculating internal parameter and external parameter; the behavior recognition unit is used for recognizing the behaviors of operators in the camera video; the target tracking unit is used for positioning, tracking and identifying a preset target.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, the operation site video is recorded through the multi-view camera, so that the accurate positioning of operators can be effectively obtained, the interference of shielding, distance and other problems on the camera is avoided, the three-dimensional coordinate positioning is realized through calculating internal and external parameters, and the working condition of the operators is intuitively displayed through feature recognition; in addition, the invention also identifies targets other than operators, and can show the condition of the operation site in detail; the safety monitoring information provided by the invention is detailed and accurate, and can ensure the safe and stable operation on site.
Drawings
FIG. 1 is a flow chart of a power field security monitoring and tracking method according to an embodiment;
FIG. 2 is a flow chart of an embodiment behavior recognition process.
Detailed Description
The invention will now be described in more detail with reference to the accompanying drawings, to which it should be noted that the description is given below by way of illustration only and not by way of limitation. Various embodiments may be combined with one another to form further embodiments not shown in the following description.
Example 1
An embodiment provides a power field security monitoring tracking method, which aims to realize tracking of staff and other things on an operation field by three-dimensional positioning through a multi-view camera.
It should be noted that, the method described in this embodiment is based on a plurality of view angle cameras, and the cameras may be a plurality of cameras at different view angles of the operation site, or may be a camera capable of realizing multiple view angles, so as to provide image information of multiple view angles, and realize coordinate positioning of personnel in the operation site by calculating internal and external parameters.
Referring to fig. 1, a power field security monitoring and tracking method includes the following steps:
s1, calculating internal parameter and external parameter of a plurality of view cameras; performing personnel positioning according to the internal parameter and the external parameter;
the internal parameters refer to calculating the focal length and distortion parameters of each camera, each camera needs to record a video containing a checkerboard, all corner points of the checkerboard are ensured to be visible to the camera, the corner points of the checkerboard are detected, and the internal matrix of each camera and the distortion parameters of a camera lens are calculated by combining a camera imaging model with a camera image distortion model. Specifically, calculating internal parameters of a plurality of view cameras includes:
receiving a video which is recorded by the camera and contains a checkerboard;
and inputting the video into a camera imaging model to obtain internal parameters and distortion parameters.
The imaging model of the camera is a pinhole camera model in this embodiment, and the corresponding internal reference matrix is as follows:
wherein f x ，c x ，f y ，c y Is a corresponding internal reference;
the distortion model generally refers to radial and tangential distortions due to the camera lens, as follows:
x distorted ＝x(1+k 1 r 2 +k 2 r 4 +k 3 r 6 )+2p 1 xy+p 2 (r 2 +2x 2 )
y distorted ＝y(1+k 1 r 2 +k 2 r 4 +k 3 r 6 )+2p 2 xy+p 1 (r 2 +2y 2 )
where (x, y) is the normalized coordinates before distortion, (x) distorted ,y distorted ) Is the normalized coordinate after distortion, k 1 ,k 2 ,k 3 Is a radial distortion parameter, p 1 ,p 2 Is a tangential distortion parameter;
by calibrating each camera in a checkerboard manner, video recording is input, and internal parameters and distortion parameters are output
S1, calculating external parameters of a plurality of view cameras, wherein the external parameters comprise:
receiving an image sequence sent by the camera;
outputting the checkerboard coordinate pose of the camera through a pose algorithm according to the image sequence;
and selecting one camera as a world coordinate system, and carrying out three-dimensional coordinate transformation on the checkerboard coordinate poses of other cameras to obtain world coordinate system poses serving as the external parameter.
The easy MoCap algorithm is adopted for calculating the external parameters of the embodiment, specifically, the ground control points corresponding to multiple cameras are erected by utilizing a checkerboard, and the control points are ensured to be visible to all cameras at the same time; and then inputting the recorded image sequences of the cameras, and outputting the pose of each camera under the established checkerboard coordinate system. Defining a checkerboard coordinate system as O, and defining each camera coordinate system as C i I=1, 2,..n, the pose of the camera in the checkerboard coordinate system is obtained through calibration
Then with camera C 1 The coordinate system is a world coordinate system, and the pose of other cameras under the world coordinate system is obtained through three-dimensional coordinate transformation>
I.e. external parameters. The transformation process satisfies
S21, receiving a video sent by the camera, extracting skeleton characteristics according to the video, and obtaining a behavior recognition result according to the skeleton characteristics;
the skeleton characteristic behavior recognition is used for detecting and recording time sequence behaviors of power field operators.
Specifically, referring to fig. 2, the extracting of the skeleton feature and the matching of the behavior recognition result include:
s211, preprocessing the video;
the preprocessing is used for preprocessing the picture streams of the multiple cameras, specifically, aligning the video streams from the multiple cameras according to time stamps, respectively intercepting 10 frames of pictures in the same time window, carrying out normalization processing, and uniformly clipping to 256×256 pixels.
S212, inputting the preprocessed video into an openpost human body posture estimation network to obtain a skeleton feature map;
s213, inputting the skeleton feature map into the ST-GCN map convolution neural network to obtain 256-dimensional features.
S214, connecting 256-dimensional features of a plurality of view cameras in series, and performing downsampling for 2 times;
taking 4 cameras as an example, the serial connection is carried out around the scene, 256-dimensional features from 4 visual angles are respectively connected in series, 256-dimensional features are obtained again after 2 times of downsampling, and the 256-dimensional features are input into a SoftMax classifier.
S215, inputting the 256-dimensional features subjected to downsampling into a SoftMax classifier;
s216, outputting the behavior category.
In this embodiment, behavior categories include standing, walking, running, climbing, and walking over. Of course, the behavior category may be increased or decreased according to the actual requirement.
S22, identifying a preset target in the camera and tracking a track to obtain a target detection result;
the above-mentioned preset targets include, but are not limited to: personnel's head, safety helmet, birds, small-size inspection robot.
The target tracking is used for detecting and tracking a specific small target of the power field so as to provide a basis for safety judgment for the work field.
In this embodiment, a Tracking-by-detection strategy is used as an overall strategy for implementing target Tracking. The strategy is to find out a target object in a single frame image through target detection and then associate the same detection target among different frames so as to realize tracking, so that the whole module comprises a small target detection network model VSSA-Net and a target tracking model ByteTrack, and the small target tracking function is realized in a combined way.
Specifically, a preset target in the camera is identified and track tracking is performed to obtain a target detection result, which comprises the following steps:
preprocessing video in the camera; preprocessing mainly comprises normalizing and scaling picture data in pictures, video sequences to 640 x 640 resolution.
Inputting the preprocessed video into a detection neural network model VSSA-Net, and detecting a specific preset target existing in a current picture to obtain a target class; in addition, matching of the detection categories output from the detection model is also required for tracked objects that simultaneously appear in a plurality of cameras.
And inputting the target category into a target tracking model ByteTrack to track a motion track, tracking the motion track of the target in a corresponding camera view, and outputting a historical motion track of the tracked target to obtain a target motion track detection result.
The VSSA-Net model is selected because the VSSA-Net model is an efficient single-shot detector and has the characteristic of multi-scale fusion, and a multi-scale characteristic diagram is obtained by using a deconvolution layer and a jump connection which are densely connected, so that compared with a common target detector, the detection performance of a small target, namely the preset target is improved. For the ByteTrack model for target tracking, the model tracks by associating almost all detection boxes instead of high score detection boxes, and determines the real target object and filters the background by using the similarity of the detection boxes to the track.
In order to further increase the accuracy of the target detection, the target detection result further comprises target positioning, and the detection of the target positioning comprises:
and calculating world coordinate system coordinates of the same target through a multi-view geometric algorithm according to pixel coordinates of the same target in a plurality of cameras.
Specifically, the pixel coordinates of the same target under different cameras are utilized, and the coordinates [ x, y, z ] of the target in the world coordinate system are calculated by using a multi-view geometric method in combination with camera calibration information] T Denoted as P w . In particular, it is desirable to utilize the detection results of objects that exist stably and are not deformed, such as helmets. The detection frame is first centered at point M and considered to be the same point in space. The pixel coordinates of the M points in different camera images are marked as pixel coordinates (u i ,v i ) I represents a camera number. Coordinate P w Conversion from camera coordinate system to respective camera coordinate system P i ＝[x i ,y i ,z i ] T The following are provided:
wherein the method comprises the steps of
Is the camera external parameter calculated before, in addition, the conversion of the homogeneous coordinates and the non-homogeneous coordinates is involved in the formula, and the conversion is needed to be carried out at P w And P i Adding 1 at the tail of the vector;
the projection model by the camera should:
wherein K is i Is a camera internal reference calculated before;
simultaneous camera equationsResolvable P w ；
And S3, outputting the behavior identification result, the target detection result and the operator positioning result as security monitoring information.
And through the behavior recognition result, the target detection result and the operator positioning result which are output in the S3, the three-dimensional positioning and tracking of the operator in the operation scene can be realized, a safety warning basis is provided for a supervisory or warning system and the like, and the safety problem of the operation scene is effectively avoided.
Example two
A second embodiment is an explanation and explanation of an electric power field safety monitoring tracking system.
The system comprises a multi-camera internal and external parameter calibration unit, a behavior recognition unit based on skeleton characteristics and a target tracking unit; the multi-camera internal and external parameter calibration unit is used for recording the relative pose transformation among the multiple cameras and calculating internal parameter and external parameter; the behavior recognition unit is used for recognizing the behaviors of operators in the camera video; the target tracking unit is used for positioning, tracking and identifying a preset target.
And the camera internal parameter calibration module calibrates the focal length and distortion parameters of each camera. Firstly, each camera records a video containing a checkerboard, and ensures that all corner points of the checkerboard are visible to the camera; and then, calculating an internal reference matrix of each camera by combining the camera imaging model with the camera distortion model. The camera extrinsic calibration unit is capable of calculating relative pose transformations between multiple cameras.
The behavior recognition unit based on the skeleton characteristics can also comprise a preprocessing module, a characteristic extraction module and a fusion output module which are connected in series; the operation logic and the processing method of each module refer to the related description in the first embodiment.
The target tracking unit may include an image preprocessing sub-module, a small target detection sub-module, a target tracking sub-module and a multi-view positioning sub-module that are connected to each other, and the specific operation logic and processing method of the unit are described in the first embodiment.
It will be apparent to those skilled in the art from this disclosure that various other changes and modifications can be made which are within the scope of the invention as defined in the appended claims.
Claims (10)
1. The electric power site safety monitoring and tracking method is characterized by comprising the following steps of:
calculating internal parameter and external parameter of multiple view cameras; performing personnel positioning according to the internal parameter and the external parameter;
receiving a video sent by the camera, extracting skeleton characteristics according to the video, and obtaining a behavior recognition result according to the skeleton characteristics;
identifying a preset target in the camera and tracking a track to obtain a target detection result;
and outputting the behavior identification result, the target detection result and the operator positioning result as security monitoring information.
2. The power field security monitoring and tracking method of claim 1, wherein calculating internal parameters of a plurality of view cameras comprises:
receiving a video which is recorded by the camera and contains a checkerboard;
and inputting the video into a camera imaging model to obtain internal parameters and distortion parameters.
3. The power field security monitoring and tracking method of claim 1 or 2, wherein calculating the extrinsic parameters of the plurality of view cameras comprises:
receiving an image sequence sent by the camera;
outputting the checkerboard coordinate pose of the camera through a pose algorithm according to the image sequence;
and selecting one camera as a world coordinate system, and carrying out three-dimensional coordinate transformation on the checkerboard coordinate poses of other cameras to obtain world coordinate system poses serving as the external parameter.
4. The power site security monitoring and tracking method of claim 1, wherein the extraction of the skeleton feature comprises:
preprocessing the video;
inputting the preprocessed video into an openpost human body posture estimation network to obtain a skeleton feature map;
inputting the skeleton feature map into an ST-GCN map convolution neural network to obtain 256-dimensional features.
5. The power field security monitoring and tracking method according to claim 4, wherein the behavior recognition result is obtained according to the skeleton feature matching, comprising the steps of:
the 256-dimensional characteristics of the multiple view cameras are connected in series, and downsampling is carried out for 2 times;
inputting the 256-dimensional characteristics subjected to downsampling into a SoftMax classifier;
and outputting the behavior category.
6. The power site security monitoring and tracking method of claim 5, wherein the behavior categories include standing, walking, running, climbing, and climbing.
7. The power field security monitoring and tracking method as set forth in claim 1, wherein the steps of identifying a preset target in the camera and tracking a trajectory to obtain a target detection result, include:
preprocessing video in the camera;
inputting the preprocessed video into a detection neural network model VSSA-Net to obtain a target class;
and inputting the target category into a target tracking model ByteTrack to track the motion trail, and obtaining a target motion trail detection result.
8. The power field security monitoring and tracking method according to claim 1 or 7, wherein the target detection result further includes target positioning, and the detection of the target positioning includes:
and calculating world coordinate system coordinates of the same target through a multi-view geometric algorithm according to pixel coordinates of the same target in a plurality of cameras.
9. The power field security monitoring and tracking method of claim 7, wherein the target class comprises: personnel's head, safety helmet, birds, small-size inspection robot.
10. The electric power field safety monitoring and tracking system is characterized by comprising a multi-camera internal and external parameter calibration unit, a behavior recognition unit based on skeleton characteristics and a target tracking unit; the multi-camera internal and external parameter calibration unit is used for recording the relative pose transformation among the multiple cameras and calculating internal parameter and external parameter; the behavior recognition unit is used for recognizing the behaviors of operators in the camera video; the target tracking unit is used for positioning, tracking and identifying a preset target.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310150307.9A CN115994953A (en) | 2023-02-15 | 2023-02-15 | Power field security monitoring and tracking method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310150307.9A CN115994953A (en) | 2023-02-15 | 2023-02-15 | Power field security monitoring and tracking method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115994953A true CN115994953A (en) | 2023-04-21 |
Family
ID=85992076
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310150307.9A Pending CN115994953A (en) | 2023-02-15 | 2023-02-15 | Power field security monitoring and tracking method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115994953A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117437595A (en) * | 2023-11-27 | 2024-01-23 | 哈尔滨航天恒星数据***科技有限公司 | Fishing boat boundary crossing early warning method based on deep learning |
-
2023
- 2023-02-15 CN CN202310150307.9A patent/CN115994953A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117437595A (en) * | 2023-11-27 | 2024-01-23 | 哈尔滨航天恒星数据***科技有限公司 | Fishing boat boundary crossing early warning method based on deep learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108564065B (en) | Cable tunnel open fire identification method based on SSD | |
| CN104754302B (en) | A kind of target detection tracking method based on rifle ball linked system | |
| CN111079600A (en) | Pedestrian identification method and system with multiple cameras | |
| CN106384106A (en) | Anti-fraud face recognition system based on 3D scanning | |
| CN111860352B (en) | Multi-lens vehicle track full tracking system and method | |
| CN109218667B (en) | Public place safety early warning system and method | |
| KR101645959B1 (en) | The Apparatus and Method for Tracking Objects Based on Multiple Overhead Cameras and a Site Map | |
| CN114905512B (en) | Panoramic tracking and obstacle avoidance method and system for intelligent inspection robot | |
| CN113486697B (en) | Forest smoke and fire monitoring method based on space-based multimode image fusion | |
| CN114973028B (en) | Aerial video image real-time change detection method and system | |
| CN108074265A (en) | A kind of tennis alignment system, the method and device of view-based access control model identification | |
| CN113688817A (en) | Instrument identification method and system for automatic inspection | |
| CN115994953A (en) | Power field security monitoring and tracking method and system | |
| CN113256731A (en) | Target detection method and device based on monocular vision | |
| CN113763484A (en) | Ship target positioning and speed estimation method based on video image analysis technology | |
| CN114511592B (en) | Personnel track tracking method and system based on RGBD camera and BIM system | |
| CN102291568A (en) | Accelerated processing method of large-view-field intelligent video monitoring system | |
| Dai et al. | Geometry-based object association and consistent labeling in multi-camera surveillance | |
| CN112488022B (en) | Method, device and system for monitoring panoramic view | |
| CN111860378A (en) | Market fire-fighting equipment inspection method based on gun-ball linkage and video event perception | |
| CN111899289B (en) | Infrared image and visible light image registration method based on image characteristic information | |
| CN116152945A (en) | Under-mine inspection system and method based on AR technology | |
| CN113743380B (en) | Active tracking method based on video image dynamic monitoring | |
| CN112766033B (en) | Method for estimating common attention targets of downlinks in scene based on multi-view camera | |
| CN112860946B (en) | Method and system for converting video image information into geographic information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |