WO2019127274A1 - Alarm method and device for criminal activity, storage medium and server - Google Patents

Alarm method and device for criminal activity, storage medium and server Download PDF

Info

Publication number
WO2019127274A1
WO2019127274A1 PCT/CN2017/119570 CN2017119570W WO2019127274A1 WO 2019127274 A1 WO2019127274 A1 WO 2019127274A1 CN 2017119570 W CN2017119570 W CN 2017119570W WO 2019127274 A1 WO2019127274 A1 WO 2019127274A1
Authority
WO
WIPO (PCT)
Prior art keywords
training
criminal activity
feature
module
test
Prior art date
Application number
PCT/CN2017/119570
Other languages
French (fr)
Chinese (zh)
Inventor
李恒
刘光军
Original Assignee
深圳市锐明技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市锐明技术股份有限公司 filed Critical 深圳市锐明技术股份有限公司
Priority to CN201780002311.3A priority Critical patent/CN108351968B/en
Priority to PCT/CN2017/119570 priority patent/WO2019127274A1/en
Publication of WO2019127274A1 publication Critical patent/WO2019127274A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G08B21/0297—Robbery alarms, e.g. hold-up alarms, bag snatching alarms
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
    • G08B25/01—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems characterised by the transmission medium
    • G08B25/012—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems characterised by the transmission medium using recorded signals, e.g. speech
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search

Definitions

  • the present invention relates to the field of video information processing technologies, and in particular, to an alarm method, apparatus, storage medium, and server for criminal activities.
  • the solution to the driver's personal and property safety alarms in the market is divided into two categories: 1. Based on the manual quick slamming of the emergency button, the remote control button sends a real-time alarm control signal for voice alarm; 2. Based on the safety seat equipped with conductive liquid emission The gun is subjected to electric shocks and physical acts of tearing gas tears are used to prevent anecdotes (such as robbery).
  • anecdotes such as robbery.
  • the hardware cost is high. There is a big cost increase in the seat equipped with conductive liquid launching gun and tear gas, which is not conducive to promotion. Secondly, the perpetrators generally hold the weapon top. Living with the driver and being closer to the driver, this type of physical attack is very easy to damage the robber and at the same time it is very risky and uncertain for the driver himself.
  • the embodiment of the invention provides an alarm method, device, storage medium and server for criminal activities, which can realize automatic identification of criminal activities and issue alarm information, without requiring the driver to take any initiative to reduce the possibility of the perpetrators taking excessive behavior. Sex, while reducing the burden on the victim (driver), can achieve safe and effective timely warning.
  • an alert method for criminal activities including:
  • the criminal activity classifier is pre-trained by the following steps:
  • Pre-collecting training group samples including a plurality of sets of first video images and first audio information for training;
  • Pre-marking the standard recognition result corresponding to each group of video images and audio information in the training group sample, and the standard recognition result is that there is criminal activity or no criminal activity;
  • test set samples including a plurality of sets of second video images and second audio information for testing
  • the warning method for criminal activities further includes:
  • test accuracy of the test is lower than a preset test threshold, adjusting the classifier parameter in the crime activity classifier, returning to perform performing the extracting the first behavior feature of the region of interest in the first video image Step, start the next training;
  • test accuracy rate of the test is higher than or equal to the preset test threshold, the step of determining that the criminal activity classifier training is completed is performed.
  • behavioral features include visual features and motion trajectory features
  • the points of interest in the region of interest are described as visual features and motion trajectory features using a Tracklet descriptor.
  • the sending the alarm information includes:
  • the preset alarm information, the real-time positioning information, and the video image and audio information collected in real time are sent to a designated alarm terminal.
  • an alerting device for criminal activities including:
  • Real-time acquisition module for real-time collection of video images and audio information of the location of the driver on the vehicle
  • a behavior feature extraction module configured to extract behavior characteristics of the region of interest in the video image
  • a voice feature extraction module configured to extract voice features in the collected audio information
  • a feature normalization module configured to normalize the extracted behavior feature and the voice feature
  • a classification identification module configured to classify and identify the normalized behavior feature and the voice feature into a pre-trained criminal activity classifier, to obtain an output classification recognition result, where the classification recognition result is a criminal activity Or there is no criminal activity;
  • the alarm module is configured to send an alarm message if the classification and recognition result is that a criminal activity exists.
  • the criminal activity classifier is pre-trained by the following modules:
  • a training sample collection module configured to pre-collect a training group sample, where the training group sample includes a plurality of sets of first video images and first audio information for training;
  • a training sample marking module configured to pre-mark standard identification results corresponding to each group of video images and audio information in the training group sample, and the standard recognition result is that there is criminal activity or no criminal activity;
  • a first behavior feature module configured to extract a first behavior feature of the region of interest in the first video image
  • a first voice feature module configured to extract a first voice feature in the first audio information
  • a first normalization processing module configured to normalize the extracted first behavior feature and the first speech feature
  • a first classifier identification module configured to classify the first behavior feature and the first voice feature normalized in the training group sample into a criminal activity classifier to obtain an output classification recognition result
  • a first comparison module configured to compare the output classification recognition result with a standard recognition result corresponding to the training group sample, to obtain an accuracy rate of the output of the criminal activity classifier in the training;
  • a first parameter adjustment module configured to: if the accuracy of the current training is lower than a preset threshold, adjust a classifier parameter in the crime activity classifier, return to trigger the first behavior feature module, and start a next training;
  • the training completion determining module is configured to determine that the criminal activity classifier training is completed if the accuracy of the current training is higher than or equal to the preset threshold.
  • test sample collection module configured to pre-collect test group samples, the test set samples including a plurality of sets of second video images and second audio information for testing;
  • test sample marking module configured to pre-mark standard identification results corresponding to each group of video images and audio information in the test group samples
  • the following modules are also triggered before the training completion determination module determines that the criminal activity classifier training is completed:
  • a second behavior feature module configured to extract a second behavior feature of the region of interest in the second video image
  • a second voice feature module configured to extract a second voice feature in the second audio information
  • a second normalization processing module configured to normalize the extracted second behavior feature and the second speech feature
  • a second classifier identification module configured to classify the second behavior feature and the second voice feature normalized in the test group sample into a criminal activity classifier to obtain an output classification recognition result
  • a second comparison module configured to compare the output classification recognition result with the standard recognition result corresponding to the test group sample, to obtain a test accuracy rate of the criminal activity classifier output result in the test;
  • a second parameter adjustment module configured to adjust a classifier parameter in the crime activity classifier if the test accuracy rate of the test is lower than a preset test threshold, and return to trigger the first behavior feature module to start the next time training;
  • the training completion determining module is configured to trigger the training completion determining module to determine that the criminal activity classifier training is completed if the test accuracy rate of the current test is higher than or equal to the preset test threshold.
  • an alert server for criminal activity comprising a memory, a processor, and a computer program stored in the memory and operative on the processor, the processor executing the computer program The steps of implementing the above-described alerting method for criminal activities.
  • a computer readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described alerting method for criminal activity.
  • a video image and audio information of a location of a driver on a vehicle are collected in real time; then, a behavior feature of the region of interest in the video image is extracted; and the collected voice feature in the audio information is extracted. And then normalizing the extracted behavioral features and the speech features; and then performing the normalized behavioral features and the speech features into a pre-trained criminal activity classifier
  • the classification identification obtains the output classification recognition result, and the classification identification result is that there is a criminal activity or no criminal activity; if the classification recognition result is that there is a criminal activity, the alarm information is sent.
  • the video image and the audio information of the location of the driver on the vehicle are collected, and the video image and the audio information are put into a criminal activity classifier for classification and recognition, thereby realizing criminal activities.
  • Automatically identify and issue alarm information without the driver taking the initiative to do any action, reducing the possibility of the perpetrators taking excessive behavior, while reducing the burden on the victim (driver), can achieve safe and effective timely warning.
  • FIG. 1 is a flowchart of an embodiment of an alarm method for criminal activities according to an embodiment of the present invention
  • FIG. 2 is a schematic flowchart of step 102 of an alarming method for a criminal activity in an application scenario according to an embodiment of the present invention
  • FIG. 3 is a schematic flowchart of pre-training a criminal activity classifier in an application scenario according to an alarm method for criminal activities according to an embodiment of the present invention
  • FIG. 4 is a schematic flowchart of testing a criminal activity classifier in an application scenario according to an alarm method for criminal activities according to an embodiment of the present invention
  • FIG. 5 is a structural diagram of an embodiment of an alarm device for criminal activities according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of an alarm server for criminal activities according to an embodiment of the present invention.
  • the embodiment of the invention provides a warning method, device, storage medium and server for criminal activities, which are used to solve the problem of how to implement a safe and effective timely alarm in the case of a driver suffering from doctrine of criminal activities.
  • an embodiment of an alerting method for criminal activities in an embodiment of the present invention includes:
  • the camera can be installed in the front or side direction of the driver's position on the vehicle, the camera is positioned at the driver's position for video shooting, and the video stream information of the driver's location can be collected in real time, and the video stream information is performed.
  • the video is imaged by frame segmentation and sampling.
  • a microphone can be installed near the driver's location to collect audio information. The video image and audio information collected by the camera and the microphone are transmitted to the execution body of the solution, so that the executed subject can obtain the captured video image and audio information in real time.
  • the execution entity of the solution may specifically be a terminal, a system, or a remote server installed on a vehicle.
  • the following is uniformly expressed as an execution subject.
  • the execution subject can extract behavioral features of the region of interest in the video image.
  • the behavior feature may include a visual feature and a motion trajectory feature.
  • the foregoing step 102 may include:
  • the Up body detector or the HOG descriptor is used to extract the region of interest in the video image, such as the outline of the robbery behavior.
  • the Cuboid detector can be used to extract the points of interest/blocks in the video, and the tracklet description sub-detection is used to describe the visual features and the motion trajectory features.
  • the hybrid deformable multi-scale split model is used to implement the action of interest (such as the commonly used action of criminal activities, pulling the driver's neck from behind).
  • Body detector First, establish a deformable multi-scale fractal model at different resolutions of the image: use a complete structure of a target premise to predict a bounding box for the target object, that is, use the function to map the feature vector to the upper left of the bounding box and On the lower right and the corner of the bounding box, an implicit support vector machine is trained to classify the model to detect the target action area in the image, and the region of interest can be obtained.
  • the main idea of the HOG descriptor is that the representation and shape of an object in an image can be well described by the direction distribution of the edges or the pixel intensity gradient.
  • the method is implemented by first dividing the image into a connected area of the square unit, and then collecting the edge direction or the gradient direction histogram of each pixel in the square unit, and finally combining the histograms to form a feature descriptor.
  • the Tracklet descriptor When using the Tracklet descriptor to detect and describe the human motion information in the video image, it mainly describes the preset landmark action. For example, the robber's head with the face is close to the driver's head from the rear, holding the object against it. Driver's neck or head, and so on.
  • the models of these landmark actions may be pre-stored in the action model database, and when the tracklet description sub-description is used, the action models in the action model database are extracted for matching and description.
  • the background noise may be removed by using a wavelet coefficient threshold method, and then MFCC (mel. frequency cepstral coefficients) features are extracted for each frame, and the features are connected together to form a voice feature.
  • MFCC mel. frequency cepstral coefficients
  • the audio open source software JAudio can extract 14 kinds of voice characteristics (specifically changed) in the spectral domain and time domain for each audio information, so that it can distinguish and identify the vocabulary related to criminal activities.
  • vocabulary related to criminal activities can include: robbers (do not move, do not move, take money out, etc.), drivers (with monitoring, you be careful, etc.).
  • step 104 these partial histograms can be normalized in a larger interval of the image. For example, the density of each local histogram in this interval can be calculated first, and then the individual grid cells in the interval are normalized according to the density value.
  • the behavioral features and the phonetic features are normalized, the behavioral features and the phonetic features are put into the pre-trained criminal activity classifier for classification and recognition, and the output classification recognition result is obtained, wherein the classification recognition result is the existence of criminal activities. Or there is no criminal activity.
  • the above-mentioned criminal activity classifier is obtained by training a large number of training samples in advance, and can classify and recognize the behavioral features and the speech features of the feature fusion, and real-time whether the criminal activity behavior exists in the current video image. Judge and output the corresponding classification recognition result.
  • the alarm information is sent.
  • the classification and recognition result is that there is a criminal activity
  • the alarm information should be issued in time.
  • there may be multiple forms of alerting information such as issuing a prompt to the criminals, "Please stop your illegal actions immediately"; or, don't make any prompts to the criminals to avoid the criminals making excessive acts and performing The subject quietly alerted the nearest public security system, waiting for the police to come to handle it; and so on.
  • the GPS positioning module can be installed on the vehicle, so that the execution body can acquire the real-time positioning information of the vehicle in real time; then, when the alarm information needs to be sent, the preset alarm information, the real-time positioning information, and the real-time collection information are collected.
  • the video image and audio information are sent to a designated alarm terminal.
  • the alarm terminal mentioned here can be the alarm server of the Public Security Bureau. Among them, the video image and audio information sent together with the alarm information can be used as evidence to prove the criminal behavior, so that the law enforcement officers can punish the criminals.
  • the criminal activity classifier is pre-trained by the following steps:
  • pre-collecting training group samples where the training group samples include multiple sets of first video images and first audio information for training;
  • the first behavior feature normalized in the training group sample and the first speech feature are classified into a criminal activity classifier for classification and identification, to obtain an output classification recognition result;
  • step 308 determining whether the accuracy of the current training is lower than a preset threshold, and if so, executing step 309, and if not, executing step 310;
  • a plurality of sets of video images and audio information for training that is, the first video image and the first audio information described above, need to be collected in advance before training the criminal activity classifier.
  • the first video image and the first audio information appear in pairs, and the same set of first video images and first audio information are collected at the same time from the same vehicle. Among them, the greater the amount of data in the collected training group samples, the better the training effect on the criminal activity classifier.
  • the normalized first behavior feature and the first speech feature in the training group sample are classified into a criminal activity classifier for classification and recognition, because the criminal activity classifier is fashionable.
  • the training is not completed, so the classification recognition result of the output will deviate from the standard recognition result.
  • step 307 it can be understood that since each set of video image and audio information in the training group sample is marked with a corresponding standard recognition result, the result output by the criminal activity classifier can be performed with the pre-marked standard recognition result.
  • the accuracy of the output of the criminal activity classifier in this training is known. For example, suppose the standard recognition results of three groups of samples are existence, existence, and non-existent criminal activities, and the classification recognition results output by the three groups of samples after inputting the criminal activity classifier are non-existent, exist, and non-existent criminal activities. The comparison shows that in this training of the three groups of samples, the recognition accuracy rate is 66.7%.
  • the criminal activity classifier After obtaining the accuracy of the output result in the current training, it can be determined whether the criminal activity classifier is completed by verifying whether the accuracy meets the requirement. If the accuracy of the training is lower than the preset threshold, adjust the classifier parameter in the crime activity classifier, and return to step 303 to start the next training; if the accuracy of the training is higher than or equal to the preset The threshold determines that the criminal activity classifier training is completed.
  • the classifier parameters refer to attribute data such as weights and thresholds preset in the criminal activity classifier, and the classifier parameters may be different for different types of classifiers. By adjusting the classifier parameters, the criminal activity classifier can be better judged which criminal images and audio information have criminal activities. Since the accuracy of the crime activity classifier output is lower than the preset threshold, it is necessary to return to step 303 and continue.
  • the training group sample is used for the next training of the criminal activity classifier.
  • the above preset threshold can be set according to specific conditions, for example, set to 95%.
  • a test group sample different from the training group sample may be prepared to test and test the criminal activity classifier.
  • a test group sample may be collected in advance, the test set sample including a plurality of sets of second video images and second audio information for testing; and then, each set of video images and audio in the test set samples is pre-marked The standard identification result corresponding to the information.
  • the alarming method for criminal activities further includes:
  • the second behavior feature normalized in the test group sample and the second speech feature are classified into a criminal activity classifier for classification and identification, to obtain an output classification recognition result;
  • step 406 determining whether the accuracy of the current test is lower than the preset test threshold, and if so, executing step 407, and if not, executing step 310;
  • step 405 the approximate content is substantially the same as step 307, except that in step 405, the obtained test accuracy rate is used to evaluate the degree of completion of the training of the criminal activity classifier, because the test test group samples are different. In the training group sample, it is more strange to the criminal activity classifier, so the evaluation effect will be better than the evaluation effect in the training phase.
  • Step 310 is performed to determine that the criminal activity classifier training is completed.
  • a video image and audio information of a location of a driver on a vehicle are collected in real time; then, a behavior feature of the region of interest in the video image is extracted; and the collected voice feature in the audio information is extracted. And then normalizing the extracted behavioral features and the speech features; and then performing the normalized behavioral features and the speech features into a pre-trained criminal activity classifier
  • the classification identification obtains the output classification recognition result, and the classification identification result is that there is a criminal activity or no criminal activity; if the classification recognition result is that there is a criminal activity, the alarm information is sent.
  • the video image and the audio information of the location of the driver on the vehicle are collected, and the video image and the audio information are put into a criminal activity classifier for classification and recognition, thereby realizing criminal activities.
  • Automatically identify and issue alarm information without the driver taking the initiative to do any action, reducing the possibility of the perpetrators taking excessive behavior, while reducing the burden on the victim (driver), can achieve safe and effective timely warning.
  • FIG. 5 is a structural diagram showing an embodiment of an alarm device for criminal activities in an embodiment of the present invention.
  • an alerting device for criminal activities includes:
  • the real-time collection module 501 is configured to collect video images and audio information of a location of the driver on the vehicle in real time;
  • the behavior feature extraction module 502 is configured to extract behavior characteristics of the region of interest in the video image
  • the voice feature extraction module 503 is configured to extract voice features in the collected audio information.
  • a feature normalization module 504 configured to normalize the extracted behavior feature and the voice feature
  • the classification and identification module 505 is configured to classify and identify the normalized behavior feature and the voice feature into a pre-trained criminal activity classifier to obtain an output classification recognition result, where the classification recognition result is a crime Activity or no criminal activity;
  • the alarm module 506 is configured to send an alarm message if the classification and recognition result is that a criminal activity exists.
  • the criminal activity classifier can be pre-trained by the following modules:
  • a training sample collection module configured to pre-collect a training group sample, where the training group sample includes a plurality of sets of first video images and first audio information for training;
  • a training sample marking module configured to pre-mark standard identification results corresponding to each group of video images and audio information in the training group sample, and the standard recognition result is that there is criminal activity or no criminal activity;
  • a first behavior feature module configured to extract a first behavior feature of the region of interest in the first video image
  • a first voice feature module configured to extract a first voice feature in the first audio information
  • a first normalization processing module configured to normalize the extracted first behavior feature and the first speech feature
  • a first classifier identification module configured to classify the first behavior feature and the first voice feature normalized in the training group sample into a criminal activity classifier to obtain an output classification recognition result
  • a first comparison module configured to compare the output classification recognition result with a standard recognition result corresponding to the training group sample, to obtain an accuracy rate of the output of the criminal activity classifier in the training;
  • a first parameter adjustment module configured to: if the accuracy of the current training is lower than a preset threshold, adjust a classifier parameter in the crime activity classifier, return to trigger the first behavior feature module, and start a next training;
  • the training completion determining module is configured to determine that the criminal activity classifier training is completed if the accuracy of the current training is higher than or equal to the preset threshold.
  • the alarm device for criminal activities may further include:
  • test sample collection module configured to pre-collect test group samples, the test set samples including a plurality of sets of second video images and second audio information for testing;
  • test sample marking module configured to pre-mark standard identification results corresponding to each group of video images and audio information in the test group samples
  • the following modules may also be triggered before the training completion determination module determines that the criminal activity classifier training is completed:
  • a second behavior feature module configured to extract a second behavior feature of the region of interest in the second video image
  • a second voice feature module configured to extract a second voice feature in the second audio information
  • a second normalization processing module configured to normalize the extracted second behavior feature and the second speech feature
  • a second classifier identification module configured to classify the second behavior feature and the second voice feature normalized in the test group sample into a criminal activity classifier to obtain an output classification recognition result
  • a second comparison module configured to compare the output classification recognition result with the standard recognition result corresponding to the test group sample, to obtain a test accuracy rate of the criminal activity classifier output result in the test;
  • a second parameter adjustment module configured to adjust a classifier parameter in the crime activity classifier if the test accuracy rate of the test is lower than a preset test threshold, and return to trigger the first behavior feature module to start the next time training;
  • the training completion determining module is configured to trigger the training completion determining module to determine that the criminal activity classifier training is completed if the test accuracy rate of the current test is higher than or equal to the preset test threshold.
  • behavioral features include visual features and motion trajectory features
  • the behavior feature extraction module may include:
  • a region extracting unit configured to extract a region of interest in the video image
  • a point of interest detecting unit configured to detect a point of interest in the region of interest
  • a feature description unit is configured to describe a point of interest in the region of interest as a visual feature and a motion trajectory feature by using a Tracklet descriptor.
  • the alarm module may include:
  • a positioning information acquiring unit configured to acquire real-time positioning information of the vehicle
  • the information sending unit is configured to send the preset alarm information, the real-time positioning information, and the video image and audio information collected in real time to the designated alarm terminal.
  • FIG. 6 is a schematic diagram of an alarm server for criminal activities according to an embodiment of the present invention.
  • the alarm server 6 for criminal activities of this embodiment includes a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and operable on the processor 60, for example, executing The above procedure for the warning method of criminal activity.
  • the processor 60 executes the computer program 62 to implement the steps in the various embodiments of the alerting method for criminal activities described above, such as steps 101 through 106 shown in FIG.
  • the processor 60 executes the computer program 62, the functions of the modules/units in the above various device embodiments are implemented, such as the functions of the modules 501 to 506 shown in FIG.
  • the computer program 62 can be partitioned into one or more modules/units that are stored in the memory 61 and executed by the processor 60 to complete this invention.
  • the one or more modules/units may be a series of computer program instruction segments capable of performing a particular function, the instruction segments being used to describe the execution of the computer program 62 in the alerting server 6 for criminal activity.
  • the alarm server 6 for criminal activities may be a computing device such as a local server or a cloud server.
  • the alert server for criminal activity may include, but is not limited to, processor 60, memory 61. It will be understood by those skilled in the art that FIG. 6 is merely an example of an alert server 6 for criminal activities, and does not constitute a limitation to the alert server 6 for criminal activities, may include more or fewer components than illustrated, or a combination Certain components, or different components, such as the alerting server for criminal activity, may also include input and output devices, network access devices, buses, and the like.
  • the processor 60 can be a central processing unit (Central Processing Unit, CPU), can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the memory 61 may be an internal storage unit of the alert server 6 for criminal activity, such as a hard disk or memory for the alert server 6 of criminal activity.
  • the memory 61 may also be an external storage device of the alarm server 6 for criminal activities, such as a plug-in hard disk equipped with a smart media card (SMC) provided on the alarm server 6 for criminal activities. Secure Digital (SD) card, flash card (Flash Card) and so on.
  • SMC smart media card
  • SD Secure Digital
  • flash card Flash Card
  • the memory 61 is used to store the computer program and other programs and data required by the alert server for criminal activity.
  • the memory 61 can also be used to temporarily store data that has been output or is about to be output.
  • modules, units, and/or method steps of the various embodiments described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.
  • the disclosed system, apparatus, and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the present invention implements all or part of the processes in the foregoing embodiments, and may also be completed by a computer program to instruct related hardware.
  • the computer program may be stored in a computer readable storage medium. The steps of the various method embodiments described above may be implemented when the program is executed by the processor.
  • the computer program comprises computer program code, which may be in the form of source code, object code form, executable file or some intermediate form.
  • the computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard drive, a magnetic disk, an optical disk, a computer memory, a read only memory (ROM, Read-Only) Memory), random access memory (RAM, Random) Access Memory), electrical carrier signals, telecommunications signals, and software distribution media.
  • ROM Read Only memory
  • RAM Random Access Memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Emergency Management (AREA)
  • Business, Economics & Management (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Alarm Systems (AREA)

Abstract

An alarm method for criminal activity, wherein same is used for solving the problem of how to safely and effectively give a alarm in a timely manner when a driver suffers from persecution during criminal activity. The alarm method for criminal activity comprises: collecting, in real time, a video image and audio information of a location where a driver is located in a vehicle; extracting a behavior feature of an area of interest in the video image; extracting a voice feature in the collected audio information; carrying out normalization processing on the extracted behavior feature and voice feature; putting the normalized behavior feature and voice feature into a pre-trained criminal activity classifier for classification and recognition, and thus obtaining an output classification and recognition result, wherein the classification and recognition result is that there is a criminal activity or that there is no criminal activity; and if the classification and recognition result is that there is a criminal activity, sending out alarm information.

Description

一种针对犯罪活动的告警方法、装置、存储介质及服务器Alarm method, device, storage medium and server for criminal activities 技术领域Technical field
本发明涉及视频信息处理技术领域，尤其涉及一种针对犯罪活动的告警方法、装置、存储介质及服务器。The present invention relates to the field of video information processing technologies, and in particular, to an alarm method, apparatus, storage medium, and server for criminal activities.
背景技术Background technique
随着车载行业向着数字化和智能化方向的迈进，以及现实中会存在因司机被挟持抢劫等而威胁到其人身和财产安全的事件，人们渴望能有一套可以对类似抢劫司机行为进行实时监控和报警的装置，以便社会向着更加和谐安全的方向发展，减少司机出行忧虑、财产忧虑和生命威胁。As the automotive industry moves toward digitalization and intelligence, and in reality there are events that threaten the safety of people and property as drivers are robbed, people are eager to have a set of real-time monitoring of similar robbery drivers. The device of the alarm, so that the society can develop in a more harmonious and safe direction, reducing drivers' worries, property concerns and life threats.
目前市面上针对司机人身和财产安全报警的解决方法多分为两类：1、基于手动迅速偷按紧急按钮、遥控器按键发出实时报警控制信号进行语音报警；2、基于安全座椅装备导电液发射枪进行电击和用催泪瓦斯催泪的物理伤害方式进行肇事行为（如抢劫）阻止。但这两种方式都存在一些较明显的缺陷：对第1种方法而言，当肇事者实施抢劫前，肇事者本身一般也是处于高度紧张状态，最先做的动作一般都是死死把控手中的武器并紧盯司机的一举一动，这时若让司机去手动按紧急按钮或找遥控器按键进行触发，不但操作不方便，而且非常容易引起肇事者情绪激动而造成误伤或刻意伤害的可能性；对第2种方法而言，首先是硬件成本较高，在座位上装备导电液发射枪及催泪瓦斯等会有很大的成本提升，不利于推广，其次是肇事者一般都手持武器顶住司机且距离司机较近，这种类型的物理攻击在伤害抢劫者的同时非常容易对司机自己带有连同伤害，风险性和不确定性太大。At present, the solution to the driver's personal and property safety alarms in the market is divided into two categories: 1. Based on the manual quick slamming of the emergency button, the remote control button sends a real-time alarm control signal for voice alarm; 2. Based on the safety seat equipped with conductive liquid emission The gun is subjected to electric shocks and physical acts of tearing gas tears are used to prevent anecdotes (such as robbery). However, there are some obvious defects in the two methods: For the first method, before the perpetrators commit the robbery, the perpetrators themselves are generally in a state of high tension, and the first action is generally controlled. The weapon in your hand keeps an eye on the driver's every move. If you let the driver manually press the emergency button or find the remote control button to trigger, it is not only inconvenient to operate, but also very likely to cause the accident of the perpetrator to cause accidental injury or intentional injury. For the second method, first of all, the hardware cost is high. There is a big cost increase in the seat equipped with conductive liquid launching gun and tear gas, which is not conducive to promotion. Secondly, the perpetrators generally hold the weapon top. Living with the driver and being closer to the driver, this type of physical attack is very easy to damage the robber and at the same time it is very risky and uncertain for the driver himself.
可见，如何在司机遭受犯罪活动迫害的情况下实现安全有效的及时告警成为本领域技术人员亟需解决的问题。It can be seen that how to implement safe and effective timely warning in the case of a driver suffering from persecution of criminal activities has become an urgent problem to be solved by those skilled in the art.
技术问题technical problem
本发明实施例提供了一种针对犯罪活动的告警方法、装置、存储介质及服务器，能够实现犯罪活动的自动识别并发出告警信息，无需司机主动做出任何动作，减少肇事者采取过激行为的可能性，同时减轻被害人（司机）的负担，可以做到安全有效的及时告警。The embodiment of the invention provides an alarm method, device, storage medium and server for criminal activities, which can realize automatic identification of criminal activities and issue alarm information, without requiring the driver to take any initiative to reduce the possibility of the perpetrators taking excessive behavior. Sex, while reducing the burden on the victim (driver), can achieve safe and effective timely warning.
技术解决方案Technical solution
第一方面，提供了一种针对犯罪活动的告警方法，包括：In the first aspect, an alert method for criminal activities is provided, including:
实时采集车辆上司机所处位置的视频图像和音频信息；Real-time collection of video images and audio information of the location of the driver on the vehicle;
提取所述视频图像中感兴趣区域的行为特征；Extracting behavioral characteristics of the region of interest in the video image;
提取采集到的所述音频信息中的语音特征；Extracting the collected voice features in the audio information;
将提取得到的所述行为特征和所述语音特征进行归一化处理；And normalizing the extracted behavior characteristics and the voice features;
将归一化后的所述行为特征和所述语音特征投入预训练完成的犯罪活动分类器进行分类识别，得到输出的分类识别结果，所述分类识别结果为存在犯罪活动或者不存在犯罪活动；And classifying the normalized behavioral feature and the speech feature into a pre-trained criminal activity classifier to obtain an output classification recognition result, where the classification recognition result is that there is a criminal activity or no criminal activity;
若所述分类识别结果为存在犯罪活动，则发出告警信息。If the classification and recognition result is that there is a criminal activity, an alarm message is sent.
进一步地，所述犯罪活动分类器通过以下步骤预先训练得到：Further, the criminal activity classifier is pre-trained by the following steps:
预先收集训练组样本，所述训练组样本包括用于训练的多组第一视频图像和第一音频信息；Pre-collecting training group samples, the training group samples including a plurality of sets of first video images and first audio information for training;
预先标记所述训练组样本中各组视频图像和音频信息对应的标准识别结果，标准识别结果为存在犯罪活动或者不存在犯罪活动；Pre-marking the standard recognition result corresponding to each group of video images and audio information in the training group sample, and the standard recognition result is that there is criminal activity or no criminal activity;
提取所述第一视频图像中感兴趣区域的第一行为特征；Extracting a first behavior characteristic of the region of interest in the first video image;
提取所述第一音频信息中的第一语音特征；Extracting a first voice feature in the first audio information;
将提取得到的所述第一行为特征和所述第一语音特征进行归一化处理；And normalizing the extracted first behavior feature and the first speech feature;
将所述训练组样本中归一化后的所述第一行为特征和所述第一语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；And normalizing the first behavior feature and the first speech feature in the training group sample into a criminal activity classifier for classification and identification, to obtain an output classification recognition result;
将输出的分类识别结果与所述训练组样本对应的标准识别结果进行对比，得到本次训练所述犯罪活动分类器输出结果的准确率；Comparing the output classification recognition result with the standard recognition result corresponding to the training group sample, and obtaining the accuracy of the output result of the criminal activity classifier in the training;
若本次训练的准确率低于预设阈值，则调整所述犯罪活动分类器中的分类器参数，返回执行所述提取所述第一视频图像中感兴趣区域的第一行为特征的步骤，开始下一次训练；If the accuracy of the current training is lower than a preset threshold, adjusting the classifier parameter in the criminal activity classifier, and returning to performing the step of extracting the first behavior feature of the region of interest in the first video image, Start the next training;
若本次训练的准确率高于或等于预设阈值，则确定所述犯罪活动分类器训练完成。If the accuracy of the training is higher than or equal to the preset threshold, it is determined that the criminal activity classifier training is completed.
进一步地，还包括：Further, it also includes:
预先收集测试组样本，所述测试组样本包括用于测试的多组第二视频图像和第二音频信息；Collecting test group samples in advance, the test set samples including a plurality of sets of second video images and second audio information for testing;
预先标记所述测试组样本中各组视频图像和音频信息对应的标准识别结果；Pre-marking the standard recognition result corresponding to each group of video images and audio information in the test group sample;
在确定所述犯罪活动分类器训练完成之前，所述针对犯罪活动的告警方法还包括：Before determining that the criminal activity classifier training is completed, the warning method for criminal activities further includes:
提取所述第二视频图像中感兴趣区域的第二行为特征；Extracting a second behavior characteristic of the region of interest in the second video image;
提取所述第二音频信息中的第二语音特征；Extracting a second voice feature in the second audio information;
将提取得到的所述第二行为特征和所述第二语音特征进行归一化处理；And normalizing the extracted second behavior feature and the second speech feature;
将所述测试组样本中归一化后的所述第二行为特征和所述第二语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；And normalizing the second behavior feature and the second speech feature normalized in the test group sample into a criminal activity classifier for classification and identification, to obtain an output classification recognition result;
将输出的分类识别结果与所述测试组样本对应的标准识别结果进行对比，得到本次测试中所述犯罪活动分类器输出结果的测试准确率；Comparing the output classification recognition result with the standard recognition result corresponding to the test group sample, and obtaining the test accuracy rate of the criminal activity classifier output result in the test;
若本次测试的测试准确率低于预设测试阈值，则调整所述犯罪活动分类器中的分类器参数，返回执行所述提取所述第一视频图像中感兴趣区域的第一行为特征的步骤，开始下一次训练；If the test accuracy of the test is lower than a preset test threshold, adjusting the classifier parameter in the crime activity classifier, returning to perform performing the extracting the first behavior feature of the region of interest in the first video image Step, start the next training;
若本次测试的测试准确率高于或等于预设测试阈值，则执行所述确定所述犯罪活动分类器训练完成的步骤。If the test accuracy rate of the test is higher than or equal to the preset test threshold, the step of determining that the criminal activity classifier training is completed is performed.
进一步地，所述行为特征包括视觉特征和动作轨迹特征；Further, the behavioral features include visual features and motion trajectory features;
所述提取所述视频图像中感兴趣区域的行为特征包括：The extracting behavior characteristics of the region of interest in the video image includes:
提取所述视频图像中感兴趣区域；Extracting a region of interest in the video image;
检测得到在所述感兴趣区域中的兴趣点；Detecting points of interest in the region of interest;
采用Tracklet描述子将所述感兴趣区域中的兴趣点描述成视觉特征和动作轨迹特征。The points of interest in the region of interest are described as visual features and motion trajectory features using a Tracklet descriptor.
进一步地，所述发出告警信息包括：Further, the sending the alarm information includes:
获取所述车辆的实时定位信息；Obtaining real-time positioning information of the vehicle;
将预设的报警信息、所述实时定位信息和实时采集的所述视频图像、音频信息发送至指定的告警终端。The preset alarm information, the real-time positioning information, and the video image and audio information collected in real time are sent to a designated alarm terminal.
第二方面，提供了一种针对犯罪活动的告警装置，包括：In a second aspect, an alerting device for criminal activities is provided, including:
实时采集模块，用于实时采集车辆上司机所处位置的视频图像和音频信息；Real-time acquisition module for real-time collection of video images and audio information of the location of the driver on the vehicle;
行为特征提取模块，用于提取所述视频图像中感兴趣区域的行为特征；a behavior feature extraction module, configured to extract behavior characteristics of the region of interest in the video image;
语音特征提取模块，用于提取采集到的所述音频信息中的语音特征；a voice feature extraction module, configured to extract voice features in the collected audio information;
特征归一模块，用于将提取得到的所述行为特征和所述语音特征进行归一化处理；a feature normalization module, configured to normalize the extracted behavior feature and the voice feature;
分类识别模块，用于将归一化后的所述行为特征和所述语音特征投入预训练完成的犯罪活动分类器进行分类识别，得到输出的分类识别结果，所述分类识别结果为存在犯罪活动或者不存在犯罪活动；a classification identification module, configured to classify and identify the normalized behavior feature and the voice feature into a pre-trained criminal activity classifier, to obtain an output classification recognition result, where the classification recognition result is a criminal activity Or there is no criminal activity;
告警模块，用于若所述分类识别结果为存在犯罪活动，则发出告警信息。The alarm module is configured to send an alarm message if the classification and recognition result is that a criminal activity exists.
进一步地，所述犯罪活动分类器通过以下模块预先训练得到：Further, the criminal activity classifier is pre-trained by the following modules:
训练样本采集模块，用于预先收集训练组样本，所述训练组样本包括用于训练的多组第一视频图像和第一音频信息；a training sample collection module, configured to pre-collect a training group sample, where the training group sample includes a plurality of sets of first video images and first audio information for training;
训练样本标记模块，用于预先标记所述训练组样本中各组视频图像和音频信息对应的标准识别结果，标准识别结果为存在犯罪活动或者不存在犯罪活动；a training sample marking module, configured to pre-mark standard identification results corresponding to each group of video images and audio information in the training group sample, and the standard recognition result is that there is criminal activity or no criminal activity;
第一行为特征模块，用于提取所述第一视频图像中感兴趣区域的第一行为特征；a first behavior feature module, configured to extract a first behavior feature of the region of interest in the first video image;
第一语音特征模块，用于提取所述第一音频信息中的第一语音特征；a first voice feature module, configured to extract a first voice feature in the first audio information;
第一归一处理模块，用于将提取得到的所述第一行为特征和所述第一语音特征进行归一化处理；a first normalization processing module, configured to normalize the extracted first behavior feature and the first speech feature;
第一分类器识别模块，用于将所述训练组样本中归一化后的所述第一行为特征和所述第一语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；a first classifier identification module, configured to classify the first behavior feature and the first voice feature normalized in the training group sample into a criminal activity classifier to obtain an output classification recognition result;
第一对比模块，用于将输出的分类识别结果与所述训练组样本对应的标准识别结果进行对比，得到本次训练所述犯罪活动分类器输出结果的准确率；a first comparison module, configured to compare the output classification recognition result with a standard recognition result corresponding to the training group sample, to obtain an accuracy rate of the output of the criminal activity classifier in the training;
第一参数调整模块，用于若本次训练的准确率低于预设阈值，则调整所述犯罪活动分类器中的分类器参数，返回触发所述第一行为特征模块，开始下一次训练；a first parameter adjustment module, configured to: if the accuracy of the current training is lower than a preset threshold, adjust a classifier parameter in the crime activity classifier, return to trigger the first behavior feature module, and start a next training;
训练完成确定模块，用于若本次训练的准确率高于或等于预设阈值，则确定所述犯罪活动分类器训练完成。The training completion determining module is configured to determine that the criminal activity classifier training is completed if the accuracy of the current training is higher than or equal to the preset threshold.
进一步地，还包括：Further, it also includes:
测试样本采集模块，用于预先收集测试组样本，所述测试组样本包括用于测试的多组第二视频图像和第二音频信息；a test sample collection module, configured to pre-collect test group samples, the test set samples including a plurality of sets of second video images and second audio information for testing;
测试样本标记模块，用于预先标记所述测试组样本中各组视频图像和音频信息对应的标准识别结果；a test sample marking module, configured to pre-mark standard identification results corresponding to each group of video images and audio information in the test group samples;
在所述训练完成确定模块确定所述犯罪活动分类器训练完成之前，还触发以下模块：The following modules are also triggered before the training completion determination module determines that the criminal activity classifier training is completed:
第二行为特征模块，用于提取所述第二视频图像中感兴趣区域的第二行为特征；a second behavior feature module, configured to extract a second behavior feature of the region of interest in the second video image;
第二语音特征模块，用于提取所述第二音频信息中的第二语音特征；a second voice feature module, configured to extract a second voice feature in the second audio information;
第二归一处理模块，用于将提取得到的所述第二行为特征和所述第二语音特征进行归一化处理；a second normalization processing module, configured to normalize the extracted second behavior feature and the second speech feature;
第二分类器识别模块，用于将所述测试组样本中归一化后的所述第二行为特征和所述第二语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；a second classifier identification module, configured to classify the second behavior feature and the second voice feature normalized in the test group sample into a criminal activity classifier to obtain an output classification recognition result;
第二对比模块，用于将输出的分类识别结果与所述测试组样本对应的标准识别结果进行对比，得到本次测试中所述犯罪活动分类器输出结果的测试准确率；a second comparison module, configured to compare the output classification recognition result with the standard recognition result corresponding to the test group sample, to obtain a test accuracy rate of the criminal activity classifier output result in the test;
第二参数调整模块，用于若本次测试的测试准确率低于预设测试阈值，则调整所述犯罪活动分类器中的分类器参数，返回触发所述第一行为特征模块，开始下一次训练；a second parameter adjustment module, configured to adjust a classifier parameter in the crime activity classifier if the test accuracy rate of the test is lower than a preset test threshold, and return to trigger the first behavior feature module to start the next time training;
训练完成确定模块，用于若本次测试的测试准确率高于或等于预设测试阈值，则触发所述训练完成确定模块来确定所述犯罪活动分类器训练完成。The training completion determining module is configured to trigger the training completion determining module to determine that the criminal activity classifier training is completed if the test accuracy rate of the current test is higher than or equal to the preset test threshold.
第三方面，提供了一种针对犯罪活动的告警服务器，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述的针对犯罪活动的告警方法的步骤。In a third aspect, an alert server for criminal activity is provided, comprising a memory, a processor, and a computer program stored in the memory and operative on the processor, the processor executing the computer program The steps of implementing the above-described alerting method for criminal activities.
第四方面，提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现上述的针对犯罪活动的告警方法的步骤。In a fourth aspect, a computer readable storage medium is provided, the computer readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described alerting method for criminal activity.
有益效果Beneficial effect
从以上技术方案可以看出，本发明实施例具有以下优点：It can be seen from the above technical solutions that the embodiments of the present invention have the following advantages:
本发明实施例中，首先，实时采集车辆上司机所处位置的视频图像和音频信息；然后，提取所述视频图像中感兴趣区域的行为特征；提取采集到的所述音频信息中的语音特征；接着，将提取得到的所述行为特征和所述语音特征进行归一化处理；再之，将归一化后的所述行为特征和所述语音特征投入预训练完成的犯罪活动分类器进行分类识别，得到输出的分类识别结果，所述分类识别结果为存在犯罪活动或者不存在犯罪活动；若所述分类识别结果为存在犯罪活动，则发出告警信息。在本发明实施例中，当犯罪活动发生时，通过采集车辆上司机所处位置的视频图像和音频信息，将这些视频图像和音频信息投入到犯罪活动分类器中进行分类识别，实现犯罪活动的自动识别并发出告警信息，无需司机主动做出任何动作，减少肇事者采取过激行为的可能性，同时减轻被害人（司机）的负担，可以做到安全有效的及时告警。In the embodiment of the present invention, first, a video image and audio information of a location of a driver on a vehicle are collected in real time; then, a behavior feature of the region of interest in the video image is extracted; and the collected voice feature in the audio information is extracted. And then normalizing the extracted behavioral features and the speech features; and then performing the normalized behavioral features and the speech features into a pre-trained criminal activity classifier The classification identification obtains the output classification recognition result, and the classification identification result is that there is a criminal activity or no criminal activity; if the classification recognition result is that there is a criminal activity, the alarm information is sent. In the embodiment of the present invention, when a criminal activity occurs, the video image and the audio information of the location of the driver on the vehicle are collected, and the video image and the audio information are put into a criminal activity classifier for classification and recognition, thereby realizing criminal activities. Automatically identify and issue alarm information, without the driver taking the initiative to do any action, reducing the possibility of the perpetrators taking excessive behavior, while reducing the burden on the victim (driver), can achieve safe and effective timely warning.
附图说明DRAWINGS
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments or the description of the prior art will be briefly described below. It is obvious that the drawings in the following description are only the present invention. For some embodiments, other drawings may be obtained from those of ordinary skill in the art in light of the inventive workability.
图1为本发明实施例中一种针对犯罪活动的告警方法一个实施例流程图；FIG. 1 is a flowchart of an embodiment of an alarm method for criminal activities according to an embodiment of the present invention;
图2为本发明实施例中一种针对犯罪活动的告警方法步骤102在一个应用场景下的流程示意图；FIG. 2 is a schematic flowchart of step 102 of an alarming method for a criminal activity in an application scenario according to an embodiment of the present invention;
图3为本发明实施例中一种针对犯罪活动的告警方法在一个应用场景下预先训练犯罪活动分类器的流程示意图；3 is a schematic flowchart of pre-training a criminal activity classifier in an application scenario according to an alarm method for criminal activities according to an embodiment of the present invention;
图4为本发明实施例中一种针对犯罪活动的告警方法在一个应用场景下测试犯罪活动分类器的流程示意图；4 is a schematic flowchart of testing a criminal activity classifier in an application scenario according to an alarm method for criminal activities according to an embodiment of the present invention;
图5为本发明实施例中一种针对犯罪活动的告警装置一个实施例结构图；FIG. 5 is a structural diagram of an embodiment of an alarm device for criminal activities according to an embodiment of the present invention; FIG.
图6为本发明一实施例提供的针对犯罪活动的告警服务器的示意图。FIG. 6 is a schematic diagram of an alarm server for criminal activities according to an embodiment of the present invention.
本发明的实施方式Embodiments of the invention
本发明实施例提供了一种针对犯罪活动的告警方法、装置、存储介质及服务器，用于解决如何在司机遭受犯罪活动迫害的情况下实现安全有效的及时告警的问题。The embodiment of the invention provides a warning method, device, storage medium and server for criminal activities, which are used to solve the problem of how to implement a safe and effective timely alarm in the case of a driver suffering from persecution of criminal activities.
为使得本发明的发明目的、特征、优点能够更加的明显和易懂，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，下面所描述的实施例仅仅是本发明一部分实施例，而非全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本发明保护的范围。In order to make the object, the features and the advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
请参阅图1，本发明实施例中一种针对犯罪活动的告警方法一个实施例包括：Referring to FIG. 1, an embodiment of an alerting method for criminal activities in an embodiment of the present invention includes:
101、实时采集车辆上司机所处位置的视频图像和音频信息；101. Collecting video images and audio information of the location of the driver on the vehicle in real time;
在本实施例中，可以在车辆上司机所处位置的前方或者侧面方向安装摄像头，摄像头对准司机位置进行视频拍摄，可以实时采集到司机所处位置的视频流信息，对这些视频流信息进行帧分割和采样，即可得到视频图像。另外，可以在司机所在位置附近安装麦克风来采集音频信息。摄像头和麦克风采集到的视频图像和音频信息传输到本方案的执行主体中，使得执行主体可以实时获取到这些采集的视频图像和音频信息。In this embodiment, the camera can be installed in the front or side direction of the driver's position on the vehicle, the camera is positioned at the driver's position for video shooting, and the video stream information of the driver's location can be collected in real time, and the video stream information is performed. The video is imaged by frame segmentation and sampling. In addition, a microphone can be installed near the driver's location to collect audio information. The video image and audio information collected by the camera and the microphone are transmitted to the execution body of the solution, so that the executed subject can obtain the captured video image and audio information in real time.
需要说明的是，本方案的执行主体具体可以是安装在车辆上的终端、***或者远程服务器，为便于描述，下面统一表述为执行主体。It should be noted that the execution entity of the solution may specifically be a terminal, a system, or a remote server installed on a vehicle. For convenience of description, the following is uniformly expressed as an execution subject.
102、提取所述视频图像中感兴趣区域的行为特征；102. Extract behavior characteristics of the region of interest in the video image.
在采集到视频图像之后，执行主体可以提取该视频图像中感兴趣区域的行为特征。具体地，所述行为特征可以包括视觉特征和动作轨迹特征，如图2所示，上述步骤102可以包括：After the video image is acquired, the execution subject can extract behavioral features of the region of interest in the video image. Specifically, the behavior feature may include a visual feature and a motion trajectory feature. As shown in FIG. 2, the foregoing step 102 may include:
201、提取所述视频图像中感兴趣区域；201. Extract a region of interest in the video image.
202、检测得到在所述感兴趣区域中的兴趣点；202. Detecting a point of interest in the region of interest;
203、采用Tracklet描述子将所述感兴趣区域中的兴趣点描述成视觉特征和动作轨迹特征。203. Using a Tracklet descriptor to describe the points of interest in the region of interest as visual features and motion trajectory features.
对于上述步骤201，采用Up body检测子或者HOG描述子提取视频图像中的感兴趣区域，如抢劫行为的轮廓。For the above step 201, the Up body detector or the HOG descriptor is used to extract the region of interest in the video image, such as the outline of the robbery behavior.
对于上述步骤202和203，在提取到感兴趣区域的基础上，可以利用Cuboid检测子提取视频中的兴趣点／块，并利用Tracklet描述子检测，描述视觉特征和动作轨迹特征。For the above steps 202 and 203, on the basis of extracting the region of interest, the Cuboid detector can be used to extract the points of interest/blocks in the video, and the tracklet description sub-detection is used to describe the visual features and the motion trajectory features.
其中，关于Up body检测子，采用基于混合可变形多尺度分体模型来实现该感兴趣动作（比如犯罪活动常用的动作，从背后勒住司机脖子等）的Up body检测子：首先在图像不同分辨率上建立起可变形多尺度分体模型：利用一个目标前提的完整结构为目标物体预测一个边界框，即使用函数将特征向量映射到边界框的左上方和右下方以及边界框的角点上，然后训练一个隐含支持向量机对模型进行分类，从而检测出图像中目标动作区域，即可得到感兴趣区域。Among them, regarding the Up body detector, the hybrid deformable multi-scale split model is used to implement the action of interest (such as the commonly used action of criminal activities, pulling the driver's neck from behind). Body detector: First, establish a deformable multi-scale fractal model at different resolutions of the image: use a complete structure of a target premise to predict a bounding box for the target object, that is, use the function to map the feature vector to the upper left of the bounding box and On the lower right and the corner of the bounding box, an implicit support vector machine is trained to classify the model to detect the target action area in the image, and the region of interest can be obtained.
关于HOG描述子，HOG描述子的主要思想是一幅图像中的物体的表象和形状可以被边缘的方向分布或像素强度梯度很好地描述。其实现方法是先将图像分成方格单元连通区域，然后采集方格单元中各像素点的边缘方向或者梯度方向直方图，最后把这些直方图组合起来就构成了特征描述子。Regarding the HOG descriptor, the main idea of the HOG descriptor is that the representation and shape of an object in an image can be well described by the direction distribution of the edges or the pixel intensity gradient. The method is implemented by first dividing the image into a connected area of the square unit, and then collecting the edge direction or the gradient direction histogram of each pixel in the square unit, and finally combining the histograms to form a feature descriptor.
在利用Tracklet描述子跟踪检测视频图像中人体动作信息并加以描述时，主要描述预设的标志性动作，比如：带有遮脸物的抢劫者头部从后方靠近司机头部、手持东西顶住司机脖子或头部，等等。特别地，这些标示性动作的模型可以预先存储在动作模型数据库中，在使用Tracklet描述子描述时，提取动作模型数据库中的这些动作模型进行匹配、描述。When using the Tracklet descriptor to detect and describe the human motion information in the video image, it mainly describes the preset landmark action. For example, the robber's head with the face is close to the driver's head from the rear, holding the object against it. Driver's neck or head, and so on. In particular, the models of these landmark actions may be pre-stored in the action model database, and when the tracklet description sub-description is used, the action models in the action model database are extracted for matching and description.
103、提取采集到的所述音频信息中的语音特征；103. Extract voice features in the collected audio information.
本实施例中，对于音频信息，可以先利用小波系数阈值方法去除背景杂声，然后对于每一帧提取MFCC(mel．frequency cepstral coefficients)特征，将这些特征连接在一起组成语音特征。另外，音频开源软件JAudio可对每一个音频信息提取14种（根据具体情况具体变动）频谱域和时间域的语音特征，从而很好得对与犯罪活动相关词汇进行区分和鉴别。In this embodiment, for audio information, the background noise may be removed by using a wavelet coefficient threshold method, and then MFCC (mel. frequency cepstral coefficients) features are extracted for each frame, and the features are connected together to form a voice feature. In addition, the audio open source software JAudio can extract 14 kinds of voice characteristics (specifically changed) in the spectral domain and time domain for each audio information, so that it can distinguish and identify the vocabulary related to criminal activities.
比如，与犯罪活动相关的词汇可以包括：抢劫者（别动、不许动、把钱拿出来等）、司机（有监控、你小心点儿等）。For example, vocabulary related to criminal activities can include: robbers (do not move, do not move, take money out, etc.), drivers (with monitoring, you be careful, etc.).
104、将提取得到的所述行为特征和所述语音特征进行归一化处理；104. Perform normalization processing on the extracted behavior feature and the voice feature.
可以理解的是，根据从上述步骤的Up body检测子、HOG描述子和Tracklet描述子提取得到的行为特征和语音特征，通常以局部直方图的形式出现。在步骤104中，可以把这些局部直方图在图像的更大的区间中进行对比归一化。比如，可以先通过计算各局部直方图在这个区间中的密度，然后根据这个密度值对区间中的各个方格单元做归一化处理。It can be understood that the behavioral features and the speech features extracted from the Up body detector, the HOG descriptor and the Tracklet descriptor extracted from the above steps usually appear in the form of a partial histogram. In step 104, these partial histograms can be normalized in a larger interval of the image. For example, the density of each local histogram in this interval can be calculated first, and then the individual grid cells in the interval are normalized according to the density value.
105、将归一化后的所述行为特征和所述语音特征投入预训练完成的犯罪活动分类器进行分类识别，得到输出的分类识别结果；105. Perform, after the normalized behavior feature and the voice feature into a pre-trained criminal activity classifier, classify and identify, and obtain an output classification recognition result;
在对行为特征和语音特征归一化后，将这些行为特征和语音特征投入预训练完成的犯罪活动分类器进行分类识别，得到输出的分类识别结果，其中，所述分类识别结果为存在犯罪活动或者不存在犯罪活动。After the behavioral features and the phonetic features are normalized, the behavioral features and the phonetic features are put into the pre-trained criminal activity classifier for classification and recognition, and the output classification recognition result is obtained, wherein the classification recognition result is the existence of criminal activities. Or there is no criminal activity.
可以理解的是，上述的犯罪活动分类器是预先经过大量的训练样本训练完成得到的，可以对特征融合的行为特征和语音特征进行分类识别，对当前的视频图像中是否存在犯罪活动行为作出实时判断，并输出相应的分类识别结果。It can be understood that the above-mentioned criminal activity classifier is obtained by training a large number of training samples in advance, and can classify and recognize the behavioral features and the speech features of the feature fusion, and real-time whether the criminal activity behavior exists in the current video image. Judge and output the corresponding classification recognition result.
其中，犯罪活动分类器的预训练过程将在下述内容中进行详细描述。Among them, the pre-training process of the criminal activity classifier will be described in detail below.
106、若所述分类识别结果为存在犯罪活动，则发出告警信息。106. If the classification and recognition result is that there is a criminal activity, the alarm information is sent.
在本实施例中，若所述分类识别结果为存在犯罪活动，则可以认为车辆中司机所在位置正在发生犯罪活动，此时应当及时发出告警信息。具体地，发出告警信息的形式可以有多种，比如，向犯罪分子发出提示语音“请立即停止你的违法行为”；或者，不向犯罪分子作出任何提示，以避免犯罪分子作出过激行为，执行主体悄然向最近的公安***进行报警，等待警察前来处理；等等。In this embodiment, if the classification and recognition result is that there is a criminal activity, it can be considered that a criminal activity is occurring at the location of the driver in the vehicle, and the alarm information should be issued in time. Specifically, there may be multiple forms of alerting information, such as issuing a prompt to the criminals, "Please stop your illegal actions immediately"; or, don't make any prompts to the criminals to avoid the criminals making excessive acts and performing The subject quietly alerted the nearest public security system, waiting for the police to come to handle it; and so on.
优选地，车辆上可以安装有GPS定位模块，从而执行主体可以实时获取到辆的实时定位信息；然后，在需要发出告警信息时，将预设的报警信息、所述实时定位信息和实时采集的所述视频图像、音频信息发送至指定的告警终端。这里说的告警终端可以是公安局的报警服务器。其中，与报警信息一并发送的视频图像和音频信息可以用作证明犯罪行为的证据，以便于执法人员对犯罪分子进行定罪处罚。Preferably, the GPS positioning module can be installed on the vehicle, so that the execution body can acquire the real-time positioning information of the vehicle in real time; then, when the alarm information needs to be sent, the preset alarm information, the real-time positioning information, and the real-time collection information are collected. The video image and audio information are sent to a designated alarm terminal. The alarm terminal mentioned here can be the alarm server of the Public Security Bureau. Among them, the video image and audio information sent together with the alarm information can be used as evidence to prove the criminal behavior, so that the law enforcement officers can punish the criminals.
下面，将对犯罪活动分类器的预训练过程进行详细介绍。如图3所示，所述犯罪活动分类器通过以下步骤预先训练得到：In the following, the pre-training process of the criminal activity classifier will be described in detail. As shown in FIG. 3, the criminal activity classifier is pre-trained by the following steps:
301、预先收集训练组样本，所述训练组样本包括用于训练的多组第一视频图像和第一音频信息；301, pre-collecting training group samples, where the training group samples include multiple sets of first video images and first audio information for training;
302、预先标记所述训练组样本中各组视频图像和音频信息对应的标准识别结果，标准识别结果为存在犯罪活动或者不存在犯罪活动；302. Pre-marking a standard recognition result corresponding to each group of video images and audio information in the training group sample, and the standard recognition result is that there is a criminal activity or no criminal activity;
303、提取所述第一视频图像中感兴趣区域的第一行为特征；303. Extract a first behavior feature of the region of interest in the first video image.
304、提取所述第一音频信息中的第一语音特征；304. Extract a first voice feature in the first audio information.
305、将提取得到的所述第一行为特征和所述第一语音特征进行归一化处理；305. Perform normalization processing on the extracted first behavior feature and the first voice feature.
306、将所述训练组样本中归一化后的所述第一行为特征和所述第一语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；306. The first behavior feature normalized in the training group sample and the first speech feature are classified into a criminal activity classifier for classification and identification, to obtain an output classification recognition result;
307、将输出的分类识别结果与所述训练组样本对应的标准识别结果进行对比，得到本次训练所述犯罪活动分类器输出结果的准确率；307. Compare the output classification recognition result with the standard recognition result corresponding to the training group sample, and obtain an accuracy rate of the output result of the criminal activity classifier in the training.
308、判断本次训练的准确率是否低于预设阈值，若是，则执行步骤309，若否，则执行步骤310；308, determining whether the accuracy of the current training is lower than a preset threshold, and if so, executing step 309, and if not, executing step 310;
309、调整所述犯罪活动分类器中的分类器参数，返回执行步骤303，开始下一次训练；309. Adjust the classifier parameter in the criminal activity classifier, and return to step 303 to start the next training;
310、确定所述犯罪活动分类器训练完成。310. Determine that the criminal activity classifier training is completed.
对于步骤301和步骤302，在训练所述犯罪活动分类器之前，需要预先收集用于训练的多组视频图像和音频信息，即上述的第一视频图像和第一音频信息。这些第一视频图像和第一音频信息是成对出现的，同一组第一视频图像和第一音频信息是从同一车辆的同一时间收集得到。其中，收集的训练组样本中的数据量越大，则对犯罪活动分类器的训练效果越好。For steps 301 and 302, a plurality of sets of video images and audio information for training, that is, the first video image and the first audio information described above, need to be collected in advance before training the criminal activity classifier. The first video image and the first audio information appear in pairs, and the same set of first video images and first audio information are collected at the same time from the same vehicle. Among them, the greater the amount of data in the collected training group samples, the better the training effect on the criminal activity classifier.
在收集到这些训练组样本之后，还需要标记这些训练组样本中每组视频图像和音频信息对应的标准识别结果，即哪些组的视频图像和音频信息是采集自存在犯罪活动的现场，哪些组的视频图像和音频信息是采集自不存在犯罪活动的现场。After collecting these training group samples, it is also necessary to mark the standard recognition results corresponding to each group of video images and audio information in the training group samples, that is, which groups of video images and audio information are collected from the scene where the criminal activities exist, and which groups The video images and audio information are collected from the scene where there is no criminal activity.
上述步骤303~305与上述步骤102~104的内容相似，原理基本相同，此处不再赘述。The foregoing steps 303 to 305 are similar to the contents of the foregoing steps 102 to 104, and the principles are basically the same, and details are not described herein again.
对于上述步骤306，在本次训练中，将训练组样本中归一化后的所述第一行为特征和所述第一语音特征投入犯罪活动分类器进行分类识别，由于犯罪活动分类器此时尚未训练完成，因此其输出的分类识别结果与标准的识别结果会存在偏差。For the above step 306, in the current training, the normalized first behavior feature and the first speech feature in the training group sample are classified into a criminal activity classifier for classification and recognition, because the criminal activity classifier is fashionable. The training is not completed, so the classification recognition result of the output will deviate from the standard recognition result.
对于上述步骤307，可以理解的是，由于训练组样本中每组视频图像和音频信息均标记有对应的标准识别结果，因此，可以将犯罪活动分类器输出的结果与预先标记的标准识别结果进行对比，得知本次训练中该犯罪活动分类器输出结果的准确率。比如，假设其中3组样本的标准识别结果依次为存在、存在、不存在犯罪活动，而这3组样本输入犯罪活动分类器后输出的分类识别结果依次为不存在、存在、不存在犯罪活动，对比可知，在3组样本的本次训练中，识别准确率为66.7%。For the above step 307, it can be understood that since each set of video image and audio information in the training group sample is marked with a corresponding standard recognition result, the result output by the criminal activity classifier can be performed with the pre-marked standard recognition result. In contrast, the accuracy of the output of the criminal activity classifier in this training is known. For example, suppose the standard recognition results of three groups of samples are existence, existence, and non-existent criminal activities, and the classification recognition results output by the three groups of samples after inputting the criminal activity classifier are non-existent, exist, and non-existent criminal activities. The comparison shows that in this training of the three groups of samples, the recognition accuracy rate is 66.7%.
对于上述步骤308~310，在得到本次训练时输出结果的准确率后，可以通过验证该准确率是否满足要求来确定该犯罪活动分类器是否训练完成。若本次训练的准确率低于预设阈值，则调整所述犯罪活动分类器中的分类器参数，返回执行步骤303，开始下一次训练；若本次训练的准确率高于或等于预设阈值，则确定所述犯罪活动分类器训练完成。For the above steps 308-310, after obtaining the accuracy of the output result in the current training, it can be determined whether the criminal activity classifier is completed by verifying whether the accuracy meets the requirement. If the accuracy of the training is lower than the preset threshold, adjust the classifier parameter in the crime activity classifier, and return to step 303 to start the next training; if the accuracy of the training is higher than or equal to the preset The threshold determines that the criminal activity classifier training is completed.
其中，上述分类器参数是指该犯罪活动分类器中预置的各个权值、阈值等属性数据，对于不同类型的分类器来说，这些分类器参数也会有所区别。通过调整分类器参数，可以使得犯罪活动分类器更加善于判断哪些视频图像和音频信息中存在犯罪活动，由于犯罪活动分类器输出结果的准确率低于预设阈值，因此需要返回执行步骤303，继续采用训练组样本对犯罪活动分类器进行下一次训练。上述的预设阈值可以根据具体情况进行设定，比如设定为95%。The classifier parameters refer to attribute data such as weights and thresholds preset in the criminal activity classifier, and the classifier parameters may be different for different types of classifiers. By adjusting the classifier parameters, the criminal activity classifier can be better judged which criminal images and audio information have criminal activities. Since the accuracy of the crime activity classifier output is lower than the preset threshold, it is necessary to return to step 303 and continue. The training group sample is used for the next training of the criminal activity classifier. The above preset threshold can be set according to specific conditions, for example, set to 95%.
对于上述步骤310，为了更进一步验证犯罪活动分类器的训练完成程度，还可以准备一套不同于训练组样本的测试组样本对犯罪活动分类器进行测试、检验。在测试之前，可以先预先收集测试组样本，所述测试组样本包括用于测试的多组第二视频图像和第二音频信息；然后，预先标记所述测试组样本中各组视频图像和音频信息对应的标准识别结果。如图4所示，在确定所述犯罪活动分类器训练完成之前，所述针对犯罪活动的告警方法还包括：For the above step 310, in order to further verify the degree of training completion of the criminal activity classifier, a test group sample different from the training group sample may be prepared to test and test the criminal activity classifier. Before testing, a test group sample may be collected in advance, the test set sample including a plurality of sets of second video images and second audio information for testing; and then, each set of video images and audio in the test set samples is pre-marked The standard identification result corresponding to the information. As shown in FIG. 4, before determining that the criminal activity classifier training is completed, the alarming method for criminal activities further includes:
401、提取所述第二视频图像中感兴趣区域的第二行为特征；401. Extract a second behavior feature of the region of interest in the second video image.
402、提取所述第二音频信息中的第二语音特征；402. Extract a second voice feature in the second audio information.
403、将提取得到的所述第二行为特征和所述第二语音特征进行归一化处理；403. Perform normalization processing on the extracted second behavior feature and the second speech feature.
404、将所述测试组样本中归一化后的所述第二行为特征和所述第二语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；404. The second behavior feature normalized in the test group sample and the second speech feature are classified into a criminal activity classifier for classification and identification, to obtain an output classification recognition result;
405、将输出的分类识别结果与所述测试组样本对应的标准识别结果进行对比，得到本次测试中所述犯罪活动分类器输出结果的测试准确率；405. Compare the output classification identification result with the standard identification result corresponding to the test group sample, and obtain a test accuracy rate of the criminal activity classifier output result in the test.
406、判断本次测试的准确率是否低于预设测试阈值，若是，则执行步骤407，若否，则执行步骤310；406, determining whether the accuracy of the current test is lower than the preset test threshold, and if so, executing step 407, and if not, executing step 310;
407、调整所述犯罪活动分类器中的分类器参数，返回开始下一次训练。407. Adjust the classifier parameter in the criminal activity classifier and return to start the next training.
上述步骤401~404与上述步骤303~306的内容相似，原理基本相同，此处不再赘述。The foregoing steps 401 to 404 are similar to the contents of the foregoing steps 303 to 306, and the principles are basically the same, and details are not described herein again.
对于上述步骤405，大致内容与步骤307基本相同，不同之处在于，在步骤405中，得到的测试准确率用于评估犯罪活动分类器的训练完成程度的，由于测试用的测试组样本有别于训练组样本，其对于该犯罪活动分类器来说更为陌生，因此评估的效果也会好于训练阶段的评估效果。For the above step 405, the approximate content is substantially the same as step 307, except that in step 405, the obtained test accuracy rate is used to evaluate the degree of completion of the training of the criminal activity classifier, because the test test group samples are different. In the training group sample, it is more strange to the criminal activity classifier, so the evaluation effect will be better than the evaluation effect in the training phase.
对于上述步骤406和407，若本次测试的准确率低于预设测试阈值，则可以认为该犯罪活动分类器仍未满足实际使用的需求，训练仍未完成，从而可以调整所述犯罪活动分类器中的分类器参数，返回开始下一次训练。反之，若本次测试的准确率高于或等于预设测试阈值，则认为该犯罪活动分类器已满足实际使用的需求，训练完成，执行步骤310确定所述犯罪活动分类器训练完成。For the above steps 406 and 407, if the accuracy of the test is lower than the preset test threshold, it can be considered that the criminal activity classifier still does not meet the actual use requirement, and the training is still not completed, so that the criminal activity classification can be adjusted. The classifier parameter in the device returns to start the next training. On the other hand, if the accuracy of the test is higher than or equal to the preset test threshold, the criminal activity classifier is considered to have met the actual use requirement, and the training is completed. Step 310 is performed to determine that the criminal activity classifier training is completed.
本发明实施例中，首先，实时采集车辆上司机所处位置的视频图像和音频信息；然后，提取所述视频图像中感兴趣区域的行为特征；提取采集到的所述音频信息中的语音特征；接着，将提取得到的所述行为特征和所述语音特征进行归一化处理；再之，将归一化后的所述行为特征和所述语音特征投入预训练完成的犯罪活动分类器进行分类识别，得到输出的分类识别结果，所述分类识别结果为存在犯罪活动或者不存在犯罪活动；若所述分类识别结果为存在犯罪活动，则发出告警信息。在本发明实施例中，当犯罪活动发生时，通过采集车辆上司机所处位置的视频图像和音频信息，将这些视频图像和音频信息投入到犯罪活动分类器中进行分类识别，实现犯罪活动的自动识别并发出告警信息，无需司机主动做出任何动作，减少肇事者采取过激行为的可能性，同时减轻被害人（司机）的负担，可以做到安全有效的及时告警。In the embodiment of the present invention, first, a video image and audio information of a location of a driver on a vehicle are collected in real time; then, a behavior feature of the region of interest in the video image is extracted; and the collected voice feature in the audio information is extracted. And then normalizing the extracted behavioral features and the speech features; and then performing the normalized behavioral features and the speech features into a pre-trained criminal activity classifier The classification identification obtains the output classification recognition result, and the classification identification result is that there is a criminal activity or no criminal activity; if the classification recognition result is that there is a criminal activity, the alarm information is sent. In the embodiment of the present invention, when a criminal activity occurs, the video image and the audio information of the location of the driver on the vehicle are collected, and the video image and the audio information are put into a criminal activity classifier for classification and recognition, thereby realizing criminal activities. Automatically identify and issue alarm information, without the driver taking the initiative to do any action, reducing the possibility of the perpetrators taking excessive behavior, while reducing the burden on the victim (driver), can achieve safe and effective timely warning.
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本发明实施例的实施过程构成任何限定。It should be understood that the size of the sequence of the steps in the above embodiments does not imply a sequence of executions, and the order of execution of the processes should be determined by its function and internal logic, and should not be construed as limiting the implementation of the embodiments of the present invention.
上面主要描述了一种针对犯罪活动的告警方法，下面将对一种针对犯罪活动的告警装置进行详细描述。The above mainly describes an alarm method for criminal activities, and a warning device for criminal activities will be described in detail below.
图5示出了本发明实施例中一种针对犯罪活动的告警装置一个实施例结构图。FIG. 5 is a structural diagram showing an embodiment of an alarm device for criminal activities in an embodiment of the present invention.
本实施例中，一种针对犯罪活动的告警装置包括：In this embodiment, an alerting device for criminal activities includes:
实时采集模块501，用于实时采集车辆上司机所处位置的视频图像和音频信息；The real-time collection module 501 is configured to collect video images and audio information of a location of the driver on the vehicle in real time;
行为特征提取模块502，用于提取所述视频图像中感兴趣区域的行为特征；The behavior feature extraction module 502 is configured to extract behavior characteristics of the region of interest in the video image;
语音特征提取模块503，用于提取采集到的所述音频信息中的语音特征；The voice feature extraction module 503 is configured to extract voice features in the collected audio information.
特征归一模块504，用于将提取得到的所述行为特征和所述语音特征进行归一化处理；a feature normalization module 504, configured to normalize the extracted behavior feature and the voice feature;
分类识别模块505，用于将归一化后的所述行为特征和所述语音特征投入预训练完成的犯罪活动分类器进行分类识别，得到输出的分类识别结果，所述分类识别结果为存在犯罪活动或者不存在犯罪活动；The classification and identification module 505 is configured to classify and identify the normalized behavior feature and the voice feature into a pre-trained criminal activity classifier to obtain an output classification recognition result, where the classification recognition result is a crime Activity or no criminal activity;
告警模块506，用于若所述分类识别结果为存在犯罪活动，则发出告警信息。The alarm module 506 is configured to send an alarm message if the classification and recognition result is that a criminal activity exists.
进一步地，所述犯罪活动分类器可以通过以下模块预先训练得到：Further, the criminal activity classifier can be pre-trained by the following modules:
训练样本采集模块，用于预先收集训练组样本，所述训练组样本包括用于训练的多组第一视频图像和第一音频信息；a training sample collection module, configured to pre-collect a training group sample, where the training group sample includes a plurality of sets of first video images and first audio information for training;
训练样本标记模块，用于预先标记所述训练组样本中各组视频图像和音频信息对应的标准识别结果，标准识别结果为存在犯罪活动或者不存在犯罪活动；a training sample marking module, configured to pre-mark standard identification results corresponding to each group of video images and audio information in the training group sample, and the standard recognition result is that there is criminal activity or no criminal activity;
第一行为特征模块，用于提取所述第一视频图像中感兴趣区域的第一行为特征；a first behavior feature module, configured to extract a first behavior feature of the region of interest in the first video image;
第一语音特征模块，用于提取所述第一音频信息中的第一语音特征；a first voice feature module, configured to extract a first voice feature in the first audio information;
第一归一处理模块，用于将提取得到的所述第一行为特征和所述第一语音特征进行归一化处理；a first normalization processing module, configured to normalize the extracted first behavior feature and the first speech feature;
第一分类器识别模块，用于将所述训练组样本中归一化后的所述第一行为特征和所述第一语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；a first classifier identification module, configured to classify the first behavior feature and the first voice feature normalized in the training group sample into a criminal activity classifier to obtain an output classification recognition result;
第一对比模块，用于将输出的分类识别结果与所述训练组样本对应的标准识别结果进行对比，得到本次训练所述犯罪活动分类器输出结果的准确率；a first comparison module, configured to compare the output classification recognition result with a standard recognition result corresponding to the training group sample, to obtain an accuracy rate of the output of the criminal activity classifier in the training;
第一参数调整模块，用于若本次训练的准确率低于预设阈值，则调整所述犯罪活动分类器中的分类器参数，返回触发所述第一行为特征模块，开始下一次训练；a first parameter adjustment module, configured to: if the accuracy of the current training is lower than a preset threshold, adjust a classifier parameter in the crime activity classifier, return to trigger the first behavior feature module, and start a next training;
训练完成确定模块，用于若本次训练的准确率高于或等于预设阈值，则确定所述犯罪活动分类器训练完成。The training completion determining module is configured to determine that the criminal activity classifier training is completed if the accuracy of the current training is higher than or equal to the preset threshold.
进一步地，所述针对犯罪活动的告警装置还可以包括：Further, the alarm device for criminal activities may further include:
测试样本采集模块，用于预先收集测试组样本，所述测试组样本包括用于测试的多组第二视频图像和第二音频信息；a test sample collection module, configured to pre-collect test group samples, the test set samples including a plurality of sets of second video images and second audio information for testing;
测试样本标记模块，用于预先标记所述测试组样本中各组视频图像和音频信息对应的标准识别结果；a test sample marking module, configured to pre-mark standard identification results corresponding to each group of video images and audio information in the test group samples;
在所述训练完成确定模块确定所述犯罪活动分类器训练完成之前，还可以触发以下模块：The following modules may also be triggered before the training completion determination module determines that the criminal activity classifier training is completed:
第二行为特征模块，用于提取所述第二视频图像中感兴趣区域的第二行为特征；a second behavior feature module, configured to extract a second behavior feature of the region of interest in the second video image;
第二语音特征模块，用于提取所述第二音频信息中的第二语音特征；a second voice feature module, configured to extract a second voice feature in the second audio information;
第二归一处理模块，用于将提取得到的所述第二行为特征和所述第二语音特征进行归一化处理；a second normalization processing module, configured to normalize the extracted second behavior feature and the second speech feature;
第二分类器识别模块，用于将所述测试组样本中归一化后的所述第二行为特征和所述第二语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；a second classifier identification module, configured to classify the second behavior feature and the second voice feature normalized in the test group sample into a criminal activity classifier to obtain an output classification recognition result;
第二对比模块，用于将输出的分类识别结果与所述测试组样本对应的标准识别结果进行对比，得到本次测试中所述犯罪活动分类器输出结果的测试准确率；a second comparison module, configured to compare the output classification recognition result with the standard recognition result corresponding to the test group sample, to obtain a test accuracy rate of the criminal activity classifier output result in the test;
第二参数调整模块，用于若本次测试的测试准确率低于预设测试阈值，则调整所述犯罪活动分类器中的分类器参数，返回触发所述第一行为特征模块，开始下一次训练；a second parameter adjustment module, configured to adjust a classifier parameter in the crime activity classifier if the test accuracy rate of the test is lower than a preset test threshold, and return to trigger the first behavior feature module to start the next time training;
训练完成确定模块，用于若本次测试的测试准确率高于或等于预设测试阈值，则触发所述训练完成确定模块来确定所述犯罪活动分类器训练完成。The training completion determining module is configured to trigger the training completion determining module to determine that the criminal activity classifier training is completed if the test accuracy rate of the current test is higher than or equal to the preset test threshold.
进一步地，所述行为特征包括视觉特征和动作轨迹特征；Further, the behavioral features include visual features and motion trajectory features;
所述行为特征提取模块可以包括：The behavior feature extraction module may include:
区域提取单元，用于提取所述视频图像中感兴趣区域；a region extracting unit, configured to extract a region of interest in the video image;
兴趣点检测单元，用于检测得到在所述感兴趣区域中的兴趣点；a point of interest detecting unit, configured to detect a point of interest in the region of interest;
特征描述单元，用于采用Tracklet描述子将所述感兴趣区域中的兴趣点描述成视觉特征和动作轨迹特征。A feature description unit is configured to describe a point of interest in the region of interest as a visual feature and a motion trajectory feature by using a Tracklet descriptor.
进一步地，所述告警模块可以包括：Further, the alarm module may include:
定位信息获取单元，用于获取所述车辆的实时定位信息；a positioning information acquiring unit, configured to acquire real-time positioning information of the vehicle;
信息发送单元，用于将预设的报警信息、所述实时定位信息和实时采集的所述视频图像、音频信息发送至指定的告警终端。The information sending unit is configured to send the preset alarm information, the real-time positioning information, and the video image and audio information collected in real time to the designated alarm terminal.
图6是本发明一实施例提供的针对犯罪活动的告警服务器的示意图。如图6所示，该实施例的针对犯罪活动的告警服务器6包括：处理器60、存储器61以及存储在所述存储器61中并可在所述处理器60上运行的计算机程序62，例如执行上述针对犯罪活动的告警方法的程序。所述处理器60执行所述计算机程序62时实现上述各个针对犯罪活动的告警方法实施例中的步骤，例如图1所示的步骤101至106。或者，所述处理器60执行所述计算机程序62时实现上述各装置实施例中各模块/单元的功能，例如图5所示模块501至506的功能。FIG. 6 is a schematic diagram of an alarm server for criminal activities according to an embodiment of the present invention. As shown in FIG. 6, the alarm server 6 for criminal activities of this embodiment includes a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and operable on the processor 60, for example, executing The above procedure for the warning method of criminal activity. The processor 60 executes the computer program 62 to implement the steps in the various embodiments of the alerting method for criminal activities described above, such as steps 101 through 106 shown in FIG. Alternatively, when the processor 60 executes the computer program 62, the functions of the modules/units in the above various device embodiments are implemented, such as the functions of the modules 501 to 506 shown in FIG.
示例性的，所述计算机程序62可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器61中，并由所述处理器60执行，以完成本发明。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述所述计算机程序62在所述针对犯罪活动的告警服务器6中的执行过程。Illustratively, the computer program 62 can be partitioned into one or more modules/units that are stored in the memory 61 and executed by the processor 60 to complete this invention. The one or more modules/units may be a series of computer program instruction segments capable of performing a particular function, the instruction segments being used to describe the execution of the computer program 62 in the alerting server 6 for criminal activity.
所述针对犯罪活动的告警服务器6可以是本地服务器、云端服务器等计算设备。所述针对犯罪活动的告警服务器可包括，但不仅限于，处理器60、存储器61。本领域技术人员可以理解，图6仅仅是针对犯罪活动的告警服务器6的示例，并不构成对针对犯罪活动的告警服务器6的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述针对犯罪活动的告警服务器还可以包括输入输出设备、网络接入设备、总线等。The alarm server 6 for criminal activities may be a computing device such as a local server or a cloud server. The alert server for criminal activity may include, but is not limited to, processor 60, memory 61. It will be understood by those skilled in the art that FIG. 6 is merely an example of an alert server 6 for criminal activities, and does not constitute a limitation to the alert server 6 for criminal activities, may include more or fewer components than illustrated, or a combination Certain components, or different components, such as the alerting server for criminal activity, may also include input and output devices, network access devices, buses, and the like.
所述处理器60可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器 (Digital Signal Processor，DSP)、专用集成电路 (Application Specific Integrated Circuit，ASIC)、现成可编程门阵列 (Field-Programmable Gate Array，FPGA) 或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。The processor 60 can be a central processing unit (Central Processing Unit, CPU), can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
所述存储器61可以是所述针对犯罪活动的告警服务器6的内部存储单元，例如针对犯罪活动的告警服务器6的硬盘或内存。所述存储器61也可以是所述针对犯罪活动的告警服务器6的外部存储设备，例如所述针对犯罪活动的告警服务器6上配备的插接式硬盘，智能存储卡（Smart Media Card, SMC），安全数字（Secure Digital, SD）卡，闪存卡（Flash Card）等。进一步地，所述存储器61还可以既包括所述针对犯罪活动的告警服务器6的内部存储单元也包括外部存储设备。所述存储器61用于存储所述计算机程序以及所述针对犯罪活动的告警服务器所需的其他程序和数据。所述存储器61还可以用于暂时地存储已经输出或者将要输出的数据。The memory 61 may be an internal storage unit of the alert server 6 for criminal activity, such as a hard disk or memory for the alert server 6 of criminal activity. The memory 61 may also be an external storage device of the alarm server 6 for criminal activities, such as a plug-in hard disk equipped with a smart media card (SMC) provided on the alarm server 6 for criminal activities. Secure Digital (SD) card, flash card (Flash Card) and so on. Further, the memory 61 may also include both the internal storage unit of the alarm server 6 for criminal activities and an external storage device. The memory 61 is used to store the computer program and other programs and data required by the alert server for criminal activity. The memory 61 can also be used to temporarily store data that has been output or is about to be output.
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的***，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。In the above embodiments, the descriptions of the various embodiments are different, and the parts that are not detailed or described in a certain embodiment can be referred to the related descriptions of other embodiments.
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各实施例的模块、单元和/或方法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。Those of ordinary skill in the art will appreciate that the modules, units, and/or method steps of the various embodiments described in connection with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.
在本申请所提供的几个实施例中，应该理解到，所揭露的***，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个***，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，Random Access Memory）、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the present invention implements all or part of the processes in the foregoing embodiments, and may also be completed by a computer program to instruct related hardware. The computer program may be stored in a computer readable storage medium. The steps of the various method embodiments described above may be implemented when the program is executed by the processor. Wherein, the computer program comprises computer program code, which may be in the form of source code, object code form, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard drive, a magnetic disk, an optical disk, a computer memory, a read only memory (ROM, Read-Only) Memory), random access memory (RAM, Random) Access Memory), electrical carrier signals, telecommunications signals, and software distribution media. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in a jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, computer readable media Does not include electrical carrier signals and telecommunication signals.
以上所述，以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to be limiting; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that The technical solutions described in the embodiments are modified, or the equivalents of the technical features are replaced by the equivalents of the technical solutions of the embodiments of the present invention.

Claims (10)

  1. 一种针对犯罪活动的告警方法，其特征在于，包括：A warning method for criminal activities, characterized in that it comprises:
    实时采集车辆上司机所处位置的视频图像和音频信息；Real-time collection of video images and audio information of the location of the driver on the vehicle;
    提取所述视频图像中感兴趣区域的行为特征；Extracting behavioral characteristics of the region of interest in the video image;
    提取采集到的所述音频信息中的语音特征；Extracting the collected voice features in the audio information;
    将提取得到的所述行为特征和所述语音特征进行归一化处理；And normalizing the extracted behavior characteristics and the voice features;
    将归一化后的所述行为特征和所述语音特征投入预训练完成的犯罪活动分类器进行分类识别，得到输出的分类识别结果，所述分类识别结果为存在犯罪活动或者不存在犯罪活动；And classifying the normalized behavioral feature and the speech feature into a pre-trained criminal activity classifier to obtain an output classification recognition result, where the classification recognition result is that there is a criminal activity or no criminal activity;
    若所述分类识别结果为存在犯罪活动，则发出告警信息。If the classification and recognition result is that there is a criminal activity, an alarm message is sent.
  2. 根据权利要求1所述的针对犯罪活动的告警方法，其特征在于，所述犯罪活动分类器通过以下步骤预先训练得到：The method for alerting a criminal activity according to claim 1, wherein the criminal activity classifier is pre-trained by the following steps:
    预先收集训练组样本，所述训练组样本包括用于训练的多组第一视频图像和第一音频信息；Pre-collecting training group samples, the training group samples including a plurality of sets of first video images and first audio information for training;
    预先标记所述训练组样本中各组视频图像和音频信息对应的标准识别结果，标准识别结果为存在犯罪活动或者不存在犯罪活动；Pre-marking the standard recognition result corresponding to each group of video images and audio information in the training group sample, and the standard recognition result is that there is criminal activity or no criminal activity;
    提取所述第一视频图像中感兴趣区域的第一行为特征；Extracting a first behavior characteristic of the region of interest in the first video image;
    提取所述第一音频信息中的第一语音特征；Extracting a first voice feature in the first audio information;
    将提取得到的所述第一行为特征和所述第一语音特征进行归一化处理；And normalizing the extracted first behavior feature and the first speech feature;
    将所述训练组样本中归一化后的所述第一行为特征和所述第一语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；And normalizing the first behavior feature and the first speech feature in the training group sample into a criminal activity classifier for classification and identification, to obtain an output classification recognition result;
    将输出的分类识别结果与所述训练组样本对应的标准识别结果进行对比，得到本次训练所述犯罪活动分类器输出结果的准确率；Comparing the output classification recognition result with the standard recognition result corresponding to the training group sample, and obtaining the accuracy of the output result of the criminal activity classifier in the training;
    若本次训练的准确率低于预设阈值，则调整所述犯罪活动分类器中的分类器参数，返回执行所述提取所述第一视频图像中感兴趣区域的第一行为特征的步骤，开始下一次训练；If the accuracy of the current training is lower than a preset threshold, adjusting the classifier parameter in the criminal activity classifier, and returning to performing the step of extracting the first behavior feature of the region of interest in the first video image, Start the next training;
    若本次训练的准确率高于或等于预设阈值，则确定所述犯罪活动分类器训练完成。If the accuracy of the training is higher than or equal to the preset threshold, it is determined that the criminal activity classifier training is completed.
  3. 根据权利要求2所述的针对犯罪活动的告警方法，其特征在于，还包括：The method for alerting a criminal activity according to claim 2, further comprising:
    预先收集测试组样本，所述测试组样本包括用于测试的多组第二视频图像和第二音频信息；Collecting test group samples in advance, the test set samples including a plurality of sets of second video images and second audio information for testing;
    预先标记所述测试组样本中各组视频图像和音频信息对应的标准识别结果；Pre-marking the standard recognition result corresponding to each group of video images and audio information in the test group sample;
    在确定所述犯罪活动分类器训练完成之前，所述针对犯罪活动的告警方法还包括：Before determining that the criminal activity classifier training is completed, the warning method for criminal activities further includes:
    提取所述第二视频图像中感兴趣区域的第二行为特征；Extracting a second behavior characteristic of the region of interest in the second video image;
    提取所述第二音频信息中的第二语音特征；Extracting a second voice feature in the second audio information;
    将提取得到的所述第二行为特征和所述第二语音特征进行归一化处理；And normalizing the extracted second behavior feature and the second speech feature;
    将所述测试组样本中归一化后的所述第二行为特征和所述第二语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；And normalizing the second behavior feature and the second speech feature normalized in the test group sample into a criminal activity classifier for classification and identification, to obtain an output classification recognition result;
    将输出的分类识别结果与所述测试组样本对应的标准识别结果进行对比，得到本次测试中所述犯罪活动分类器输出结果的测试准确率；Comparing the output classification recognition result with the standard recognition result corresponding to the test group sample, and obtaining the test accuracy rate of the criminal activity classifier output result in the test;
    若本次测试的测试准确率低于预设测试阈值，则调整所述犯罪活动分类器中的分类器参数，返回执行所述提取所述第一视频图像中感兴趣区域的第一行为特征的步骤，开始下一次训练；If the test accuracy of the test is lower than a preset test threshold, adjusting the classifier parameter in the crime activity classifier, returning to perform performing the extracting the first behavior feature of the region of interest in the first video image Step, start the next training;
    若本次测试的测试准确率高于或等于预设测试阈值，则执行所述确定所述犯罪活动分类器训练完成的步骤。If the test accuracy rate of the test is higher than or equal to the preset test threshold, the step of determining that the criminal activity classifier training is completed is performed.
  4. 根据权利要求1所述的针对犯罪活动的告警方法，其特征在于，所述行为特征包括视觉特征和动作轨迹特征；The method for alerting a criminal activity according to claim 1, wherein the behavior characteristic comprises a visual feature and an action trajectory feature;
    所述提取所述视频图像中感兴趣区域的行为特征包括：The extracting behavior characteristics of the region of interest in the video image includes:
    提取所述视频图像中感兴趣区域；Extracting a region of interest in the video image;
    检测得到在所述感兴趣区域中的兴趣点；Detecting points of interest in the region of interest;
    采用Tracklet描述子将所述感兴趣区域中的兴趣点描述成视觉特征和动作轨迹特征。The points of interest in the region of interest are described as visual features and motion trajectory features using a Tracklet descriptor.
  5. 根据权利要求1至4中任一项所述的针对犯罪活动的告警方法，其特征在于，所述发出告警信息包括：The method for alerting a criminal activity according to any one of claims 1 to 4, wherein the issuing the alarm information comprises:
    获取所述车辆的实时定位信息；Obtaining real-time positioning information of the vehicle;
    将预设的报警信息、所述实时定位信息和实时采集的所述视频图像、音频信息发送至指定的告警终端。The preset alarm information, the real-time positioning information, and the video image and audio information collected in real time are sent to a designated alarm terminal.
  6. 一种针对犯罪活动的告警装置，其特征在于，包括：An alarm device for criminal activities, characterized in that it comprises:
    实时采集模块，用于实时采集车辆上司机所处位置的视频图像和音频信息；Real-time acquisition module for real-time collection of video images and audio information of the location of the driver on the vehicle;
    行为特征提取模块，用于提取所述视频图像中感兴趣区域的行为特征；a behavior feature extraction module, configured to extract behavior characteristics of the region of interest in the video image;
    语音特征提取模块，用于提取采集到的所述音频信息中的语音特征；a voice feature extraction module, configured to extract voice features in the collected audio information;
    特征归一模块，用于将提取得到的所述行为特征和所述语音特征进行归一化处理；a feature normalization module, configured to normalize the extracted behavior feature and the voice feature;
    分类识别模块，用于将归一化后的所述行为特征和所述语音特征投入预训练完成的犯罪活动分类器进行分类识别，得到输出的分类识别结果，所述分类识别结果为存在犯罪活动或者不存在犯罪活动；a classification identification module, configured to classify and identify the normalized behavior feature and the voice feature into a pre-trained criminal activity classifier, to obtain an output classification recognition result, where the classification recognition result is a criminal activity Or there is no criminal activity;
    告警模块，用于若所述分类识别结果为存在犯罪活动，则发出告警信息。The alarm module is configured to send an alarm message if the classification and recognition result is that a criminal activity exists.
  7. 根据权利要求6所述的针对犯罪活动的告警装置，其特征在于，所述犯罪活动分类器通过以下模块预先训练得到：The alarm device for criminal activities according to claim 6, wherein the criminal activity classifier is pre-trained by the following modules:
    训练样本采集模块，用于预先收集训练组样本，所述训练组样本包括用于训练的多组第一视频图像和第一音频信息；a training sample collection module, configured to pre-collect a training group sample, where the training group sample includes a plurality of sets of first video images and first audio information for training;
    训练样本标记模块，用于预先标记所述训练组样本中各组视频图像和音频信息对应的标准识别结果，标准识别结果为存在犯罪活动或者不存在犯罪活动；a training sample marking module, configured to pre-mark standard identification results corresponding to each group of video images and audio information in the training group sample, and the standard recognition result is that there is criminal activity or no criminal activity;
    第一行为特征模块，用于提取所述第一视频图像中感兴趣区域的第一行为特征；a first behavior feature module, configured to extract a first behavior feature of the region of interest in the first video image;
    第一语音特征模块，用于提取所述第一音频信息中的第一语音特征；a first voice feature module, configured to extract a first voice feature in the first audio information;
    第一归一处理模块，用于将提取得到的所述第一行为特征和所述第一语音特征进行归一化处理；a first normalization processing module, configured to normalize the extracted first behavior feature and the first speech feature;
    第一分类器识别模块，用于将所述训练组样本中归一化后的所述第一行为特征和所述第一语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；a first classifier identification module, configured to classify the first behavior feature and the first voice feature normalized in the training group sample into a criminal activity classifier to obtain an output classification recognition result;
    第一对比模块，用于将输出的分类识别结果与所述训练组样本对应的标准识别结果进行对比，得到本次训练所述犯罪活动分类器输出结果的准确率；a first comparison module, configured to compare the output classification recognition result with a standard recognition result corresponding to the training group sample, to obtain an accuracy rate of the output of the criminal activity classifier in the training;
    第一参数调整模块，用于若本次训练的准确率低于预设阈值，则调整所述犯罪活动分类器中的分类器参数，返回触发所述第一行为特征模块，开始下一次训练；a first parameter adjustment module, configured to: if the accuracy of the current training is lower than a preset threshold, adjust a classifier parameter in the crime activity classifier, return to trigger the first behavior feature module, and start a next training;
    训练完成确定模块，用于若本次训练的准确率高于或等于预设阈值，则确定所述犯罪活动分类器训练完成。The training completion determining module is configured to determine that the criminal activity classifier training is completed if the accuracy of the current training is higher than or equal to the preset threshold.
  8. 根据权利要求7所述的针对犯罪活动的告警装置，其特征在于，还包括：The alarm device for criminal activities according to claim 7, further comprising:
    测试样本采集模块，用于预先收集测试组样本，所述测试组样本包括用于测试的多组第二视频图像和第二音频信息；a test sample collection module, configured to pre-collect test group samples, the test set samples including a plurality of sets of second video images and second audio information for testing;
    测试样本标记模块，用于预先标记所述测试组样本中各组视频图像和音频信息对应的标准识别结果；a test sample marking module, configured to pre-mark standard identification results corresponding to each group of video images and audio information in the test group samples;
    在所述训练完成确定模块确定所述犯罪活动分类器训练完成之前，还触发以下模块：The following modules are also triggered before the training completion determination module determines that the criminal activity classifier training is completed:
    第二行为特征模块，用于提取所述第二视频图像中感兴趣区域的第二行为特征；a second behavior feature module, configured to extract a second behavior feature of the region of interest in the second video image;
    第二语音特征模块，用于提取所述第二音频信息中的第二语音特征；a second voice feature module, configured to extract a second voice feature in the second audio information;
    第二归一处理模块，用于将提取得到的所述第二行为特征和所述第二语音特征进行归一化处理；a second normalization processing module, configured to normalize the extracted second behavior feature and the second speech feature;
    第二分类器识别模块，用于将所述测试组样本中归一化后的所述第二行为特征和所述第二语音特征投入犯罪活动分类器进行分类识别，得到输出的分类识别结果；a second classifier identification module, configured to classify the second behavior feature and the second voice feature normalized in the test group sample into a criminal activity classifier to obtain an output classification recognition result;
    第二对比模块，用于将输出的分类识别结果与所述测试组样本对应的标准识别结果进行对比，得到本次测试中所述犯罪活动分类器输出结果的测试准确率；a second comparison module, configured to compare the output classification recognition result with the standard recognition result corresponding to the test group sample, to obtain a test accuracy rate of the criminal activity classifier output result in the test;
    第二参数调整模块，用于若本次测试的测试准确率低于预设测试阈值，则调整所述犯罪活动分类器中的分类器参数，返回触发所述第一行为特征模块，开始下一次训练；a second parameter adjustment module, configured to adjust a classifier parameter in the crime activity classifier if the test accuracy rate of the test is lower than a preset test threshold, and return to trigger the first behavior feature module to start the next time training;
    训练完成确定模块，用于若本次测试的测试准确率高于或等于预设测试阈值，则触发所述训练完成确定模块来确定所述犯罪活动分类器训练完成。The training completion determining module is configured to trigger the training completion determining module to determine that the criminal activity classifier training is completed if the test accuracy rate of the current test is higher than or equal to the preset test threshold.
  9. 一种针对犯罪活动的告警服务器，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至5中任一项所述针对犯罪活动的告警方法的步骤。An alarm server for criminal activity, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein the processor executes the computer program, such as The steps of the alerting method for criminal activity as claimed in any one of claims 1 to 5.
  10. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至5中任一项所述针对犯罪活动的告警方法的步骤。A computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the criminal activity according to any one of claims 1 to 5. The steps of the alarm method.
PCT/CN2017/119570 2017-12-28 2017-12-28 Alarm method and device for criminal activity, storage medium and server WO2019127274A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201780002311.3A CN108351968B (en) 2017-12-28 2017-12-28 Alarming method, device, storage medium and server for criminal activities
PCT/CN2017/119570 WO2019127274A1 (en) 2017-12-28 2017-12-28 Alarm method and device for criminal activity, storage medium and server

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/119570 WO2019127274A1 (en) 2017-12-28 2017-12-28 Alarm method and device for criminal activity, storage medium and server

Publications (1)

Publication Number Publication Date
WO2019127274A1 true WO2019127274A1 (en) 2019-07-04

Family

ID=62960827

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/119570 WO2019127274A1 (en) 2017-12-28 2017-12-28 Alarm method and device for criminal activity, storage medium and server

Country Status (2)

Country Link
CN (1) CN108351968B (en)
WO (1) WO2019127274A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110909699A (en) * 2019-11-28 2020-03-24 北京以萨技术股份有限公司 Video vehicle non-guide driving detection method and device and readable storage medium
CN111723239A (en) * 2020-05-11 2020-09-29 华中科技大学 Multi-mode-based video annotation method
CN112995582A (en) * 2019-12-17 2021-06-18 广州慧睿思通科技股份有限公司 Control method and device of monitoring equipment and server
CN116189668A (en) * 2023-04-24 2023-05-30 科大讯飞股份有限公司 Voice classification and cognitive disorder detection method, device, equipment and medium

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109300279A (en) * 2018-10-01 2019-02-01 厦门快商通信息技术有限公司 A kind of shop security monitoring method
CN109448287B (en) * 2018-10-25 2021-11-02 平安科技（深圳）有限公司 Safety monitoring method and device, readable storage medium and terminal equipment
CN109784525A (en) * 2018-11-13 2019-05-21 北京码牛科技有限公司 Method for early warning and device based on day vacant lot integration data
CN111222370A (en) * 2018-11-26 2020-06-02 浙江宇视科技有限公司 Case studying and judging method, system and device
CN109685131A (en) * 2018-12-20 2019-04-26 斑马网络技术有限公司 Automobile vehicle device system exception recognition methods and device
CN109815991B (en) * 2018-12-29 2021-02-19 北京城市网邻信息技术有限公司 Training method and device of machine learning model, electronic equipment and storage medium
CN109697815A (en) * 2019-01-24 2019-04-30 广州市天河区保安服务公司 Anti-theft communication network alarming method, appliance arrangement and storage medium
CN111815899B (en) * 2019-04-12 2021-02-12 汇金智融(深圳)科技有限公司 Target distance real-time measuring method and storage medium
CN110278201B (en) * 2019-06-12 2022-08-23 深圳市腾讯计算机***有限公司 Security policy evaluation method and device, computer readable medium and electronic device
CN112199974A (en) * 2019-07-08 2021-01-08 Tcl集团股份有限公司 Learning evaluation method, device and equipment
CN110503081B (en) * 2019-08-30 2022-08-26 山东师范大学 Violent behavior detection method, system, equipment and medium based on interframe difference
CN110659603A (en) * 2019-09-20 2020-01-07 北京小狗智能机器人技术有限公司 Data processing method and device
CN110706700B (en) * 2019-09-29 2022-06-14 深圳市元征科技股份有限公司 In-vehicle disturbance prevention alarm method and device, server and storage medium
CN110991289B (en) * 2019-11-25 2023-09-05 达闼机器人股份有限公司 Abnormal event monitoring method and device, electronic equipment and storage medium
CN113518201B (en) * 2020-07-14 2022-08-12 阿里巴巴集团控股有限公司 Video processing method, device and equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102014278A (en) * 2010-12-21 2011-04-13 四川大学 Intelligent video monitoring method based on voice recognition technology
CN102348101A (en) * 2010-07-30 2012-02-08 深圳市先进智能技术研究所 Examination room intelligence monitoring system and method thereof
CN102509430A (en) * 2011-10-27 2012-06-20 深圳市赛格导航科技股份有限公司 Visual GPS (global positioning system) vehicular anti-theft system
CN203206395U (en) * 2013-04-19 2013-09-18 福建亿榕信息技术有限公司 Intelligent system for monitoring crimes
CN106571055A (en) * 2016-10-28 2017-04-19 广州广电银通金融电子科技有限公司 Internet-of-things-based intelligent vehicle management system and method

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1919711A (en) * 2006-09-20 2007-02-28 浙江工业大学 Elevator inner violence-proof apparatus based on image and speech recognition technique
CN102098492A (en) * 2009-12-11 2011-06-15 上海弘视通信技术有限公司 Audio and video conjoint analysis-based fighting detection system and detection method thereof
CN102722726B (en) * 2012-06-05 2014-01-15 江苏省电力公司南京供电公司 Multi-class support vector machine classification method based on dynamic binary tree
CN102930297B (en) * 2012-11-05 2015-04-29 北京理工大学 Emotion recognition method for enhancing coupling hidden markov model (HMM) voice-vision fusion
US20150310442A1 (en) * 2014-04-25 2015-10-29 Mastercard International Incorporated Methods, systems and computer readable media for determining criminal propensities in a geographic location based on purchase card transaction data
CN104318578B (en) * 2014-11-12 2017-07-21 苏州科达科技股份有限公司 A kind of video image analysis method and system
CN105894539A (en) * 2016-04-01 2016-08-24 成都理工大学 Theft prevention method and theft prevention system based on video identification and detected moving track
CN107146096B (en) * 2017-03-07 2020-08-18 浙江工业大学 Intelligent video advertisement display method and device

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102348101A (en) * 2010-07-30 2012-02-08 深圳市先进智能技术研究所 Examination room intelligence monitoring system and method thereof
CN102014278A (en) * 2010-12-21 2011-04-13 四川大学 Intelligent video monitoring method based on voice recognition technology
CN102509430A (en) * 2011-10-27 2012-06-20 深圳市赛格导航科技股份有限公司 Visual GPS (global positioning system) vehicular anti-theft system
CN203206395U (en) * 2013-04-19 2013-09-18 福建亿榕信息技术有限公司 Intelligent system for monitoring crimes
CN106571055A (en) * 2016-10-28 2017-04-19 广州广电银通金融电子科技有限公司 Internet-of-things-based intelligent vehicle management system and method

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110909699A (en) * 2019-11-28 2020-03-24 北京以萨技术股份有限公司 Video vehicle non-guide driving detection method and device and readable storage medium
CN112995582A (en) * 2019-12-17 2021-06-18 广州慧睿思通科技股份有限公司 Control method and device of monitoring equipment and server
CN111723239A (en) * 2020-05-11 2020-09-29 华中科技大学 Multi-mode-based video annotation method
CN111723239B (en) * 2020-05-11 2023-06-16 华中科技大学 Video annotation method based on multiple modes
CN116189668A (en) * 2023-04-24 2023-05-30 科大讯飞股份有限公司 Voice classification and cognitive disorder detection method, device, equipment and medium
CN116189668B (en) * 2023-04-24 2023-07-25 科大讯飞股份有限公司 Voice classification and cognitive disorder detection method, device, equipment and medium

Also Published As

Publication number Publication date
CN108351968A (en) 2018-07-31
CN108351968B (en) 2022-04-22

Similar Documents

Publication Publication Date Title
WO2019127274A1 (en) Alarm method and device for criminal activity, storage medium and server
CN108960065B (en) Driving behavior detection method based on vision
CN108053427B (en) Improved multi-target tracking method, system and device based on KCF and Kalman
US10311289B2 (en) Face recognition method and device and apparatus
CN108052859B (en) Abnormal behavior detection method, system and device based on clustering optical flow characteristics
CN103714631B (en) ATM cash dispenser intelligent monitor system based on recognition of face
JP4569190B2 (en) Suspicious person countermeasure system and suspicious person detection device
CN108647582A (en) Goal behavior identification and prediction technique under a kind of complex dynamic environment
US20210382933A1 (en) Method and device for archive application, and storage medium
US8965068B2 (en) Apparatus and method for discriminating disguised face
WO2019128263A1 (en) Vehicle honking detection method, device, apparatus, storage medium, and system
CN110588512A (en) Dangerous driving identification and early warning device, method and system
CN112052815B (en) Behavior detection method and device and electronic equipment
Andersson et al. Fusion of acoustic and optical sensor data for automatic fight detection in urban environments
CN105809890B (en) Towards omission child's detection method of safety of school bus
CN108563978A (en) A kind of mood detection method and device
WO2019119515A1 (en) Face analysis and filtering method, device, embedded apparatus, dielectric and integrated circuit
CN109977771A (en) Verification method, device, equipment and the computer readable storage medium of driver identification
CN112744174B (en) Vehicle collision monitoring method, device, equipment and computer readable storage medium
CN110767214A (en) Speech recognition method and device and speech recognition system
CN108108651B (en) Method and system for detecting driver non-attentive driving based on video face analysis
CN109670457A (en) A kind of driver status recognition methods and device
CN109766804A (en) Item identification method, device, equipment and storage medium based on vehicle-mounted scene
CN112989950A (en) Violent video recognition system oriented to multi-mode feature semantic correlation features
CN108449514A (en) Information processing method and device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17936302

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17936302

Country of ref document: EP

Kind code of ref document: A1