CN108520572B - Real-time alarm data analysis method and system based on recorder management platform - Google Patents
Real-time alarm data analysis method and system based on recorder management platform Download PDFInfo
- Publication number
- CN108520572B CN108520572B CN201810331783.XA CN201810331783A CN108520572B CN 108520572 B CN108520572 B CN 108520572B CN 201810331783 A CN201810331783 A CN 201810331783A CN 108520572 B CN108520572 B CN 108520572B
- Authority
- CN
- China
- Prior art keywords
- alarm
- data
- vehicle
- counting
- coordinate data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0841—Registering performance data
- G07C5/085—Registering performance data using electronic data carriers
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/008—Registering or indicating the working of vehicles communicating information to a remotely located station
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Traffic Control Systems (AREA)
- Time Recorders, Dirve Recorders, Access Control (AREA)
Abstract
The invention relates to a method and a system for analyzing alarm data in real time based on a recorder management platform, which are provided for solving the defects that the existing recorder alarm information processing method is slow in query speed and difficult to respond to large data volume in real time, and the method comprises the following steps: receiving vehicle coordinate data, wherein the vehicle coordinate data comprises an alarm zone bit; analyzing the vehicle coordinate data, and packaging the analyzed alarm data into a new data packet; caching the new data packet by using a cache region; and reading the data of the buffer area at regular time according to the setting of the timer, and storing the data into an alarm temporary table in a database. Reading all data from the temporary alarm table, counting the alarm times according to the preset information type, and updating the counting result into the alarm information table in the database. The invention also comprises a real-time analysis alarm data system based on the recorder tube platform. The invention is suitable for alarm data management of the recorder.
Description
Technical Field
The invention relates to a recorder management platform-based real-time alarm data analysis method and system, and belongs to the field of data processing of automobile data recorders.
Background
The alarm data analysis refers to operations such as real-time analysis, classification, statistics, storage and the like of alarm information uploaded by a terminal, and as more and more vehicles are networked by platforms, the data volume is larger and larger, a quick and effective alarm analysis method is needed.
The recorder management platform needs to perform real-time display, query and processing operations on the vehicle alarm information, and when the data volume of the alarm information is large, the query of the original method is slow, a bottleneck exists, and real-time response to the large data volume is difficult.
Disclosure of Invention
The invention aims to solve the defects that the existing recorder alarm information processing method is low in query speed and difficult to respond to large data volume in real time, and provides a recorder management platform-based real-time alarm data analysis method.
A real-time analysis alarm data method based on a recorder management platform comprises the following steps:
the method comprises the steps of firstly, receiving vehicle coordinate data, wherein the vehicle coordinate data comprise alarm zone bits;
analyzing the vehicle coordinate data, and packaging the analyzed alarm data into a new data packet;
step three, caching the new data packet by using a cache region;
and step four, reading the data of the buffer area at regular time according to the setting of the timer, and storing the data into an alarm temporary table in the database.
And step five, reading all data from the temporary alarm table, counting the alarm times according to the preset information type, and updating the counting result into the alarm information table in the database.
A real-time analysis alarm data system based on a recorder management platform comprises:
the data receiving module is used for receiving the vehicle coordinate data; the vehicle coordinate data comprises an alarm zone bit;
the analysis module is used for analyzing the vehicle coordinate data and packaging the analyzed alarm data into a new data packet;
the cache module is used for caching the new data packet by using the cache region;
the storage module is used for regularly reading the data of the cache region according to the setting of the timer and storing the data into an alarm temporary table in the database;
and the data updating module is used for reading all data from the temporary alarm table, counting the alarm times according to the preset information type and updating the counting result into the alarm information table in the database.
The invention has the beneficial effects that:
1) the embodiment of the invention effectively solves the problem of slow speed of inquiring and processing the alarm information of the recorder management platform; compared with the existing method, the query speed of the method is improved by at least 10%.
2) In one embodiment of the invention, the coordinate data is pushed by the zeroMQ, the receiving end can receive the coordinate data in a multi-thread mode, the processing capacity of the system is increased, the thread number can be modified in the configuration file, and the system is convenient to deploy;
3) when the alarm information is stored, a cache mechanism, a timer and a timeout batch data insertion mode are adopted, so that the pressure of operating timeout is greatly reduced;
4) the embodiment of the invention supports multi-server multi-process deployment and can effectively solve the bottleneck of continuous increase of the number of real-time coordinates.
Drawings
FIG. 1 is a flow chart of one embodiment of the present invention.
Detailed Description
The first embodiment is as follows: as shown in fig. 1, the method for analyzing alarm data in real time based on a recorder management platform of the embodiment includes:
the method comprises the steps of firstly, receiving vehicle coordinate data, wherein the vehicle coordinate data comprise alarm zone bits.
And step two, analyzing the vehicle coordinate data, and packaging the analyzed alarm data into a new data packet.
And step three, caching the new data packet by using a cache region.
And step four, reading the data of the buffer area at regular time according to the setting of the timer, and storing the data into an alarm temporary table in the database.
And step five, reading all data from the temporary alarm table, counting the alarm times according to the preset information type, and updating the counting result into the alarm information table in the database.
The temporary alarm table and the alarm information table are two different tables, and the arrangement is to avoid the phenomenon of table deadlock caused by updating operation.
The specific meaning of the method is that the preset information is four items, namely, vehicle, alarm type, driver and date, the four same items are integrated into one type, then the item of the alarm frequency is added, 1 is added to each four items, and when the alarm information table is specifically updated, the four items are not only updated, but also all the rest information, such as longitude and latitude information and the like, is updated. The alarm information inquiry speed can be increased by the arrangement.
The second embodiment is as follows: the first difference between the present embodiment and the specific embodiment is: in the first step, the vehicle coordinate data is received in a multithreading mode through a multithreading network base zeroMQ based on a message queue.
Other steps and parameters are the same as those in the first embodiment.
The third concrete implementation mode: the present embodiment differs from the first or second embodiment in that: and in the second step, analyzing the vehicle coordinate data, specifically analyzing the coordinate data according to a TJ/T808 protocol.
Other steps and parameters are the same as those in the first or second embodiment.
The fourth concrete implementation mode: the difference between this embodiment mode and one of the first to third embodiment modes is: in the second step, the analyzed alarm data comprises vehicle, longitude, latitude, speed, altitude, direction, alarm time, alarm type and driver information. And step five, the preset information includes the vehicle, the alarm type, the driver and the date.
Other steps and parameters are the same as those in one of the first to third embodiments.
The fifth concrete implementation mode: the difference between this embodiment and one of the first to fourth embodiments is: counting the alarm times according to the preset information type, and updating the counting result into an alarm information table in a database specifically comprises the following steps:
and counting the alarm times according to the vehicle, the alarm type, the driver and the date, judging whether a record with the same vehicle, alarm type, driver and date as the statistical result exists in the alarm information table, if so, updating the record, and if not, inserting the statistical result into the alarm information table.
Other steps and parameters are the same as in one of the first to fourth embodiments.
The sixth specific implementation mode: the difference between this embodiment and one of the first to fifth embodiments is: in the fourth step, the database is a timesten database.
Other steps and parameters are the same as those in one of the first to fifth embodiments.
The seventh embodiment: this embodiment provides a real-time analysis alarm data system based on record appearance management platform, includes:
the data receiving module is used for receiving the vehicle coordinate data; the vehicle coordinate data includes an alarm flag bit.
And the analysis module is used for analyzing the vehicle coordinate data and packaging the analyzed alarm data into a new data packet.
And the cache module is used for caching the new data packet by using the cache region.
And the storage module is used for regularly reading the data of the cache region according to the setting of the timer and storing the data into an alarm temporary table in the database.
And the data updating module is used for reading all data from the temporary alarm table, counting the alarm times according to the preset information type and updating the counting result into the alarm information table in the database.
The specific implementation mode is eight: the seventh embodiment is different from the seventh embodiment in that: in the data receiving module, multithreading receiving is carried out on the vehicle coordinate data through a multithreading network base zeroMQ based on a message queue;
in the analysis module, coordinate data are analyzed through a TJ/T808 protocol; the analyzed alarm data comprises vehicle, longitude, latitude, speed, altitude, direction, alarm time, alarm type and driver information; in the storage module, the preset information includes a vehicle, an alarm type, a driver and a date.
Other steps and parameters are the same as those in the seventh embodiment.
The specific implementation method nine: the eighth embodiment is different from the eighth embodiment in that: in the cache module, the buffer area is a ring buffer area.
Other steps and parameters are the same as those of the seventh or eighth embodiment.
The detailed implementation mode is ten: the present embodiment differs from the ninth embodiment in that: in the storage module, the process of counting the alarm times according to the preset information type and updating the counting result into the alarm information table in the database specifically comprises the following steps:
and counting the alarm times according to the vehicle, the alarm type, the driver and the date, judging whether a record with the same vehicle, alarm type, driver and date as the statistical result exists in the alarm information table, if so, updating the record, and if not, inserting the statistical result into the alarm information table, wherein other steps and parameters are the same as those in one of the seventh to ninth specific implementation modes.
The concrete implementation mode eleven: in this embodiment, the data required by this system comes from the information push to the coordinate data of the recorder management platform, the vehicle coordinate data is received to this system by the message push of ZeroMQ, the coordinate data contains the alarm flag bit, the alarm is analyzed according to the part mark TJ/T808 protocol, the alarm information that can be analyzed includes vehicle, longitude, latitude, speed, altitude, direction, alarm time, alarm type, driver, etc., and is packaged into a new data packet, a buffer is opened up to buffer the analyzed alarm information, the alarm data in the buffer is read regularly by using a timer, and is stored in the temporary alarm table of the timeten database in batch.
The timing task is responsible for reading all data of the temporary alarm table, counting alarm times according to the vehicle, the alarm type, the driver and the date, updating the counting result into the alarm table, judging whether the alarm table has the alarm record in the updating process (the judgment of the alarm record is according to the vehicle, the alarm type, the driver and the date), if the alarm record exists, operating the record by the update, if the alarm record does not exist, deleting the new record by insert, and deleting the counted data in the temporary table to a buffer area by adopting a ring buffer area (ring buffer). The alarm analysis service adopts multithreading to receive coordinate data and multithreading to process alarm data, the number of threads can be adjusted according to the configuration of the server and the actual data volume, and only the configuration file needs to be modified, so that the method is fast in aspect.
The present invention is capable of other embodiments and its several details are capable of modifications in various obvious respects, all without departing from the spirit and scope of the present invention.
Claims (3)
1. A real-time analysis alarm data method based on a recorder management platform is characterized by comprising the following steps:
the method comprises the steps of firstly, receiving vehicle coordinate data, wherein the vehicle coordinate data comprise alarm zone bits;
receiving the vehicle coordinate data specifically comprises the steps of receiving the vehicle coordinate data in a multithreading mode through a multithreading network base zeroMQ based on a message queue;
analyzing the vehicle coordinate data, and packaging the analyzed alarm data into a new data packet; resolving the vehicle coordinate data specifically to resolve the coordinate data according to a TJ/T808 protocol;
the analyzed alarm data comprises vehicle, longitude, latitude, speed, altitude, direction, alarm time, alarm type and driver information;
step three, caching the new data packet by using a cache region;
reading the data of the cache region at regular time according to the setting of the timer, and storing the data into an alarm temporary table of the database; the database is a timestest database;
reading all data from the temporary alarm table, counting the alarm times according to the preset information type, and updating the counting result into an alarm information table in a database;
the preset information comprises a vehicle, an alarm type, a driver and a date;
the process of counting the alarm times according to the preset information type and updating the counting result into the alarm information table in the database specifically comprises the following steps:
and counting the alarm times according to the vehicle, the alarm type, the driver and the date, judging whether a record with the same vehicle, alarm type, driver and date as the statistical result exists in the alarm information table, if so, updating the record, and if not, inserting the statistical result into the alarm information table.
2. A real-time analysis alarm data system based on a recorder management platform is characterized by comprising:
the data receiving module is used for receiving the vehicle coordinate data; the vehicle coordinate data comprises an alarm zone bit;
the analysis module is used for analyzing the vehicle coordinate data and packaging the analyzed alarm data into a new data packet;
the cache module is used for caching the new data packet by using the cache region;
the storage module is used for regularly reading the data of the cache region according to the setting of the timer and storing the data into an alarm temporary table in the database;
the data updating module is used for reading all data from the alarm temporary table, counting the alarm times according to the preset information type and updating the counting result into the alarm information table in the database;
in the data receiving module, multithreading receiving is carried out on the vehicle coordinate data through a multithreading network base zeroMQ based on a message queue;
in the analysis module, coordinate data are analyzed through a TJ/T808 protocol; the analyzed alarm data comprises vehicle, longitude, latitude, speed, altitude, direction, alarm time, alarm type and driver information; in the storage module, the preset information includes a vehicle, an alarm type, a driver and a date;
in the storage module, the process of counting the alarm times according to the preset information type and updating the counting result into the alarm information table in the database specifically comprises the following steps:
and counting the alarm times according to the vehicle, the alarm type, the driver and the date, judging whether a record with the same vehicle, alarm type, driver and date as the statistical result exists in the alarm information table, if so, updating the record, and if not, inserting the statistical result into the alarm information table.
3. The recorder management platform based real-time analysis alarm data system according to claim 2, wherein:
in the cache module, the buffer area is a ring buffer area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810331783.XA CN108520572B (en) | 2018-04-13 | 2018-04-13 | Real-time alarm data analysis method and system based on recorder management platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810331783.XA CN108520572B (en) | 2018-04-13 | 2018-04-13 | Real-time alarm data analysis method and system based on recorder management platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108520572A CN108520572A (en) | 2018-09-11 |
| CN108520572B true CN108520572B (en) | 2021-12-17 |
Family
ID=63432598
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810331783.XA Active CN108520572B (en) | 2018-04-13 | 2018-04-13 | Real-time alarm data analysis method and system based on recorder management platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108520572B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109739148A (en) * | 2019-01-23 | 2019-05-10 | 上海智共荟智能科技有限公司 | For industrial processing equipment large data sets at optimization processing transmission system and method |
| CN113132464B (en) * | 2021-03-17 | 2022-07-01 | 中创智维科技有限公司 | Method for collecting geometrical parameter alarm data of contact network |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6853919B2 (en) * | 2003-02-04 | 2005-02-08 | General Motors Corporation | Method for reducing repeat false alarm indications in vehicle impact detection systems |
| CN104077897A (en) * | 2014-06-30 | 2014-10-01 | 广东威创视讯科技股份有限公司 | Alarm method and system for automatic positioning |
| CN106856034A (en) * | 2015-12-09 | 2017-06-16 | 西安网清生物科技有限公司 | A kind of alarm of intelligence 110 board |
| CN107357839A (en) * | 2017-06-23 | 2017-11-17 | 深圳市盛路物联通讯技术有限公司 | A kind of bicycle location tracking method and system |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8508359B2 (en) * | 2010-12-17 | 2013-08-13 | Simplexgrinnell Lp | Method and system for wireless configuration, control, and status reporting of devices in a fire alarm system |
| US20150370854A1 (en) * | 2013-01-31 | 2015-12-24 | Hewlett-Packard Development Company, L.P. | Approximate query processing |
| CN103472780B (en) * | 2013-08-30 | 2016-03-16 | 中国神华能源股份有限公司 | To the system and method that locomotive and synchronous device thereof are monitored |
| US9240082B2 (en) * | 2013-10-22 | 2016-01-19 | At&T Intellectual Property I, L.P. | Crowd sourced optimization of vehicle performance based on cloud based data |
| CN103730007B (en) * | 2013-12-20 | 2016-04-20 | 南威软件股份有限公司 | A kind of bayonet socket crosses the real-time compare of analysis alarm method of car and warning system thereof |
| CN103646496A (en) * | 2013-12-27 | 2014-03-19 | 深圳市众鸿科技股份有限公司 | Method, device and system for monitoring vehicle information |
| CN104700645B (en) * | 2015-03-23 | 2017-01-11 | 航天科技控股集团股份有限公司 | Method for verifying electronic fences based on recorder management platform ends |
| SG10201508064TA (en) * | 2015-09-28 | 2017-04-27 | Mastercard International Inc | Vehicle Telematics Policing System |
| CN105590419A (en) * | 2016-03-24 | 2016-05-18 | 贵州网恒科技股份有限公司 | Police calling and sending platform based on Internet |
| CN106292496B (en) * | 2016-08-31 | 2019-01-11 | 浙江浙大中控信息技术有限公司 | A kind of subway monitoring system alarming management method |
| CN106971432A (en) * | 2017-04-11 | 2017-07-21 | 中国人民解放军海军航空工程学院青岛校区 | A kind of airplane data management record system and data presentation technique |
| CN107277614A (en) * | 2017-06-27 | 2017-10-20 | 深圳市爱培科技术股份有限公司 | Audio and video remote player method, storage device and the mobile terminal of drive recorder |
| CN107767641A (en) * | 2017-09-13 | 2018-03-06 | 新丝绸之路科技有限公司 | Alarm method, mobile terminal and computer-readable storage medium |
| CN107742417B (en) * | 2017-09-13 | 2019-12-17 | 成都路行通信息技术有限公司 | Vehicle accident alarm method and device |
| CN107564272B (en) * | 2017-09-25 | 2020-07-03 | 航天科技控股集团股份有限公司 | Terminal reporting alarm automatic alarm handling device and method based on recorder management platform |
-
2018
- 2018-04-13 CN CN201810331783.XA patent/CN108520572B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6853919B2 (en) * | 2003-02-04 | 2005-02-08 | General Motors Corporation | Method for reducing repeat false alarm indications in vehicle impact detection systems |
| CN104077897A (en) * | 2014-06-30 | 2014-10-01 | 广东威创视讯科技股份有限公司 | Alarm method and system for automatic positioning |
| CN106856034A (en) * | 2015-12-09 | 2017-06-16 | 西安网清生物科技有限公司 | A kind of alarm of intelligence 110 board |
| CN107357839A (en) * | 2017-06-23 | 2017-11-17 | 深圳市盛路物联通讯技术有限公司 | A kind of bicycle location tracking method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108520572A (en) | 2018-09-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107273506B (en) | Method for multi-table combined query of database | |
| US8145859B2 (en) | Method and system for spilling from a queue to a persistent store | |
| CN108520572B (en) | Real-time alarm data analysis method and system based on recorder management platform | |
| CN111046049B (en) | Truck GPS track data compression method | |
| CN102306194B (en) | Methods for storing and querying GPS (global positioning system) data | |
| CN103744913A (en) | Database retrieval method based on search engine technology | |
| CN110191109B (en) | Message sampling method and device | |
| CN106528649A (en) | Massive data storage and retrieval system and massive data storage and retrieval methods for new energy vehicles | |
| GB0423793D0 (en) | A method and system for gathering and processing data for road use charging | |
| CN106682167B (en) | Statistical device and method for user behavior data | |
| CN103246653A (en) | Data processing method and device | |
| CN107862068B (en) | Data processing method, device and computer readable storage medium | |
| CN107357885A (en) | Method for writing data and device, electronic equipment, computer-readable storage medium | |
| CN103761255A (en) | Method and system for optimizing data storage of NoSQL mode | |
| CN107241251B (en) | Software implementation method for real-time receiving of multi-path CAN messages | |
| EP1271362A3 (en) | A method of handling a data request | |
| EP1538537A3 (en) | Method of storing fast throughput experimentation information in a database | |
| CN108156230B (en) | Real-time data synchronization method, system and framework | |
| CN114189455B (en) | Container network flow monitoring and counting method and system based on ebpf technology | |
| CN116010400A (en) | Method, system, device, equipment and storage medium for storing time series data of vehicle | |
| CN108401003B (en) | Radar data synchronization method, device, equipment and computer storage medium | |
| CN114637929A (en) | Trajectory collision method based on elastic search | |
| CN109308293B (en) | Database and table dividing method for large concurrent database | |
| CN115328818A (en) | Data stream storage display rapid addressing method | |
| CN113806374A (en) | Big data-oriented data stream type caching method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Li Dexian Inventor after: Liu Mingyue Inventor after: Jiang Haifeng Inventor after: Yin Lu Inventor after: Yang Chunyan Inventor after: Chen Yufeng Inventor before: Yang Chunyan Inventor before: Li Dexian Inventor before: Liu Mingyue Inventor before: Jiang Haifeng Inventor before: Chen Yufeng |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |