CN112532644A - Real-time data analysis system based on Tbox new energy automobile - Google Patents
Real-time data analysis system based on Tbox new energy automobile Download PDFInfo
- Publication number
- CN112532644A CN112532644A CN202011423398.1A CN202011423398A CN112532644A CN 112532644 A CN112532644 A CN 112532644A CN 202011423398 A CN202011423398 A CN 202011423398A CN 112532644 A CN112532644 A CN 112532644A
- Authority
- CN
- China
- Prior art keywords
- data
- real
- module
- tbox
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/08—Protocols for interworking; Protocol conversion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Small-Scale Networks (AREA)
Abstract
A real-time data analysis system based on a Tbox new energy automobile belongs to the technical field of new energy automobile data analysis. The problems of long processing period and low data processing speed in the existing real-time data analysis of the Tbox new energy automobile are solved. The method adopts the gateway to receive the data reported by the Tbox terminal and transmits the data to the data bus; the data bus is used for transmitting data uploaded by the Tbox terminal through the gateway to the protocol analysis module; the protocol analysis module outputs the received data to the protocol analysis module; the protocol analysis module transmits the analyzed data to the data packaging conversion module; the data packaging conversion module is used for packaging the analyzed data into a json data packet and a CSV file, analyzing the category of the uploaded data, the vehicle identification code and the data encryption mode through the json data packet, analyzing the real-time data through the data encryption mode, and analyzing the uploading time of the data through the CSV file. The invention is suitable for data analysis.
Description
Technical Field
The invention belongs to the technical field of new energy automobile data analysis.
Background
The data flow of the Tbox new energy automobile real-time data analysis is that a terminal accesses a platform gateway, the gateway unpacks and verifies the data, the gateway pushes the data which conform to the protocol to a real-time data analyzer in an enterprise level message bus mode, the real-time data analyzer is responsible for protocol analysis of the data, and the protocol refers to a third part of communication protocols and data formats of GB/T32960 electric automobile remote service and management system technical specifications. When receiving data uploaded by the terminal, firstly carrying out adaptation of a data protocol, data conversion and final encapsulation to obtain a standard json message, and pushing the json message to a data storage module, a front-end display module and the like through a message bus.
However, the existing protocol analysis has the problems of long processing period and slow data processing speed.
Disclosure of Invention
The invention aims to solve the problems of long processing period and low data processing speed in the existing real-time data analysis of the Tbox new energy automobile. A real-time data analysis system based on a Tbox new energy automobile is provided.
The invention relates to a real-time data analysis system based on a Tbox new energy automobile, which comprises a gateway, a data bus and a real-time data analysis module;
the gateway is used for being connected with the Tbox terminal and the data bus, receiving data reported by the Tbox terminal and transmitting the data to the data bus;
the data bus is used for transmitting data uploaded by the Tbox terminal through the gateway to the real-time data analysis module;
the real-time data analysis module comprises a data receiving module 301, a data packing and converting module and a protocol analysis module 302;
the data receiving module is used for receiving data transmitted by the data bus and outputting the received data to the protocol analysis module;
the protocol analysis module is used for carrying out protocol analysis on the received data and transmitting the analyzed data to the data packaging conversion module;
the data packaging conversion module is used for packaging the analyzed data into a json data packet and a CSV file, analyzing the category of the uploaded data, the vehicle identification code and the data encryption mode through the json data packet, analyzing the real-time data through the data encryption mode, and analyzing the uploading time of the data through the CSV file.
Furthermore, the categories of the uploaded data analyzed by the json data packet comprise login/logout data, real-time data and supplementary transmission data.
Further, the protocol parsing module is used for parsing the received data according to a data format agreed in the GB/T32960 protocol.
Further, the data receiving module receives data using multiple threads.
According to the invention, an independent data analysis layer is specially used for data analysis in the system from the overall architecture of the system, so that the whole platform is more convenient to maintain, and the data analysis is independent of the gateway, so that the platform gateway is divided into a data access module and data analysis, and the pressure of the gateway is effectively solved. A lightweight data exchange format of json is selected as a data transmission format of the front end and the back end of the platform, so that the development work of a front-end page is simpler, the json can realize the rapid display of data, the original data is analyzed into standard json, the structure is complete and unified, and the calling of other services is facilitated. The mode of data publishing and subscribing of the enterprise-level data bus is fully utilized, the mode supports distributed deployment, has a message caching function, and greatly improves the reliability of the system. With the gradual increase of terminals accessed by the platform, the bearing capacity of the platform is an important index, and the platform is extremely simple to expand in the mode, and only the nodes of the data analysis system need to be increased.
Drawings
FIG. 1 is a schematic structural diagram of an axisymmetric vector polarized light obtaining apparatus according to the present invention;
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
The first embodiment is as follows: the embodiment is described below with reference to fig. 1, and the Tbox new energy vehicle-based real-time data analysis system according to the present invention includes a gateway 1, a data bus 2, and a real-time data analysis module;
the gateway 1 is used for being connected with the Tbox terminal and the data bus 2, receiving data reported by the Tbox terminal and transmitting the data to the data bus 2;
the data bus 2 is used for transmitting data uploaded by the Tbox terminal through the gateway 1 to the real-time data analysis module;
the real-time data analysis module comprises a data receiving module 301, a data packing and converting module 303 and a protocol analysis module 302;
the data receiving module 301 is configured to receive data transmitted by the data bus 2, and output the received data to the protocol analyzing module 302;
the protocol analysis module 302 is configured to perform protocol analysis on the received data, and transmit the analyzed data to the data packing and converting module 303;
the data packing and converting module 303 is configured to pack the parsed data into a json data packet and a CSV file, parse the category, the vehicle identification code, and the data encryption mode of the uploaded data through the json data packet, parse the real-time data through the data encryption mode, and parse the uploading time of the data through the CSV file.
The present invention converts data into CSV files (Comma-Separated Values, CSV, sometimes also called character-Separated Values, since the Separated characters may not be commas as well), which files store tabular data in plain text form, as numbers and text, enabling the data to be stored in a tabular manner.
Furthermore, the categories of the uploaded data analyzed by the json data packet comprise login/logout data, real-time data and supplementary transmission data.
Further, the protocol parsing module 302 is configured to parse the received data according to a data format agreed in the GB/T32960 protocol.
Further, the data receiving module 301 receives data using multiple threads.
Further, the device further comprises a thread configuration module, and the thread configuration module is used for setting the number of threads of the data receiving module 301.
The invention realizes the rapid analysis process from the original data (standard protocol uploaded by the terminal) to the analysis result which is ready to read. The data receiving part adopts multi-thread receiving, the number of receiving threads can be configured according to actual conditions, the data receiving module 301 completes the data packet dividing process, one packet of data is directly rotated to the data analysis module, the data analysis module divides the one packet of data into two parts for analysis, as a complete data packet consists of a message header and a message body, the message header is analyzed in the first step, the type of the message can be judged according to a command unit in the message header, a corresponding message body analysis method is called according to the type of the message, and a corresponding structural body is analyzed according to each field defined in a protocol in the message body analysis method. After the packet data is analyzed, the analyzed data structure is transferred to the data packaging conversion module 303, JSON packaging is realized (JSON Object Notation) is a lightweight data exchange format), and the packaged JSON has readability and real significance. This is done from the data source to parsing into json data, next considering how the data should be distributed for use by other systems.
The access method of the data can be well expanded, and the access mode of the enterprise-level data bus which is realized in the invention is called by other systems on the data bus which sends the json data packet; and moreover, json data packets are stored in kafka (a high-throughput distributed publishing and subscribing message system), and other subsystems need the json data and can acquire the data only by accessing to an enterprise-level data bus or the kafka. The design has the advantage that the data access mode is diversified, for example, when a real-time monitoring system, a big data system or a mobile phone APP needs data, the data can be obtained according to the requirement.
Although the invention herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that features described in different dependent claims and herein may be combined in ways different from those described in the original claims. It is also to be understood that features described in connection with individual embodiments may be used in other described embodiments.
Claims (4)
1. A real-time data analysis system based on a Tbox new energy automobile is characterized by comprising a gateway (1), a data bus (2) and a real-time data analysis module;
the gateway (1) is used for being connected with the Tbox terminal and the data bus (2), receiving data reported by the Tbox terminal and transmitting the data to the data bus (2);
the data bus (2) is used for transmitting data uploaded by the Tbox terminal through the gateway (1) to the real-time data analysis module;
the real-time data analysis module comprises a data receiving module (301), a data packing conversion module (303) and a protocol analysis module (302);
the data receiving module (301) is used for receiving data transmitted by the data bus (2) and outputting the received data to the protocol analyzing module (302);
the protocol analysis module (302) is used for carrying out protocol analysis on the received data and transmitting the analyzed data to the data packing and converting module (303);
the data packaging conversion module (303) is used for packaging the analyzed data into a json data packet and a CSV file, analyzing the category, the vehicle identification code and the data encryption mode of the uploaded data through the json data packet, analyzing the real-time data through the data encryption mode, and analyzing the uploading time of the data through the CSV file.
2. The Tbox-based new energy automobile real-time data analysis system as claimed in claim 1, wherein the categories of the uploaded data analyzed by the json data packet include login/logout data, real-time data and supplementary transmission data.
3. The real-time data analysis system based on the Tbox new energy automobile of claim 1, characterized in that the protocol parsing module (302) is used for parsing the received data according to a data format agreed in a GB/T32960 protocol.
4. The real-time data analysis system based on the Tbox new energy automobile as claimed in claim 1, 2 or 3, wherein the data receiving module (301) receives data in a multi-thread mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011423398.1A CN112532644A (en) | 2020-12-08 | 2020-12-08 | Real-time data analysis system based on Tbox new energy automobile |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011423398.1A CN112532644A (en) | 2020-12-08 | 2020-12-08 | Real-time data analysis system based on Tbox new energy automobile |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112532644A true CN112532644A (en) | 2021-03-19 |
Family
ID=74998236
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011423398.1A Pending CN112532644A (en) | 2020-12-08 | 2020-12-08 | Real-time data analysis system based on Tbox new energy automobile |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112532644A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113436364A (en) * | 2021-06-22 | 2021-09-24 | 广汽埃安新能源汽车有限公司 | Tbox invalid signal value judgment method and device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103412556A (en) * | 2013-07-18 | 2013-11-27 | 江苏中科天安智联科技有限公司 | On-line detection system for vehicle |
| CN105721482A (en) * | 2016-03-03 | 2016-06-29 | 天津大学 | Mobile terminal handheld vehicle management method based on Internet of Vehicles |
| CN106027605A (en) * | 2016-04-30 | 2016-10-12 | 北京智驾互联信息服务有限公司 | Data processing device of vehicle state |
| CN106027606A (en) * | 2016-04-30 | 2016-10-12 | 北京智驾互联信息服务有限公司 | Data processing device of vehicle OBD (On-Board Diagnostic) |
| CN107636602A (en) * | 2017-07-14 | 2018-01-26 | 深圳市元征科技股份有限公司 | A kind of data processing method and data processing equipment |
| CN107846432A (en) * | 2016-09-19 | 2018-03-27 | 北京智驾互联信息服务有限公司 | Data processing equipment and system |
| CN109117313A (en) * | 2018-08-28 | 2019-01-01 | 成都信息工程大学 | A kind of band isolation calamity for mechanism of control vehicle wisdom security gateway and management-control method |
| CN109246090A (en) * | 2018-08-22 | 2019-01-18 | 北京九五智驾信息技术股份有限公司 | Data processing equipment, intelligent back vision mirror and data processing system |
| CN109542866A (en) * | 2018-11-20 | 2019-03-29 | 深圳市博实结科技有限公司 | Vehicle location monitor cloud framework, method, server and medium |
| CN111049937A (en) * | 2019-12-31 | 2020-04-21 | 长城汽车股份有限公司 | Data processing system and data transmission method of intelligent networked automobile |
| CN112020034A (en) * | 2020-10-30 | 2020-12-01 | 深圳市爱夫卡科技股份有限公司 | 4G/5 GT-Box Internet of vehicles environmental protection monitoring server, system and method |
-
2020
- 2020-12-08 CN CN202011423398.1A patent/CN112532644A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103412556A (en) * | 2013-07-18 | 2013-11-27 | 江苏中科天安智联科技有限公司 | On-line detection system for vehicle |
| CN105721482A (en) * | 2016-03-03 | 2016-06-29 | 天津大学 | Mobile terminal handheld vehicle management method based on Internet of Vehicles |
| CN106027605A (en) * | 2016-04-30 | 2016-10-12 | 北京智驾互联信息服务有限公司 | Data processing device of vehicle state |
| CN106027606A (en) * | 2016-04-30 | 2016-10-12 | 北京智驾互联信息服务有限公司 | Data processing device of vehicle OBD (On-Board Diagnostic) |
| CN107846432A (en) * | 2016-09-19 | 2018-03-27 | 北京智驾互联信息服务有限公司 | Data processing equipment and system |
| CN107636602A (en) * | 2017-07-14 | 2018-01-26 | 深圳市元征科技股份有限公司 | A kind of data processing method and data processing equipment |
| CN109246090A (en) * | 2018-08-22 | 2019-01-18 | 北京九五智驾信息技术股份有限公司 | Data processing equipment, intelligent back vision mirror and data processing system |
| CN109117313A (en) * | 2018-08-28 | 2019-01-01 | 成都信息工程大学 | A kind of band isolation calamity for mechanism of control vehicle wisdom security gateway and management-control method |
| CN109542866A (en) * | 2018-11-20 | 2019-03-29 | 深圳市博实结科技有限公司 | Vehicle location monitor cloud framework, method, server and medium |
| CN111049937A (en) * | 2019-12-31 | 2020-04-21 | 长城汽车股份有限公司 | Data processing system and data transmission method of intelligent networked automobile |
| CN112020034A (en) * | 2020-10-30 | 2020-12-01 | 深圳市爱夫卡科技股份有限公司 | 4G/5 GT-Box Internet of vehicles environmental protection monitoring server, system and method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113436364A (en) * | 2021-06-22 | 2021-09-24 | 广汽埃安新能源汽车有限公司 | Tbox invalid signal value judgment method and device |
| CN113436364B (en) * | 2021-06-22 | 2022-04-08 | 广汽埃安新能源汽车有限公司 | Tbox invalid signal value judgment method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108965484B (en) | Internet of things data transmission method, system and terminal | |
| EP2571224B1 (en) | Method for processing messages on m2m platform and m2m platform system | |
| CN102223731A (en) | Interface equipment compatible with various terminals in internet of things and realization method thereof | |
| CN108769017B (en) | Data communication method and device | |
| CN110908363A (en) | Data acquisition method and device for vehicle-mounted terminal | |
| CN111294235B (en) | Data processing method, device, gateway and readable storage medium | |
| CN111131231A (en) | Method and equipment for accessing data of vehicle-mounted terminal into Internet of vehicles monitoring platform | |
| CN110166485B (en) | Protocol scheduling and using method and device | |
| CN112532644A (en) | Real-time data analysis system based on Tbox new energy automobile | |
| CN115568012A (en) | Positioning data transmission method, device and system, electronic equipment and storage medium | |
| CN114490467B (en) | Message processing DMA system and method of multi-core network processor | |
| CN110850452A (en) | Method, device and system for processing satellite telemetry data | |
| CN114040018A (en) | Data receiving and transmitting method and device based on JSON data format | |
| CN107342920B (en) | Internet of things gateway access equipment and data interaction method thereof | |
| CN109358820B (en) | Data access method and device, electronic equipment and computer readable storage medium | |
| CN114726674B (en) | PROFIBUS bus process data extraction method based on FPGA | |
| CN113746850B (en) | Multichannel data acquisition method for Internet of things | |
| CN116156012A (en) | Universal heterogeneous Internet of things equipment standardized access and data analysis system and method | |
| CN107517094B (en) | FPGA-based S-mode secondary radar information server and working method thereof | |
| CN112583839B (en) | Protocol conversion device, method, equipment and medium for Autbus bus and Hart bus | |
| CN109617853B (en) | Gigabit network port auto-negotiation device and method based on FPGA | |
| CN112910830B (en) | Data transmission system, data uploading method and data issuing method of Internet of vehicles | |
| CN110161468B (en) | Method for designing satellite-borne SAR transmission channel data-to-imaging receiving channel functional module | |
| CN112671758A (en) | Protocol conversion device, method, equipment and medium for Autbus bus and Modbus | |
| CN112650597A (en) | Processing system and method for high-concurrency acquired data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210319 |