CN116932626B - Data analysis method, device, equipment and storage medium - Google Patents
Data analysis method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116932626B CN116932626B CN202310930995.0A CN202310930995A CN116932626B CN 116932626 B CN116932626 B CN 116932626B CN 202310930995 A CN202310930995 A CN 202310930995A CN 116932626 B CN116932626 B CN 116932626B
- Authority
- CN
- China
- Prior art keywords
- data
- target
- data item
- generating
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000007405 data analysis Methods 0.000 title claims abstract description 12
- 230000006870 function Effects 0.000 claims description 64
- 238000004458 analytical method Methods 0.000 claims description 46
- 238000012545 processing Methods 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 16
- 238000010276 construction Methods 0.000 claims description 7
- 238000012423 maintenance Methods 0.000 abstract description 7
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data analysis method, a device, equipment and a storage medium. The method comprises the following steps: acquiring original binary data, and splitting the original binary data to obtain target data; determining a target description file based on a preset parsing rule of the data item; generating a tool package according to the target description file, and storing the tool package to the target file; according to the technical scheme, the complex and changeable data frame formats of the target data can be effectively processed, the maintainability and the expandability of the system are improved, and the maintenance cost of the system is reduced.
Description
Technical Field
The embodiment of the invention relates to the technical field of aerospace information, in particular to a data analysis method, a device, equipment and a storage medium.
Background
The low-orbit satellite data acquisition system can acquire data such as weather and hydrology of the terminal to the low-orbit satellite, then the satellite transmits the data back to the ground operation control center for processing, and the processed data is provided for a user terminal, so that the terminal scattered in different regions is convenient to use, the terminal management effect is particularly outstanding for unmanned areas or oceans, and the system has great significance for scenes such as environment monitoring, rescue and relief work and the like.
However, DCS (Data Collecting System, satellite data collection system) data transmitted back from the satellite are binary format data, the format of DCS data does not prescribe a standard data frame format in a protocol layer, manufacturers of different terminals can customize the data frame format according to the conditions of the manufacturers, so that great challenges are brought to binary data analysis processing, and different analysis rules need to be customized according to different manufacturer terminals, so that the subsequent maintenance and expansion of the system are not facilitated.
Disclosure of Invention
The embodiment of the invention provides a data analysis method, a device, equipment and a storage medium, which solve the problem that subsequent system maintenance and expansion are difficult due to inconvenient binary data analysis.
According to an aspect of the present invention, there is provided a data parsing method, including:
acquiring original binary data, and splitting the original binary data to obtain target data;
determining a target description file based on a preset parsing rule of the data item;
generating a tool package according to the target description file, and storing the tool package to the target file;
and reading the target data based on the target file to obtain an analysis result corresponding to the target data.
According to another aspect of the present invention, there is provided a data parsing apparatus including:
the acquisition module is used for acquiring original binary data and splitting the original binary data to obtain target data;
the determining module is used for determining a target description file based on a preset parsing rule of the data item;
the generating module is used for generating a tool kit according to the target description file and storing the tool kit into the target file;
the obtaining module is used for reading the target data based on the target file and obtaining an analysis result corresponding to the target data.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data parsing method according to any one of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute a data parsing method according to any embodiment of the present invention.
According to the embodiment of the invention, the original binary data is obtained, and split is carried out on the original binary data to obtain the target data; determining a target description file based on a preset parsing rule of the data item; generating a tool package according to the target description file, and storing the tool package to the target file; the target data is read based on the target file, the analysis result corresponding to the target data is obtained, the problem that subsequent system maintenance and expansion are difficult due to inconvenience in binary data analysis is solved, the complexity and the variability of the data frame format of the target data can be effectively improved, the maintainability and the expandability of the system are improved, and the maintenance cost of the system is reduced.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a data parsing method according to a first embodiment of the invention;
fig. 2 is a schematic structural diagram of a data analysis device in a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It will be appreciated that prior to using the technical solutions disclosed in the embodiments of the present disclosure, the user should be informed and authorized of the type, usage range, usage scenario, etc. of the personal information related to the present disclosure in an appropriate manner according to the relevant legal regulations.
Example 1
Fig. 1 is a flowchart of a data parsing method in a first embodiment of the present invention, where the method may be applied to the case of parsing binary data, and the method may be performed by a data parsing apparatus in the embodiment of the present invention, where the apparatus may be implemented in software and/or hardware, as shown in fig. 1, and the method specifically includes the following steps:
s110, acquiring original binary data, and splitting the original binary data to obtain target data.
The original binary data are original data which are collected by a ground operation control center and are returned by satellites, wherein the original binary data comprise various load data. The target data is DCS data obtained after splitting the original data.
Specifically, the method for obtaining the original binary data and splitting the original binary data to obtain the target data may be: the method comprises the steps of collecting original binary data returned by a satellite through a ground operation control center, splitting and framing the original binary data and reverse traversal to obtain load data of different types of protocols, such as AIS (Automatic Identification System ) data, DCS (Data Collecting System, satellite data collection system) data and VDES (VHF Data Exchange System, very high frequency data exchange system) data, and determining the DCS data obtained by splitting the original binary data as target data.
S120, determining a target description file based on a preset analysis rule of the data item.
It should be noted that one frame of target data may include a plurality of data items. The preset analysis rules of the data items are used for analyzing the analysis rules of different data items. The target description file is used for describing different data items in the target data based on the analysis rule.
Specifically, the method for determining the target description file based on the preset parsing rule of the data item may be: and setting a preset analysis rule for the data item, and determining a target description file capable of analyzing the target data according to the preset analysis rule of the data item.
S130, generating a tool package according to the target description file, and storing the tool package to the target file.
The tool package may be SDK (Software Development Kit, development tool package) code, and the target file may be a project code file of the system, for example, may be a target js file.
Specifically, the method for generating the tool package according to the target description file and storing the tool package in the target file may be: and reading the target description file based on the command line tool to obtain a tool package, and adding the obtained tool package to the target file to facilitate the subsequent analysis of the target data.
And S140, reading target data based on the target file to obtain an analysis result corresponding to the target data.
The analysis result corresponding to the target data is data information obtained by analyzing the target data.
Specifically, the method for reading the target data based on the target file to obtain the analysis result corresponding to the target data may be: and reading the target data through the target file, namely directly acquiring an analysis result corresponding to the target data, writing the analysis result into the CSV file in real time, and importing the CSV file into a database in batches for persistent storage, so that the analysis result in the database can be conveniently and subsequently acquired for application.
Optionally, determining the target description file based on a preset parsing rule of the data item includes:
acquiring initial attribute information corresponding to a data item, wherein the initial attribute information comprises at least one of an attribute name, a data type, occupation byte information, description information, a processing mode and decryption information corresponding to each data type;
determining a preset analysis rule according to initial attribute information corresponding to the data item;
generating descriptive content of at least one data item according to a preset parsing rule;
a target description file is generated based on the description content of the at least one data item.
The initial attribute information corresponding to the data item is all attribute information that may be included in the data item, for example, the initial attribute information may include at least one of an attribute name, a data type, occupied byte information corresponding to each data type, description information, a processing mode, and decryption information, where the data type and occupied byte information corresponding to each data type may be set according to a protocol of the target data. The processing mode is a size end mode.
Specifically, the method for obtaining the initial attribute information corresponding to the data item may be: and acquiring initial attribute information corresponding to the data item according to the analysis result of the historical target data.
Specifically, the method for determining the preset parsing rule according to the initial attribute information corresponding to the data item may be: and determining a preset analysis rule corresponding to each attribute information according to the initial attribute information corresponding to the data item. For example, the preset parsing rule may be:
id represents the attribute name corresponding to the data item, and can be customized according to actual requirements;
dataType represents the data type of the data item, and the occupied byte information or binary bit number corresponding to each data type, wherein byte represents bytes; float represents a floating point number; unsigned represents an unsigned integer; signed represents a signed integer; bit represents binary bit, data after bit can represent bit number, and other cases represent byte number;
desc represents description information of the data item;
endian represents a processing mode, i.e., a size end mode, where be represents the size end; le represents the small end;
decryptFn represents data item decryption information that may be used to implement the data item decryption function.
Specifically, the manner of generating the description content of at least one data item according to the preset parsing rule may be: descriptive content of at least one data item is generated based on at least one preset parsing rule. For example, the description content of one data item may be:
{
"id":"version",
"dataType":"byte2",
"desc" terminal version number "
}。
Specifically, the manner of generating the target description file according to the description content of at least one data item may be: and splicing the descriptive contents of at least one data item to generate a target descriptive file, wherein the target descriptive file can contain the descriptive contents of a plurality of data items. For example, the object description file may be:
[{
"id":"version",
"dataType":"byte2",
"desc" terminal version number "
},{
"id":"lat",
"dataType":"float4",
"desc" dimension "
},{
"id":"lng",
"dataType":"float4",
"desc": "longitude"
},{
"id":"crtTime",
"dataType":"byte4",
"desc": "DCS message Transmission time"
},{
"id":"terminalPower",
"dataType":"unsigned4",
"endian":"be",
"decryptFn": "the specific code of decryption information",
"desc" means "residual quantity"
},{
"id":"status",
"dataType":"bit1",
"desc" terminal operating state "
}]。
The description content of the file is an array, and can represent all data items owned by one frame of data in the target data.
By acquiring the initial attribute information corresponding to the data items, determining a preset analysis rule according to the initial attribute information corresponding to the data items, generating description content of at least one data item according to the preset analysis rule, generating a target description file according to the description content of at least one data item, different preset analysis rules can be provided for target data of different protocols, flexibility of the target description file is improved, a frame structure of changeable target data is rapidly dealt with, and customization of different decryption information can be facilitated.
Optionally, generating a tool package according to the target description file, and storing the tool package to the target file, including:
acquiring command line parameters in a command line tool;
acquiring path information corresponding to the target description file and the tool kit based on the command line parameters;
analyzing the target description file based on the command line tool to generate a calling function corresponding to at least one data item;
and generating a tool package according to the calling function corresponding to the at least one data item, and storing the tool package to the target file according to the path information corresponding to the tool package.
The command line tool can be realized based on a npm package of node. Js, and can be operated in a command line mode in windows and linux by packaging a standard npm command line tool. The command line parameters are stored in the command line tool. The path information corresponding to the tool pack is stored information of the tool pack after the tool pack is generated. The calling function corresponding to the at least one data item may be a calling function of an API (Application Programming Interface, application program interface) in the toolkit.
Specifically, the manner of obtaining the command line parameters in the command line tool may be: and analyzing and acquiring command line parameters.
Specifically, the manner of acquiring the path information corresponding to the target description file and the tool package based on the command line parameter may be: and acquiring corresponding binary description files and path information corresponding to the tool kit based on the command line parameters. For example, if the command name is json-to-code, -f bin-desc. Json indicates the object description file, and-t./dcs-parameter-sdk. Js indicates the path information corresponding to the tool pack.
Specifically, the manner of generating the calling function corresponding to the at least one data item based on the parsing of the target description file by the command line tool may be: and reading and analyzing the target description file based on the command line tool to obtain a calling function corresponding to at least one data item of each frame of data in the target data.
Specifically, the method for generating the tool package according to the calling function corresponding to the at least one data item and storing the tool package to the target file according to the path information corresponding to the tool package may be: and obtaining a tool package corresponding to the target data according to the calling function corresponding to at least one data item corresponding to each frame of data in the target data, and storing the tool package into a corresponding path in the target file according to the path information.
The command line parameters in the command line tool are acquired, the path information corresponding to the target description file and the tool kit is acquired based on the command line parameters, the target description file is analyzed based on the command line tool, the calling function corresponding to at least one data item is generated, the tool kit is generated according to the calling function corresponding to the at least one data item, and the tool kit is stored to the target file according to the path information corresponding to the tool kit, so that the tool kit generation efficiency can be improved, and the aging cost can be reduced.
Optionally, reading the target data based on the target file to obtain an analysis result corresponding to the target data, including:
reading target data based on a target file to obtain a byte array of each frame corresponding to the target data;
calling each frame of byte array through a calling function corresponding to at least one data item in the target file, and obtaining an analysis result corresponding to each frame of byte array;
and generating an analysis result corresponding to the target data according to the analysis result corresponding to the byte array of each frame.
The byte array of each frame is the representation result of each frame of data in the target data.
Specifically, the method for reading the target data based on the target file to obtain the byte array of each frame corresponding to the target data may be: and reading target data in the original binary data based on the target file, and analyzing the target data to obtain a byte array of each frame.
Specifically, the method for calling the byte array of each frame through the calling function corresponding to at least one data item in the target file to obtain the analysis result corresponding to the byte array of each frame may be: and transferring the byte array of each frame as a parameter to a calling function corresponding to at least one data item in the target file to obtain an analysis result of the data item corresponding to the byte array of each frame.
Optionally, the parsing the object description file based on the command line tool generates a calling function corresponding to at least one data item, including:
reading a target description file based on a command line tool, and acquiring target attribute information corresponding to at least one data item;
generating a frame sequence according to the target attribute information;
and calling the construction function according to the frame sequence, and generating a calling function corresponding to at least one data item.
The target attribute information may be at least one of an attribute name, a data type, occupied byte information, description information, a processing mode, and decryption information corresponding to the data item. Wherein the frame sequence is a byte sequence array of the target data frame. The constructor may be a DataView function.
Specifically, the manner of reading the target description file based on the command line tool and obtaining the target attribute information corresponding to at least one data item may be: when the command line tool reads the target description file, each frame of data in the target data is processed according to different data types in the target description file, and the target attribute information corresponding to at least one data item is obtained. For example, if the datatype attribute value in the read target description file is byte2, the data type is byte, and the occupied byte information is 2 bytes; if the dateType attribute value is float4, the data type is float and the occupied byte information is 4 bytes.
Specifically, the manner of generating the frame sequence according to the target attribute information may be: and generating a corresponding byte sequence array according to the target attribute information in each data frame.
Specifically, the manner of calling the constructor according to the frame sequence to generate the calling function corresponding to the at least one data item may be: and transferring the frame sequence as a parameter to a DataView construction function, and respectively calling different instance methods of the DataView according to each item of occupied byte information and the processing mode to convert the workflow of the corresponding type so as to generate a calling function corresponding to at least one data item.
The target description file is read based on the command line tool, the target attribute information corresponding to at least one data item is obtained, the frame sequence is generated according to the target attribute information, the constructor is called according to the frame sequence, and the calling function corresponding to at least one data item is generated, so that different data types can be rapidly processed, and meanwhile, the problem that the size end and binary bit are difficult to intercept accurately can be more conveniently processed. For example, the process of processing the different data types and sizes may be:
const frame = frame sequence;
const dv＝new DataView(frame)；
const result；
if (small end) {
result＝dv.getUint32(3，true)；
}else{
result＝dv.getFloat64(5，false)；
}；
Where getUint32 (3, true) indicates that the 32-bit length is calculated from the third byte, getflow 64 (5, false) indicates that the 64-bit length is calculated from the fifth byte, true indicates low endian, and false indicates high endian.
Optionally, the target attribute information includes: decryption information and first attribute information;
calling the construction function according to the frame sequence to generate a calling function corresponding to at least one data item, wherein the calling function comprises the following steps:
generating a decryption workflow according to the decryption information;
generating an attribute workflow according to first attribute information, wherein the first attribute information comprises: at least one of attribute name, data type, occupied byte information corresponding to each data type, description information and processing mode;
and generating a calling function corresponding to at least one data item according to the decryption workflow and the attribute workflow.
The first attribute information comprises at least one of an attribute name of a data item, a data type, occupied byte information corresponding to each data type, description information and a processing mode, and the decryption information is processing information of a decryption function.
The decryption workflow is a packaged workflow generated by reading the decryption information, and the attribute workflow is a workflow generated by reading each attribute information.
Specifically, the manner of generating the decryption workflow according to the decryption information may be: if the target attribute information comprises decryption information, the content of the decryption information is required to be called a Function construction Function to generate a decryption workflow. For example, in the reading process, a corresponding configuration is performed according to a decryption function corresponding to the decryption information configured in the target description file, and the decryption function is dynamically generated according to the configuration processing of the decryption function, as follows:
{
"id":"terminalPower",
"dataType":"unsigned4",
"endian":"be",
"decryptFn":"x*36*36+y*36+z",
"desc" means "residual quantity"
}；
An encryption function is generated from the value of the decryptFn attribute,
const decFn＝new Function(obj.decryptFn,“x"y""z")；
the following processing can be performed according to the encryption function:
function readFloat4(){
if(obj.decryptFn){
result＝decFn(result，1，2)；
return；
}else{
return result；
}
}。
specifically, the manner of generating the attribute workflow according to the first attribute information may be: and calling a DataView constructor according to the content corresponding to the first attribute information to generate an attribute workflow corresponding to each attribute information.
Specifically, the manner of generating the calling function corresponding to the at least one data item according to the decryption workflow and the attribute workflow may be: combining the decryption workflow and the attribute workflow to generate a calling function corresponding to at least one data item, for example, the generated calling function may be:
readInfo(){
version number of/(terminal)
this.version＝io.readByte2()；
Dimension//
this.lat＝io.readFloat4()；
/(longitude)
this.lng＝io.readFloat4()；
Time of transmission of/(DCS) message
this.crtTime＝io.readByte4()；
Charge/residual quantity
this.terminalPower＝io.readUnsigned4()；
Operation state of the terminal
this.status＝io.readBit1()；
}。
By generating the decryption workflow according to the decryption information, generating the attribute workflow according to the first attribute information, and generating the calling function corresponding to at least one data item according to the decryption workflow and the attribute workflow, the calling function corresponding to the data item can be generated according to the decryption information and the first attribute information, and further, the target data containing the decryption information is analyzed through the calling function, so that the accuracy of analyzing the target data is improved.
According to the technical scheme, original binary data are obtained, and split is carried out on the original binary data to obtain target data; determining a target description file based on a preset parsing rule of the data item; generating a tool package according to the target description file, and storing the tool package to the target file; the target data is read based on the target file, the analysis result corresponding to the target data is obtained, the problem that subsequent system maintenance and expansion are difficult due to inconvenience in binary data analysis is solved, the complexity and the variability of the data frame format of the target data can be effectively improved, the maintainability and the expandability of the system are improved, and the maintenance cost of the system is reduced.
Example two
Fig. 2 is a schematic structural diagram of a data analysis device in a second embodiment of the present invention. The embodiment may be applicable to the case of resolving binary data, where the device may be implemented in software and/or hardware, and the device may be integrated in any device that provides a function of data resolution, as shown in fig. 2, where the data resolving device specifically includes: an acquisition module 210, a determination module 220, a generation module 230, and an acquisition module 240.
The acquiring module 210 is configured to acquire original binary data, split the original binary data, and obtain target data;
a determining module 220, configured to determine a target description file based on a preset parsing rule of the data item;
a generating module 230, configured to generate a tool package according to the target description file, and store the tool package to the target file;
the obtaining module 240 is configured to read the target data based on the target file, and obtain an analysis result corresponding to the target data.
Optionally, the determining module is specifically configured to:
acquiring initial attribute information corresponding to a data item, wherein the initial attribute information comprises at least one of an attribute name, a data type, occupation byte information, description information, a processing mode and decryption information corresponding to each data type;
determining a preset analysis rule according to initial attribute information corresponding to the data item;
generating descriptive content of at least one data item according to a preset parsing rule;
a target description file is generated based on the description content of the at least one data item.
Optionally, the generating module is specifically configured to:
acquiring command line parameters in a command line tool;
acquiring path information corresponding to the target description file and the tool kit based on the command line parameters;
analyzing the target description file based on the command line tool to generate a calling function corresponding to at least one data item;
and generating a tool package according to the calling function corresponding to the at least one data item, and storing the tool package to the target file according to the path information corresponding to the tool package.
Optionally, the obtaining module is specifically configured to:
reading target data based on a target file to obtain a byte array of each frame corresponding to the target data;
calling each frame of byte array through a calling function corresponding to at least one data item in the target file, and obtaining an analysis result corresponding to each frame of byte array;
and generating an analysis result corresponding to the target data according to the analysis result corresponding to the byte array of each frame.
Optionally, the generating module is specifically configured to:
reading a target description file based on a command line tool, and acquiring target attribute information corresponding to at least one data item;
generating a frame sequence according to the target attribute information;
and calling the construction function according to the frame sequence, and generating a calling function corresponding to at least one data item.
Optionally, the target attribute information includes: decryption information and first attribute information;
the generating module is specifically configured to:
generating a decryption workflow according to the decryption information;
generating an attribute workflow according to first attribute information, wherein the first attribute information comprises: at least one of attribute name, data type, occupied byte information corresponding to each data type, description information and processing mode;
and generating a calling function corresponding to at least one data item according to the decryption workflow and the attribute workflow.
The product can execute the method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example III
Fig. 3 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 3, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM12 and the RAM13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the data parsing method.
In some embodiments, the data parsing method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM12 and/or the communication unit 19. When the computer program is loaded into RAM13 and executed by processor 11, one or more steps of the data parsing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the data parsing method in any other suitable way (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (8)
1. A data parsing method, comprising:
acquiring original binary data, and splitting the original binary data to obtain target data;
determining a target description file based on a preset parsing rule of the data item;
generating a tool package according to the target description file, and storing the tool package to the target file;
reading target data based on the target file to obtain an analysis result corresponding to the target data;
generating a tool package according to the target description file, and storing the tool package to the target file, wherein the method comprises the following steps:
acquiring command line parameters in a command line tool;
acquiring path information corresponding to the target description file and the tool kit based on the command line parameters;
analyzing the target description file based on the command line tool to generate a calling function corresponding to at least one data item;
generating a tool package according to the calling function corresponding to the at least one data item, and storing the tool package to the target file according to the path information corresponding to the tool package;
reading target data based on the target file to obtain an analysis result corresponding to the target data, wherein the method comprises the following steps:
reading target data based on a target file to obtain a byte array of each frame corresponding to the target data;
calling each frame of byte array through a calling function corresponding to at least one data item in the target file, and obtaining an analysis result corresponding to each frame of byte array;
and generating an analysis result corresponding to the target data according to the analysis result corresponding to the byte array of each frame.
2. The method of claim 1, wherein determining the target description file based on the preset parsing rule of the data item comprises:
acquiring initial attribute information corresponding to a data item, wherein the initial attribute information comprises at least one of an attribute name, a data type, occupation byte information, description information, a processing mode and decryption information corresponding to each data type;
determining a preset analysis rule according to initial attribute information corresponding to the data item;
generating descriptive content of at least one data item according to a preset parsing rule;
a target description file is generated based on the description content of the at least one data item.
3. The method of claim 1, wherein parsing the object description file based on the command line tool generates a calling function corresponding to at least one data item, comprising:
reading a target description file based on a command line tool, and acquiring target attribute information corresponding to at least one data item;
generating a frame sequence according to the target attribute information;
and calling the construction function according to the frame sequence, and generating a calling function corresponding to at least one data item.
4. A method according to claim 3, wherein the target attribute information comprises: decryption information and first attribute information;
calling the construction function according to the frame sequence to generate a calling function corresponding to at least one data item, wherein the calling function comprises the following steps:
generating a decryption workflow according to the decryption information;
generating an attribute workflow according to first attribute information, wherein the first attribute information comprises: at least one of attribute name, data type, occupied byte information corresponding to each data type, description information and processing mode;
and generating a calling function corresponding to at least one data item according to the decryption workflow and the attribute workflow.
5. A data analysis device, comprising:
the acquisition module is used for acquiring original binary data and splitting the original binary data to obtain target data;
the determining module is used for determining a target description file based on a preset parsing rule of the data item;
the generating module is used for generating a tool kit according to the target description file and storing the tool kit into the target file;
the obtaining module is used for reading the target data based on the target file and obtaining an analysis result corresponding to the target data;
the generating module is specifically configured to:
acquiring command line parameters in a command line tool;
acquiring path information corresponding to the target description file and the tool kit based on the command line parameters;
analyzing the target description file based on the command line tool to generate a calling function corresponding to at least one data item;
generating a tool package according to the calling function corresponding to the at least one data item, and storing the tool package to the target file according to the path information corresponding to the tool package;
the obtaining module is specifically configured to:
reading target data based on a target file to obtain a byte array of each frame corresponding to the target data;
calling each frame of byte array through a calling function corresponding to at least one data item in the target file, and obtaining an analysis result corresponding to each frame of byte array;
and generating an analysis result corresponding to the target data according to the analysis result corresponding to the byte array of each frame.
6. The apparatus of claim 5, wherein the determining module is specifically configured to:
acquiring initial attribute information corresponding to a data item, wherein the initial attribute information comprises at least one of an attribute name, a data type, occupation byte information, description information, a processing mode and decryption information corresponding to each data type;
determining a preset analysis rule according to initial attribute information corresponding to the data item;
generating descriptive content of at least one data item according to a preset parsing rule;
a target description file is generated based on the description content of the at least one data item.
7. An electronic device, the electronic device comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the data parsing method of any one of claims 1-4.
8. A computer readable storage medium storing computer instructions for causing a processor to perform the data parsing method of any one of claims 1-4 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310930995.0A CN116932626B (en) | 2023-07-27 | 2023-07-27 | Data analysis method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310930995.0A CN116932626B (en) | 2023-07-27 | 2023-07-27 | Data analysis method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116932626A CN116932626A (en) | 2023-10-24 |
| CN116932626B true CN116932626B (en) | 2024-04-02 |
Family
ID=88382410
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310930995.0A Active CN116932626B (en) | 2023-07-27 | 2023-07-27 | Data analysis method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116932626B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1859268A (en) * | 2006-02-20 | 2006-11-08 | 华为技术有限公司 | Method and system for backward-forward analytic message at background |
| US8756695B1 (en) * | 2010-10-26 | 2014-06-17 | Emc Corporation | Analysis of binary code |
| CN109814870A (en) * | 2018-12-26 | 2019-05-28 | 东软集团股份有限公司 | Data analysis method, device, storage medium and electronic equipment |
| CN110457526A (en) * | 2019-07-31 | 2019-11-15 | 南京理工大学 | Unitized data analytic method based on xml document |
| CN111090417A (en) * | 2019-11-14 | 2020-05-01 | 杭州中恒电气股份有限公司 | Binary file analysis method, device, equipment and medium |
| CN115544304A (en) * | 2022-10-12 | 2022-12-30 | 东软睿驰汽车技术(大连)有限公司 | File analysis method and device, readable storage medium and file analysis equipment |
| CN115604365A (en) * | 2022-11-15 | 2023-01-13 | 南京芯传汇电子科技有限公司（Cn） | Data encoding and decoding method and device, electronic equipment and readable storage medium |
| CN115701088A (en) * | 2021-07-30 | 2023-02-07 | 比亚迪股份有限公司 | Data packet parsing method, system, storage medium and electronic device |
| CN115794106A (en) * | 2022-11-09 | 2023-03-14 | 北京京投亿雅捷交通科技有限公司 | Method and system for analyzing configuration of binary protocol data of rail transit |
| CN116055305A (en) * | 2022-12-27 | 2023-05-02 | 建信信托有限责任公司 | Method, device, equipment and storage medium for acquiring running state data |
| CN116166845A (en) * | 2022-12-05 | 2023-05-26 | 北京无线电测量研究所 | Universal analysis system and method for radar admission data |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014062194A1 (en) * | 2012-10-19 | 2014-04-24 | Halliburton Energy Services, Inc. | Self-defining configuration apparatus, methods, and systems |
-
2023
- 2023-07-27 CN CN202310930995.0A patent/CN116932626B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1859268A (en) * | 2006-02-20 | 2006-11-08 | 华为技术有限公司 | Method and system for backward-forward analytic message at background |
| US8756695B1 (en) * | 2010-10-26 | 2014-06-17 | Emc Corporation | Analysis of binary code |
| CN109814870A (en) * | 2018-12-26 | 2019-05-28 | 东软集团股份有限公司 | Data analysis method, device, storage medium and electronic equipment |
| CN110457526A (en) * | 2019-07-31 | 2019-11-15 | 南京理工大学 | Unitized data analytic method based on xml document |
| CN111090417A (en) * | 2019-11-14 | 2020-05-01 | 杭州中恒电气股份有限公司 | Binary file analysis method, device, equipment and medium |
| CN115701088A (en) * | 2021-07-30 | 2023-02-07 | 比亚迪股份有限公司 | Data packet parsing method, system, storage medium and electronic device |
| CN115544304A (en) * | 2022-10-12 | 2022-12-30 | 东软睿驰汽车技术(大连)有限公司 | File analysis method and device, readable storage medium and file analysis equipment |
| CN115794106A (en) * | 2022-11-09 | 2023-03-14 | 北京京投亿雅捷交通科技有限公司 | Method and system for analyzing configuration of binary protocol data of rail transit |
| CN115604365A (en) * | 2022-11-15 | 2023-01-13 | 南京芯传汇电子科技有限公司（Cn） | Data encoding and decoding method and device, electronic equipment and readable storage medium |
| CN116166845A (en) * | 2022-12-05 | 2023-05-26 | 北京无线电测量研究所 | Universal analysis system and method for radar admission data |
| CN116055305A (en) * | 2022-12-27 | 2023-05-02 | 建信信托有限责任公司 | Method, device, equipment and storage medium for acquiring running state data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116932626A (en) | 2023-10-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113808231B (en) | Information processing method and device, image rendering method and device, and electronic device | |
| CN114417780B (en) | State synchronization method and device, electronic equipment and storage medium | |
| CN115222444A (en) | Method, apparatus, device, medium and product for outputting model information | |
| CN115311469A (en) | Image labeling method, training method, image processing method and electronic equipment | |
| CN114389969A (en) | Client test method and device, electronic equipment and storage medium | |
| CN112035401A (en) | Model data processing method and device, electronic equipment and readable medium | |
| CN116932626B (en) | Data analysis method, device, equipment and storage medium | |
| CN114880422B (en) | Interest point information processing method based on robot process automation and related device | |
| CN116248772A (en) | Data transmission method, device, equipment and medium under virtualization management | |
| CN115665285A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN114997329A (en) | Method, apparatus, device, medium and product for generating a model | |
| CN115269418A (en) | Coverage rate acquisition method, device, equipment and storage medium | |
| CN114581711A (en) | Target object detection method, apparatus, device, storage medium, and program product | |
| CN114218313A (en) | Data management method, device, electronic equipment, storage medium and product | |
| CN114791996B (en) | Information processing method, device, system, electronic equipment and storage medium | |
| CN113836291B (en) | Data processing method, device, equipment and storage medium | |
| CN116954518B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN113157360B (en) | Method, apparatus, device, medium, and article for processing an API | |
| CN116225568A (en) | Management method, device and equipment of application system interface and storage medium | |
| CN113360382A (en) | Test method, device, equipment and storage medium | |
| CN114064002A (en) | Software development kit generation method, device, equipment and storage medium | |
| CN114968455A (en) | Report generation method and device of application interface and electronic equipment | |
| CN115629939A (en) | Data acquisition method and device for edge device, electronic device and medium | |
| CN115098565A (en) | Method, device, equipment and medium for processing infrared inspection data of power transmission line | |
| CN115265801A (en) | Innovative transformer substation temperature measurement method, device and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |