CN109739821A - Daily record data bedding storage method, apparatus and storage medium - Google Patents
Daily record data bedding storage method, apparatus and storage medium Download PDFInfo
- Publication number
- CN109739821A CN109739821A CN201811546324.XA CN201811546324A CN109739821A CN 109739821 A CN109739821 A CN 109739821A CN 201811546324 A CN201811546324 A CN 201811546324A CN 109739821 A CN109739821 A CN 109739821A
- Authority
- CN
- China
- Prior art keywords
- data
- field
- log
- fractionation
- daily record
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention discloses a kind of daily record data bedding storage method, apparatus and storage mediums, are related to network technique field.The method comprise the steps that obtaining log source data to be stored in response to receiving log storage request；Data prediction is carried out to the log source data, the data prediction includes data redundancy filtration treatment and standard data format processing；Algorithm is split based on data, field fractionation is carried out to pretreated data, obtains multiple fractionation fields；Coded treatment is carried out to the multiple fractionation field respectively, obtains multiple code field data；The multiple code field data are stored in field spacing.The memory space when present invention can reduce storing daily record data occupies, and improves log query efficiency.
Description
Technical field
The present invention relates to network technique field more particularly to a kind of daily record data bedding storage method, apparatus and storage to be situated between
Matter.
Background technique
With the continuous development of network technology, the log type and quantity of generation are also increasing, and log are stored empty
Between demand also increase rapidly.
It is stored by the way of text file in traditional logs storage scheme, the log for directly generating Log Source
Local log file is written in data.Log server timing detects and collects each journal file, and the file of collection is carried out
Compression storage.In query log files, needs first to decompress compressed package, then decompressing files is retrieved, with
Obtain the journal file needed.In face of massive logs data, this mode is unable to satisfy storage demand and search efficiency.
Summary of the invention
The embodiment of the present invention provides a kind of daily record data bedding storage method, apparatus and storage medium, can reduce pair
The space hold that massive logs data are stored.
In order to achieve the above objectives, the embodiment of the present invention adopts the following technical scheme that
In a first aspect, the embodiment of the present invention provides a kind of daily record data bedding storage method, comprising:
In response to receiving log storage request, log source data to be stored is obtained；
Data prediction is carried out to the log source data, the data prediction includes data redundancy filtration treatment and number
It is handled according to standardized format；
Algorithm is split based on data, field fractionation is carried out to pretreated data, obtains multiple fractionation fields；
Coded treatment is carried out to the multiple fractionation field respectively, obtains multiple code field data；
The multiple code field data are stored in field spacing.
With reference to first aspect, in the first possible implementation of the first aspect, described to the multiple fractionation word
Section carries out coded treatment respectively, obtains multiple code field data, comprising:
According to key-value form, coded treatment is carried out to the multiple fractionation field respectively, obtains multiple code fields
Data, the key-value are data type-field data, and the code field data include data type information and field data
Information；
It is described that the multiple code field data are stored in field spacing, comprising:
According to key-value form, the multiple code field data are stored in field spacing.
It is with reference to first aspect, in the second possible implementation of the first aspect, described to split algorithm based on data,
Field fractionation is carried out to pretreated data, before obtaining multiple fractionation fields, further includes:
Obtain the Log Types of the log source data；
It determines in the field spacing with the presence or absence of the Log Types；
If the Log Types are not present in the field spacing, the log class is created in the field spacing
Type；
It is described to split algorithm based on data, field fractionation is carried out to pretreated data, obtains multiple fractionation field packets
It includes:
If there are the Log Types in the field spacing, described to split algorithm based on data, to pretreated
Data carry out field fractionation, obtain multiple fractionation fields.
With reference to first aspect, in a third possible implementation of the first aspect, described to the multiple fractionation word
Section carries out coded treatment respectively, after obtaining multiple code field data, further includes:
Format analysis processing is carried out to the multiple code field data, in the multiple code field data after format analysis processing
The data type information of carrying is orderly numerical string.
With reference to first aspect, in a fourth possible implementation of the first aspect, the method also includes:
In response to receiving log query request, determine that the log query requests corresponding data type query information；
Corresponding daily record data is inquired in the field spacing according to the data type query information；
Processing is decoded to the corresponding daily record data of the data type query information, and is sent to the initiation log and looks into
Ask the client of request.
Second aspect, the embodiment of the present invention provide a kind of daily record data bedding storage device, comprising:
Module is obtained, for obtaining log source data to be stored in response to receiving log storage request；
Preprocessing module, for carrying out data prediction to the log source data, the data prediction includes data
Redundant filtration processing and standard data format processing；
Module is split, for splitting algorithm based on data, field fractionation is carried out to pretreated data, obtains multiple tear open
Divide field；
Coding module obtains multiple code field data for carrying out coded treatment respectively to the multiple fractionation field；
Memory module, for the multiple code field data to be stored in field spacing.
In conjunction with second aspect, in the first possible implementation of the second aspect,
The coding module is also used to carry out at coding the multiple fractionation field respectively according to key-value form
Reason obtains multiple code field data, and the key-value is data type-field data, and the code field data include number
According to type information and field data information；
The memory module, is also used to according to key-value form, and the multiple code field data are stored in field
In space.
In conjunction with second aspect, in a second possible implementation of the second aspect, described device further include:
The acquisition module is also used to obtain the Log Types of the log source data；
Determining module, for determining in the field spacing with the presence or absence of the Log Types；
Creation module is used for when the Log Types are not present in the field spacing, then in the field spacing
Create the Log Types；
The fractionation module, be also used to when in the field spacing there are when the Log Types, then it is described to be based on data
Algorithm is split, field fractionation is carried out to pretreated data, obtains multiple fractionation fields.
In conjunction with second aspect, in the third possible implementation of the second aspect, described device further include:
Format analysis processing module, it is described after format analysis processing for carrying out format analysis processing to the multiple code field data
The data type information carried in multiple code field data is orderly numerical string.
In conjunction with second aspect, in the fourth possible implementation of the second aspect, described device further include:
The determining module is also used to determine that the log query request corresponds in response to receiving log query request
Data type query information；
Enquiry module, for inquiring corresponding log number in the field spacing according to the data type query information
According to；
Decoder module for being decoded processing to the corresponding daily record data of the data type query information, and is sent to
Initiate the client of the log query request.
The third aspect, the embodiment of the present invention provide a kind of computer readable storage medium, are stored thereon with computer journey
Sequence, which is characterized in that the step of method that first aspect provides is realized when described program is executed by processor.
Daily record data bedding storage method, apparatus provided in an embodiment of the present invention and storage medium, by response to receiving
It stores and requests to log, obtain log source data to be stored；Data prediction, the data are carried out to the log source data
Pretreatment includes that data redundancy filtration treatment and standard data format are handled；Algorithm is split based on data, to pretreated
Data carry out field fractionation, obtain multiple fractionation fields；Coded treatment is carried out to the multiple fractionation field respectively, is obtained multiple
Code field data；The multiple code field data are stored in field spacing.Redundant data and some can be filtered out
Extraneous data, it can memory space when reducing storing daily record data under the premise of guaranteeing critical data integrality occupies；
Also, data format is standardized, field disassembled coding and is stored, inquiry item can be passed through in log query
Part quickly finds corresponding daily record data field, so as to effectively improve the efficiency of log query.
Detailed description of the invention
It to describe the technical solutions in the embodiments of the present invention more clearly, below will be to needed in the embodiment
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for ability
For the those of ordinary skill of domain, without creative efforts, it can also be obtained according to these attached drawings other attached
Figure.
Fig. 1 is the flow diagram of the daily record data bedding storage method of the embodiment of the present invention；
Fig. 2 is another flow diagram of the daily record data bedding storage method of the embodiment of the present invention；
Fig. 3 is the system architecture diagram of the daily record data bedding storage method of the embodiment of the present invention；
Fig. 4 is the daily record data bedding storage apparatus structure schematic diagram of the embodiment of the present invention；
Fig. 5 is another structural schematic diagram of the daily record data bedding storage device of the embodiment of the present invention；
Fig. 6 is the structural schematic diagram of the daily record data bedding storage device 600 of the embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts all other
Embodiment shall fall within the protection scope of the present invention.
One embodiment of the invention provides a kind of daily record data bedding storage method, as shown in Figure 1, which comprises
101, in response to receiving log storage request, log source data to be stored is obtained.
102, data prediction is carried out to the log source data, the data prediction includes data redundancy filtration treatment
And standard data format processing.
Wherein, data redundancy filtration treatment includes the mistake of the filtering of duplicate message and garbage in the source data to day
Filter.In embodiments of the present invention, by advance to Log Source data information data redundant filtration processing can reduce to log
The occupancy of data space.
103, algorithm is split based on data, field fractionation is carried out to pretreated data, obtains multiple fractionation fields.
104, coded treatment is carried out to the multiple fractionation field respectively, obtains multiple code field data.
105, the multiple code field data are stored in field spacing.
Compared with prior art, the embodiment of the present invention can filter out redundant data and some extraneous datas, it can
Guarantee that memory space when reducing storing daily record data under the premise of critical data integrality occupies；Also, to data format into
Row standardization, field disassembled coding simultaneously store, and can quickly be found by querying condition corresponding in log query
Daily record data field, so as to effectively improve the efficiency of log query.
Further embodiment of this invention provides a kind of daily record data bedding storage method, as shown in Figure 2, which comprises
201, in response to receiving log storage request, log source data to be stored is obtained.
202, data prediction is carried out to the log source data, the data prediction includes data redundancy filtration treatment
And standard data format processing.
Wherein, data redundancy filtration treatment includes the mistake of the filtering of duplicate message and garbage in the source data to day
Filter.In embodiments of the present invention, by advance to Log Source data information data redundant filtration processing can reduce to log
The occupancy of data space.
203, the Log Types of the log source data are obtained.
204, it determines in the field spacing with the presence or absence of the Log Types.
If can also include: that there is no the Log Types in the field spacing after step 204, in the field
The Log Types are created in space.
If 205, there are the Log Types in the field spacing, described to split algorithm based on data, to pretreatment
Data afterwards carry out field fractionation, obtain multiple fractionation fields.
206, according to key-value form, coded treatment is carried out to the multiple fractionation field respectively, obtains multiple fields
Coded data.
Wherein, the key-value be data type-field data, the code field data include data type information and
Field data information.
It can also include: that format analysis processing carried out to the multiple code field data after step 206, after format analysis processing
The data type information carried in the multiple code field data is orderly numerical string.
207, according to key-value form, the multiple code field data are stored in field spacing.
208, in response to receiving log query request, determine that the log query requests corresponding data type inquiry letter
Breath.
209, corresponding daily record data is inquired in the field spacing according to the data type query information.
210, processing is decoded to the corresponding daily record data of the data type query information, and is sent to and initiates the day
The client of will inquiry request.
Illustratively, as shown in figure 3, being a kind of application of daily record data bedding storage method provided in an embodiment of the present invention
System architecture diagram.In embodiments of the present invention, in log source data (including: input module, custom block, output module) and
Data processing module (i.e. part in Fig. 3 between two " journal queue "), the data processing module are increased between database
It is divided into three parts: data encoding submodule, field spacing submodule and data decoding sub-module.
Wherein, data encoding submodule encodes daily record data, is split, is compiled by pretreatment, log classification, field
Code and five steps of format analysis processing become a daily record data from character type to occupy the lesser numeric type of memory space.Pre- place
Reason, log classification and field tear open and are applied to NLP in three steps.The following are the concrete operations modes of five steps:
(1) pre-treatment step, which refers to, is filtered simultaneously format analysis processing for extraneous datas some in daily record data.
(2) log classifying step refers to the type for judging log, and type is passed to field management module and judges whether to deposit
Among field spacing, the type, then create the field spacing of the type if it does not exist；The type if it exists then directly carries out
Field is split.
(3) field splitting step, which refers to, splits into field for daily record data according to related algorithm.
(4) coding step refers to code field at key-value (numeric type-field), and the key-value is stored
To among corresponding field spacing.
(5) format analysis processing step refers to that the key for the key-value that will be obtained after coding forms orderly numerical string and is passed to day
Will queue is simultaneously stored in database.
Field spacing submodule is made of field management and field memory space.Field management is responsible in field memory space
The storage of field key-value in the creation of field type and each field type.
Decoding sub-module is the inverse process of coding module.Change the field of query statement into key first；Then again by data
The result that library returns changes value into；Result is finally returned into client.
Compared with prior art, the embodiment of the present invention can filter out redundant data and some extraneous datas, it can
Guarantee that memory space when reducing storing daily record data under the premise of critical data integrality occupies；Also, to data format into
Row standardization, field disassembled coding simultaneously store, and can quickly be found by querying condition corresponding in log query
Daily record data field, so as to effectively improve the efficiency of log query.
Further embodiment of this invention provides a kind of daily record data bedding storage device, as shown in figure 4, described device includes:
Module 41 is obtained, for obtaining log source data to be stored in response to receiving log storage request；
Preprocessing module 42, for carrying out data prediction to the log source data, the data prediction includes number
It is handled according to redundant filtration and standard data format is handled；
Module 43 is split, for splitting algorithm based on data, field fractionation is carried out to pretreated data, is obtained multiple
Split field；
Coding module 44 obtains multiple code field numbers for carrying out coded treatment respectively to the multiple fractionation field
According to；
Memory module 45, for the multiple code field data to be stored in field spacing.
The coding module 44 is also used to encode the multiple fractionation field respectively according to key-value form
Processing obtains multiple code field data, and the key-value is data type-field data, and the code field data include
Data type information and field data information；
The memory module 45, is also used to according to key-value form, and the multiple code field data are stored in word
In section space.
Further, as shown in figure 5, described device further include:
The acquisition module 41, is also used to obtain the Log Types of the log source data；
Determining module 51, for determining in the field spacing with the presence or absence of the Log Types；
Creation module 52 is used for when the Log Types are not present in the field spacing, then in the field spacing
The middle creation Log Types；
The fractionation module 43, be also used to when in the field spacing there are when the Log Types, then it is described based on number
According to algorithm is split, field fractionation is carried out to pretreated data, obtains multiple fractionation fields.
Format analysis processing module 53, for carrying out format analysis processing to the multiple code field data, the institute after format analysis processing
Stating the data type information carried in multiple code field data is orderly numerical string.
The determining module 51 is also used to determine the log query request pair in response to receiving log query request
The data type query information answered；
Enquiry module 54, for inquiring corresponding log number in the field spacing according to the data type query information
According to；
Decoder module 55 for being decoded processing to the corresponding daily record data of the data type query information, and is sent
To the client for initiating the log query request.
Compared with prior art, the embodiment of the present invention can filter out redundant data and some extraneous datas, it can
Guarantee that memory space when reducing storing daily record data under the premise of critical data integrality occupies；Also, to data format into
Row standardization, field disassembled coding simultaneously store, and can quickly be found by querying condition corresponding in log query
Daily record data field, so as to effectively improve the efficiency of log query.
The embodiment of the present invention also provides another computer readable storage medium, which can be
Computer readable storage medium included in memory in above-described embodiment；It is also possible to individualism, eventually without supplying
Computer readable storage medium in end.The computer-readable recording medium storage has one or more than one program, institute
State that one or more than one program by one or more than one processor are used to execute Fig. 1, embodiment illustrated in fig. 2 provides
Daily record data bedding storage method.
The embodiment of the method for above-mentioned offer may be implemented in daily record data bedding storage device provided in an embodiment of the present invention, tool
Body function realizes the explanation referred in embodiment of the method, and details are not described herein.Daily record data provided in an embodiment of the present invention point
Layer storage method, device and storage medium can be adapted for storing massive logs data, but be not limited only to this.
As shown in fig. 6, daily record data bedding storage device 600 can be mobile phone, and computer, digital broadcast terminal,
Messaging devices, game console, tablet device, personal digital assistant etc..
Referring to Fig. 6, daily record data bedding storage device 600 may include following one or more components: processing component
502, memory 504, power supply module 506, multimedia component 508, audio component 510, the interface 512 of input/output (I/O),
Sensor module 514 and communication component 516.
Processing component 502 usually control unmanned aerial vehicle (UAV) control device 500 integrated operation, such as with display, call, number
According to communication, camera operation and record operate associated operation.Processing component 502 may include one or more processors 520
To execute instruction.
In addition, processing component 502 may include one or more modules, convenient between processing component 502 and other assemblies
Interaction.For example, processing component 502 may include multi-media module, with facilitate multimedia component 508 and processing component 502 it
Between interaction.
Memory 504 is configured as storing various types of data to support the operation in unmanned aerial vehicle (UAV) control device 500.This
The example of a little data includes the instruction of any application or method for operating on unmanned aerial vehicle (UAV) control device 500, connection
Personal data, telephone book data, message, picture, video etc..Memory 504 can be by any kind of volatibility or non-volatile
It stores equipment or their combination is realized, such as static random access memory (SRAM), the read-only storage of electrically erasable
Device (EEPROM), Erasable Programmable Read Only Memory EPROM (EPROM), programmable read only memory (PROM), read-only memory
(ROM), magnetic memory, flash memory, disk or CD.
Power supply module 506 provides electric power for the various assemblies of unmanned aerial vehicle (UAV) control device 500.Power supply module 506 may include
Power-supply management system, one or more power supplys and other with for unmanned aerial vehicle (UAV) control device 500 generate, manage, and distribute electric power phase
Associated component.
Multimedia component 508 includes one output interface of offer between the unmanned aerial vehicle (UAV) control device 500 and user
Screen.In some embodiments, screen may include liquid crystal display (LCD) and touch panel (TP).If screen includes
Touch panel, screen may be implemented as touch screen, to receive input signal from the user.Touch panel includes one or more
A touch sensor is to sense the gesture on touch, slide, and touch panel.The touch sensor can not only sense touch
Or the boundary of sliding action, but also detect duration and pressure associated with the touch or slide operation.In some realities
It applies in example, multimedia component 508 includes a front camera and/or rear camera.When unmanned aerial vehicle (UAV) control device 500 is in
Operation mode, such as in a shooting mode or a video mode, front camera and/or rear camera can receive external multimedia
Data.Each front camera and rear camera can be a fixed optical lens system or there is focal length and optics to become
Burnt ability.
Audio component 510 is configured as output and/or input audio signal.For example, audio component 510 includes a Mike
Wind (MIC), when unmanned aerial vehicle (UAV) control device 500 is in operation mode, when such as call mode, recording mode, and voice recognition mode,
Microphone is configured as receiving external audio signal.The received audio signal can be further stored in memory 504 or
It is sent via communication component 516.In some embodiments, audio component 510 further includes a loudspeaker, for exporting audio letter
Number.
I/O interface 512 provides interface between processing component 502 and peripheral interface module, and above-mentioned peripheral interface module can
To be keyboard, click wheel, button etc..These buttons may include, but are not limited to: home button, volume button, start button and lock
Determine button.
Sensor module 514 includes one or more sensors, for providing various aspects for unmanned aerial vehicle (UAV) control device 500
Status assessment.For example, sensor module 514 can detecte the state that opens/closes of unmanned aerial vehicle (UAV) control device 500, component
Relative positioning, such as the component is the display and keypad of unmanned aerial vehicle (UAV) control device 500, and sensor module 514 may be used also
To detect the position change of 500 1 components of unmanned aerial vehicle (UAV) control device 500 or unmanned aerial vehicle (UAV) control device, user and unmanned aerial vehicle (UAV) control
The existence or non-existence that device 500 contacts, 500 orientation of unmanned aerial vehicle (UAV) control device or acceleration/deceleration and unmanned aerial vehicle (UAV) control device 500
Temperature change.Sensor module 514 may include proximity sensor, be configured to examine without any physical contact
Survey presence of nearby objects.Sensor module 514 can also include that optical sensor is used for such as CMOS or ccd image sensor
It is used in imaging applications.In some embodiments, which can also include acceleration transducer, and gyroscope passes
Sensor, Magnetic Sensor, pressure sensor or temperature sensor.
Communication component 516 is configured to facilitate wired or wireless way between unmanned aerial vehicle (UAV) control device 500 and other equipment
Communication.Unmanned aerial vehicle (UAV) control device 500 can access the wireless network based on communication standard, such as WiFi, 2G or 3G or they
Combination.In one exemplary embodiment, communication component 516 is received via broadcast channel from the wide of external broadcasting management system
Broadcast signal or broadcast related information.In one exemplary embodiment, the communication component 516 further includes near-field communication (NFC)
Module, to promote short range communication.For example, radio frequency identification (RFID) technology, Infrared Data Association (IrDA) can be based in NFC module
Technology, ultra wide band (UWB) technology, bluetooth (BT) technology and other technologies are realized.
In the exemplary embodiment, unmanned aerial vehicle (UAV) control device 500 can be by one or more application specific integrated circuit
(ASIC), digital signal processor (DSP), digital signal processing appts (DSPD), programmable logic device (PLD), scene can
Gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are programmed to realize.
All the embodiments in this specification are described in a progressive manner, same and similar portion between each embodiment
Dividing may refer to each other, and each embodiment focuses on the differences from other embodiments.Especially for equipment reality
For applying example, since it is substantially similar to the method embodiment, so describing fairly simple, related place is referring to embodiment of the method
Part explanation.
Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with
Relevant hardware is instructed to complete by computer program, the program can be stored in a computer-readable storage medium
In, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, the storage medium can be magnetic
Dish, CD, read-only memory (Read-Only Memory, ROM) or random access memory (Random Access
Memory, RAM) etc..
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
In the technical scope disclosed by the present invention, any changes or substitutions that can be easily thought of by those familiar with the art, all answers
It is included within the scope of the present invention.Therefore, protection scope of the present invention should be subject to the protection scope in claims.
Claims (11)
1. a kind of daily record data bedding storage method characterized by comprising
In response to receiving log storage request, log source data to be stored is obtained；
Data prediction is carried out to the log source data, the data prediction includes data redundancy filtration treatment and data lattice
Formula standardization；
Algorithm is split based on data, field fractionation is carried out to pretreated data, obtains multiple fractionation fields；
Coded treatment is carried out to the multiple fractionation field respectively, obtains multiple code field data；
The multiple code field data are stored in field spacing.
2. daily record data bedding storage method according to claim 1, which is characterized in that described to the multiple fractionation word
Section carries out coded treatment respectively, obtains multiple code field data, comprising:
According to key-value form, coded treatment is carried out to the multiple fractionation field respectively, obtains multiple code field numbers
Be data type-field data according to, the key-value, the code field data include data type information and Field Count it is believed that
Breath；
It is described that the multiple code field data are stored in field spacing, comprising:
According to key-value form, the multiple code field data are stored in field spacing.
3. daily record data bedding storage method according to claim 1, which is characterized in that described split based on data is calculated
Method carries out field fractionation to pretreated data, before obtaining multiple fractionation fields, further includes:
Obtain the Log Types of the log source data；
It determines in the field spacing with the presence or absence of the Log Types；
If the Log Types are not present in the field spacing, the Log Types are created in the field spacing；
It is described to split algorithm based on data, field fractionation is carried out to pretreated data, obtaining multiple fractionation fields includes:
If there are the Log Types in the field spacing, described to split algorithm based on data, to pretreated data
Field fractionation is carried out, multiple fractionation fields are obtained.
4. daily record data bedding storage method according to claim 1, which is characterized in that described to the multiple fractionation word
Section carries out coded treatment respectively, after obtaining multiple code field data, further includes:
Format analysis processing is carried out to the multiple code field data, is carried in the multiple code field data after format analysis processing
Data type information be orderly numerical string.
5. daily record data bedding storage method according to claim 1, which is characterized in that the method also includes:
In response to receiving log query request, determine that the log query requests corresponding data type query information；
Corresponding daily record data is inquired in the field spacing according to the data type query information；
Processing is decoded to the corresponding daily record data of the data type query information, and is sent to the initiation log query and asks
The client asked.
6. a kind of daily record data bedding storage device characterized by comprising
Module is obtained, for obtaining log source data to be stored in response to receiving log storage request；
Preprocessing module, for carrying out data prediction to the log source data, the data prediction includes data redundancy
Filtration treatment and standard data format processing；
Module is split, for splitting algorithm based on data, field fractionation is carried out to pretreated data, obtains multiple fractionation words
Section；
Coding module obtains multiple code field data for carrying out coded treatment respectively to the multiple fractionation field；
Memory module, for the multiple code field data to be stored in field spacing.
7. daily record data bedding storage device according to claim 6, which is characterized in that
The coding module, is also used to according to key-value form, carries out coded treatment respectively to the multiple fractionation field,
Multiple code field data are obtained, the key-value is data type-field data, and the code field data include data
Type information and field data information；
The memory module, is also used to according to key-value form, and the multiple code field data are stored in field spacing
In.
8. daily record data bedding storage device according to claim 6, which is characterized in that described device further include:
The acquisition module is also used to obtain the Log Types of the log source data；
Determining module, for determining in the field spacing with the presence or absence of the Log Types；
Creation module, for when the Log Types are not present in the field spacing, then being created in the field spacing
The Log Types；
The fractionation module is also used to when there are when the Log Types, then the data that are based on are split in the field spacing
Algorithm carries out field fractionation to pretreated data, obtains multiple fractionation fields.
9. daily record data bedding storage device according to claim 6, which is characterized in that described device further include:
Format analysis processing module, it is the multiple after format analysis processing for carrying out format analysis processing to the multiple code field data
The data type information carried in code field data is orderly numerical string.
10. daily record data bedding storage device according to claim 6, which is characterized in that described device further include:
The determining module is also used to determine that the log query requests corresponding number in response to receiving log query request
According to type query information；
Enquiry module, for inquiring corresponding daily record data in the field spacing according to the data type query information；
Decoder module for being decoded processing to the corresponding daily record data of the data type query information, and is sent to initiation
The client of the log query request.
11. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that described program is processed
The step of claim 1-5 the method is realized when device executes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811546324.XA CN109739821A (en) | 2018-12-18 | 2018-12-18 | Daily record data bedding storage method, apparatus and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811546324.XA CN109739821A (en) | 2018-12-18 | 2018-12-18 | Daily record data bedding storage method, apparatus and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109739821A true CN109739821A (en) | 2019-05-10 |
Family
ID=66360487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811546324.XA Pending CN109739821A (en) | 2018-12-18 | 2018-12-18 | Daily record data bedding storage method, apparatus and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109739821A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110958024A (en) * | 2019-12-13 | 2020-04-03 | 深圳市道通智能航空技术有限公司 | Serial port data encoding method and device, embedded device and communication system |
| CN111818011A (en) * | 2020-05-29 | 2020-10-23 | 中国平安财产保险股份有限公司 | Abnormal access behavior recognition method and device, computer equipment and storage medium |
| CN112069048A (en) * | 2020-09-09 | 2020-12-11 | 北京明略昭辉科技有限公司 | Log processing method, device and storage medium |
| CN112569606A (en) * | 2020-12-24 | 2021-03-30 | 厦门极致互动网络技术股份有限公司 | Management system for online game behavior logs |
| CN113099271A (en) * | 2021-04-08 | 2021-07-09 | 天津天地伟业智能安全防范科技有限公司 | Video auxiliary information encoding and decoding methods and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102523131A (en) * | 2011-12-07 | 2012-06-27 | 上海海高通信发展有限公司 | User internet behavior collecting method and system and user internet behavior analyzing method and system |
| CN103178982A (en) * | 2011-12-23 | 2013-06-26 | 阿里巴巴集团控股有限公司 | Method and device for analyzing log |
| CN103929321A (en) * | 2013-01-15 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Log processing method and device |
| CN105138592A (en) * | 2015-07-31 | 2015-12-09 | 武汉虹信技术服务有限责任公司 | Distributed framework-based log data storing and retrieving method |
| US10078654B2 (en) * | 2014-03-18 | 2018-09-18 | Facebook, Inc. | Data logging framework |
-
2018
- 2018-12-18 CN CN201811546324.XA patent/CN109739821A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102523131A (en) * | 2011-12-07 | 2012-06-27 | 上海海高通信发展有限公司 | User internet behavior collecting method and system and user internet behavior analyzing method and system |
| CN103178982A (en) * | 2011-12-23 | 2013-06-26 | 阿里巴巴集团控股有限公司 | Method and device for analyzing log |
| CN103929321A (en) * | 2013-01-15 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Log processing method and device |
| US10078654B2 (en) * | 2014-03-18 | 2018-09-18 | Facebook, Inc. | Data logging framework |
| CN105138592A (en) * | 2015-07-31 | 2015-12-09 | 武汉虹信技术服务有限责任公司 | Distributed framework-based log data storing and retrieving method |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110958024A (en) * | 2019-12-13 | 2020-04-03 | 深圳市道通智能航空技术有限公司 | Serial port data encoding method and device, embedded device and communication system |
| CN110958024B (en) * | 2019-12-13 | 2024-01-19 | 深圳市道通智能航空技术股份有限公司 | Method and device for encoding serial data, embedded equipment and communication system |
| CN111818011A (en) * | 2020-05-29 | 2020-10-23 | 中国平安财产保险股份有限公司 | Abnormal access behavior recognition method and device, computer equipment and storage medium |
| CN112069048A (en) * | 2020-09-09 | 2020-12-11 | 北京明略昭辉科技有限公司 | Log processing method, device and storage medium |
| CN112569606A (en) * | 2020-12-24 | 2021-03-30 | 厦门极致互动网络技术股份有限公司 | Management system for online game behavior logs |
| CN113099271A (en) * | 2021-04-08 | 2021-07-09 | 天津天地伟业智能安全防范科技有限公司 | Video auxiliary information encoding and decoding methods and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109739821A (en) | Daily record data bedding storage method, apparatus and storage medium | |
| US8463025B2 (en) | Distributed artificial intelligence services on a cell phone | |
| CN109656980A (en) | Data processing method, electronic equipment, device and readable storage medium storing program for executing | |
| CN104834665A (en) | Target picture acquiring method and device | |
| CN106488251A (en) | Realize the live middle even method and device of wheat, main broadcaster's client and subscription client | |
| CN104933170A (en) | Information exhibition method and device | |
| CN104967683A (en) | Message pushing method, terminal and server | |
| CN105703932A (en) | Log recording and uploading method, log recording and reception method, terminal, and server | |
| CN104636453A (en) | Illegal user data identification method and device | |
| CN104616241A (en) | Video screen-shot method and device | |
| CN110942143A (en) | Toy detection acceleration method and device based on convolutional neural network | |
| CN108491282A (en) | The method and apparatus that webpage and operating system are communicated | |
| CN104636164A (en) | Generating method and device for starting page | |
| CN104539812A (en) | Recommendation information acquisition method, terminal and server | |
| US20230388416A1 (en) | Emergency communication system with contextual snippets | |
| CN105205153A (en) | File storage method and device | |
| CN105513101B (en) | Image processing method and device | |
| CN109981624B (en) | Intrusion detection method, device and storage medium | |
| CN104317911A (en) | File uploading method and device | |
| CN114446318A (en) | Audio data separation method and device, electronic equipment and storage medium | |
| CN114333804A (en) | Audio classification identification method and device, electronic equipment and storage medium | |
| CN106296150A (en) | Reimbursement tracking process method and device | |
| CN106060138A (en) | Method and apparatus for updating data | |
| CN107402762B (en) | Fingerprint navigation implementation method and device | |
| CN105812530A (en) | Method and device for adding contact means by using fingerprint information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |