CN108763461A - Data processing method, device, system and storage medium - Google Patents
Data processing method, device, system and storage medium Download PDFInfo
- Publication number
- CN108763461A CN108763461A CN201810525826.8A CN201810525826A CN108763461A CN 108763461 A CN108763461 A CN 108763461A CN 201810525826 A CN201810525826 A CN 201810525826A CN 108763461 A CN108763461 A CN 108763461A
- Authority
- CN
- China
- Prior art keywords
- data processing
- data
- processing equipment
- target
- parsing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/163—In-band adaptation of TCP data exchange; In-band control procedures
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
Abstract
A kind of data processing method of the embodiment of the present application offer, device, system and storage medium；The data processing method includes：Long connection is carried out with data processing equipment, and obtains multiple parsing data modes of the data processing equipment；Determine that target parses data mode from the multiple parsing data mode；The corresponding parsing mark of target parsing data mode is sent to the data processing equipment；The target journaling information that the data processing equipment is sent is received, the target journaling information is for the data processing equipment by its collected log information according to the log information after the parsing identification (RNC-ID) analytic.Can be with real-time control data processing equipment, and allow different data processing equipments according to unified parsing data mode parsing log information and upload server.
Description
Technical field
This application involves technical field of data processing, more particularly to a kind of data processing method, device, system and storage are situated between
Matter.
Background technology
With the fast development of electronic technology, the function of the electronic equipments such as smart mobile phone is more and more.For example, user
It can select network connection mode, the standby mode etc. of electronic equipment, user that can also use the various applications on electronic equipment
To realize different functions.
During user uses electronic equipment, it is often necessary to send out various instructions to electronic equipment.And user is usual
It needs to select required instruction from the setting menu for the application installed on the System menu or electronic equipment of electronic equipment.
The mode of this selection instruction is cumbersome, and efficiency is low.
With the development of big data technology, pure far can not meet big data by source data collection to system
The requirement of analysis, present big data platform more require the data that collector uploads to be labelled with set form.Citing comes
It says, for daily record " [info] 2018/12/23, an Xiao Ming：Hello.", it can be seen that this daily record is by can essentially be simple
Divide four parts, the meaning expressed by each section all different.First part indicates the rank of daily record in bracket, second
It is a date to divide, and Part III indicates talker, and Part IV indicates the content spoken, then big data platform is more desirable to
It is what respectively that the data received, which are according to this good four part of format description, is then transmitted up again, and remove symbol useless
Number, the content after parsing identification is similar as follows.Rank：info；Date：2018/12/23；User：Xiao Ming；Speech content：You
It is good.It is envisioned that the daily record of computer system printout at present, format is multifarious, how accurately to know these daily records
It does not parse, becomes a big difficulty again of present data collection.
Invention content
A kind of data processing method of the embodiment of the present application offer, device, system and storage medium, can improve data processing
Accuracy.
The embodiment of the present application provides a kind of data processing method comprising：
Long connection is carried out with data processing equipment, and obtains multiple parsing data modes of the data processing equipment；
Determine that target parses data mode from the multiple parsing data mode；
The corresponding parsing mark of target parsing data mode is sent to the data processing equipment；
The target journaling information that the data processing equipment is sent is received, the target journaling information is the data processing
Equipment is by its collected log information according to the log information after the parsing identification (RNC-ID) analytic.
The embodiment of the present application also provides a kind of data processing equipment comprising：
First acquisition module for carrying out long connection with data processing equipment, and obtains the more of the data processing equipment
A parsing data mode；
Target parses data mode determining module, for determining that target parses number from the multiple parsing data mode
According to mode；
Sending module is set for the corresponding parsing mark of target parsing data mode to be sent to the data processing
It is standby；
Receiving module, the target journaling information sent for receiving the data processing equipment, the target journaling information
For the data processing equipment by its collected log information according to the log information after the parsing identification (RNC-ID) analytic.
The embodiment of the present application also provides a kind of data processing system comprising server and data processing equipment；
The server carries out long connection with data processing equipment；
The server obtains multiple parsing data modes of the data processing equipment；
The server determines that target parses data mode from the multiple parsing data mode；
The corresponding parsing mark of target parsing data mode is sent to the data processing equipment by the server；
The data processing equipment receives the parsing mark, and is identified from multiple parsing data modes according to the parsing
Middle determining target parses data mode；
The data processing equipment acquires log information, and parses data mode according to the target and parse the daily record letter
Breath, obtains target journaling information；
The target journaling information is sent to the server by the data processing equipment；
The server receives the target journaling information that the data processing equipment is sent.
The embodiment of the present application also provides a kind of storage medium, computer program is stored in the storage medium, when described
When computer program is run on computers so that the computer executes above-mentioned data processing method.
Data processing method provided by the embodiments of the present application carries out long connection with data processing equipment first, and obtains number
According to multiple parsing data modes of processing equipment；Then determine that target parses data mode from multiple parsing data modes；
Then the corresponding parsing mark of target parsing data mode is sent to data processing equipment；Finally receive data processing equipment hair
The target journaling information sent, target journaling information are that data processing equipment solves its collected log information according to parsing mark
Log information after analysis.Can be with real-time control data processing equipment, and allow different data processing equipments according to unified parsing
Data mode parses log information and upload server.
Description of the drawings
In order to more clearly explain the technical solutions in the embodiments of the present application, make required in being described below to embodiment
Attached drawing is briefly described.It should be evident that the accompanying drawings in the following description is only some embodiments of the present application, for
For those skilled in the art, without creative efforts, it can also be obtained according to these attached drawings other attached
Figure.
Fig. 1 is the first flow diagram of data processing method provided by the embodiments of the present application.
Fig. 2 is second of flow diagram of data processing method provided by the embodiments of the present application.
Fig. 3 is the third flow diagram of data processing method provided by the embodiments of the present application.
Fig. 4 is the 4th kind of flow diagram of data processing method provided by the embodiments of the present application.
Fig. 5 is the 5th kind of flow diagram of data processing method provided by the embodiments of the present application.
Fig. 6 is the structural schematic diagram of data processing equipment provided by the embodiments of the present application.
Fig. 7 is another structural schematic diagram of data processing equipment provided by the embodiments of the present application.
Fig. 8 is the structural schematic diagram of data processing system provided by the embodiments of the present application.
Specific implementation mode
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete
Site preparation describes.Obviously, described embodiments are only a part of embodiments of the present application, instead of all the embodiments.It is based on
Embodiment in the application, the every other implementation that those skilled in the art are obtained under the premise of not making the creative labor
Example, belongs to the protection domain of the application.
Term " first ", " second ", " third " in the description and claims of this application and above-mentioned attached drawing etc.
(if present) is for distinguishing similar object, without being used to describe specific sequence or precedence.It should be appreciated that this
The object of sample description can be interchanged in the appropriate case.In addition, term " comprising " and " having " and their any deformation, meaning
Figure, which is to cover, non-exclusive includes.For example, containing the process of series of steps, method or containing a series of modules or list
The device of member, electronic equipment, system those of are not necessarily limited to clearly to list step or module or unit, can also include not having
The step of clearly listing or module or unit can also include for these processes, method, apparatus, electronic equipment or system
Intrinsic other steps or module or unit.
The embodiment of the present application provides a kind of data processing method, and data processing method can be applied to server end, also may be used
To be applied to data processing equipment, server can be cloud server, remote server, local server etc., data processing
Equipment can be the electronic equipments such as computer, mobile phone.
In some embodiments, with reference to figure 1, which can be based on server, and this method can specifically wrap
Include following steps：
101, long connection is carried out with data processing equipment, and obtain multiple parsing data modes of data processing equipment.
Server carries out long connection with data processing equipment, such as can carry out long connection by Transmission Control Protocol.In this way, clothes
Business device and data processing equipment can in real time send data to data processing equipment with real-time communication, server, can also be real
When receive the data that are sent from data processing equipment.
Multiple parsing data modes of server or data processing equipment, specifically, server can send control instruction
To data processing equipment, data processing equipment is sent to service according to multiple parsing data modes that control instruction is had
Device.The identity information of data processing equipment can also be sent to server, server by data processing equipment according to control instruction
Multiple parsing data modes that the data processing equipment has are obtained according to the identity information of data processing equipment.
Can also be after server and data processing equipment establish connection, multiple solutions that data processing equipment is included actively
Analysis data mode is sent to server.
102, determine that target parses data mode from multiple parsing data modes.
Server determines that target parses data mode from multiple parsing data modes.Specifically, user can be detected
Setting information, and determine that from multiple parsing data modes, target parses data mode according to the setting information of user.?
It can determine that target parses data mode from multiple parsing data modes according to the preset setting information of server.It can also
Server first obtains the parsing data mode of the log information and the log information that have been stored in server, then from multiple
It parses and determines with the identical parsing data mode of the log information of server to be that target parses data mode in data mode.
103, the corresponding parsing mark of target parsing data mode is sent to data processing equipment.
After obtaining target parsing data mode, the corresponding parsing mark of target parsing data mode is sent at data
Manage equipment.Parsing is identified as the identification information that server and data processing equipment have been made an appointment.
104, receive data processing equipment send target journaling information, target journaling information be data processing equipment by its
Collected log information is according to the log information after parsing identification (RNC-ID) analytic.
Finally receive the target journaling information that data processing equipment is sent, the acquisition modes of the target journaling information can be with
For：Data processing equipment first obtains target analysis mode according to parsing mark, then will be collected according to target analysis mode
Log information is parsed, the target journaling information after being parsed.
In some embodiments, it after server obtains target journaling information, saves it in corresponding memory, may be used also
To be shown.Specifically, can be illustrated on the display screen being connect with server.
In some embodiments, with reference to figure 2, which can specifically include following steps：
201, long connection is carried out with data processing equipment, and obtain multiple parsing data modes of data processing equipment.
Server carries out long connection with data processing equipment, such as can carry out long connection by Transmission Control Protocol.In this way, clothes
Business device and data processing equipment can in real time send data to data processing equipment with real-time communication, server, can also be real
When receive the data that are sent from data processing equipment.
Server obtains multiple parsing data modes of data processing equipment, refers to specifically, server can send control
It enables to data processing equipment, data processing equipment is sent to service according to multiple parsing data modes that control instruction is had
Device.The identity information of data processing equipment can also be sent to server, server by data processing equipment according to control instruction
Multiple parsing data modes that the data processing equipment has are obtained according to the identity information of data processing equipment.
Can also be after server and data processing equipment establish connection, multiple solutions that data processing equipment is included actively
Analysis data mode is sent to server.
202, determine that target parses data mode from multiple parsing data modes.
Server determines that target parses data mode from multiple parsing data modes.Specifically, user can be detected
Setting information, and determine that from multiple parsing data modes, target parses data mode according to the setting information of user.?
It can determine that target parses data mode from multiple parsing data modes according to the preset setting information of server.It can also
Server first obtains the parsing data mode of the log information and the log information that have been stored in server, then from multiple
It parses and determines with the identical parsing data mode of the log information of server to be that target parses data mode in data mode.
203, the corresponding parsing mark of target parsing data mode is sent to data processing equipment.
After obtaining target parsing data mode, the corresponding parsing mark of target parsing data mode is sent at data
Manage equipment.Parsing is identified as the identification information that server and data processing equipment have been made an appointment.
204, obtain multiple data transfer methods of data processing equipment.
Server obtains multiple data transfer methods of data processing equipment, refers to specifically, server can send control
It enables to data processing equipment, data processing equipment is sent to service according to multiple data transfer methods that control instruction is had
Device.The identity information of data processing equipment can also be sent to server, server by data processing equipment according to control instruction
Multiple data transfer methods that the data processing equipment has are obtained according to the identity information of data processing equipment.
Can also be after server and data processing equipment establish connection, multiple numbers that data processing equipment is included actively
It is sent to server according to conversion regime.
In some embodiments, the step that 204 steps corresponded to can be corresponding with 201 is closed in one step, and 204 correspond to
The step of can be arranged 201 correspond to step before.
205, target data conversion regime is determined from multiple data transfer methods.
Server determines target data conversion regime from multiple data transfer methods.Specifically, user can be detected
Setting information, and target data conversion regime is determined from multiple data transfer methods according to the setting information of user.?
Target data conversion regime can be determined from multiple data transfer methods according to the preset setting information of server.It can also
Server first obtains the data transfer method of the log information and the log information that have been stored in server, then from multiple
Determine that with the identical data transfer method of the log information of server be target data conversion regime in data transfer method.
In some embodiments, the step that 205 steps corresponded to can be corresponding with 202 is closed in one step, and 205 correspond to
The step of can be arranged 202 correspond to step before.
206, the corresponding switch signs of data transfer method are sent to data processing equipment.
After obtaining target data conversion regime, which is sent at data
Manage equipment.Switch signs are the identification information that server and data processing equipment have been made an appointment.
In some embodiments, the step that 206 steps corresponded to can be corresponding with 203 is closed in one step, and 206 correspond to
The step of can be arranged 203 correspond to step before.
207, receive data processing equipment send target journaling information, target journaling information be data processing equipment by its
Collected log information is according to parsing identification (RNC-ID) analytic and according to the transformed log information of switch signs.
Finally receive the target journaling information that data processing equipment is sent, the acquisition modes of the target journaling information can be with
For：Data processing equipment first obtains target parsing data mode and obtains target according to switch signs to turn according to parsing mark
Data mode is changed, then parsing data mode according to target parses collected log information, the day after being parsed
Will information converts the log information after parsing then according to target data conversion regime, obtains transformed daily record letter
Breath, the transformed log information are target journaling information.
Can also data processing equipment target parsing data mode and according to switch signs is first obtained according to parsing mark
Target data conversion regime is obtained, then log information is converted according to target data conversion regime, obtains transformed daily record
Information parses transformed log information then according to target parsing data mode, the log information after being parsed,
Log information after the parsing is target journaling information.
In some embodiments, it after server obtains target journaling information, saves it in corresponding memory, may be used also
To be shown.Specifically, can be illustrated on the display screen being connect with server.
In some embodiments, before the step of receiving the target journaling information that data processing equipment is sent, further include：
Obtain multiple data sender's formulas of data processing equipment；Target data is determined from multiple data sender's formulas
Sending method；
After the step of receiving the target journaling information that data processing equipment is sent, further include：
Receive the target journaling information that data processing equipment is sent according to target data sending method.
In some embodiments, with reference to figure 3, which can specifically include following steps：
301, long connection is carried out with data processing equipment, and obtain multiple parsing data modes of data processing equipment.
Server obtains multiple parsing data modes of data processing equipment, refers to specifically, server can send control
It enables to data processing equipment, data processing equipment is sent to service according to multiple parsing data modes that control instruction is had
Device.The identity information of data processing equipment can also be sent to server, server by data processing equipment according to control instruction
Multiple parsing data modes that the data processing equipment has are obtained according to the identity information of data processing equipment.
Can also be after server and data processing equipment establish connection, multiple solutions that data processing equipment is included actively
Analysis data mode is sent to server.
302, determine that target parses data mode from multiple parsing data modes.
Server determines that target parses data mode from multiple parsing data modes.Specifically, user can be detected
Setting information, and determine that from multiple parsing data modes, target parses data mode according to the setting information of user.?
It can determine that target parses data mode from multiple parsing data modes according to the preset setting information of server.It can also
Server first obtains the parsing data mode of the log information and the log information that have been stored in server, then from multiple
It parses and determines with the identical parsing data mode of the log information of server to be that target parses data mode in data mode.
303, the corresponding parsing mark of target parsing data mode is sent to data processing equipment, while generating control and appointing
Business.
After obtaining target parsing data mode, the corresponding parsing mark of target parsing data mode is sent at data
Manage equipment.Parsing is identified as the identification information that server and data processing equipment have been made an appointment.
When parsing mark is sent to data processing equipment by server, while it is corresponding to generate a control task (task ID)
Data processing equipment execute as a result, on the one hand allow data processing equipment execute data parsing task.It is returned if running succeeded
If returning successful as a result, executing failure, the wrong content of failure is returned.On the other hand before control task (task ID) being returned to
Hold interface.
In some embodiments, data processing equipment can execute data collection, parsing, conversion, transmission task.
304, the target journaling information that real-time reception data processing equipment is sent, and by the target journaling information storage in number
According in library.
It is, for example, possible to use MongoDB databases are stored.
Target journaling information is for data processing equipment by its collected log information according to the day after parsing identification (RNC-ID) analytic
Will information.
305, pass through control task polling data library.
Server constantly passes through control task polling data library.Server may include display interface, and display interface is real-time
Pass through control task polling data library.
306, when increasing new target journaling information in database, obtain new target journaling information.
When database looks for that increase new target journaling information, illustrate, data processing equipment is to log information processing
Success, and return to the target journaling information after handling successfully.
307, control task is shown new target journaling information is increased.
Server corresponds to a display interface, and control task shows new target journaling information by the display interface.
If data processing equipment returns to the information of failure, there is no log information such as, to log information processing failure etc., equally by failure
Information is shown also by display interface.
It should be noted that in above-mentioned all embodiments, data processing equipment can be multiple, and multiple data processings are set
An Agent (agent) is arranged in each data processing equipment in standby, and server is controlled each by the Agent
Data processing equipment.The Agent is the control program in above-described embodiment.
The Agent can have data collection, data parsing, data conversion, data transmission capabilities.Server is arranged
One server program, server program establish long connection with Agent.
Server can configure an interface, which can control the data collection of data processing equipment, parsing, conversion
And transmission, the interface can be with real-time exhibition data processing equipments to the result of data processing.
When user is after completion configuration (and authorized agency's program) is filled at interface, configuration file is sent to specified by server
Agent, and generate a task ID corresponds to Agent execution as a result, on the one hand allow Agent execution data collection,
Parsing, sends task at conversion.If being returned if running succeeded successful as a result, executing failure, return in the mistake of failure
Hold.On the other hand task ID is returned into front-end interface.
In some embodiments, with reference to figure 4, which can be applied to data processing equipment, at the data
Reason method can specifically include following steps：
401, long connection is carried out with server, and receive the configuration information of server transmission.
Data processing equipment carries out long connection with server, such as can carry out long connection by Transmission Control Protocol.In this way, clothes
Business device and data processing equipment can in real time send data to server with real-time communication, data processing equipment, can also be real
When receive the data that are sent from server.
Server obtains multiple parsing data modes of data processing equipment, refers to specifically, server can send control
It enables to data processing equipment, data processing equipment is sent to service according to multiple parsing data modes that control instruction is had
Device.The identity information of data processing equipment can also be sent to server, server by data processing equipment according to control instruction
Multiple parsing data modes that the data processing equipment has are obtained according to the identity information of data processing equipment.
Can also be after server and data processing equipment establish connection, multiple solutions that data processing equipment is included actively
Analysis data mode is sent to server.
Data processing equipment receives the configuration information that server is sent.
It can be collected, parsing, conversion, be sent with configuration data specifically, server may provide the user with one, and in real time
Show the interface of result.Then when user fills in completion with postponing at interface, configuration file is sent to specified data by server
Processing equipment.
402, log information is acquired, and determine that target parses data side from multiple parsing data modes according to configuration information
Formula.
The local log information of data processing equipment acquisition, and the configuration information sent according to server is from multiple parsing numbers
According to determining one in mode data mode is parsed for target.
403, data mode is parsed according to target and parses log information, obtains target journaling information.
404, target journaling information is sent to server.
Using the log information after parsing as target journaling information, and occurred to server.
In some embodiments, with reference to figure 5, which can be applied to data processing equipment, at the data
Reason method can specifically include following steps：
501, setting and the associated processing routine of server on data processing equipment.
A processing routine is disposed on data processing equipment in advance, which has data collection, data parsing, number
According to conversion, data transmission capabilities.The processing routine carries out long connection with server.
In some embodiments, server also disposes a service routine, which connects with processing routine length.
502, processing routine carries out long connection with server, and receives the configuration information of server transmission；Configuration information packet
Include and the corresponding parsing mark of target analysis mode and switch signs corresponding with target conversion regime.
Data processing equipment carries out long connection with server, such as can carry out long connection by Transmission Control Protocol.In this way, clothes
Business device and data processing equipment can in real time send data to server with real-time communication, data processing equipment, can also be real
When receive the data that are sent from server.
Server obtains multiple parsing data modes of data processing equipment, refers to specifically, server can send control
It enables to data processing equipment, data processing equipment is sent to service according to multiple parsing data modes that control instruction is had
Device.The identity information of data processing equipment can also be sent to server, server by data processing equipment according to control instruction
Multiple parsing data modes that the data processing equipment has are obtained according to the identity information of data processing equipment.
Can also be after server and data processing equipment establish connection, multiple solutions that data processing equipment is included actively
Analysis data mode is sent to server.
Data processing equipment receives the configuration information that server is sent.
It can be collected, parsing, conversion, be sent with configuration data specifically, server may provide the user with one, and in real time
Show the interface of result.Then when user fills in completion with postponing at interface, configuration file is sent to specified data by server
Processing equipment.
Wherein, configuration information includes corresponding with target analysis mode parsing mark and corresponding with target conversion regime
Switch signs.
In some embodiments, configuration information can also include transmission mark corresponding with target sending method.
503, it is identified according to parsing, target analysis mode is determined from multiple data analysis modes of processing routine.
504, according to switch signs, target conversion regime is determined from multiple data transfer methods of processing routine.
Wherein 503 steps corresponded to and 504 steps that correspond to can 503 steps corresponded to preceding, can also be 504 corresponding
Step is preceding.
505, the log information of acquisition is handled according to target analysis mode and target conversion regime, obtains target day
Will information.
Specifically, log information can be parsed, the log information after being parsed, then by the daily record after parsing
Information is converted, and transformed log information is obtained, and the transformed log information is as target journaling information.
Log information can also be converted, obtain transformed log information, then by transformed log information
It is parsed, the log information after being parsed, the log information after the parsing is as target journaling information.
506, target journaling information is sent to server.
After parsing and transformed log information is as target journaling information, and is occurred to server.
From the foregoing, it will be observed that the data processing method of the embodiment of the present application, carries out long connection, and obtain with data processing equipment first
Take multiple parsing data modes of data processing equipment；Then determine that target parses data side from multiple parsing data modes
Formula；Then the corresponding parsing mark of target parsing data mode is sent to data processing equipment；Data processing is finally received to set
The target journaling information that preparation is sent, target journaling information are that data processing equipment marks its collected log information according to parsing
Know the log information after parsing.Can be with real-time control data processing equipment, and allow different data processing equipments according to unification
Parse data mode parsing log information and upload server.The application can assist user accurately to configure identification not in real time
With the system log of format, the log content after collection is regular.By long connection can in real time to digital independent, parsing, turn
It changes, send this complete procedure and carry out rehearsal feedback, precisely configured in real time, greatly improve the effect of data collector configuration
Rate.Originally the data analytic ability for existing only in data processing equipment is integrated into server unified management, to identical situation,
It need to only configure primary；Different configurations can also be carried out in server for different in the case of.So that user can with batch operation,
But true property without misalignment, reduces complexity.
The new demand that big data platform is generated for collecting data at present it is expected to carry out accurately the data of collection
Parsing, while carrying out different configurations for different collection formats, facilitates and unified has managed different log collection processes.
The application can be in unified server-side (server) to being deployed on different computer nodes (data processing equipment) agency
Program (agent) is controlled, and carries out data collection, data parsing, data turn in server-side in real time for different machines
It changes, the different configuration of debugging such as data are sent and issue process.
The embodiment of the present application also provides a kind of data processing equipment, and data processing equipment can integrate in the server.
In some embodiments, with reference to figure 6, data processing equipment 600 may include the first acquisition module 601, target solution
Analyse data mode determining module 602, sending module 603 and receiving module 604.
First acquisition module 601 for carrying out long connection with data processing equipment, and obtains the multiple of data processing equipment
Parse data mode；
Target parses data mode determining module 602, for determining that target parses number from multiple parsing data modes
According to mode；
Sending module 603, for the corresponding parsing mark of target parsing data mode to be sent to data processing equipment；
Receiving module 604, the target journaling information for receiving data processing equipment transmission, target journaling information are data
Processing equipment is by its collected log information according to the log information after parsing identification (RNC-ID) analytic.
In some embodiments, data processing equipment can be with the second acquisition module.
Second acquisition module, multiple data transfer methods for obtaining data processing equipment；From multiple data conversion sides
Target data conversion regime is determined in formula.
Sending module is additionally operable to the corresponding switch signs of data transfer method being sent to data processing equipment.
Receiving module is additionally operable to receive the target journaling information that data processing equipment is sent, and target journaling information is data
Processing equipment is by its collected log information according to parsing identification (RNC-ID) analytic and according to the transformed log information of switch signs.
In some embodiments, data processing equipment can also include third acquisition module.
Third acquisition module, multiple data sender's formulas for obtaining data processing equipment；From multiple data senders
Target data sending method is determined in formula.
Receiving module is additionally operable to receive the target journaling letter that data processing equipment is sent according to target data sending method
Breath.
In some embodiments, data processing equipment can also include display module.
Sending module, when being additionally operable to the corresponding parsing mark of target parsing data mode being sent to data processing equipment,
Generate control task；
Display module shows target journaling information for control task.
In some embodiments, data processing equipment can also include poller module and third acquisition module.
Receiving module, is additionally operable to the target journaling information of real-time reception data processing equipment transmission, and target journaling is believed
Breath storage is in the database；
Poller module, for passing through control task polling data library；
Third acquisition module, for when increasing new target journaling information in database, obtaining new target journaling letter
Breath；
Display module is additionally operable to show new target journaling information.
In some embodiments, with reference to figure 7, data processing equipment 700 can be arranged on data processing equipment, the data
Processing unit 700 may include configuration information receiving module 701, the second determining module 702, third determining module 703 and second
Sending module 704.
Configuration information receiving module 701, for carrying out long connection with server, and receive server transmission match confidence
Breath；
Second determining module 702, for acquiring log information, and it is true from multiple parsing data modes according to configuration information
Set the goal parsing data mode；
Log information is parsed for parsing data mode according to target, obtains target day by third determining module 703
Will information；
Second sending module 704, for target journaling information to be sent to server.
In some embodiments, data processing equipment can also include processing routine setup module.
Processing routine setup module, for the setting on data processing equipment and the associated processing routine of server.
Second determining module is additionally operable to be identified according to parsing, mesh is determined from multiple data analysis modes of processing routine
Mark analysis mode；According to switch signs, target conversion regime is determined from multiple data transfer methods of processing routine.
Third determining module is additionally operable to be handled log information according to target analysis mode and target conversion regime,
Obtain target journaling information.
From the foregoing, it will be observed that the embodiment of the present application device, first the first acquisition module 601 carry out long company with data processing equipment
It connects, and obtains multiple parsing data modes of data processing equipment；Then target parsing data mode determining module 602 is from multiple
Determine that target parses data mode in parsing data mode；Then sending module 603 is corresponding by target parsing data mode
Parsing mark is sent to data processing equipment；Last receiving module 604 receives the target journaling information that data processing equipment is sent,
Target journaling information is for data processing equipment by its collected log information according to the log information after parsing identification (RNC-ID) analytic.It can
With real-time control data processing equipment, and allow different data processing equipments according to unified parsing data mode parsing daily record letter
Cease simultaneously upload server.The application can assist user accurately to configure the system log of identification different-format in real time, collect
Log content after regular.By long connection can in real time to digital independent, parsing, conversion, send this complete procedure into
Row rehearsal feedback, is precisely configured, greatly improves the efficiency of data collector configuration in real time.Data will be existed only in originally
The data analytic ability of processing equipment is integrated into server unified management, to identical situation, need to only configure primary；For difference
The case where different configurations can also be carried out in server.So that user can be with batch operation, but true property without misalignment reduces multiple
Miscellaneous degree.
In some embodiments, with reference to figure 8, which includes server 810 and data processing equipment
820.Server 810 and data processing equipment 820 are connected by communication network 830.
Server 810 carries out long connection with data processing equipment 820；
Server 810 obtains multiple parsing data modes of data processing equipment 820；
Server 810 determines that target parses data mode from multiple parsing data modes；
The corresponding parsing mark of target parsing data mode is sent to data processing equipment 820 by server 810；
Data processing equipment 820 receives parsing mark, and mesh is determined from multiple parsing data modes according to parsing mark
Mark parsing data mode；
Data processing equipment 820 acquires log information, and parses data mode according to target and parse log information, obtains mesh
Mark log information；
Target journaling information is sent to server 810 by data processing equipment 820；
Server 810 receives the target journaling information that data processing equipment is sent.
The embodiment of the present application also provides a kind of storage medium, computer program is stored in the storage medium, when described
When computer program is run on computers, the computer executes the data processing method described in any of the above-described embodiment.
It should be noted that one of ordinary skill in the art will appreciate that whole in the various methods of above-described embodiment or
Part steps are relevant hardware can be instructed to complete by computer program, and the computer program can be stored in meter
In calculation machine readable storage medium storing program for executing, the storage medium can include but is not limited to：Read-only memory (ROM, Read Only
Memory), random access memory (RAM, Random Access Memory), disk or CD etc..
Data processing method, device, system and the storage medium provided above the embodiment of the present application has carried out in detail
It introduces, specific examples are used herein to illustrate the principle and implementation manner of the present application, the explanation of above example
It is merely used to help understand the present processes and its core concept；Meanwhile for those skilled in the art, according to the application
Thought, there will be changes in the specific implementation manner and application range, in conclusion the content of the present specification should not be understood
For the limitation to the application.
Claims (10)
1. a kind of data processing method, which is characterized in that including：
Long connection is carried out with data processing equipment, and obtains multiple parsing data modes of the data processing equipment；
Determine that target parses data mode from the multiple parsing data mode；
The corresponding parsing mark of target parsing data mode is sent to the data processing equipment；
The target journaling information that the data processing equipment is sent is received, the target journaling information is the data processing equipment
By its collected log information according to the log information after the parsing identification (RNC-ID) analytic.
2. data processing method according to claim 1, which is characterized in that described to receive the data processing equipment transmission
Target journaling information the step of before, further include：
Obtain multiple data transfer methods of the data processing equipment；
Target data conversion regime is determined from the multiple data transfer method；
The corresponding switch signs of the data transfer method are sent to the data processing equipment；
The target journaling information for receiving the data processing equipment and sending, the target journaling information are the data processing
Equipment by its collected log information according to it is described parsing identification (RNC-ID) analytic after log information the step of, including：
The target journaling information that the data processing equipment is sent is received, the target journaling information is the data processing equipment
By its collected log information according to the parsing identification (RNC-ID) analytic and according to the transformed log information of switch signs.
3. data processing method according to claim 1, which is characterized in that described to receive the data processing equipment transmission
Target journaling information the step of before, further include：
Obtain multiple data sender's formulas of the data processing equipment；
Target data sending method is determined from the multiple data sender's formula；
Described the step of receiving the target journaling information that the data processing equipment is sent, including：
Receive the target journaling information that the data processing equipment is sent according to the target data sending method.
4. data processing method according to claim 1, which is characterized in that
When the corresponding parsing mark of target parsing data mode is sent to the data processing equipment, generates control and appoint
Business；
After the step of receiving the target journaling information that the data processing equipment is sent, further include：
The control task shows the target journaling information.
5. data processing method according to claim 4, which is characterized in that further include：
The target journaling information that data processing equipment described in real-time reception is sent, and by the target journaling information storage in data
In library；
Pass through database described in the control task poll；
When increasing new target journaling information in the database, new target journaling information is obtained；
By the new target journaling information displaying.
6. a kind of data processing method is applied to data processing equipment, which is characterized in that the method includes：
Long connection is carried out with server, and receives the configuration information that the server is sent；
Log information is acquired, and determines that target parses data mode from multiple parsing data modes according to the configuration information；
Data mode is parsed according to the target to parse the log information, obtains target journaling information；
The target journaling information is sent to server.
7. data processing method according to claim 6, which is characterized in that the configuration information includes and target parsing side
The corresponding parsing mark of formula and switch signs corresponding with target conversion regime；The method further includes：
Setting and the associated processing routine of server on data processing equipment；
It is identified according to the parsing, target analysis mode is determined from multiple data analysis modes of the processing routine；
According to the switch signs, target conversion regime is determined from multiple data transfer methods of the processing routine；
It is described to be parsed the log information according to target parsing data mode, obtain the step of target journaling information
Suddenly, including：
The log information is handled according to the target analysis mode and the target conversion regime, obtains target journaling
Information.
8. a kind of data processing equipment, which is characterized in that including：
First acquisition module for carrying out long connection with data processing equipment, and obtains multiple solutions of the data processing equipment
Analyse data mode；
Target parses data mode determining module, for determining that target parses data side from the multiple parsing data mode
Formula；
Sending module, for the corresponding parsing mark of target parsing data mode to be sent to the data processing equipment；
Receiving module, the target journaling information sent for receiving the data processing equipment, the target journaling information is institute
Data processing equipment is stated by its collected log information according to the log information after the parsing identification (RNC-ID) analytic.
9. a kind of data processing system, which is characterized in that including server and data processing equipment；
The server carries out long connection with data processing equipment；
The server obtains multiple parsing data modes of the data processing equipment；
The server determines that target parses data mode from the multiple parsing data mode；
The corresponding parsing mark of target parsing data mode is sent to the data processing equipment by the server；
The data processing equipment receives the parsing mark, and true from multiple parsing data modes according to parsing mark
Set the goal parsing data mode；
The data processing equipment acquires log information, and parses data mode according to the target and parse the log information,
Obtain target journaling information；
The target journaling information is sent to the server by the data processing equipment；
The server receives the target journaling information that the data processing equipment is sent.
10. a kind of storage medium, which is characterized in that computer program is stored in the storage medium, when the computer journey
When sequence is run on computers so that the computer perform claim requires 1 to 7 any one of them data processing method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810525826.8A CN108763461A (en) | 2018-05-28 | 2018-05-28 | Data processing method, device, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810525826.8A CN108763461A (en) | 2018-05-28 | 2018-05-28 | Data processing method, device, system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108763461A true CN108763461A (en) | 2018-11-06 |
Family
ID=64003009
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810525826.8A Pending CN108763461A (en) | 2018-05-28 | 2018-05-28 | Data processing method, device, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108763461A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130054680A1 (en) * | 2011-08-23 | 2013-02-28 | Fujitsu Limited | Apparatus and method for collecting log information from a plurality of servers |
| CN103116690A (en) * | 2012-12-14 | 2013-05-22 | 东软熙康健康科技有限公司 | Method and intelligent terminal for obtaining medical data from medical device |
| CN106202324A (en) * | 2016-06-30 | 2016-12-07 | 北京奇虎科技有限公司 | The data processing method of a kind of real-time calculating platform and device |
| CN107273269A (en) * | 2017-06-12 | 2017-10-20 | 北京奇虎科技有限公司 | Daily record analysis method and device |
-
2018
- 2018-05-28 CN CN201810525826.8A patent/CN108763461A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130054680A1 (en) * | 2011-08-23 | 2013-02-28 | Fujitsu Limited | Apparatus and method for collecting log information from a plurality of servers |
| CN103116690A (en) * | 2012-12-14 | 2013-05-22 | 东软熙康健康科技有限公司 | Method and intelligent terminal for obtaining medical data from medical device |
| CN106202324A (en) * | 2016-06-30 | 2016-12-07 | 北京奇虎科技有限公司 | The data processing method of a kind of real-time calculating platform and device |
| CN107273269A (en) * | 2017-06-12 | 2017-10-20 | 北京奇虎科技有限公司 | Daily record analysis method and device |
Non-Patent Citations (1)
| Title |
|---|
| 毕红双等: "多源日志的远程存储和分析管理", 《信息通信》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2017041406A1 (en) | Failure positioning method and device | |
| CN103441861B (en) | A kind of data record generation method and device | |
| CN110932918B (en) | Log data acquisition method and device and storage medium | |
| CN111211934B (en) | Cluster remote communication test method and system | |
| CN110932887A (en) | BMC debugging method, system and device | |
| CN110321531A (en) | Laboratory Instruments information management system | |
| CN110334147A (en) | A kind of method of data synchronization and device | |
| CN109347812A (en) | A kind of industry control bug excavation method and system | |
| CN109783330B (en) | Log processing method, log display method, and related device and system | |
| CN115615732A (en) | Quality detector abnormal state monitoring method and system | |
| CN112188319A (en) | Data processing method of optical fiber distribution frame and rear-end monitoring platform | |
| CN113067710A (en) | Online user query method and device, computer equipment and storage medium | |
| CN106130787B (en) | Service processing system based on software defined network | |
| CN108763461A (en) | Data processing method, device, system and storage medium | |
| KR100276882B1 (en) | Integrated operation management device and method of broadband access device | |
| CN110969082A (en) | Clock synchronization test inspection method and system | |
| CN101232405B (en) | IPMI message monitoring system and method under ATCA platform | |
| EP4373047A1 (en) | Network fault root cause determining method and apparatus, device, and storage medium | |
| KR100358156B1 (en) | Converting Method of Managing Operation from service management system to Switching Command in a Integrated Network | |
| JP5312124B2 (en) | Network management method and system, and network management system program | |
| CN109613423A (en) | High-voltage circuitbreaker on-line monitoring and trouble analysis system based on vibration signal | |
| CN110401576A (en) | Network interaction test method and device | |
| CN117254854A (en) | Processing method and device of remote measurement message and nonvolatile computer readable storage medium | |
| CN113411410B (en) | Inertial navigation equipment testing method, communication server, communication device and storage medium | |
| CN114374812B (en) | Communication information monitoring equipment, method and system of DP (data processing) interface auxiliary channel |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181106 |
|
| WD01 | Invention patent application deemed withdrawn after publication |