CN107846327A - A kind of processing method and processing device of network management performance data - Google Patents

A kind of processing method and processing device of network management performance data Download PDF

Info

Publication number
CN107846327A
CN107846327A CN201711115345.1A CN201711115345A CN107846327A CN 107846327 A CN107846327 A CN 107846327A CN 201711115345 A CN201711115345 A CN 201711115345A CN 107846327 A CN107846327 A CN 107846327A
Authority
CN
China
Prior art keywords
dimension
file
data
analysis
network element
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711115345.1A
Other languages
Chinese (zh)
Inventor
刘睿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Tianyuan Communication Information System Co Ltd
Original Assignee
Inspur Tianyuan Communication Information System Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Tianyuan Communication Information System Co Ltd filed Critical Inspur Tianyuan Communication Information System Co Ltd
Priority to CN201711115345.1A priority Critical patent/CN107846327A/en
Publication of CN107846327A publication Critical patent/CN107846327A/en
Pending legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/02—Capturing of monitoring data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/02—Capturing of monitoring data
    • H04L43/028—Capturing of monitoring data by filtering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/04—Processing captured monitoring data, e.g. for logfile generation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/06—Generation of reports

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a kind of processing method and processing device of network management performance data, this method, including：Pre-set at least one analysis dimension；Obtain at least one network management performance data file of at least one network element；According at least one analysis dimension, each network management performance data file is split, generates at least one dimension file, wherein, the network management performance data in each dimension file belongs to the same analysis dimension；Store at least one dimension file.The invention provides a kind of processing method and processing device of network management performance data, it is possible to increase inquiry velocity.

Description

A kind of processing method and processing device of network management performance data
Technical field
The present invention relates to field of computer technology, more particularly to a kind of processing method and processing device of network management performance data.
Background technology
Network management performance data is generally referred to as network element by network management system taken at regular intervals, the management object of network management system.Webmaster System forms a network management performance number when gathering network management performance data typically in units of the larger network element in overlay area According to file, these network management performance data files are stored for inquiry.
It is visible by foregoing description, it is one network management performance data text of formation in units of network element in the prior art Part, these network management performance data files are simply stored according to a dimension, in the network management performance data that inquiry needs, Need to inquire about each network management performance data file, the network management performance of needs is searched from each network management performance data file Data, inquiry velocity are slower.
The content of the invention
The embodiments of the invention provide a kind of processing method and processing device of network management performance data, it is possible to increase inquiry velocity.
On the one hand, the embodiments of the invention provide a kind of processing method of network management performance data, including：
Pre-set at least one analysis dimension；
Obtain at least one network management performance data file of at least one network element；
According at least one analysis dimension, each network management performance data file is split, generation is at least One dimension file, wherein, the network management performance data in each dimension file belongs to the same analysis dimension；
Store at least one dimension file.
Further,
This method further comprises：
In advance according at least one analysis dimension, storage catalogue is set；
Storage at least one dimension file, including：
For each dimension file, according to present analysis dimension corresponding to current dimension file, by the leading dimension File storage is spent to arrive described in the storage catalogue under store path corresponding to present analysis dimension.
Further,
This method further comprises：Pre-set buffer queue；
At least one dimension file of generation, including：
For each analysis dimension, perform：
A1：To be split out from the network management performance data file data that belong to present analysis dimension be saved in it is described slow Deposit in queue；
A2：Data are read from the buffer queue, the data of reading are written to corresponding to the present analysis dimension In current dimension file；
A3：Judge whether the data in the buffer queue are successfully written in the current dimension file, if it is, The dequeuing data in the current dimension file will be written in the buffer queue, otherwise, returns to A2.
Further,
This method further comprises：
When receiving the querying condition of outside input, perform：
According to the storage catalogue and the querying condition, at least one target storage of the querying condition is determined for compliance with Path；
A thread is distributed for each target store path；
For each thread, using continuously being read under target store path corresponding to current thread from the current thread Take the dimension file for meeting the querying condition.
Further,
Storage at least one dimension file, including：
At least one dimension file for belonging to the same analysis dimension is subjected to Coutinuous store.
Further,
This method further comprises：
For memory cell corresponding to each network element setting；
Storage at least one dimension file, including：
For each network element, at least one dimension file for belonging to current device network element is stored to institute State in the memory cell corresponding to current device network element.
Further,
At least one dimension file of generation, including：
For each analysis dimension, it will be split out from the network management performance data file and belong to present analysis dimension Data compression be binary data stream, the binary data stream is generated into the dimension corresponding to the present analysis dimension File.
Further,
At least one analysis dimension includes：The dimension of acquisition time, network element dimension, depend on the facility network One or more of the dimension of the lower level network element of member, dimension of time granularity of analysis.
Further,
After at least one dimension file of generation, further comprise：
Index file corresponding to each dimension file of generation, wherein, each index file is used to identify correspondingly The dimension file in data original position.
On the other hand, the embodiments of the invention provide a kind of processing unit of network management performance data, including：
Dimension setup module, for setting at least one analysis dimension；
Acquisition module, for obtaining at least one network management performance data file of at least one network element；
Generation module, for according at least one analysis dimension, being carried out to each network management performance data file Split, generate at least one dimension file, wherein, the network management performance data in each dimension file belongs to same described Analyze dimension；
Memory module, for storing at least one dimension file.
Further,
The device further comprises：
Catalogue setup module, for according at least one analysis dimension, setting storage catalogue；
The memory module, for for each dimension file, according to present analysis corresponding to current dimension file Dimension, by under the current dimension file storage to store path corresponding to present analysis dimension described in the storage catalogue.
Further,
The device further comprises：Buffer queue；
The generation module, for for each analysis dimension, performing：
A1：To be split out from the network management performance data file data that belong to present analysis dimension be saved in it is described slow Deposit in queue；
A2：Data are read from the buffer queue, the data of reading are written to corresponding to the present analysis dimension In current dimension file；
A3：Judge whether the data in the buffer queue are successfully written in the current dimension file, if it is, The dequeuing data in the current dimension file will be written in the buffer queue, otherwise, returns to A2.
Further,
The device further comprises：
Enquiry module, for when receiving the querying condition of outside input, performing：
According to the storage catalogue and the querying condition, at least one target storage of the querying condition is determined for compliance with Path；
A thread is distributed for each target store path；
For each thread, using continuously being read under target store path corresponding to current thread from the current thread Take the dimension file for meeting the querying condition.
Further,
The memory module, at least one dimension file for belonging to the same analysis dimension to be connected Renew storage.
Further,
The device further comprises：
Setup module is stored, for memory cell corresponding to being set for each network element；
The memory module, for for each network element, at least one institute of current device network element will to be belonged to The storage of dimension file is stated into the memory cell corresponding to the current device network element.
Further,
The generation module, for that for each analysis dimension, will be split from the network management performance data file The data compression for going out to belong to present analysis dimension is binary data stream, and the binary data stream is generated into the present analysis The dimension file corresponding to dimension；
Further,
At least one analysis dimension includes：The dimension of acquisition time, network element dimension, depend on the facility network One or more of the dimension of the lower level network element of member, dimension of time granularity of analysis.
Further,
The generation module, for after at least one dimension file of generation, generating each dimension file Corresponding index file, wherein, each index file is used for the starting for identifying the data in the corresponding dimension file Position.
In embodiments of the present invention, network management performance data file is split, at least one dimension text is generated after fractionation Part so that the network management performance data in each dimension file belongs to same analysis dimension, when user needs to inquire about some analysis During the network management performance data of dimension, because network management performance data is stored according to analysis dimension, can more quickly it look into The data of needs are ask, improve inquiry velocity.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are the present invention Some embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis These accompanying drawings obtain other accompanying drawings.
Fig. 1 is a kind of flow chart of the processing method for network management performance data that one embodiment of the invention provides；
Fig. 2 is a kind of schematic diagram for storage catalogue that one embodiment of the invention provides；
Fig. 3 is the flow chart of the processing method for another network management performance data that one embodiment of the invention provides；
Fig. 4 is a kind of schematic diagram of the processing unit for network management performance data that one embodiment of the invention provides；
Fig. 5 is the schematic diagram of the processing unit for another network management performance data that one embodiment of the invention provides.
Embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention In accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is Part of the embodiment of the present invention, rather than whole embodiments, based on the embodiment in the present invention, those of ordinary skill in the art The every other embodiment obtained on the premise of creative work is not made, belongs to the scope of protection of the invention.
As shown in figure 1, the embodiments of the invention provide a kind of processing method of network management performance data, this method can include Following steps：
Step 101：Pre-set at least one analysis dimension；
Step 102：Obtain at least one network management performance data file of at least one network element；
Step 103：According at least one analysis dimension, each network management performance data file is split, At least one dimension file is generated, wherein, the network management performance data in each dimension file belongs to the same analysis Dimension；
Step 104：Store at least one dimension file.
In embodiments of the present invention, network management performance data file is split, at least one dimension text is generated after fractionation Part so that the network management performance data in each dimension file belongs to same analysis dimension, when user needs to inquire about some analysis During the network management performance data of dimension, because network management performance data is stored according to analysis dimension, can more quickly it look into The data of needs are ask, improve inquiry velocity.
In an embodiment of the present invention, this method further comprises：
In advance according at least one analysis dimension, storage catalogue is set；
Storage at least one dimension file, including：
For each dimension file, according to present analysis dimension corresponding to current dimension file, by the leading dimension File storage is spent to arrive described in the storage catalogue under store path corresponding to present analysis dimension.
In embodiments of the present invention, storage catalogue is provided with for each dimension file, when needing to inquire about some network management performances During data, the store path of the network management performance data of needs can be quickly found by storage catalogue, improves inquiry velocity.
In embodiments of the present invention, each dimension file can be embodied by store path corresponding to each dimension file Analysis dimension.
In an embodiment of the present invention, this method further comprises：Pre-set buffer queue；
At least one dimension file of generation, including：
For each analysis dimension, perform：
A1：To be split out from the network management performance data file data that belong to present analysis dimension be saved in it is described slow Deposit in queue；
A2：Data are read from the buffer queue, the data of reading are written to corresponding to the present analysis dimension In current dimension file；
A3：Judge whether the data in the buffer queue are successfully written in the current dimension file, if it is, The dequeuing data in the current dimension file will be written in the buffer queue, otherwise, returns to A2.
In embodiments of the present invention, it would be desirable to which the data being written in dimension file are saved in buffer queue, work as caching When data in queue are successfully written in dimension file, just by the dequeuing data in buffer queue, when the number in buffer queue During according to not being successfully written in dimension file, again to dimension file write data, it can so ensure not to be successfully written It will not be lost to the data in dimension file, ensure that the reliability of data.
During to dimension file write data, buffer queue is used for the temporary data to be write, certain reaching After the threshold value of size (such as：1GB), then by data it is continuously written into dimension file, ensures that continuously raising gulps down in storage The amount of telling, reduce write-in number, reduce IOPS (Input/Output Operations Per Second, it is per second to be written and read behaviour The number of work).Technically mainly in combination with two kinds of File Technologies of sparse file and Memory Mapping File.
Sparse file technology：Sparse file technology is operating system Floor layer Technology, it is allowed to the cavity of no data be present in file, Purpose is to reduce the actual disk size taken of file, in buffer queue, writes data in batches in units of 128KB every time, As the write-in inadequate 128KB of data actual size, the disk size actually taken in sparse file can be less than 128KB.
Storage mapping file technique：By on one piece of region of memory Map Disk fixed size (such as 1GB) it is slow File is deposited, realizes the file operation based on internal storage access, the method binding cache file, which rolls, to be used, and once rolls fixed size File, that is, correspond to a fixed size Memory Mapping File.
In an embodiment of the present invention, this method further comprises：
When receiving the querying condition of outside input, perform：
According to the storage catalogue and the querying condition, at least one target storage of the querying condition is determined for compliance with Path；
A thread is distributed for each target store path；
For each thread, using continuously being read under target store path corresponding to current thread from the current thread Take the dimension file for meeting the querying condition.
In embodiments of the present invention, meeting the target store path of querying condition may have multiple, be each target at this moment Store path distributes a thread, realizes multiple threads, improves treatment effeciency, real for each target store path Existing single thread batch processing, improve the handling capacity of disk.
In an embodiment of the present invention, storage at least one dimension file, including：
At least one dimension file for belonging to the same analysis dimension is subjected to Coutinuous store.
In embodiments of the present invention, the dimension file of same analysis dimension is subjected to Coutinuous store, these are same reading and writing During the data of one analysis dimension, it is possible to achieve it is continuous to read, substantially increase the handling capacity of disk.
In an embodiment of the present invention, this method further comprises：
For memory cell corresponding to each network element setting；
Storage at least one dimension file, including：
For each network element, at least one dimension file for belonging to current device network element is stored to institute State in the memory cell corresponding to current device network element.
In embodiments of the present invention, disk is divided into each memory cell, is only stored in each memory cell and belong to same The data storage of one network element can more easily inquire about the webmaster of each equipment webpage in same memory cell Can data.Each memory cell can be named according to the ID of corresponding network element.
In an embodiment of the present invention, at least one dimension file of generation, including：
For each analysis dimension, it will be split out from the network management performance data file and belong to present analysis dimension Data compression be binary data stream, the binary data stream is generated into the dimension corresponding to the present analysis dimension File.
In embodiments of the present invention, can be compressed by GZIP algorithms.Network management performance in dimension file is obtained , it is necessary to first be decompressed to the binary data stream in dimension file during data.
In an embodiment of the present invention, at least one analysis dimension includes：Dimension, the network element dimension of acquisition time Spend, depend on one or more of dimension, dimension of time granularity of analysis of the lower level network element of the network element.
In embodiments of the present invention, for the dimension of acquisition time, mesh can be created in units of week in storage catalogue Record, belong under the data flow of the analysis dimension this catalogue.The catalogue can be named with date of first day weekly, form can To be YYYYMMDD, such as：20171021.On acquisition time by Zhou Youxu store, can single file it is excessive, avoid because Single file is excessive and reduces recall precision.
In embodiments of the present invention, for network element dimension, creatied directory in storage catalogue with network element, can be with Named by the hashed value of the numbering of network element come catalogue, such as：By the numbering step-by-step and 255, Neng Goufang of network element Only file is excessive under same catalogue, influences recall precision.
In embodiments of the present invention, for the dimension for the lower level network element for depending on network element, network element is depended on Lower level network element can include：CPU, port, internal memory etc..Can be that each lower level network element establishes catalogue in storage catalogue.
In embodiments of the present invention, for analysis time granularity dimension, data are entered according to the time cycle of analysis Row classification, such as：According to it is per minute, time granularity is stored per hour etc., for subsequent query.
In embodiments of the present invention, dimension according to the above analysis, in same all scopes, same network element, same depend on The data of the time granularity of lower level network element, same analysis in network element, which are flowed under same catalogue, to be stored, according to looking into , only need to be according to querying condition stitching path when inquiry condition is inquired about.
Corresponding to corresponding each network element for memory cell, identical all, identical network elements, identical equipment is depended on Performance data under the lower level network element of network element, the time granularity of same analysis, same network element ID is deposited under storage catalogue Same network element ID name memory cell under.
When newly-increased analysis dimension, storage catalogue is also required to be changed accordingly.
As shown in Fig. 2 the schematic diagram based on a kind of storage catalogue that dimension is analyzed in the embodiment of the present invention.As shown in Figure 2 Storage catalogue shown in, the one-level of the storage catalogue is catalogue corresponding to the dimension of acquisition time (being YYYYMMDD in figure), is adopted The next stage for collecting catalogue corresponding to the dimension of time is catalogue corresponding to network element dimension, catalogue corresponding to network element dimension Next stage be the dimension for the lower level network element for depending on network element corresponding to catalogue (specifically, depend under network element Level network element includes CPU, port, internal memory etc.), the subordinate of catalogue corresponding to the dimension for the lower level network element for depending on network element is point Catalogue corresponding to the dimension of the time granularity of analysis, wherein, M5 refers to that the time granularity of analysis is 5 minutes, and H refers to the time of analysis Granularity is 1 hour.Each dimension file is stored with the dimension of the time granularity of analysis.
In an embodiment of the present invention, after at least one dimension file of generation, further comprise：
Index file corresponding to each dimension file of generation, wherein, each index file is used to identify correspondingly The dimension file in data original position.
In embodiments of the present invention, the original position of the data in dimension file can be found by index file, is being looked into During inquiry, the network management performance data of needs can be found according to the original position in index file.
In embodiments of the present invention, dimension file can be " .dat " file, and index file can be " .idx " file.Dimension Physical length of 8 bytes of beginning of degree file for the data after storing this write-in, every time by file actual (tube) length after write-in Degree is written to preceding four bytes, and backups to the 5-8 byte, for the positioning before write-in next time to file writing position.Under The one before byte is read before secondary write-in, every 4 bytes are compared, take larger byte number as last time write-in length according to According to enhancing robustness.
In each memory cell, dimension file and index file are generally comprised.
When data are written to memory cell, be first written to dat files, then safeguard this write-in after Data Position, after will Position in dat files is written to idx index files, ensures dirty data, data will not occur because abnormal in dat ablation process Write successfully, by the dequeuing data in buffer queue, whole process, which has, exception occurs at one, data at least can be based on file Exist in buffer queue, ensure that data are not lost.
To the data being repeatedly written, when reading, the index occurred using last time can ensure to read data as most The valid data being once stored in afterwards, ensure the reliability of digital independent.
In embodiments of the present invention, inquiry has two levels, and whole implementation method is all based on file, is used in deployment (SuSE) Linux OS, Linux can be existed in internal memory always for last time using file, just be discharged until memory usage is higher, One layer of hot spot data caching naturally has been done for this method, query caching is done to idx the and dat files frequently used.Another layer Secondary, inquiry data, which are to ensure that, occurs without the problem of write-in data are unreadable, first the data in the buffer queue of file is entered Row retrieval, after the data in memory cell are retrieved, ensure data it is readable all the time.
In embodiments of the present invention, buffer queue can use the mentality of designing of FIFO, can continuously be read when reading More datas are taken, ensure handling capacity during write-in.
When the actual use configuration of cache file threshold size (such as：When 1G), legacy data file is sealed up for safekeeping, raw The .cache file new into one provides data write-in.
In embodiments of the present invention, optimized for mass data storage and inquiry, add and be based in logic is write The buffer queue of sparse file and Memory Mapping File, data buffer storage is realized, data cached run up to after a certain size again will be slow Batch data write-in dimension file in depositing, realizes file Coutinuous store, improves write-in and inquiry handling capacity；Based on data cached Copy distribution realize at one write, many places storage, realize that data calamity is standby.
In embodiments of the present invention, by optimizing for file design Storage, compression storing data mode and read-write, for In network management performance data time, space, logical relation multidimensional analysis, scene is equally analyzed, the lower read-write of same hardware configuration gulps down The amount of telling is dominant compared with relevant database and Key-Value database performances.
As shown in figure 3, the embodiments of the invention provide a kind of processing method of network management performance data, this method can include Following steps：
Step 301：At least one analysis dimension is pre-set, in advance according at least one analysis dimension, storage is set Catalogue, pre-set buffer queue.
Specifically, the storage catalogue that storage catalogue can be as shown in Figure 2.
Step 302：Obtain at least one network management performance data file of at least one network element.
Specifically, network management performance data file is obtained in units of network element.Such as：Obtaining the network management performance of this week During data file, the network management performance data of each network element forms a file.
Here network element is the larger network element in overlay area, such as：Base station controller.
Step 303：According at least one analysis dimension, each network management performance data file is split.
Because network management performance data file is obtained in units of network element, wherein, may include multiple points The data of dimension are analysed, by the step, the data of each analysis dimension can be splitted out.
Step 304：For each analysis dimension, it will be split out from network management performance data file and belong to present analysis dimension Data be saved in buffer queue, data are read from buffer queue, the data of reading are written to present analysis dimension pair In the current dimension file answered, when the data in buffer queue are successfully written in current dimension file, by buffer queue The dequeuing data being written in current dimension file, wherein, the network management performance data in each dimension file belongs to same point Analyse dimension.
Specifically, network management performance data is stored respectively according to analysis dimension, can more facilitates inquiry.
Step 305：For each dimension file, according to present analysis dimension corresponding to current dimension file, by leading dimension File storage is spent into storage catalogue under store path corresponding to present analysis dimension, wherein, belong to same analysis dimension At least one dimension file carries out Coutinuous store.
Specifically, each dimension file is subjected to Coutinuous store according to storage catalogue, can be according to storage mesh in inquiry Record is quickly found out the network management performance data of needs.
As shown in Figure 4, Figure 5, the embodiments of the invention provide a kind of processing unit of network management performance data.Device embodiment It can be realized, can also be realized by way of hardware or software and hardware combining by software.For hardware view, such as Fig. 4 It is shown, it is a kind of hardware structure diagram of equipment where a kind of processing unit of network management performance data provided in an embodiment of the present invention, In addition to the processor shown in Fig. 4, internal memory, network interface and nonvolatile memory, setting where device in embodiment It is standby generally to include other hardware, such as it is responsible for the forwarding chip of processing message.Exemplified by implemented in software, such as Fig. 5 institutes Show, be by corresponding calculating in nonvolatile memory by the CPU of equipment where it as the device on a logical meaning Machine programmed instruction reads what operation in internal memory was formed.A kind of processing unit for network management performance data that the present embodiment provides, bag Include：
Dimension setup module 501, for setting at least one analysis dimension；
Acquisition module 502, for obtaining at least one network management performance data file of at least one network element；
Generation module 503, for according at least one analysis dimension, entering to each network management performance data file Row is split, and generates at least one dimension file, wherein, the network management performance data in each dimension file belongs to same institute State analysis dimension；
Memory module 504, for storing at least one dimension file.
In an embodiment of the present invention, the device further comprises：
Catalogue setup module, for according at least one analysis dimension, setting storage catalogue；
The memory module, for for each dimension file, according to present analysis corresponding to current dimension file Dimension, by under the current dimension file storage to store path corresponding to present analysis dimension described in the storage catalogue.
In an embodiment of the present invention, the device further comprises：Buffer queue；
The generation module, for for each analysis dimension, performing：
A1：To be split out from the network management performance data file data that belong to present analysis dimension be saved in it is described slow Deposit in queue；
A2：Data are read from the buffer queue, the data of reading are written to corresponding to the present analysis dimension In current dimension file；
A3：Judge whether the data in the buffer queue are successfully written in the current dimension file, if it is, The dequeuing data in the current dimension file will be written in the buffer queue, otherwise, returns to A2.
In an embodiment of the present invention, the device further comprises：
Enquiry module, for when receiving the querying condition of outside input, performing：
According to the storage catalogue and the querying condition, at least one target storage of the querying condition is determined for compliance with Path；
A thread is distributed for each target store path；
For each thread, using continuously being read under target store path corresponding to current thread from the current thread Take the dimension file for meeting the querying condition.
In an embodiment of the present invention, the memory module, at least the one of the same analysis dimension will to be belonged to The individual dimension file carries out Coutinuous store.
In an embodiment of the present invention, the device further comprises：
Setup module is stored, for memory cell corresponding to being set for each network element；
The memory module, for for each network element, at least one institute of current device network element will to be belonged to The storage of dimension file is stated into the memory cell corresponding to the current device network element.
In an embodiment of the present invention, the generation module, will be from the webmaster for for each analysis dimension Split out that to belong to the data compression of present analysis dimension be binary data stream in performance data files, by the binary data Stream generates the dimension file corresponding to the present analysis dimension.
In an embodiment of the present invention, at least one analysis dimension includes：Dimension, the network element dimension of acquisition time Spend, depend on one or more of dimension, dimension of time granularity of analysis of the lower level network element of the network element.
In an embodiment of the present invention, the generation module, for after at least one dimension file of generation, giving birth to Into index file corresponding to each dimension file, wherein, each index file is used to identify the corresponding dimension The original position of data in file.
The contents such as the information exchange between each unit, implementation procedure in said apparatus, due to implementing with the inventive method Example is based on same design, and particular content can be found in the narration in the inventive method embodiment, and here is omitted.
The embodiments of the invention provide a kind of computer-readable recording medium, including execute instruction, when the computing device of storage control During the execute instruction, the storage control performs the processing of any one network management performance data provided in an embodiment of the present invention Method.
The embodiments of the invention provide a kind of storage control, including：Processor, memory and bus；
The memory is used to store execute instruction, and the processor is connected with the memory by the bus, when During the storage control operation, the execute instruction of memory storage described in the computing device, so that the storage control Device performs the processing method of any one network management performance data provided in an embodiment of the present invention.
The each embodiment of the present invention at least has the advantages that：
1st, in embodiments of the present invention, network management performance data file is split, at least one dimension is generated after fractionation File so that the network management performance data in each dimension file belongs to same analysis dimension, when user needs to inquire about some point , can be more quick because network management performance data is stored according to analysis dimension when analysing the network management performance data of dimension The data of needs are inquired, improve inquiry velocity.
2nd, in embodiments of the present invention, storage catalogue is provided with for each dimension file, when needing to inquire about some webmasters During energy data, the store path of the network management performance data of needs can be quickly found by storage catalogue, improves inquiry speed Degree.
3rd, in embodiments of the present invention, it would be desirable to which the data being written in dimension file are saved in buffer queue, when slow When depositing the data in queue and being successfully written in dimension file, just by the dequeuing data in buffer queue, when in buffer queue When data are not successfully written in dimension file, again to dimension file write data, it can so ensure not write successfully Entering to the data in dimension file to lose, and ensure that the reliability of data.
4th, in embodiments of the present invention, meeting the target store path of querying condition may have multiple, be each mesh at this moment Mark store path and distribute a thread, realize multiple threads, improve treatment effeciency, for each target store path, Single thread batch processing is realized, improves the handling capacity of disk.
5th, in embodiments of the present invention, the dimension file of same analysis dimension is subjected to Coutinuous store, reads and writes these During the data of same analysis dimension, it is possible to achieve it is continuous to read, substantially increase the handling capacity of disk.
It should be noted that herein, such as first and second etc relational terms are used merely to an entity Or operation makes a distinction with another entity or operation, and not necessarily require or imply and exist between these entities or operation Any this actual relation or order.Moreover, term " comprising ", "comprising" or its any other variant be intended to it is non- It is exclusive to include, so that process, method, article or equipment including a series of elements not only include those key elements, But also the other element including being not expressly set out, or also include solid by this process, method, article or equipment Some key elements.In the absence of more restrictions, the key element limited by sentence " including one ", is not arranged Except other identical factor in the process including the key element, method, article or equipment being also present.
One of ordinary skill in the art will appreciate that：Realizing all or part of step of above method embodiment can pass through Programmed instruction related hardware is completed, and foregoing program can be stored in computer-readable storage medium, the program Upon execution, the step of execution includes above method embodiment；And foregoing storage medium includes：ROM, RAM, magnetic disc or light Disk etc. is various can be with the medium of store program codes.
It is last it should be noted that：Presently preferred embodiments of the present invention is the foregoing is only, is merely to illustrate the skill of the present invention Art scheme, is not intended to limit the scope of the present invention.Any modification for being made within the spirit and principles of the invention, Equivalent substitution, improvement etc., are all contained in protection scope of the present invention.

Claims (10)

  1. A kind of 1. processing method of network management performance data, it is characterised in that
    At least one analysis dimension is pre-set, including：
    Obtain at least one network management performance data file of at least one network element；
    According at least one analysis dimension, each network management performance data file is split, generation is at least one Dimension file, wherein, the network management performance data in each dimension file belongs to the same analysis dimension；
    Store at least one dimension file.
  2. 2. according to the method for claim 1, it is characterised in that
    Further comprise：
    In advance according at least one analysis dimension, storage catalogue is set；
    Storage at least one dimension file, including：
    For each dimension file, according to present analysis dimension corresponding to current dimension file, by the current dimension text Part storage is arrived described in the storage catalogue under store path corresponding to present analysis dimension.
  3. 3. according to the method for claim 1, it is characterised in that
    Further comprise：Pre-set buffer queue；
    At least one dimension file of generation, including：
    For each analysis dimension, perform：
    A1：The data that belong to present analysis dimension will be split out from the network management performance data file and are saved in the caching team In row；
    A2：Data are read from the buffer queue, the data of reading are written to current corresponding to the present analysis dimension In dimension file；
    A3：Judge whether the data in the buffer queue are successfully written in the current dimension file, if it is, by institute The dequeuing data being written in buffer queue in the current dimension file is stated, otherwise, returns to A2.
  4. 4. according to the method for claim 2, it is characterised in that
    Further comprise：
    When receiving the querying condition of outside input, perform：
    According to the storage catalogue and the querying condition, at least one target for being determined for compliance with the querying condition stores road Footpath；
    A thread is distributed for each target store path；
    For each thread, symbol is read using continuous under target store path corresponding to current thread from the current thread Close the dimension file of the querying condition.
  5. 5. according to any described method in claim 1-4, it is characterised in that
    Storage at least one dimension file, including：
    At least one dimension file for belonging to the same analysis dimension is subjected to Coutinuous store；
    And/or
    Further comprise：
    For memory cell corresponding to each network element setting；
    Storage at least one dimension file, including：
    For each network element, at least one dimension file storage for belonging to current device network element is worked as to described In the memory cell corresponding to preceding network element；
    And/or
    At least one dimension file of generation, including：
    For each analysis dimension, the number that belongs to present analysis dimension will be split out from the network management performance data file According to boil down to binary data stream, the binary data stream is generated into the dimension text corresponding to the present analysis dimension Part；
    And/or
    At least one analysis dimension includes：The dimension of acquisition time, network element dimension, depend on the network element One or more of the dimension of lower level network element, dimension of time granularity of analysis；
    And/or
    After at least one dimension file of generation, further comprise：
    Index file corresponding to each dimension file of generation, wherein, each index file is used to identify corresponding institute State the original position of the data in dimension file.
  6. A kind of 6. processing unit of network management performance data, it is characterised in that including：
    Dimension setup module, for setting at least one analysis dimension；
    Acquisition module, for obtaining at least one network management performance data file of at least one network element；
    Generation module, for according at least one analysis dimension, being split to each network management performance data file, At least one dimension file is generated, wherein, the network management performance data in each dimension file belongs to the same analysis Dimension；
    Memory module, for storing at least one dimension file.
  7. 7. device according to claim 6, it is characterised in that
    Further comprise：
    Catalogue setup module, for according at least one analysis dimension, setting storage catalogue；
    The memory module, for for each dimension file, according to present analysis dimension corresponding to current dimension file, By under the current dimension file storage to store path corresponding to present analysis dimension described in the storage catalogue.
  8. 8. device according to claim 6, it is characterised in that
    Further comprise：Buffer queue；
    The generation module, for for each analysis dimension, performing：
    A1：The data that belong to present analysis dimension will be split out from the network management performance data file and are saved in the caching team In row；
    A2：Data are read from the buffer queue, the data of reading are written to current corresponding to the present analysis dimension In dimension file；
    A3：Judge whether the data in the buffer queue are successfully written in the current dimension file, if it is, by institute The dequeuing data being written in buffer queue in the current dimension file is stated, otherwise, returns to A2.
  9. 9. device according to claim 7, it is characterised in that
    Further comprise：
    Enquiry module, for when receiving the querying condition of outside input, performing：
    According to the storage catalogue and the querying condition, at least one target for being determined for compliance with the querying condition stores road Footpath；
    A thread is distributed for each target store path；
    For each thread, symbol is read using continuous under target store path corresponding to current thread from the current thread Close the dimension file of the querying condition.
  10. 10. according to any described device in claim 6-9, it is characterised in that
    The memory module, at least one dimension file for belonging to the same analysis dimension continuously to be deposited Storage；
    And/or
    Further comprise：
    Setup module is stored, for memory cell corresponding to being set for each network element；
    The memory module, for for each network element, at least one dimension of current device network element will to be belonged to File storage is spent into the memory cell corresponding to the current device network element；
    And/or
    The generation module, for for each analysis dimension, category will to be split out from the network management performance data file It is binary data stream in the data compression of present analysis dimension, the binary data stream is generated into the present analysis dimension The corresponding dimension file；
    And/or
    At least one analysis dimension includes：The dimension of acquisition time, network element dimension, depend on the network element One or more of the dimension of lower level network element, dimension of time granularity of analysis；
    And/or
    The generation module, for after at least one dimension file of generation, each dimension file of generation to be corresponding Index file, wherein, each index file be used to identifying corresponding to data in the dimension file original position.
CN201711115345.1A 2017-11-13 2017-11-13 A kind of processing method and processing device of network management performance data Pending CN107846327A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711115345.1A CN107846327A (en) 2017-11-13 2017-11-13 A kind of processing method and processing device of network management performance data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711115345.1A CN107846327A (en) 2017-11-13 2017-11-13 A kind of processing method and processing device of network management performance data

Publications (1)

Publication Number Publication Date
CN107846327A true CN107846327A (en) 2018-03-27

Family

ID=61681054

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711115345.1A Pending CN107846327A (en) 2017-11-13 2017-11-13 A kind of processing method and processing device of network management performance data

Country Status (1)

Country Link
CN (1) CN107846327A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020129612A1 (en) * 2018-12-19 2020-06-25 ソニー株式会社 Information processing apparatus, information processing method, and information processing program
CN112069270A (en) * 2020-09-07 2020-12-11 青岛海信传媒网络技术有限公司 File-based multi-dimensional data reading and writing method and display device

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102750377A (en) * 2012-06-21 2012-10-24 深圳市谷米科技有限公司 Massive data storage and retrieval method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102750377A (en) * 2012-06-21 2012-10-24 深圳市谷米科技有限公司 Massive data storage and retrieval method

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020129612A1 (en) * 2018-12-19 2020-06-25 ソニー株式会社 Information processing apparatus, information processing method, and information processing program
JP7435470B2 (en) 2018-12-19 2024-02-21 ソニーグループ株式会社 Information processing device, information processing method, and information processing program
CN112069270A (en) * 2020-09-07 2020-12-11 青岛海信传媒网络技术有限公司 File-based multi-dimensional data reading and writing method and display device

Similar Documents

Publication Publication Date Title
US11068455B2 (en) Mapper tree with super leaf nodes
TWI719281B (en) A system, machine readable medium, and machine-implemented method for stream selection
TWI702503B (en) Systems, methods, and computer readable media to implement merge tree modifications for maintenance operations
TWI702506B (en) System, machine readable medium, and machine-implemenated method for merge tree garbage metrics
US10642515B2 (en) Data storage method, electronic device, and computer non-volatile storage medium
CN103635900B (en) Time-based data partitioning
US9298726B1 (en) Techniques for using a bloom filter in a duplication operation
US9223660B2 (en) Storage device to backup content based on a deduplication system
KR20170054299A (en) Reference block aggregating into a reference set for deduplication in memory management
CN103955530B (en) Data reconstruction and optimization method of on-line repeating data deletion system
CN108009008A (en) Data processing method and system, electronic equipment
CN103562914B (en) The type that economizes on resources extends file system
US11232073B2 (en) Method and apparatus for file compaction in key-value store system
US11157453B2 (en) Parallel deduplication using automatic chunk sizing
CN107045531A (en) A kind of system and method for optimization HDFS small documents access
CN108319654A (en) Computing system, cold and hot data separation method and device, computer readable storage medium
CN107766374B (en) Optimization method and system for storage and reading of massive small files
CN105117351A (en) Method and apparatus for writing data into cache
CN113867627B (en) Storage system performance optimization method and system
CN112148217B (en) Method, device and medium for caching deduplication metadata of full flash memory system
CN105493080B (en) The method and apparatus of data de-duplication based on context-aware
CN107665219A (en) A kind of blog management method and device
CN113094374A (en) Distributed storage and retrieval method and device and computer equipment
CN109407985B (en) Data management method and related device
CN107846327A (en) A kind of processing method and processing device of network management performance data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20180327

RJ01 Rejection of invention patent application after publication