CN104102664B - Data processing method and system for physical machine resource information - Google Patents

Data processing method and system for physical machine resource information Download PDF

Info

Publication number
CN104102664B
CN104102664B CN201310123750.3A CN201310123750A CN104102664B CN 104102664 B CN104102664 B CN 104102664B CN 201310123750 A CN201310123750 A CN 201310123750A CN 104102664 B CN104102664 B CN 104102664B
Authority
CN
China
Prior art keywords
data
information
usage amount
file system
accumulated value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201310123750.3A
Other languages
Chinese (zh)
Other versions
CN104102664A (en
Inventor
王博
孙毓忠
展旭升
杨庆林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zhongke Flux Technology Co ltd
Original Assignee
Institute of Computing Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Computing Technology of CAS filed Critical Institute of Computing Technology of CAS
Priority to CN201310123750.3A priority Critical patent/CN104102664B/en
Publication of CN104102664A publication Critical patent/CN104102664A/en
Application granted granted Critical
Publication of CN104102664B publication Critical patent/CN104102664B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention provides a data processing method and a data processing system for physical machine resource information. The method comprises the following steps of 1, acquiring the static information of physical machines and the dynamic information, which is represented in form of a current state value or an accumulated value at a current moment, of the physical machines; 2, re-acquiring the new dynamic information of the physical machines after 1 second, and subtracting the data of the dynamic information acquired in the previous second from data in the new dynamic information represented in form of an accumulated value to obtain information about resource utilization within a second; 3, creating a low-redundancy data table, wherein DTI is used for identifying data types, CI is used for identifying data positions, DRB is used for identifying subsequently-transmitted data items of the data, and DV is used for identifying a sequential arrangement result of values of the data items; 4, representing the static information, the new dynamic information and the information about resource utilization within a second by using the low-redundancy data table respectively, transmitting the low-redundancy data table to a destination node, repeating the steps, and acquiring the real-time resource information of an acquired node in real time by using the destination node.

Description

A kind of data processing method and system of physical machine resource information
Technical field
The present invention relates to numeral as the large-scale data process field of main transmission data, more particularly to a kind of physical machine The data processing method and system of resource information.
Background technology
Extensibility is one of key challenge of large scope software system.For large scale system, the network bandwidth is which is soft One of major limitation condition of the part system expandability.So, optimization software system internal transmission number is the presentation format energy of evidence Transmission volume is reduced effectively, so as to improve the autgmentability of software system.
Extensible markup language (Extensible Markup Language, XML) is the common language of data transfer, is Using a kind of extensive data transmission format.The independence between software can be improved using XML.But XML is to each data item Two labels are used all（Start-tag and end-tag）Give and define, this considerably increases the redundancy of data.But big rule The software system of mould needs enhanced scalability, and the XML data presentation format with more redundancy can increase network transmission number According to amount, so XML should not be used to the form of large scope software internal system transmission data.
Chukwa:A large‐scale monitoring system.Jerome Boulon,Andy Konwinski And Runping Qi propose one and operate in Hadoop（Increase income for one and distributed system and realize thereon MapReduce:A kind of programming model for Large-scale parallel computing）On extensive monitoring system.The internal system with File is transmitted, and this method does later stage process suitable for Hadoop, but carries out internal system data transfer shape with file Formula can increase unnecessary redundancy（Such as, file attribute etc..When data are less, file attribute is sometimes more than useful Information）.
Aggregation of Real‐Time System Monitoring Data for Analyzing Large‐ Scale Parallel and Distributed Computing Environments.S.C.Engelmann, The form of the large-scale monitoring system data transfer inside that and S.L.Scott are proposed is a data item labelling （Identification, ID）The value of one data item of correspondence.What to have made the data that represent in this way fewer than XML many is superfluous Remaining information, however, for the data based on numeral, value and data item ID of data item take essentially identical size, so Data represented by the method also have higher redundancy.
The content of the invention
To solve the above problems, the present invention provides a kind of data processing method and system of physical machine resource information, to subtract Redundancy in few transmission data.
To achieve the above object of the invention, the present invention provides a kind of data processing method of physical machine resource information, the method Including：
Step 1, obtains the static information of physical machine from the file of the resource information and proc file system of linux orders And the multidate information that current time is represented in the form of current state value or accumulated value；
Step 2, obtains the new multidate information of physical machine again after 1 second, by the new dynamic letter represented in the form of accumulated value Data in breath deduct the data of the multidate information for obtaining for a second, obtain the resource using information in 1 second；
Step 3, creates low-redundancy data form：DTI is used for mark data type, and CI is used for mark data position, and DRB is used In the data item of mark data subsequent transmission, DV is used for the result that the value of identification data item is arranged in order；
Step 4, by the static information, the new multidate information and it is described 1 second in resource using information use respectively Low-redundancy data form is indicated, and is subsequently transmitted to destination node, repeats above-mentioned steps, obtains the destination node Real time resources information.
Further, the static information acquisition methods in the step 1 include：
Step 11：IP is extracted in the output information of the ifconfig orders for checking network card configuration information provided from Linux Address；
Step 12, reads the cpuinfo files in proc file system, obtains the check figure and frequency of processor；
Step 13, reads the meminfo files in proc file system, obtains internal memory total size；
Step 14, reads the partitions files in proc file system, obtains disk total size；
Step 15, Netowrk tape is obtained in the output information of the dmesg orders for checking each unit status provided from Linux It is wide；
Step 16, reads the stat file in proc file system, the processor usage amount being obtained from since start it is tired It is value added；
Step 17, reads the net/dev files in proc file system, the network bandwidth usage amount being obtained from since start Accumulated value.
Further, the multidate information acquisition methods in the step 1 include：
Step 111, reads the stat file in proc file system, the processor usage amount being obtained from since start it is tired It is value added；
Step 112, reads the meminfo files in proc file system, obtains internal memory usage amount；
Step 113, extracts disk usage amount in the output information of the df orders for checking disk resource provided from Linux；
Step 114, reads the net/dev files in proc file system, has been obtained from the network bandwidth since start and uses The accumulated value of amount.
Further, the step 2 includes：
Step 21, the processor that the accumulated value of the processor usage amount since start deducts acquisition upper one second is made Consumption accumulated value, obtains the usage amount of the processor in this second；
The network bandwidth usage amount accumulated value since start is deducted the network bandwidth of acquisition upper one second by step 22 Usage amount accumulated value, obtains the usage amount of the network bandwidth in this second.
Further, in the step 3：
If data type has NDTIKind, then using every data beforePosition as DTI, whereinRepresent not Less than the smallest positive integral of m；
The CI, is represented with one, if the position is 1, then it represents that the data is not the last item in data message, if The position is 0, then it represents that the data is the last item of data message；
The value of the data item in the DV is adjustable length character string, then add a byte before the character string Represent the length of the character string；
N is up to per the data item number of class dataDRB, then using at least N of the every data for transmittingDRBPosition is used as DRB, institute Each one data item of correspondence of DRB is stated, if certain position is 1, then it represents that values of the DV comprising the corresponding data item, if should Position is 0, then values of the DV not comprising the data item.
Further, in the step 3：
If the data of software system internal transmission only have a class, DTI can be left out；If the number of the data transmitted every time According to item all same, then DRB can be left out.
For achieving the above object, the present invention also provides a kind of data handling system of physical machine resource information, and this is System includes：
Data obtaining module, obtains physical machine from the file of the resource information and proc file system of linux orders The multidate information that static information and current time are represented in the form of current state value or accumulated value；
Message processing module, obtains the new multidate information of physical machine again after 1 second, new by what is represented in the form of accumulated value Data in multidate information deduct the data of the multidate information for obtaining for a second, obtain the resource using information in 1 second；
Form creation module, creates low-redundancy data form：DTI is used for mark data type, and CI is used for mark data position Put, DRB is used for the data item of mark data subsequent transmission, DV is used for the result that the value of identification data item is arranged in order；
As a result acquisition module, by the static information, the new multidate information and it is described 1 second in resource using information It is indicated with low-redundancy data form respectively, is subsequently transmitted to destination node, repeat above-mentioned module, obtains the mesh The real time resources information of mark node.
Further, described information acquisition module includes：
IP address module is extracted, the output information of the ifconfig orders for checking network card configuration information provided from Linux Middle extraction IP address；
First reads proc file system modules, reads the cpuinfo files in proc file system, obtains processor Check figure and frequency, read the meminfo files in proc file system, obtain internal memory total size, read in proc file system Partitions files, obtain disk total size, read proc file system in stat file, be obtained from start since The accumulated value of processor usage amount, reads the net/dev files in proc file system, the network bandwidth being obtained from since start The accumulated value of usage amount；
Operation dmesg command modules, in the output information of dmesg orders for checking each unit status provided from Linux Obtain the network bandwidth.
Further, described information acquisition module also includes：
Second reads proc file system modules, reads the stat file in proc file system, since being obtained from start Processor usage amount accumulated value, read proc file system in meminfo files, obtain internal memory usage amount, read Net/dev files in proc file system, the accumulated value of the network bandwidth usage amount being obtained from since start；
Df modules are performed, disk is provided in the output information of the df orders for checking disk resource provided from Linux and is used Amount.
Further, it is characterised in that described information processing module includes：
The accumulated value of the processor usage amount since start is deducted the place of acquisition upper one second by first processing module Reason device usage amount accumulated value, obtains the usage amount of the processor in this second；
The network bandwidth usage amount accumulated value since start is deducted the net of acquisition upper one second by Second processing module Network bandwidth usage amount accumulated value, obtains the usage amount of the network bandwidth in this second.
The beneficial effects of the present invention is：Greatly reduce the redundancy in data transfer.If the present invention is used on a large scale Monitoring system in, can pass through reduce by monitoring data produce network transmission, so as to increased the autgmentability of monitoring system.And And, the method for gathered data of the present invention is the data for directly being provided using operating system, does not exist and gathers work using intermediate data The expense that tool brings.
Describe the present invention below in conjunction with the drawings and specific embodiments, but it is not as a limitation of the invention.
Description of the drawings
Fig. 1 is to represent a complete data using the data presentation technique of the present invention；
Fig. 2 is to represent a datagram with many datas using the data presentation technique of the present invention；
Fig. 3 is to realize the data comprising a static data with complete information with the data presentation technique of the present invention Report；
Fig. 4 is to realize the data comprising a dynamic data with complete information with the data presentation technique of the present invention Report；
Fig. 5 is realized comprising a static data with complete information and a tool with the data presentation technique of the present invention There is the datagram of the dynamic data of complete information；
Fig. 6 is to realize the data comprising a dynamic data with partial information with the data presentation technique of the present invention Report；
Fig. 7 is the data processing method flow chart of the physical machine resource information of the present invention；
Fig. 8 is the data handling system schematic diagram of the physical machine resource information of the present invention.
Specific embodiment
Fig. 7 is the data processing method flow chart of the physical machine resource information of the present invention.As shown in fig. 7, the method includes：
S1, from the file of the resource information and proc file system of linux orders obtain physical machine static information with And the multidate information that current time is represented in the form of current state value or accumulated value；
S2, obtains the new multidate information of physical machine again after 1 second, by the new multidate information represented in the form of accumulated value Data deduct one second obtain multidate information data, obtain the resource using information in 1 second；
S3, creates low-redundancy data form：DTI is used for mark data type, and CI is used for mark data position, and DRB is used for The data item of mark data subsequent transmission, DV are used for the result that the value of identification data item is arranged in order；
S4, by the static information, the new multidate information and it is described 1 second in resource using information respectively with low superfluous Remaining data form is indicated, and is subsequently transmitted to destination node, repeats above-mentioned steps, obtains the reality of the destination node When resource information.
Further, the static information acquisition methods in the S1 include：
S11：, in the output information of the ifconfig orders of the information for checking network card configuration provided from Linux, extract IP Address；
S12, reads the cpuinfo files in proc file system, obtains the check figure and frequency of processor；
S13, reads the meminfo files in proc file system, obtains internal memory total size；
S14, reads the partitions files in proc file system, obtains disk total size；
S15, obtains Netowrk tape in the output information of the dmesg orders for checking each unit status provided from Linux；
S16, reads the stat file in proc file system, the processor usage amount being obtained from since start it is cumulative Value；
S17, reads the net/dev files in proc file system, the network bandwidth usage amount being obtained from since start Accumulated value.
Further, the multidate information acquisition methods in the S1 include：
S111, reads the stat file in proc file system, the processor usage amount being obtained from since start it is cumulative Value；
S112, reads the meminfo files in proc file system, obtains internal memory usage amount；
S113, disk usage amount is provided in the output information of the df orders for checking disk resource provided from Linux；
S114, reads the net/dev files in proc file system, the network bandwidth usage amount being obtained from since start Accumulated value.
Further, the S2 includes：
S21, the processor that the accumulated value of the processor usage amount since start deducts acquisition upper one second is used Amount accumulated value, obtains the usage amount of the processor in this second；
S22, the network bandwidth that the network bandwidth usage amount accumulated value since start deducts acquisition upper one second is made Consumption accumulated value, obtains the usage amount of the network bandwidth in this second.
Further, in the S3：
If data type has NDTIKind, then using every data beforePosition as DTI, whereinRepresent not Less than the smallest positive integral of m；
The CI, is represented with one, if the position is 1, then it represents that the data is not the last item in data message, if The position is 0, then it represents that the data is the last item of data message；
The value of the data item in the DV is adjustable length character string, then add a byte before the character string Represent the length of the character string；
N is up to per the data item number of class dataDRB, then using at least N of the every data for transmittingDRBPosition is used as DRB, institute Each one data item of correspondence of DRB is stated, if certain position is 1, then it represents that values of the DV comprising the corresponding data item, if should Position is 0, then values of the DV not comprising the data item.
Further, in the S3：
If the data of software system internal transmission only have a class, DTI can be left out；If the number of the data transmitted every time According to item all same, then DRB can be left out.
Fig. 8 is the data handling system schematic diagram of the physical machine resource information of the present invention.As shown in figure 8, the system includes：
Data obtaining module 100, obtains physical machine from the file of the resource information and proc file system of linux orders Static information and the multidate information that represented in the form of current state value or accumulated value of current time；
Message processing module obtains the new multidate information of physical machine again after 200,1 second, will be represented in the form of accumulated value New multidate information in data deduct one second obtain multidate information data, obtain the resource using information in 1 second；
Form creation module 300, creates low-redundancy data form：DTI is used for mark data type, and CI is used for mark data Position, DRB are used for the data item of mark data subsequent transmission, and DV is used for the result that the value of identification data item is arranged in order；
As a result acquisition module 400, by the static information, the new multidate information and it is described 1 second in resource use Information is indicated with low-redundancy data form respectively, is subsequently transmitted to destination node, repeats above-mentioned module, is obtained The real time resources information of the destination node.
Further, described information acquisition module 100 includes：
IP address module is extracted, is carried in the output of the information ifconfig order for checking network card configuration provided from Linux Take IP address；
First reads proc file system modules, reads the cpuinfo files in proc file system, obtains processor Check figure and frequency, read the meminfo files in proc file system, obtain internal memory total size, read in proc file system Partitions files, obtain disk total size, read proc file system in stat file, be obtained from start since The accumulated value of processor usage amount, reads the net/dev files in proc file system, the network bandwidth being obtained from since start The accumulated value of usage amount；
Operation dmesg command modules, obtain Netowrk tape in the dmesg orders for checking each unit status provided from Linux It is wide.
Further, described information acquisition module 100 also includes：
Second reads proc file system modules, reads the stat file in proc file system, since being obtained from start Processor usage amount accumulated value, read proc file system in meminfo files, obtain internal memory usage amount, read Net/dev files in proc file system, the accumulated value of the network bandwidth usage amount being obtained from since start；
Df modules are performed, disk is provided in the output information of the df orders for checking disk resource provided from Linux and is used Amount.
Further, it is characterised in that described information processing module 200 includes：
The accumulated value of the processor usage amount since start is deducted the place of acquisition upper one second by first processing module Reason device usage amount accumulated value, obtains the usage amount of the processor in this second；
The network bandwidth usage amount accumulated value since start is deducted the net of acquisition upper one second by Second processing module Network bandwidth usage amount accumulated value, obtains the usage amount of the network bandwidth in this second.
Further, in the form creation module 300：
If data type has NDTIKind, then using every data beforePosition as DTI, whereinRepresent not Less than the smallest positive integral of m；
The CI, is represented with one, if the position is 1, then it represents that the data is not the last item in data message, if The position is 0, then it represents that the data is the last item of data message；
The value of the data item in the DV is adjustable length character string, then add a byte before the character string Represent the length of the character string；
N is up to per the data item number of class dataDRB, then using at least N of the every data for transmittingDRBPosition is used as DRB, institute Each one data item of correspondence of DRB is stated, if certain position is 1, then it represents that values of the DV comprising the corresponding data item, if should Position is 0, then values of the DV not comprising the data item.
Further, in the form creation module 300：
If the data of software system internal transmission only have a class, DTI can be left out；If the number of the data transmitted every time According to item all same, then DRB can be left out.
The resource information acquisition method of the present invention towards be (SuSE) Linux OS, the life that information source is provided for Linux Order and proc file system, proc file system mainly include the service condition of total resources and resource, and resource service condition is With the accumulated value since the start（The such as load of CPU and transmission volume etc.）And current state（Such as internal memory usage amount and disk Usage amount etc.）Form represent.
One data message can include one or more data.Using the data presentation technique in the invention, one complete The expression of whole data such as Fig. 1, expression such as Fig. 2 of a datagram with many datas.
The monitoring system of one large scale system, the data of transmission totally two class：Static data and dynamic data.It is therein every The data item included by class data（Assume the size and unit of each data item occurrence, numeral is represented with 10 systems below）It is as follows：
Static data（Totally 6）：The network address（10.10.10.10, account for 4 bytes）, (8, account for one to processor check figure Byte), processor frequencies (2048MHZ accounts for two bytes), internal memory total size (4192MB accounts for two bytes), disk total size (140272MB accounts for four bytes) and network interface card bandwidth (1000Mbps accounts for two bytes).
Dynamic data（Totally 5）：The network address（10.10.10.10 4 bytes are accounted for）, (123MHZ accounts for two to processor load Individual byte), internal memory usage amount (2001MB accounts for two bytes), disk usage amount (2013MB accounts for four bytes) and network interface card bandwidth Usage amount (28Mbps accounts for two bytes).
Wherein processor load and network interface card bandwidth usage amount have been represented in the form of the accumulated value since from start, other Represented in the form of current state.Because the data of the software system internal transmission totally two class, the DTI per data isPosition, 1 represents static data, and 0 represents dynamic data.DRB is 6：For static data, 6 of DRB（First Position is to the 6th）It is corresponding in turn to the network address, processor check figure, processor frequencies, internal memory total size, disk total size and network interface card Bandwidth；For dynamic data, first 5 of DRB（First to the 5th）It is corresponding in turn to the network address, processor load, internal memory Usage amount, disk usage amount and network interface card bandwidth usage amount.
Conveniently, representing DTI using a byte（8th）、CI（7th）And DRB（First 6）.
Data acquisition is comprised the following steps that：
Step A：The number order provided using Linux and the static information of the file acquisition physical machine of proc file system The multidate information represented in the form of accumulated value with the moment；
Step B：The static information of acquisition is processed into into the data message of low redundancy, being sent to needs to obtain physical machine letter The destination node of breath；
Step C：From the multidate information that physical machine is newly read from the file of proc file system after 1 second；
Step D：For the multidate information represented in the form of accumulated value, the data that step C is obtained were individually subtracted before 1 second The data of acquisition, obtain the resource service condition in this 1 second；
Step E：It is dynamic that the multidate information with current state as representation and step D that step C is obtained is calculated Data message of the state information processing into low redundancy, is sent to the destination node for needing to obtain the physics machine information；
Step F：Repeat step C-E, destination node can just obtain the real-time resource information of the node.
Wherein step A is concretely comprised the following steps：
Step A1：Operation ifconfig（The information of network card configuration is checked in Linux）, extract the IP address in output information （10.10.10.10）；
Step A2：The cpuinfo files in proc file system are read, the check figure of processor is obtained（8）And frequency （2048MHZ）；
Step A3：The meminfo files in proc file system are read, internal memory total size is obtained（4192MB）；
Step A4：The partitions files in proc file system are read, disk total size is obtained（140272MB）；
Step A5：Operation dmesg orders（The order for checking each unit status that Linux is provided）, obtain the network bandwidth （1000Mbps）；
Step A6：Read the stat file in proc file system, the processor usage amount being obtained from since start it is tired It is value added；
Step A7：The net/dev files in proc file system are read, the network bandwidth usage amount being obtained from since start Accumulated value.
Process the data message for obtaining such as Fig. 3 in step B, wherein BF (binary system is 10111111) for represent DTI, CI and 8 digits of DRB, remaining represents the value of each static data item respectively.
Step C includes step again：
Step C1：Read the stat file in proc file system, the processor usage amount being obtained from since start it is tired It is value added；
Step C2：The meminfo files in proc file system are read, internal memory usage amount (2001MB) is obtained；
Step C3：Perform df（The order for checking disk resource that Linux is provided）, magnetic is extracted from the order output information Disk usage amount (2013MB)；
Step C4：The net/dev files in proc file system are read, the network bandwidth usage amount being obtained from since start Accumulated value；
Step D includes step：
Step D1：The processor usage amount accumulated value that step C1 is obtained is deducted into upper one second（From step A6 or step C1）Obtain The processor usage amount accumulated value for taking, obtains the usage amount of the processor in this second（123MHZ）；
Step D2：The network bandwidth usage amount accumulated value that step C4 is obtained is deducted into upper one second（From step A7 or step C4） The network bandwidth usage amount accumulated value of acquisition, obtains the usage amount of the network bandwidth in this second（28Mbps）；
The data message that the process of step E is obtained such as Fig. 4, wherein 1F (binary system is 00011111) for represent DTI, CI and 8 digits of DRB, remaining represents the value of each Dynamic Data Items respectively.
If sent together with the dynamic data that the static data that step A is obtained is obtained with step C, by these data Data message such as Fig. 5, wherein FF (binary system is 11111111) and 1F (binary system is 00011111) that process is obtained is respectively Front 8 digit of static DTI, CI and DRB with dynamic data is represented, the CI of static data is 1（The 7th of FF）Represent the number There are data according to rear（Dynamic data）, and the CI of dynamic data is 0（The 7th of 1F）Represent that the data is the last item.
If destination node only needs processor load, internal memory usage amount and the disk usage amount for obtaining the physical machine node Dynamic data, then datagram such as Fig. 6 that the process of step E is obtained, wherein 0F (binary system is 00001111) be represent DTI, CI With 8 digits of DRB.DRB is 0F (binary system is 01111), represent DV in the data of the data message only containing the network address, The value of processor load, internal memory usage amount and disk usage amount, not comprising network interface card bandwidth usage amount.
Need to know the data class number of software system internal transmission and per class using the data presentation technique of the present invention The number of the data item that packet contains.For value is that the data item of numeral needs to know the data item size.
DTI per data is used for the type of mark data.The total N of the data of one software system internal transmissionDTIClass, Before then using per data(Representing the smallest positive integral not less than m) position, as DTI, represents the number According to type.
That follow DTI closely is CI, is represented with one.If the position is 1, then it represents that the data is not last in data message One, otherwise, it means that the data is the last item of data message.
It is DRB after CI, for identifying its follow-up data item.N is up to per the data item number of class dataDRB, then use At least N of every data of transmissionDRBPosition is used as DRB.Each one data item of correspondence of DRB.In DRB, if certain position is 1, Values of the DV comprising the corresponding data item is represented then, conversely, values of the DV not comprising the data item.
Last DV is by the order formulated according to the CI result arranged by the value of each data item.If the value is length Variable character string, the then length before the character string plus the byte representation character string.
If the data of software system internal transmission only have a class, DTI can be left out；If the number of the data transmitted every time According to item all same, then DRB can be left out.So as to further reduce the redundancy of software inhouse transmission data.
Certainly, the present invention can also have other various embodiments, in the case of without departing substantially from spirit of the invention and its essence, ripe Know those skilled in the art and work as and various corresponding changes and deformation, but these corresponding changes and change can be made according to the present invention Shape should all belong to the protection domain of appended claims of the invention.

Claims (10)

1. a kind of data processing method of physical machine resource information, it is characterised in that include：
Step 1, from the file of the resource information and proc file system of linux orders obtain physical machine static information and The multidate information that current time is represented in the form of current state value or accumulated value；
Step 2, obtains the new multidate information of physical machine again after 1 second, by the new multidate information represented in the form of accumulated value Data deduct one second obtain multidate information data, obtain the resource using information in 1 second；
Step 3, creates low-redundancy data form：DTI is used for mark data type, and CI is used for mark data position, and DRB is used to mark Know the data item of data subsequent transmission, DV is used for the result that the value of identification data item is arranged in order；
Step 4, by the static information, the new multidate information and it is described 1 second in resource using information respectively with low superfluous Remaining data form is indicated, and is subsequently transmitted to destination node, repeats above-mentioned steps, obtains the reality of the destination node When resource information.
2. the data processing method of physical machine resource information as claimed in claim 1, it is characterised in that in the step 1 Static information acquisition methods include：
Step 11, extracts IP address in the output information of the ifconfig orders for checking network card configuration information provided from Linux；
Step 12, reads the cpuinfo files in proc file system, obtains the check figure and frequency of processor；
Step 13, reads the meminfo files in proc file system, obtains internal memory total size；
Step 14, reads the partitions files in proc file system, obtains disk total size；
Step 15, obtains the network bandwidth in the output information of the dmesg orders for checking each unit status provided from Linux；
Step 16, reads the stat file in proc file system, the accumulated value of the processor usage amount being obtained from since start；
Step 17, reads the net/dev files in proc file system, the network bandwidth usage amount being obtained from since start it is tired It is value added.
3. the data processing method of physical machine resource information as claimed in claim 1, it is characterised in that in the step 1 Multidate information acquisition methods include：
Step 111, reads the stat file in proc file system, the processor usage amount being obtained from since start it is cumulative Value；
Step 112, reads the meminfo files in proc file system, obtains internal memory usage amount；
Step 113, extracts disk usage amount in the output information of the df orders for checking disk resource provided from Linux；
Step 114, reads the net/dev files in proc file system, the network bandwidth usage amount being obtained from since start Accumulated value.
4. the data processing method of physical machine resource information as claimed in claim 2, it is characterised in that the step 2 includes：
The accumulated value of the processor usage amount since start is deducted the processor usage amount of acquisition upper one second by step 21 Accumulated value, obtains the usage amount of the processor in this second；
Step 22, the network bandwidth that the network bandwidth usage amount accumulated value since start deducts acquisition upper one second is used Amount accumulated value, obtains the usage amount of the network bandwidth in this second.
5. the data processing method of physical machine resource information as claimed in claim 1, it is characterised in that in the step 3：
If data type has NDTIKind, then using every data beforePosition as DTI, whereinExpression is not less than The smallest positive integral of m；
The CI, is represented with one, if the position is 1, then it represents that the data is not the last item in data message, if the position For 0, then it represents that the data is the last item of data message；
The value of the data item in the DV is adjustable length character string, then add a byte representation before the character string The length of the character string；
N is up to per the data item number of class dataDRB, then using at least N of the every data for transmittingDRBPosition is as DRB, described Each one data item of correspondence of DRB, if certain position is 1, then it represents that values of the DV comprising the corresponding data item, if the position For 0, then values of the DV not comprising the data item.
6. the data processing method of physical machine resource information as claimed in claim 1, it is characterised in that in the step 3：
If the data of software system internal transmission only have a class, DTI can be left out；If the data item of the data transmitted every time All same, then can leave out DRB.
7. a kind of data handling system of physical machine resource information, it is characterised in that include：
Data obtaining module, obtains the static state of physical machine from the file of the resource information and proc file system of linux orders The multidate information that information and current time are represented in the form of current state value or accumulated value；
Message processing module, obtains the new multidate information of physical machine again after 1 second, new dynamic letter will be represented in the form of accumulated value Data in breath deduct the data of the multidate information for obtaining for a second, obtain the resource using information in 1 second；
Form creation module, creates low-redundancy data form：DTI is used for mark data type, and CI is used for mark data position, DRB is used for the data item of mark data subsequent transmission, and DV is used for the result that the value of identification data item is arranged in order；
As a result acquisition module, by the static information, the new multidate information and it is described 1 second in resource using information difference It is indicated with low-redundancy data form, is subsequently transmitted to destination node, repeat above-mentioned module, obtains the target section The real time resources information of point.
8. the data handling system of physical machine resource information as claimed in claim 7, it is characterised in that described information obtains mould Block includes：
IP address module is extracted, is carried in the output information of the ifconfig orders for checking network card configuration information provided from Linux Take IP address；
First reads proc file system modules, reads the cpuinfo files in proc file system, obtains the check figure of processor And frequency, the meminfo files in proc file system are read, internal memory total size is obtained, is read in proc file system Partitions files, obtain disk total size, read the stat file in proc file system, the place being obtained from since start The accumulated value of reason device usage amount, reads the net/dev files in proc file system, and the network bandwidth since being obtained from start makes The accumulated value of consumption；
Operation dmesg command modules, obtain in the output information of the dmesg orders for checking each unit status provided from Linux The network bandwidth.
9. the data handling system of physical machine resource information as claimed in claim 7, it is characterised in that described information obtains mould Block also includes：
Second reads proc file system modules, reads the stat file in proc file system, the place being obtained from since start The accumulated value of reason device usage amount, reads the meminfo files in proc file system, obtains internal memory usage amount, reads proc literary Net/dev files in part system, the accumulated value of the network bandwidth usage amount being obtained from since start；
Df modules are performed, disk usage amount is provided in the output information of the df orders for checking disk resource provided from Linux.
10. the data handling system of physical machine resource information as claimed in claim 8, it is characterised in that described information process Module includes：
The accumulated value of the processor usage amount since start is deducted the processor of acquisition upper one second by first processing module Usage amount accumulated value, obtains the usage amount of the processor in this second；
The network bandwidth usage amount accumulated value since start is deducted the Netowrk tape of acquisition upper one second by Second processing module Wide usage amount accumulated value, obtains the usage amount of the network bandwidth in this second.
CN201310123750.3A 2013-04-10 2013-04-10 Data processing method and system for physical machine resource information Active CN104102664B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310123750.3A CN104102664B (en) 2013-04-10 2013-04-10 Data processing method and system for physical machine resource information

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310123750.3A CN104102664B (en) 2013-04-10 2013-04-10 Data processing method and system for physical machine resource information

Publications (2)

Publication Number Publication Date
CN104102664A CN104102664A (en) 2014-10-15
CN104102664B true CN104102664B (en) 2017-04-12

Family

ID=51670823

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310123750.3A Active CN104102664B (en) 2013-04-10 2013-04-10 Data processing method and system for physical machine resource information

Country Status (1)

Country Link
CN (1) CN104102664B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107203478B (en) * 2017-06-21 2020-07-24 苏州浪潮智能科技有限公司 Data storage method and device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102103518A (en) * 2011-02-23 2011-06-22 运软网络科技(上海)有限公司 System for managing resources in virtual environment and implementation method thereof
CN102271145A (en) * 2010-06-04 2011-12-07 国云科技股份有限公司 Virtual computer cluster and enforcement method thereof
CN102681899A (en) * 2011-03-14 2012-09-19 金剑 Virtual computing resource dynamic management system of cloud computing service platform

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8776050B2 (en) * 2003-08-20 2014-07-08 Oracle International Corporation Distributed virtual machine monitor for managing multiple virtual resources across multiple physical nodes

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102271145A (en) * 2010-06-04 2011-12-07 国云科技股份有限公司 Virtual computer cluster and enforcement method thereof
CN102103518A (en) * 2011-02-23 2011-06-22 运软网络科技(上海)有限公司 System for managing resources in virtual environment and implementation method thereof
CN102681899A (en) * 2011-03-14 2012-09-19 金剑 Virtual computing resource dynamic management system of cloud computing service platform

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
云计算平台中多虚拟机内存协同优化策略研究;张伟哲等;《计算机学报》;20111231;第34卷(第12期);第2265-2277页 *

Also Published As

Publication number Publication date
CN104102664A (en) 2014-10-15

Similar Documents

Publication Publication Date Title
US10498355B2 (en) Searchable, streaming text compression and decompression using a dictionary
US20190188417A1 (en) Data masking systems and methods
CN111404883B (en) Method and device for analyzing Internet of things equipment universal protocol
CN104753540B (en) Data compression method, data decompression method and apparatus
CN106407201A (en) Data processing method and apparatus
CN106959948A (en) The system and its preprocess method pre-processed for distributed nature to big data
CN111970353A (en) Data processing method and device for heterogeneous equipment in Internet of things of cloud computing platform
CN107241305B (en) Network protocol analysis system based on multi-core processor and analysis method thereof
CN107145556B (en) Universal distributed acquisition system
CN105868364B (en) Byte stream-based structured data representation method
CN103209153A (en) Method, device and system for processing messages
CN103246484B (en) A kind of date storage method, Apparatus and system
CN103885829A (en) Virtual machine cross-data-center dynamic migration optimization method based on statistics
CN105574032A (en) Rule matching operation method and device
CN112702228A (en) Service current limiting response method and device, electronic equipment and readable storage medium
CN107124396B (en) Method for structuring processing of IEC61850 GOOSE message machine data
CN104102664B (en) Data processing method and system for physical machine resource information
CN111831540A (en) Debugging information processing method, processing device and storage medium
CN107368300B (en) MapReduce-based data summarization system and method
CN108694205B (en) Method and device for matching target field
CN107454456B (en) Management control method and system for floating layer elements
CN109284088A (en) A kind of signaling big data processing method and electronic equipment
CN103106144B (en) A kind of internal memory index compression method and apparatus
KR101218087B1 (en) Method for Extracting InputFormat for Binary Format Data in Hadoop MapReduce and Binary Data Analysis Using the Same
CN109408577B (en) ORACLE database JSON analysis method, system, device and storable medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20240320

Address after: Room 711C, Floor 7, Building A, Yard 19, Ronghua Middle Road, Daxing District, Beijing Economic-Technological Development Area, 100176

Patentee after: Beijing Zhongke Flux Technology Co.,Ltd.

Country or region after: China

Address before: 100190 No. 6 South Road, Zhongguancun Academy of Sciences, Beijing, Haidian District

Patentee before: Institute of Computing Technology, Chinese Academy of Sciences

Country or region before: China