CN113505165A - Sensor data processing method and device, computer equipment and storage medium - Google Patents
Sensor data processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113505165A CN113505165A CN202110756769.6A CN202110756769A CN113505165A CN 113505165 A CN113505165 A CN 113505165A CN 202110756769 A CN202110756769 A CN 202110756769A CN 113505165 A CN113505165 A CN 113505165A
- Authority
- CN
- China
- Prior art keywords
- data
- index
- index data
- target
- sensors
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title description 13
- 238000006243 chemical reaction Methods 0.000 claims abstract description 88
- 238000012545 processing Methods 0.000 claims abstract description 77
- 230000000007 visual effect Effects 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims abstract description 31
- 230000001960 triggered effect Effects 0.000 claims abstract description 22
- 239000000758 substrate Substances 0.000 claims description 27
- 230000006870 function Effects 0.000 claims description 14
- 238000004891 communication Methods 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 claims description 8
- 230000004044 response Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 2
- 108010028984 3-isopropylmalate dehydratase Proteins 0.000 claims 3
- 238000007726 management method Methods 0.000 description 123
- 238000010586 diagram Methods 0.000 description 4
- 238000013480 data collection Methods 0.000 description 2
- 238000007667 floating Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013501 data transformation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Fuzzy Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for processing sensor data, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring index data of a plurality of sensors through an intelligent platform management interface; carrying out format conversion according to the index data to obtain structured data; and responding to a data processing instruction triggered by a user through the visual interface, wherein the data processing instruction is used for processing the structured data. The index data query can be provided for the user based on the structured data, and the query can comprise various operations such as backtracking query and the like. The server can unify the index data obtained by the sensors of different types into the structured data by performing format conversion on the index data, and can provide the structural data which is easier to read for the user when responding to a data processing instruction triggered by the user based on the structured data, so that the user can provide the result data which is easier to read for the user while performing backtracking query on the index data of the sensors, and the data query efficiency is improved.
Description
Technical Field
The embodiment of the invention relates to a sensor monitoring technology, in particular to a sensor data processing method and device, computer equipment and a storage medium.
Background
With the upgrade of server hardware, the server implements parameter detection of hardware components by configuring a Baseboard Management Controller (BMC). The sensors of the baseboard management controller can measure parameters of the hardware components by embedding the hardware components.
Currently, a server adopts centralized collection when counting index data measured by a sensor of a substrate management controller. The centralized collection is to obtain index data collected by a sensor of the baseboard management controller at a fixed time and output the index data. However, the centralized collection method cannot realize data backtracking, and the user cannot quickly query the index data, so that the index data query efficiency is low.
Disclosure of Invention
The invention provides a sensor data processing method and device, computer equipment and a storage medium, and aims to improve the query efficiency of index data.
In a first aspect, an embodiment of the present invention provides a method for processing sensor data, including:
acquiring index data of a plurality of sensors through an intelligent platform management interface;
carrying out format conversion according to the index data to obtain structured data;
and responding to a data processing instruction triggered by a user through the visual interface, wherein the data processing instruction is used for processing the structured data.
On the basis of the above embodiment, acquiring index data of a plurality of sensors through an intelligent platform management interface includes:
and configuring a plurality of hierarchical management nodes, wherein each hierarchical management node is connected with the sensors of the plurality of substrate management controllers through an intelligent platform management interface, and the hierarchical management nodes are used for respectively acquiring index data from the sensors of the plurality of substrate management controllers through the intelligent platform management interface according to a preset time interval and writing the index data into a target file.
In the above embodiment, index data of sensors of different bmcs can be collected by a plurality of hierarchical management nodes at preset time intervals. By configuring different hierarchical management nodes, the sampling frequency of a sensor of the baseboard management controller can be set, and meanwhile, index data can be collected in parallel among the hierarchical management nodes, so that the configurability and the collection efficiency of index data collection are improved.
In addition to the above embodiment, configuring a plurality of hierarchical management nodes includes:
establishing an intelligent platform management interface communication link between a hierarchical management node and sensors of a plurality of baseboard management controllers through a Free IPMI tool;
configuring the acquisition frequency of a sensor of each baseboard management controller;
establishing a target file and configuring a storage address of the target file;
respectively acquiring index data of sensors of a plurality of substrate management controllers through an out-of-band grabbing function of a Free IPMI tool according to the acquisition frequency;
and writing the acquired index data into a target file.
According to the embodiment, the quick configuration of the hierarchical management nodes can be realized by means of the FreeI PMI tool, the index data of the sensors of the plurality of substrate management controllers can be quickly acquired by means of the out-of-band capture function of the FreeI PMI tool, and the configuration efficiency and the acquisition efficiency of the index data are improved.
On the basis of the above embodiment, performing format conversion according to the index data to obtain structured data includes:
configuring a listener through Free IPMI, wherein the listener is used for monitoring the content of a target file;
and when the listener monitors that the content of the target file is updated, performing format conversion according to the updated index data to obtain structured data.
According to the embodiment, the target file is monitored through the monitor, when the condition that the content of the target file is updated is monitored, format conversion is carried out on the index data to obtain the structured data, the condition that the content of the target file is overstocked is avoided, the data volume of each conversion is reduced, and the conversion efficiency of the structured data is improved for different data. The monitor can monitor the content update of the target file in real time, so that the timeliness of the structured data conversion can be improved.
On the basis of the above embodiment, when the listener monitors that the content of the target file is updated, performing format conversion according to the updated index data to obtain structured data, including:
when the listener monitors that the content of the target file is updated, the updated index data is sent to the logstack cluster;
and calling a data conversion template through the logstack cluster, and performing format conversion according to the index data to obtain structured data.
In the embodiment, the data conversion template is called to realize the conversion of the index data through the logstack cluster. The formats of the index data output by the sensors of different substrate management controllers are different, so that different data conversion templates can be configured for the sensors of different substrate management controllers, the index data are converted into the structured data with a uniform format through the data conversion template, the data of the sensors of different types of substrate management controllers are converted into the data with the uniform format, and the structured data have the uniform format, so that indexes and searches are easy to establish, and the data access efficiency is improved.
On the basis of the above embodiment, the index data includes a plurality of index parameters sorted according to a preset order, and the data conversion template includes parameter type configuration information of the plurality of index parameters sorted according to the preset order. Correspondingly, calling a data conversion template through the logstack cluster, and performing format conversion according to the index data to obtain structured data, wherein the format conversion comprises the following steps:
acquiring a plurality of index parameters contained in the index data through the logstack cluster;
configuring the parameter type of a target parameter according to the parameter type configuration information in the data conversion template, wherein the target parameter is any one index parameter;
determining structured data according to the target parameters and the configured parameter types;
an index is generated from the structured data.
The above embodiment can configure a parameter type for the index parameter, and form the structured data according to the parameter type and the index parameter. According to the coding mode of different index data, the index parameters in the index data output by the sensors of different baseboard management controllers can be read, so that the parameter types can be configured for the index parameters more accurately, and the accuracy of the structured data is improved.
On the basis of the above embodiment, the response to the data processing instruction triggered by the user through the visual interface includes:
receiving a data processing instruction sent by a client through a visual interface, wherein the data processing instruction is used for inquiring target parameters of the structured data according to target time;
determining target structured data according to the target time and the index;
determining a query result according to the target structured data and the target parameters;
and feeding back the query result to the client so that the client can output the query structure through the visual interface.
According to the embodiment, the data processing instruction sent by the user through the visual interface can be received, and the server feeds the query result back to the client after obtaining the query result according to the data processing instruction, so that the client provides the query result for the user through the visual interface. The visual interface can simplify user operation and improve usability.
In a second aspect, an embodiment of the present invention further provides a device for processing sensor data, including:
the index data acquisition module is used for acquiring index data of the sensors through the intelligent platform management interface;
the structured data conversion module is used for carrying out format conversion according to the index data to obtain structured data;
and the response module is used for responding to a data processing instruction triggered by a user through the visual interface, and the data processing instruction is used for processing the structured data.
In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor, when executing the program, implements the method for processing sensor data as shown in the embodiment of the present application.
In a fourth aspect, embodiments of the present invention further provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are used to perform a method for processing sensor data as shown in the embodiments of the present application.
According to the sensor data processing method provided by the embodiment of the invention, the index data of a plurality of sensors can be acquired through the intelligent platform management interface, the structured data is obtained after the format conversion is carried out on the index data, and a data processing instruction triggered by a client through the visual interface is responded. Compared with the prior art that index data can not be collected in a centralized manner, the method for processing sensor data provided by the embodiment of the invention can be used for converting the format of the index data obtained by the sensor in the server, storing the structured data and providing index data query for a user based on the structured data, wherein the query can comprise various operations such as backtracking query and the like. The server can unify the index data obtained by the sensors of different types into the structured data by performing format conversion on the index data, and can provide the structural data which is easier to read for the user when responding to a data processing instruction triggered by the user based on the structured data, so that the user can provide the result data which is easier to read for the user while performing backtracking query on the index data of the sensors, and the data query efficiency is improved.
Drawings
FIG. 1 is a flow chart of a method for processing sensor data according to a first embodiment of the present invention;
FIG. 2 is a flow chart of a method for processing sensor data according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a sensor data processing apparatus according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device in the fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of a sensor data processing method according to an embodiment of the present invention, where this embodiment is applicable to a situation of sensor data acquisition and query of a Baseboard Management Controller (BMC), and the method may be executed by a server, and specifically includes the following steps:
and 110, acquiring index data of a plurality of sensors through an intelligent platform management interface.
An Intelligent Platform Management Interface (IPMI) is used to collect data collected by a sensor of a substrate Management controller. An intelligent platform management interface may be established in advance between a sensor of the substrate management controller and a collection node, and the collection node may be located in a server. Alternatively, a connection may be established between one collection node and the sensors of multiple baseboard management controllers. The method realizes that one acquisition node acquires the sensing data of the sensors of the plurality of substrate management controllers through the intelligent platform management interface. The sensing data includes time information, serial number information, sensor network address (IP address), event and other parameters.
Optionally, in step 110, index data of a plurality of sensors is acquired through the intelligent platform management interface, and the method may be implemented in the following manner:
and configuring a plurality of hierarchical management nodes, wherein each hierarchical management node is connected with the sensors of the plurality of substrate management controllers through an intelligent platform management interface, and the hierarchical management nodes are used for respectively acquiring index data from the sensors of the plurality of substrate management controllers through the intelligent platform management interface according to a preset time interval and writing the index data into a target file.
A plurality of hierarchical management nodes, each of which corresponds to an acquisition node, may be configured in advance. A hierarchical management node is configured to interface with sensors of a plurality of baseboard management controllers. The functions of the hierarchical management node comprise acquisition of index data, writing of the index data into a target file and output of the index data.
Due to the large number of sensors of the baseboard management controller, for example, in the order of tens of thousands or more, there are duplicates in the types of sensors. Therefore, the sensors can be classified according to the types of the sensors, and index data of a plurality of sensors of one type can be collected by one hierarchical management node according to the classification result.
The acquisition object of each hierarchical management node is a sensor, and the acquisition frequency of each hierarchical management node is a preset time interval. In the above embodiment, the preset time interval may be determined according to the type of the sensor. The acquisition frequency of any one hierarchical management node may also be configured by an administrator.
In the above embodiment, index data of sensors of different bmcs can be collected by a plurality of hierarchical management nodes at preset time intervals. By configuring different hierarchical management nodes, the sampling frequency of a sensor of the baseboard management controller can be set, and meanwhile, index data can be collected in parallel among the hierarchical management nodes, so that the configurability and the collection efficiency of index data collection are improved.
Further, configuring a plurality of hierarchical management nodes may be implemented as:
establishing an intelligent platform management interface communication link between a hierarchical management node and sensors of a plurality of baseboard management controllers through a Free IPMI tool; configuring the acquisition frequency of a sensor of each baseboard management controller; establishing a target file and configuring a storage address of the target file; respectively acquiring index data of sensors of a plurality of substrate management controllers through an out-of-band grabbing function of a Free IPMI tool according to the acquisition frequency; and writing the acquired index data into a target file.
Each hierarchical management node may be configured with a timer, the duration of which represents the acquisition frequency of the hierarchical management node. Through the node management function provided by the Free IPMI tool, an intelligent platform management interface communication link between the hierarchical management node and the sensor can be established quickly. The target file is established in the server, and the storage address of the target file can be configured by an administrator. Each hierarchical management node may be configured with a target file. And the hierarchical management node writes the acquired index data into a target file. The out-of-band capture function provided by the Free IPMI tool is used for collecting the sensors establishing the intelligent platform management interface communication link. After the configuration, the hierarchical management node can collect the index data of the sensors and write the collected index data into the target file. The name of the target file may be named with the identity of the hierarchical management node in order to distinguish between different target files written by different hierarchical management nodes.
According to the embodiment, the quick configuration of the hierarchical management nodes can be realized by means of the FreeI PMI tool, the index data of the sensors of the plurality of substrate management controllers can be quickly acquired by means of the out-of-band capture function of the FreeI PMI tool, and the configuration efficiency and the acquisition efficiency of the index data are improved.
And step 120, carrying out format conversion according to the index data to obtain structured data.
Since the data format of the index data is the default data format of the sensor, the server needs to convert the index data into structured data which is easy to recognize and edit in order to perform subsequent data processing.
Optionally, in step 120, format conversion is performed according to the index data to obtain structured data, which may be implemented in the following manner:
and step 121, configuring a listener through Free IPMI, wherein the listener is used for monitoring the content of the target file.
And installing a filebeat program package on the hierarchical management node, wherein a filebeat monitor is used for monitoring the content of the target file. And editing the path of the target file in the configuration file of the filebeat program package, so that the filebeat listener can acquire the target file according to the path.
And step 122, when the listener monitors that the content of the target file is updated, performing format conversion according to the updated index data to obtain structured data.
And the filebeat monitor detects the content of the target file in real time, and when the content of the target file is monitored to be updated, format conversion is triggered to be performed on the updated index data to obtain structured data.
According to the embodiment, the target file is monitored through the monitor, when the condition that the content of the target file is updated is monitored, format conversion is carried out on the index data to obtain the structured data, the condition that the content of the target file is overstocked is avoided, the data volume of each conversion is reduced, and the conversion efficiency of the structured data is improved for different data. The monitor can monitor the content update of the target file in real time, so that the timeliness of the structured data conversion can be improved.
On the basis of the foregoing embodiment, in step 122, when the listener monitors that the content of the target file is updated, format conversion is performed according to the updated index data, and the obtained structured data may be implemented as:
when the listener monitors that the content of the target file is updated, the updated index data is sent to the logstack cluster; and calling a data conversion template through the logstack cluster, and performing format conversion according to the index data to obtain structured data.
The data transformation template may be configured by the logstack cluster. The data conversion template comprises index parameters contained in the index data and data types of the index parameters which are configured in advance. By configuring the data type for the index parameter, the index parameter contained in the index data can be converted into structured data.
In the embodiment, the data conversion template is called to realize the conversion of the index data through the logstack cluster. The formats of the index data output by the sensors of different substrate management controllers are different, so that different data conversion templates can be configured for the sensors of different substrate management controllers, the index data are converted into the structured data with a uniform format through the data conversion template, the data of the sensors of different types of substrate management controllers are converted into the data with the uniform format, and the structured data have the uniform format, so that indexes and searches are easy to establish, and the data access efficiency is improved.
Further, the index data includes a plurality of index parameters sorted according to a preset order, and the data conversion template includes parameter type configuration information of the plurality of index parameters sorted according to the preset order. At this time, calling a data conversion template through the logstack cluster, and performing format conversion according to the index data to obtain structured data, wherein the format conversion comprises the following steps:
acquiring a plurality of index parameters contained in the index data through the logstack cluster; configuring the parameter type of a target parameter according to the parameter type configuration information in the data conversion template, wherein the target parameter is any one index parameter; determining structured data according to the target parameters and the configured parameter types; an index is generated from the structured data.
Optionally, the data conversion template is preconfigured according to the index parameters included in the index data. Illustratively, the index data includes index parameters such as a timestamp (timestamp), a serial number (productSerialNumber), a name (name), reading, time (event), type (type), and a network address (ipohome). And configuring corresponding type configuration for each index parameter in the data conversion template. For example, if the data type of the configured timestamp is date (date), the timestamp index parameter is converted according to the data type of the date; configuring a data type of the serial number as a keyword (keyword), and storing the serial number according to the data type of the keyword; and if the data type of the configured network address is a floating point number (float), converting the index parameter of the network address according to the data type of the floating point number to obtain structured data.
And installing an Elasticissearch cluster. The resulting structured data is stored in the Elasticsearch cluster. When stored, an index is built for each structured data.
The above embodiment can configure a parameter type for the index parameter, and form the structured data according to the parameter type and the index parameter. According to the coding mode of different index data, the index parameters in the index data output by the sensors of different baseboard management controllers can be read, so that the parameter types can be configured for the index parameters more accurately, and the accuracy of the structured data is improved.
And step 130, responding to a data processing instruction triggered by a user through a visual interface, wherein the data processing instruction is used for processing the structured data.
When a user needs to inquire index data of a certain sensor, a data processing instruction can be initiated to the server through the visual interface. The visualization interface may be a web interface. Alternatively, a Kibana service may be installed. The Kibana service is used for providing visual data query and output of query results. The query field can be set through the Kibana service, such as setting time period self-definition, sensor name, IP, keyword search and the like.
Structured data can be quickly obtained from the Elasticissearch cluster to be matched with the time period searched by a user based on the time period. When the user inputs parameters such as the name of the sensor, the query result can be screened out.
Optionally, step 130, responding to a data processing instruction triggered by the user through the visual interface, may be implemented in the following manner:
and receiving a data processing instruction sent by the client through the visual interface, wherein the data processing instruction is used for inquiring the target parameters of the structured data according to the target time. And determining the target structured data according to the target time and the index. And determining a query result according to the target structured data and the target parameters. And feeding back the query result to the client so that the client can output the query structure through the visual interface.
The user can input the target time of the expected query, and further input target parameters such as the name of the sensor and the like for screening the keywords of the query result. The Elasticissearch cluster queries based on the target time in combination with the index field to determine the target structured data. And further screening according to the target structured data and target parameters input by the user, determining a query result, and realizing backtracking query and custom query.
According to the embodiment, the data processing instruction sent by the user through the visual interface can be received, and the server feeds the query result back to the client after obtaining the query result according to the data processing instruction, so that the client provides the query result for the user through the visual interface. The visual interface can simplify user operation and improve usability.
According to the sensor data processing method provided by the embodiment of the invention, the index data of a plurality of sensors can be acquired through the intelligent platform management interface, the structured data is obtained after the format conversion is carried out on the index data, and a data processing instruction triggered by a client through the visual interface is responded. Compared with the prior art that index data can not be collected in a centralized manner, the method for processing sensor data provided by the embodiment of the invention can be used for converting the format of the index data obtained by the sensor in the server, storing the structured data and providing index data query for a user based on the structured data, wherein the query can comprise various operations such as backtracking query and the like. The server can unify the index data obtained by the sensors of different types into the structured data by performing format conversion on the index data, and can provide the structural data which is easier to read for the user when responding to a data processing instruction triggered by the user based on the structured data, so that the user can provide the result data which is easier to read for the user while performing backtracking query on the index data of the sensors, and the data query efficiency is improved.
Example two
Fig. 2 is a flowchart of a processing method of sensor data according to an embodiment of the present application, which further illustrates the foregoing embodiment, and includes:
step 210, establishing an intelligent platform management interface communication link between the hierarchical management node and a plurality of sensors of the baseboard management controller through a Free IPMI tool.
Step 220, configuring the acquisition frequency of the sensor of each baseboard management controller; establishing a target file and configuring the storage address of the target file.
And step 230, respectively acquiring index data of the sensors of the plurality of substrate management controllers through an out-of-band grabbing function of a Free IPMI tool according to the acquisition frequency. And writing the acquired index data into a target file.
And step 240, configuring a filebeat monitor through Free IPMI, wherein the monitor is used for monitoring the content of the target file. And when the filebeat monitor monitors the content update of the target file, sending the updated index data to the logstack cluster.
Step 250, acquiring a plurality of index parameters contained in the index data through the logstack cluster; and determining the structured data according to the target parameters and the configured parameter types.
Step 260, generating an index from the structured data.
And step 270, receiving a data processing instruction sent by the client through the visual interface.
The data processing instruction is used for inquiring target parameters of the structured data according to the target time;
step 280, determining target structured data according to the target time and the index; and determining a query result according to the target structured data and the target parameters.
According to the sensor data processing method provided by the embodiment of the invention, the index data of a plurality of sensors can be acquired through the intelligent platform management interface, the structured data is obtained after the format conversion is carried out on the index data, and a data processing instruction triggered by a client through the visual interface is responded. Compared with the prior art that index data can not be collected in a centralized manner, the method for processing sensor data provided by the embodiment of the invention can be used for converting the format of the index data obtained by the sensor in the server, storing the structured data and providing index data query for a user based on the structured data, wherein the query can comprise various operations such as backtracking query and the like. The server can unify the index data obtained by the sensors of different types into the structured data by performing format conversion on the index data, and can provide the structural data which is easier to read for the user when responding to a data processing instruction triggered by the user based on the structured data, so that the user can provide the result data which is easier to read for the user while performing backtracking query on the index data of the sensors, and the data query efficiency is improved.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a sensor data processing apparatus according to a second embodiment of the present invention, where this embodiment is applicable to a situation of sensor (Baseboard Management Controller, BMC) data acquisition and query of a substrate Management Controller, and the apparatus may be executed by a server, and specifically includes: an index data acquisition module 310, a structured data conversion module 320, and a response module 330.
An index data obtaining module 310, configured to obtain index data of multiple sensors through an intelligent platform management interface;
the structured data conversion module 320 is configured to perform format conversion according to the index data to obtain structured data;
and the response module 330 is configured to respond to a data processing instruction triggered by a user through the visual interface, where the data processing instruction is used to process the structured data.
On the basis of the above embodiment, the index data obtaining module 310 is configured to:
and configuring a plurality of hierarchical management nodes, wherein each hierarchical management node is connected with the sensors of the plurality of substrate management controllers through an intelligent platform management interface, and the hierarchical management nodes are used for respectively acquiring index data from the sensors of the plurality of substrate management controllers through the intelligent platform management interface according to a preset time interval and writing the index data into a target file.
On the basis of the above embodiment, the index data obtaining module 310 is configured to:
establishing an intelligent platform management interface communication link between a hierarchical management node and sensors of a plurality of baseboard management controllers through a Free IPMI tool;
configuring the acquisition frequency of a sensor of each baseboard management controller;
establishing a target file and configuring a storage address of the target file;
respectively acquiring index data of sensors of a plurality of substrate management controllers through an out-of-band grabbing function of a Free IPMI tool according to the acquisition frequency;
and writing the acquired index data into a target file.
On the basis of the above embodiment, the structured data conversion module 320 is configured to:
configuring a listener through Free IPMI, wherein the listener is used for monitoring the content of a target file;
and when the listener monitors that the content of the target file is updated, performing format conversion according to the updated index data to obtain structured data.
On the basis of the above embodiment, the structured data conversion module 320 is configured to:
when the listener monitors that the content of the target file is updated, the updated index data is sent to the logstack cluster;
and calling a data conversion template through the logstack cluster, and performing format conversion according to the index data to obtain structured data.
On the basis of the embodiment, the index data comprises a plurality of index parameters which are sorted according to a preset sequence, and the data conversion template comprises parameter type configuration information of the index parameters which are sorted according to the preset sequence; the structured data conversion module 320 is configured to:
acquiring a plurality of index parameters contained in the index data through the logstack cluster;
configuring the parameter type of a target parameter according to the parameter type configuration information in the data conversion template, wherein the target parameter is any one index parameter;
determining structured data according to the target parameters and the configured parameter types;
an index is generated from the structured data.
On the basis of the above embodiment, the response module 330 is configured to:
receiving a data processing instruction sent by a client through a visual interface, wherein the data processing instruction is used for inquiring target parameters of the structured data according to target time;
determining target structured data according to the target time and the index;
determining a query result according to the target structured data and the target parameters;
and feeding back the query result to the client so that the client can output the query structure through the visual interface.
In the sensor data processing apparatus provided in the embodiment of the present invention, the index data obtaining module 310 obtains the index data of the plurality of sensors through the intelligent platform management interface, the structured data converting module 320 performs format conversion on the index data to obtain structured data, and the responding module 330 responds to a data processing instruction triggered by a client through the visual interface, and feeds back result data to the user. Compared with the prior art that index data can not be collected in a centralized manner, the method for processing sensor data provided by the embodiment of the invention can be used for converting the format of the index data obtained by the sensor in the server, storing the structured data and providing index data query for a user based on the structured data, wherein the query can comprise various operations such as backtracking query and the like. The server can unify the index data obtained by the sensors of different types into the structured data by performing format conversion on the index data, and can provide the structural data which is easier to read for the user when responding to a data processing instruction triggered by the user based on the structured data, so that the user can provide the result data which is easier to read for the user while performing backtracking query on the index data of the sensors, and the data query efficiency is improved.
The sensor data processing device provided by the embodiment of the invention can execute the sensor data processing method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example four
Fig. 4 is a schematic structural diagram of a computer apparatus according to a fourth embodiment of the present invention, as shown in fig. 4, the computer apparatus includes a processor 40, a memory 41, an input device 42, and an output device 43; the number of processors 40 in the computer device may be one or more, and one processor 40 is taken as an example in fig. 4; the processor 40, the memory 41, the input device 42 and the output device 43 in the computer apparatus may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 4.
The memory 41 is a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the sensor data processing method in the embodiment of the present invention (for example, the index data acquisition module 310, the structured data conversion module 320, and the response module 330 in the sensor data processing device). The processor 40 executes various functional applications of the computer device and data processing, i.e., implements the above-described processing method of the sensor data, by executing software programs, instructions, and modules stored in the memory 41.
The memory 41 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 41 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 41 may further include memory located remotely from processor 40, which may be connected to a computer device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 42 is operable to receive input numeric or character information and to generate key signal inputs relating to user settings and function controls of the computer apparatus. The output device 43 may include a display device such as a display screen.
EXAMPLE five
An embodiment of the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, perform a method for processing sensor data, the method including:
acquiring index data of a plurality of sensors through an intelligent platform management interface;
carrying out format conversion according to the index data to obtain structured data;
and responding to a data processing instruction triggered by a user through the visual interface, wherein the data processing instruction is used for processing the structured data.
On the basis of the above embodiment, acquiring index data of a plurality of sensors through an intelligent platform management interface includes:
and configuring a plurality of hierarchical management nodes, wherein each hierarchical management node is connected with the sensors of the plurality of substrate management controllers through an intelligent platform management interface, and the hierarchical management nodes are used for respectively acquiring index data from the sensors of the plurality of substrate management controllers through the intelligent platform management interface according to a preset time interval and writing the index data into a target file.
In addition to the above embodiment, configuring a plurality of hierarchical management nodes includes:
establishing an intelligent platform management interface communication link between a hierarchical management node and sensors of a plurality of baseboard management controllers through a Free IPMI tool;
configuring the acquisition frequency of a sensor of each baseboard management controller;
establishing a target file and configuring a storage address of the target file;
respectively acquiring index data of sensors of a plurality of substrate management controllers through an out-of-band grabbing function of a Free IPMI tool according to the acquisition frequency;
and writing the acquired index data into a target file.
On the basis of the above embodiment, performing format conversion according to the index data to obtain structured data includes:
configuring a listener through Free IPMI, wherein the listener is used for monitoring the content of a target file;
and when the listener monitors that the content of the target file is updated, performing format conversion according to the updated index data to obtain structured data.
On the basis of the above embodiment, when the listener monitors that the content of the target file is updated, performing format conversion according to the updated index data to obtain structured data, including:
when the listener monitors that the content of the target file is updated, the updated index data is sent to the logstack cluster;
and calling a data conversion template through the logstack cluster, and performing format conversion according to the index data to obtain structured data.
On the basis of the above embodiment, the index data includes a plurality of index parameters sorted according to a preset order, and the data conversion template includes parameter type configuration information of the plurality of index parameters sorted according to the preset order;
correspondingly, calling a data conversion template through the logstack cluster, and performing format conversion according to the index data to obtain structured data, wherein the format conversion comprises the following steps:
acquiring a plurality of index parameters contained in the index data through the logstack cluster;
configuring the parameter type of a target parameter according to the parameter type configuration information in the data conversion template, wherein the target parameter is any one index parameter;
determining structured data according to the target parameters and the configured parameter types;
an index is generated from the structured data.
On the basis of the above embodiment, the response to the data processing instruction triggered by the user through the visual interface includes:
receiving a data processing instruction sent by a client through a visual interface, wherein the data processing instruction is used for inquiring target parameters of the structured data according to target time;
determining target structured data according to the target time and the index;
determining a query result according to the target structured data and the target parameters;
and feeding back the query result to the client so that the client can output the query structure through the visual interface.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the above method operations, and may also perform related operations in the sensor data processing method provided by any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the sensor data processing apparatus, the units and modules included in the apparatus are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A method of processing sensor data, comprising:
acquiring index data of a plurality of sensors through an intelligent platform management interface;
carrying out format conversion according to the index data to obtain structured data;
responding to a data processing instruction triggered by a user through a visual interface, wherein the data processing instruction is used for processing the structured data.
2. The method of claim 1, wherein the obtaining metric data for a plurality of sensors via an intelligent platform management interface comprises:
the method comprises the steps that a plurality of hierarchical management nodes are configured, each hierarchical management node is connected with sensors of a plurality of baseboard management controllers through an intelligent platform management interface, and the hierarchical management nodes are used for respectively obtaining index data from the sensors of the plurality of baseboard management controllers through the intelligent platform management interface according to preset time intervals and writing the index data into a target file.
3. The method of claim 2, wherein configuring the plurality of hierarchical management nodes comprises:
establishing an intelligent platform management interface communication link between the hierarchical management node and sensors of a plurality of baseboard management controllers through a Free IPMI tool;
configuring the acquisition frequency of a sensor of each baseboard management controller;
establishing a target file and configuring a storage address of the target file;
according to the acquisition frequency, acquiring index data of the sensors of the plurality of substrate management controllers respectively through an out-of-band grabbing function of the Free IPMI tool;
and writing the collected index data into the target file.
4. The method of claim 3, wherein converting the format of the indicator data to obtain structured data comprises:
configuring a listener through the Free IPMI, wherein the listener is used for monitoring the content of the target file;
and when the monitor monitors that the content of the target file is updated, performing format conversion according to the updated index data to obtain structured data.
5. The method of claim 4, wherein when the listener monitors the content of the target file to be updated, performing format conversion according to the updated index data to obtain the structured data, comprises:
when the listener monitors the content update of the target file, the updated index data is sent to the logstack cluster;
and calling a data conversion template through the logstack cluster, and performing format conversion according to the index data to obtain structured data.
6. The method according to claim 5, wherein the index data comprises a plurality of index parameters sorted according to a preset order, and the data conversion template comprises parameter type configuration information of the plurality of index parameters sorted according to the preset order;
correspondingly, the calling a data conversion template through the logstack cluster, and performing format conversion according to the index data to obtain structured data includes:
acquiring a plurality of index parameters contained in index data through the logstack cluster;
configuring the parameter type of a target parameter according to the parameter type configuration information in the data conversion template, wherein the target parameter is any one index parameter;
determining structured data according to the target parameters and the configured parameter types;
an index is generated from the structured data.
7. The method of claim 6, wherein responding to a data processing instruction triggered by a user via a visual interface comprises:
receiving a data processing instruction sent by a client through a visual interface, wherein the data processing instruction is used for inquiring target parameters of structured data according to target time;
determining target structured data according to the target time and the index;
determining a query result according to the target structured data and the target parameter;
and feeding back the query result to the client so that the client can output the query structure through a visual interface.
8. An apparatus for processing sensor data, comprising:
the index data acquisition module is used for acquiring index data of the sensors through the intelligent platform management interface;
the structured data conversion module is used for carrying out format conversion according to the index data to obtain structured data;
and the response module is used for responding to a data processing instruction triggered by a user through a visual interface, and the data processing instruction is used for processing the structured data.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of processing sensor data according to any of claims 1-7 when executing the program.
10. A storage medium containing computer executable instructions for performing the method of processing sensor data according to any one of claims 1-7 when executed by a computer processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110756769.6A CN113505165A (en) | 2021-07-05 | 2021-07-05 | Sensor data processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110756769.6A CN113505165A (en) | 2021-07-05 | 2021-07-05 | Sensor data processing method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113505165A true CN113505165A (en) | 2021-10-15 |
Family
ID=78011706
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110756769.6A Pending CN113505165A (en) | 2021-07-05 | 2021-07-05 | Sensor data processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113505165A (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103200199A (en) * | 2013-04-15 | 2013-07-10 | 北京搜狐新媒体信息技术有限公司 | Out of band (OOB) data collection system |
| US20140195657A1 (en) * | 2013-01-08 | 2014-07-10 | American Megatrends, Inc. | Implementation on baseboard management controller of single out-of-band communication access to multiple managed computer nodes |
| CN104104543A (en) * | 2014-07-17 | 2014-10-15 | 浪潮集团有限公司 | Server managing system and method based on SNMP and IPMI protocol |
| CN105260280A (en) * | 2015-09-23 | 2016-01-20 | 浪潮电子信息产业股份有限公司 | Method and device for detecting sensors in servers |
| CN105354135A (en) * | 2015-12-01 | 2016-02-24 | 浪潮集团有限公司 | Method for incrementally acquiring server BMC log through Logstash |
| CN107491375A (en) * | 2017-08-18 | 2017-12-19 | 国网山东省电力公司信息通信公司 | Equipment detection and fault early warning system and method under a kind of cloud computing environment |
| CN107508869A (en) * | 2017-08-11 | 2017-12-22 | 上海中信信息发展股份有限公司 | Trace back data acquisition method and client |
| CN108092813A (en) * | 2017-12-21 | 2018-05-29 | 郑州云海信息技术有限公司 | Data center's total management system server hardware Governance framework and implementation method |
| CN111984498A (en) * | 2020-07-24 | 2020-11-24 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Server cluster monitoring and management system |
| US10924350B1 (en) * | 2017-06-21 | 2021-02-16 | Amazon Technologies, Inc. | Software sensor for reporting controller metrics |
| CN112532470A (en) * | 2020-11-05 | 2021-03-19 | 苏州浪潮智能科技有限公司 | Method and system for actively reporting log information by BMC |
| CN112527605A (en) * | 2020-12-23 | 2021-03-19 | 中盈优创资讯科技有限公司 | Server management method and device based on IPMI |
-
2021
- 2021-07-05 CN CN202110756769.6A patent/CN113505165A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140195657A1 (en) * | 2013-01-08 | 2014-07-10 | American Megatrends, Inc. | Implementation on baseboard management controller of single out-of-band communication access to multiple managed computer nodes |
| CN103200199A (en) * | 2013-04-15 | 2013-07-10 | 北京搜狐新媒体信息技术有限公司 | Out of band (OOB) data collection system |
| CN104104543A (en) * | 2014-07-17 | 2014-10-15 | 浪潮集团有限公司 | Server managing system and method based on SNMP and IPMI protocol |
| CN105260280A (en) * | 2015-09-23 | 2016-01-20 | 浪潮电子信息产业股份有限公司 | Method and device for detecting sensors in servers |
| CN105354135A (en) * | 2015-12-01 | 2016-02-24 | 浪潮集团有限公司 | Method for incrementally acquiring server BMC log through Logstash |
| US10924350B1 (en) * | 2017-06-21 | 2021-02-16 | Amazon Technologies, Inc. | Software sensor for reporting controller metrics |
| CN107508869A (en) * | 2017-08-11 | 2017-12-22 | 上海中信信息发展股份有限公司 | Trace back data acquisition method and client |
| CN107491375A (en) * | 2017-08-18 | 2017-12-19 | 国网山东省电力公司信息通信公司 | Equipment detection and fault early warning system and method under a kind of cloud computing environment |
| CN108092813A (en) * | 2017-12-21 | 2018-05-29 | 郑州云海信息技术有限公司 | Data center's total management system server hardware Governance framework and implementation method |
| CN111984498A (en) * | 2020-07-24 | 2020-11-24 | 华东计算技术研究所（中国电子科技集团公司第三十二研究所） | Server cluster monitoring and management system |
| CN112532470A (en) * | 2020-11-05 | 2021-03-19 | 苏州浪潮智能科技有限公司 | Method and system for actively reporting log information by BMC |
| CN112527605A (en) * | 2020-12-23 | 2021-03-19 | 中盈优创资讯科技有限公司 | Server management method and device based on IPMI |
Non-Patent Citations (6)
| Title |
|---|
| 思月行云: ""如何使用ELK来监控性能"", pages 1 - 8, Retrieved from the Internet <URL:《https://blog.csdn.net/kenkao/article/details/96866055》> * |
| 李中男 等: ""基于云计算资源池的运维管理平台"", 《信息与电脑(理论版)》, vol. 32, no. 17, 10 September 2020 (2020-09-10), pages 13 - 15 * |
| 李宁: ""数据中心能耗数据采集方法研究与实现"", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 04, pages 137 - 16 * |
| 潘春华: ""基于ELK的高校数据中心运维多层面监控平台研究与实践"", 《中国教育信息化》, no. 07, 5 April 2020 (2020-04-05), pages 93 - 96 * |
| 胡鹤 等: ""面向集群服务器***的监控平台综述"", 《科研信息化技术与应用》, vol. 9, no. 01, pages 79 - 88 * |
| 高建平 等: ""基于Web的服务器群带外集中监控***设计与实现"", 《计算机与数字工程》, vol. 38, no. 08, pages 70 - 73 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110489391B (en) | Data processing method and related equipment | |
| RU2394262C2 (en) | System for configuring devices and method of preventing abnormal situations on production plant | |
| CN109359094B (en) | Distributed system log full-link tracking method and device | |
| CN108039959B (en) | Data situation perception method, system and related device | |
| CN108521339B (en) | Feedback type node fault processing method and system based on cluster log | |
| CN114189430A (en) | Three-dimensional log full-link monitoring system, method, medium and equipment | |
| CN108632111A (en) | Service link monitoring method based on log | |
| CN109710439B (en) | Fault processing method and device | |
| CN111736571A (en) | Fault diagnosis system and method, and storage medium | |
| CN112600719A (en) | Alarm clustering method, device and storage medium | |
| CN113792008A (en) | Method and device for acquiring network topology structure, electronic equipment and storage medium | |
| CN114221997A (en) | Interface monitoring system based on micro-service gateway | |
| CN113505165A (en) | Sensor data processing method and device, computer equipment and storage medium | |
| CN112579552A (en) | Log storage and calling method, device and system | |
| CN109710487A (en) | A kind of monitoring method and device | |
| CN113778831A (en) | Data application performance analysis method, device, equipment and medium | |
| CN114978976A (en) | SRv6 fusion network data anomaly detection method and device | |
| CN110955709B (en) | Data processing method and device and electronic equipment | |
| CN111953519A (en) | SDN network flow visualization method and device | |
| CN108829563B (en) | Alarm method and alarm device | |
| CN113055243B (en) | DPI interface data processing method and device | |
| CN112308328B (en) | Top-Down network measurement system-oriented parallel measurement task optimization method and system | |
| CN110858341A (en) | Index monitoring method, device, equipment and medium based on distributed storage system | |
| CN116401138B (en) | Operating system running state detection method and device, electronic equipment and medium | |
| CN118041761A (en) | Automatic equipment acquisition method, system, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |