CN110737655A - Method and device for reporting data - Google Patents
Method and device for reporting data Download PDFInfo
- Publication number
- CN110737655A CN110737655A CN201910998716.8A CN201910998716A CN110737655A CN 110737655 A CN110737655 A CN 110737655A CN 201910998716 A CN201910998716 A CN 201910998716A CN 110737655 A CN110737655 A CN 110737655A
- Authority
- CN
- China
- Prior art keywords
- data
- data message
- target
- reported
- check
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
Abstract
An specific implementation mode of the method comprises the steps of pulling a data message from a preset data message set as a target data message, wherein the preset data message set is used for storing the data message reported by at least clients, the data message comprises data to be reported and data acquisition time, conducting pre-check on the target data message to generate a pre-check result, conducting data repeatability check on the data to be reported in the target data message according to the data acquisition time in the target data message in response to the fact that the pre-check result indicates that the pre-check is passed, generating a data repeatability check result, and storing the target data message into a preset database in response to the fact that the data repeatability check result indicates that the target data message is not corresponding to the data message in the preset database.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of computers, in particular to a method and a device for reporting data.
Background
In the IaaS system, accuracy and timeliness of each attribute data of computing resources are used as important indexes for evaluating performance of a resource management system, verification mechanisms or update mechanisms are often needed to verify or update whether recording of each attribute of the computing resources in the IaaS system is accurate or not and whether the recording is consistent with an actual situation or not, a reporting mechanism is generated under the background, the reporting is processes of collecting, capturing and analyzing the resource attribute of the computing resources by using agents of the computing resources, and finally informing an IaaS system Service end, the reporting aims to timely verify and update the attribute of the computing resources, and accuracy of the data of the IaaS system is guaranteed.
The method is to send pull command to the managed client end through the server end to trigger the information collection program of the client end agent to execute and then to return the collected information to the server end, the second method is to report the self information to consumption queues (queue) by the client end agent, the server end pulls the information from the queue to consume, and the third method is to report the client end as the main and the server end as the auxiliary.
Disclosure of Invention
The embodiment of the disclosure provides a method and a device for reporting data.
, the embodiment of the disclosure provides a method for reporting data, the method includes pulling a data message from a preset data message set as a target data message, wherein the preset data message set is used for storing data messages reported by at least clients, the data messages include data to be reported and data acquisition time, performing pre-check on the target data message to generate a pre-check result, performing data repeatability check on the data to be reported in the target data message according to the data acquisition time in the target data message in response to determining that the pre-check result indicates that the pre-check is passed, generating a data repeatability check result, and storing the target data message in a preset database in response to determining that the data repeatability check result indicates that the target data message is not the corresponding data message in the preset database.
In , the pre-verifying the target data message to generate the pre-verification result includes matching a format of data to be reported in the target data message with a preset reported data format to generate a format matching result, and generating the pre-verification result according to the format matching result.
In , the data message in the data message set further includes an identifier of a data source end, and the performing pre-check on the target data message to generate a pre-check result includes performing check on the identifier of the data source end in the target data message according to a preset data source end check rule to generate a source end check result, and generating the pre-check result based on the source end check result.
In , the generating the pre-verification result based on the source-side verification result includes, in response to determining that the source-side verification result indicates verification pass, obtaining, from a preset database, a data message that matches the identifier of the source end of the data in the target data message and has a data acquisition time closest to the current time as a matching data message, and in response to determining that the data acquisition time in the matching data message is earlier than the data acquisition time in the target data message, generating the pre-verification result indicating that the pre-verification pass.
In , the method further includes selecting a target client from at least clients, sending information representing the acquired reported data to the target client, and storing the data message reported by the target client in a preset data message set in response to receiving the data message reported by the target client.
In , the selecting the target ue from the at least ues includes selecting a ue with a reporting time interval greater than a preset time interval from the at least ues as the target ue.
In , the method further includes, in response to receiving a data message reported by a client that meets an active reporting condition, storing the data message reported by the client that meets the active reporting condition into a preset data message set, where the active reporting condition includes at least items that a time interval between when a change in local data to be reported by the client is detected and when the time interval between when data are reported by times reaches a preset reporting period.
In , the predetermined data message set includes a predetermined message queue, the predetermined database includes a Cyclic Redundancy Check (CRC) code corresponding to the data message, and the data repetition Check includes a CRC Check.
In a second aspect, an embodiment of the present disclosure provides apparatuses for reporting data, the apparatus includes a pulling unit configured to pull a data message from a preset data message set as a target data message, where the preset data message set is used to store at least data messages reported by clients, and the data message includes data to be reported and data acquisition time, a pre-verification unit configured to perform pre-verification on the target data message to generate a pre-verification result, a repetition verification unit configured to perform data repetition verification on the data to be reported in the target data message according to the data acquisition time in the target data message in response to determining that the pre-verification result indicates that the pre-verification passes, and generate a data repetition verification result, and a storage unit configured to store the target data message in the preset database in response to determining that the data repetition verification result indicates that the target data message does not correspond to the corresponding data message in the preset database in .
In , the pre-check unit includes a generation module configured to match a format of data to be reported in the target data message with a preset reported data format to generate a format matching result, and a second generation module configured to generate a pre-check result according to the format matching result.
In , the data message in the data message set further includes an identifier of a data source end, and the pre-check unit includes a third generation module configured to check the identifier of the data source end in the target data message according to a preset data source end check rule to generate a source end check result, and a fourth generation module configured to generate a pre-check result based on the source end check result.
In , the fourth generation module includes an obtaining sub-module configured to, in response to determining that the source side verification result indicates verification pass, obtain, from a preset database, a data message that matches the identification of the source side of the data in the target data message and has a data acquisition time closest to the current time as a matching data message, and a generation sub-module configured to, in response to determining that the data acquisition time in the matching data message is earlier than the data acquisition time in the target data message, generate a pre-verification result indicating that the pre-verification pass.
In , the apparatus further includes a selecting unit configured to select a target client from at least clients, a sending unit configured to send information characterizing acquisition of reported data to the target client, and a second storing unit configured to store the data message reported by the target client in a preset data message set in response to receiving the data message reported by the target client.
In , the selecting unit is further configured to select a client with a reporting data time interval greater than a preset time interval from at least clients as a target client.
In , the apparatus further includes a third storage unit configured to, in response to receiving a data message reported by a client satisfying an active reporting condition, store the data message reported by the client satisfying the active reporting condition into a preset data message set, where the active reporting condition includes at least items that a time interval between when a change in data to be reported by the client is detected and when the time interval between when data to be reported by the client is times reaches a preset reporting period.
In , the predetermined data message set includes a predetermined message queue, the predetermined database includes CRC codes corresponding to the data messages, and the data repetition check includes CRC check.
In a third aspect, an embodiment of the present disclosure provides a system for reporting data, the system including a client configured to actively send a data message to a server in response to determining that a data active reporting condition is satisfied, where the data message includes data to be reported and a data acquisition time, the server configured to store the actively sent data message in a preset data message set in response to receiving the data message actively sent by the client, and implement the method as described in any implementation manner in aspect .
In , the active data reporting condition includes detecting that the data to be reported locally changes.
In a fourth aspect, embodiments of the disclosure provide servers comprising or more processors, a storage device having or more programs stored thereon, the or more programs, when executed by the or more processors, cause the or more processors to implement a method as described in any implementation of aspect .
In a fifth aspect, embodiments of the present disclosure provide computer readable media having stored thereon a computer program that, when executed by a processor, performs a method as described in any of the implementations of aspect .
The method and the device for reporting data provided by the embodiment of the disclosure firstly pull a data message from a preset data message set as a target data message, wherein the preset data message set is used for storing the data message reported by at least clients, the data message comprises data to be reported and data acquisition time, then, perform pre-check on the target data message to generate a pre-check result, then, in response to the fact that the pre-check result indicates that the pre-check is passed, perform data repeatability check on the data to be reported in the target data message according to the data acquisition time in the target data message to generate a data repeatability check result, and finally, in response to the fact that the data repeatability check result indicates that the target data message does not cause with the corresponding data message in a preset database, store the target data message in the preset database.
Drawings
Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture diagram to which embodiments of the present disclosure may be applied;
fig. 2 is a flow diagram of embodiments of a method for reporting data in accordance with the present disclosure;
fig. 3 is a schematic diagram of application scenarios of a method for reporting data according to an embodiment of the present disclosure;
fig. 4 is a flow diagram of still another embodiments of a method for reporting data in accordance with the present disclosure;
fig. 5 is a schematic structural diagram of embodiments of an apparatus for reporting data according to the present disclosure;
fig. 6 is a timing diagram of interactions among devices in embodiments for reporting data according to the present disclosure.
FIG. 7 is a schematic structural diagram of an electronic device suitable for use in implementing embodiments of the present disclosure.
Detailed Description
The present disclosure will now be described in further detail in with reference to the drawings and examples, it being understood that the specific embodiments herein described are merely illustrative of and not restrictive on the broad invention, and it should be further noted that for purposes of illustration only those portions which are pertinent to the present invention are shown in the drawings.
It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates an example architecture 100 to which the disclosed method for reporting data or apparatus for reporting data may be applied.
As shown in fig. 1, system architecture 100 may include terminal devices 101, 102, 103, network 104, and cloud services 105. The cloud services 105 may be provided by the servers 1051, 1052, 1053. Network 104 is used to provide a medium of communication links between terminal devices 101, 102, 103 and cloud service 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The terminal devices 101, 102, 103 interact with the cloud service 105 through the network 104 to receive or transmit messages and the like. The terminal devices 101, 102, 103 may have various communication client applications installed thereon, such as a web browser application, a shopping application, a search application, an instant messaging tool, a mailbox client, social platform software, a text editing application reading application, and the like.
The terminal apparatuses 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices having a display screen and supporting information transceiving, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers, desktop computers, and the like. When the terminal apparatuses 101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.
The cloud service 105 may provide various services, for example, support for various client applications installed on the terminal devices 101, 102, 103. The server 1051 may serve as a server for resource management in the cloud service 105, and the servers 1052 and 1053 may serve as clients for resource management in the cloud service 105. The server 1051 can check and update the resource attribute information reported by the servers 1052 and 1053 in time, so as to ensure the accuracy of the resource data in the cloud service 105.
The server may be hardware or software. When the server is hardware, it may be implemented as a distributed server cluster formed by multiple servers, or may be implemented as a single server. When the server is software, it may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.
It should be noted that the method for reporting data provided by the embodiment of the present disclosure is generally executed by the server 1051, and accordingly, the apparatus for reporting data is generally disposed in the server 1051.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Continuing to refer to fig. 2, a flow 200 of embodiments of a method for reporting data is shown, the method for reporting data comprising the steps of:
In this embodiment, an execution subject (e.g., the server 1051 shown in fig. 1) of the method for reporting data may pull a data message from a preset data message set as a target data message through a wired connection manner or a wireless connection manner, where the preset data message set may be used to store at least data messages reported by clients.
In optional implementation manners of this embodiment, the preset data message set may include a preset message queue, so that data messages reported by the client may enter the message queue according to a sequence of data collection times.
In optional implementations of this embodiment, the data message in the data message set may further include an identifier of a data source end, where the identifier of the data source end may be used to uniquely identify a client reporting the data message.
In optional implementation manners of this embodiment, in response to receiving a data message reported by a client that meets an active reporting condition, the execution main body may store the data message reported by the client that meets the active reporting condition into the preset data message set, where the active reporting condition may include at least items that a time interval between detection of a change in data to be reported by the client and reporting of the data at times reaches a preset reporting period.
In this embodiment, the execution subject may perform pre-check on the data message pulled in step 201 in various manners to generate a pre-check result, where the pre-check result may be used to indicate whether the pre-check is passed or not, and thus, portions of data messages that do not meet the requirement of reporting data are filtered out first through the pre-check, so that the data amount processed by the server may be reduced, and the resource management efficiency is improved.
In alternative implementations of the present embodiment, the execution subject may generate the pre-verification result according to the following steps:
and , matching the format of the data to be reported in the target data message with a preset reported data format to generate a format matching result.
In these implementations, the execution main body may match the format of the data to be reported in the target data message pulled in step 201 with a preset reporting data format, where the matching with the preset reporting data format may include, but is not limited to, at least items, that is, whether the number of data items is , whether fields corresponding to the data items are missing, and whether values of the data items are abnormal.
And secondly, generating a pre-check result according to the format matching result.
In alternative implementations of this embodiment, based on the identifier of the data source end included in the data message, the execution subject may generate a format matching result according to the following steps:
, according to the preset data source end check rule, checking the identifier of the data source end in the target data message to generate a source end check result.
As yet another example , in response to determining that the identifier of the data source end in the target data message exists in a predefined white list, a source end verification result indicating that the verification is passed is generated.
And secondly, generating a pre-check result based on the source end check result.
In these implementations, the execution body may generate a pre-check result in various ways based on the source-side check result generated in step .
Optionally, based on the source side verification result, the execution subject may further generate a pre-verification result according to the following steps:
and , in response to the fact that the verification result of the source terminal indicates that the verification is passed, acquiring the data message which is matched with the identifier of the source terminal of the data in the target data message and has the data acquisition time closest to the current time from a preset database as a matched data message.
And a second step of generating a pre-check result indicating that the pre-check passes in response to determining that the data acquisition time in the matching data message is earlier than the data acquisition time in the target data message.
In these implementations, the pre-check result is generated by comparing the data acquisition time in the target data message with the data acquisition time in the matching data message, which can ensure that the data is not falsely modified by earlier data.
And 203, in response to the fact that the pre-check result indicates that the pre-check is passed, performing data repeatability check on the data to be reported in the target data message according to the data acquisition time in the target data message to generate a data repeatability check result.
In this embodiment, in response to determining that the pre-check result indicates that the pre-check is passed, the execution main body may perform data repeatability check on data to be reported in the target data message according to data acquisition time in the target data message to generate a data repeatability check result, where the data repeatability check may include various methods for checking data integrity, the data repeatability check result may be used to indicate whether data messages corresponding to the target data message and a preset database are equivalent, the preset database may be used to store history data messages reported by the at least clients, and the preset database is generally persistent.
In this embodiment, the execution main body may perform data repeatability Check on data to be reported in a target data Message and reference data, where the reference data may be data to be reported in a data Message reported times before the data acquisition time in the target data Message, in practice, the data Message corresponding to the target data Message and the reference data is usually sent by the same client, where the data repeatability Check may include, but is not limited to, at least items, i.e., parity Check, BCC (Block Check code) Check, and md5(Message-digest algorithm).
In optional implementations of this embodiment, the data repeatability check may further include a CRC check.
And step 204, in response to determining that the data repeatability check result indicates that the target data message does not correspond to the data message in the preset database, storing the target data message in the preset database.
In this embodiment, in response to determining that the data duplication checking result indicates that the target data message does not correspond to the data message in the predetermined database, the executing entity may store the target data message in the predetermined database.
In optional implementations of this embodiment, the preset database may further include a CRC code corresponding to the data message.
In alternative implementations of this embodiment, the execution main body may further compare the target data message with a theoretical calculation value, and in response to determining that the data to be reported in the target data message does not correspond to the data indicated by the theoretical calculation value , the execution main body may further send information indicating correction data.
Based on the optional implementation mode, the target data message uploaded by the client serves as an objective actual value, and a theoretical value generated by the resource management system can be corrected, so that the accuracy of the data of the whole resource management system is improved.
In optional implementations of the embodiment, the execution subject may further perform various subsequent processes on the data in the preset database.
Continuing to refer to fig. 3, fig. 3 is schematic diagrams of an application scenario of a method for reporting data according to an embodiment of the present disclosure, in the application scenario of fig. 3, a user 301 uses a resource allocated by a cloud service 303 by using a terminal 302, then, the terminal 302 sends a resource usage query request 304 to the cloud service 303, a server 3031 of the cloud service 303 pulls a data message reported by a client 3032 where the resource allocated to the terminal 302 is located from a preset data message set as a target data message 305, wherein the preset data message set includes data messages reported by clients 3032 and 3033, the target data message 305 may include "200G 11:30: 08", then, the server 3031 may perform a pre-check on the target data message 305 to generate a pre-check result 306 indicating that the data message passes the check, then, the server 3031 may generate a data repeatability result 307 according to data messages "180G 11:25: 08" reported by the client 3032 recorded in a database, and then, the server 3031 may optionally store the target data message 305 into the target database 3031.
At present, in the prior art usually reports data to a server at a fixed time through a server or a client, which results in weak data reporting purposiveness and causes repeated data uploading or system congestion, however, the method provided by the above embodiment of the present disclosure reports data messages to a preset data message set actively by the client, and then reports the data messages to the server through the server system , thereby satisfying real-time data update and preventing system congestion caused by reporting of large batches of data.
referring to FIG. 4, there is shown a flow 400 of another embodiments of a method for reporting data, the flow 400 of the method for reporting data comprising the steps of:
in step 401, a target client is selected from at least clients.
In this embodiment, an execution subject of the method for reporting data (e.g., the server 1051 shown in fig. 1) may select a target client from the at least clients by various methods.
In optional implementations of this embodiment, the execution main body may select, as the target client, a client whose reporting time interval is greater than a preset time interval from at least clients.
In this embodiment, in response to receiving the data message reported by the target client, the execution main body may directly store the data message reported by the target client into a preset data message set to receive a subsequent data verification process. The data message reported by the target client is generally generated based on data collected locally from the client in response to receiving the information characterizing the obtained reported data.
And step 405, performing pre-check on the target data message to generate a pre-check result.
And step 406, in response to the fact that the pre-check result indicates that the pre-check is passed, performing data repeatability check on the data to be reported in the target data message according to the data acquisition time in the target data message to generate a data repeatability check result.
As can be seen from fig. 4, a process 400 of the method for reporting data in this embodiment includes a step of sending information representing that reported data is obtained to a target client, and a step of storing a data message reported by the target client into a preset data message set in response to receiving the data message reported by the target client. Therefore, according to the scheme described in this embodiment, the server can actively acquire the reported data message from the target client, in addition to the active reporting mechanism of the client, so that the problem that the client data stored in the server cannot be updated is avoided under the condition that the active reporting mechanism of the client fails, and the stability of the reporting mechanism is ensured.
referring to fig. 5, as an implementation of the methods shown in the above figures, the present disclosure provides embodiments of an apparatus for reporting data, where the embodiments of the apparatus correspond to the embodiments of the method shown in fig. 2, and the apparatus may be applied to various electronic devices.
As shown in fig. 5, the apparatus 500 for reporting data provided in this embodiment includes a pulling unit 501, a pre-checking unit 502, a duplicate checking unit 503, and an th storage unit 504, where the pulling unit 501 is configured to pull a data message from a preset data message set as a target data message, where the preset data message set is used to store data messages reported by at least clients, the data message includes data to be reported and a data acquisition time, the pre-checking unit 502 is configured to perform pre-checking on the target data message and generate a pre-checking result, the duplicate checking unit 503 is configured to perform data duplicate checking on the data to be reported in the target data message according to the data acquisition time in the target data message in response to determining that the pre-checking result indicates that the pre-checking passes, and generate a data duplicate checking result, the th storage unit 504 is configured to store the target data message into the preset database in response to determining that the data duplicate checking result indicates that the target data message does not correspond to a corresponding data message in the preset database.
In this embodiment, specific processes of the pull unit 501, the pre-check unit 502, the duplicate check unit 503, and the th storage unit 504 in the apparatus 500 for reporting data and technical effects thereof may refer to the related descriptions of step 201, step 202, step 203, and step 204 in the corresponding embodiment of fig. 2, and are not described herein again.
In optional implementation manners of the embodiment, the pre-check unit 502 may include a generation module (not shown in the figure) and a second generation module (not shown in the figure), where the generation module may be configured to match a format of data to be reported in the target data message with a preset reported data format, so as to generate a format matching result.
In optional implementations of the embodiment, the pre-check unit 502 may include a third generation module (not shown in the figure) and a fourth generation module (not shown in the figure), where the third generation module may be configured to check the identifier of the data source end in the target data message according to a preset data source end check rule, and generate a source end check result.
In alternative implementations of the present embodiment, the fourth generating module may include an obtaining sub-module (not shown in the figure) and a generating sub-module (not shown in the figure), where the obtaining sub-module may be configured to, in response to determining that the source side verification result indicates a verification pass, obtain, from a preset database, a data message that matches the identification of the source side from the data in the target data message and has a data acquisition time closest to the current time as a matching data message.
In alternative implementations of the embodiment, the apparatus 500 for reporting data may further include a selecting unit (not shown), a sending unit (not shown), and a second storage unit (not shown), where the selecting unit may be configured to select a target client from at least clients, the sending unit may be configured to send information representing that the reported data is obtained to the target client, and the second storage unit may be configured to store the data message reported by the target client into a preset data message set in response to receiving the data message reported by the target client.
In alternative implementations of the embodiment, the selecting unit may be further configured to select, as the target client, a client with a reporting data time interval greater than a preset time interval from at least clients.
In optional implementation manners of this embodiment, the apparatus 500 for reporting data may further include a third storage unit (not shown in the figure), configured to store, in response to receiving a data message reported by a client that meets an active reporting condition, the data message reported by the client that meets the active reporting condition into a preset data message set, where the active reporting condition includes at least that a time interval between when data to be reported locally by the client is detected to change and when data is reported times reaches a preset reporting period.
In optional implementations of this embodiment, the preset data message set may include a preset message queue, the preset database may include a CRC code corresponding to the data message, and the data repeatability check may include a CRC check.
The device provided by the above embodiment of the present disclosure uses the pulling unit 501 to pull the data message from the preset data message set as the target data message, where the preset data message set is used to store the data messages reported by at least clients, the data message includes the data to be reported and the data acquisition time, then, the pre-check unit 502 performs pre-check on the target data message to generate a pre-check result, then, the repetition check unit 503 performs data repetition check on the data to be reported in the target data message according to the data acquisition time in the target data message in response to determining that the pre-check result indicates that the pre-check is passed, and generates a data repetition check result, and finally, the th storage unit 504 stores the target data message into the preset database in response to determining that the data repetition check result indicates that the target data message does not correspond to the data message in the preset database .
referring to FIG. 6, it shows a timing sequence 600 of interactions among devices in embodiments for reporting data, the system for reporting data may include a client (e.g., servers 1052, 1053 shown in FIG. 1), a server (e.g., server 1051 shown in FIG. 1), wherein the client may be configured to send a data message to the server proactively in response to determining that a data proactively reporting condition is satisfied.
In optional implementations of the embodiment, the condition for actively reporting data may include detecting that data to be reported locally changes.
As shown in fig. 6, in step 601, in response to determining that the data active reporting condition is satisfied, the client actively sends a data message to the server.
In this embodiment, in response to determining that a data active reporting condition is satisfied, the client may actively send a data message to the server, where the data message may include data to be reported and data acquisition time, and the data active reporting condition may include that an interval duration between the data active reporting condition and the data reporting time of the last times reaches a preset data reporting time interval.
In optional implementation manners of this embodiment, the active data reporting condition may further include detecting that data to be reported locally changes, where the change of the data to be reported locally may include various situations.
In step 602, in response to receiving the data message actively sent by the client, the server stores the actively sent data message into a preset data message set.
In step 603, the server pulls the data message from the preset data message set as the target data message.
In step 604, the server performs pre-check on the target data message to generate a pre-check result.
In step 605, in response to determining that the pre-check result indicates that the pre-check is passed, the server performs data repeatability check on the data to be reported in the target data message according to the data acquisition time in the target data message, and generates a data repeatability check result.
In step 606, in response to determining that the data repeatability check result indicates that the target data message does not correspond to a data message in the preset database, the server stores the target data message in the preset database.
Step 603, step 604, step 605 and step 606 are respectively the same as step 201, step 202, step 203 and step 204 in the foregoing embodiment and their optional implementation , and the above description for step 201, step 202, step 203 and step 204 and their optional implementation also applies to step 603, step 604, step 605 and step 606, which is not described herein again.
The system for reporting data, provided by the above embodiment of the application, comprises a client, a server and a server, wherein the client actively sends a data message to the server in response to a determination that a data active reporting condition is met, the server stores the actively sent data message into a preset data message set in response to a receipt of the data message actively sent by the client, the server then pulls the data message from the preset data message set as a target data message, the server then performs pre-verification on the target data message to generate a pre-verification result, the server then performs data repeatability verification on data to be reported in the target data message according to data acquisition time in the target data message in response to a determination that the pre-verification result indicates that the target data message does not correspond to the corresponding data message in a preset database, the server stores the target data message into the preset database, and the server actively sends the data message to the server through the client, and the server efficiently stores the verified data in the preset database after the data message is pulled from the preset data message set, thereby saving resources for network resource reporting.
Referring now to FIG. 7, a block diagram of an electronic device (e.g., server 1051 of FIG. 1)700 suitable for use in implementing embodiments of the present disclosure is shown FIG. 7 illustrates merely examples and should not be taken to limit the scope of use or functionality of embodiments of the present disclosure.
As shown in fig. 7, electronic device 700 may include a processing means (e.g., central processing unit, graphics processor, etc.) 701 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from storage 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data necessary for the operation of the electronic apparatus 700 are also stored. The processing device 701, the ROM 702, and the RAM703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
In general, input devices 706 including, for example, a touch screen, touch pad, keyboard, mouse, etc., output devices 707 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, etc., storage devices 708 including, for example, magnetic tape, hard disk, etc., and communication devices 709. communication devices 709 may allow electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data although FIG. 7 illustrates electronic device 700 with various means, it is to be understood that not all of the illustrated means are required to be implemented or provided, more or less means may be implemented instead, each of the blocks illustrated in FIG. 7 may represent means, or a plurality of means as desired.
Embodiments of the present disclosure include, for example, computer program products comprising a computer program embodied on a computer-readable medium, the computer program containing program code for performing the method illustrated by the flow chart in such embodiments, the computer program may be downloaded and installed from a network through the communication means 709, or installed from the storage means 708, or installed from the ROM 702. when executed by the processing means 701, the computer program performs the above-described functions defined in the method of the embodiments of the present disclosure.
In an embodiment of the disclosure, the computer readable storage medium may be any medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device, and may be embodied in any suitable form such as, but not limited to, a computer readable signal medium, a computer readable storage medium, or any combination thereof.
The computer readable medium can be contained in the server or can exist independently without being assembled into the server, and the computer readable medium carries or more programs, when the or more programs are executed by the server, the server pulls a data message from a preset data message set as a target data message, wherein the preset data message set is used for storing the data messages reported by at least clients, the data messages comprise data to be reported and data acquisition time, performs pre-check on the target data message to generate a pre-check result, performs data repeatability check on the data to be reported in the target data message according to the data acquisition time in the target data message in response to the fact that the pre-check result indicates that the pre-check passes, generates a data repeatability check result, and stores the target data message into a preset database in response to the fact that the data repeatability check result indicates that the target data message and the corresponding data message in the preset database are not .
Computer program code for carrying out operations for embodiments of the present disclosure may be written in or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, or a combination thereof, as well as conventional procedural programming languages, such as the "C" programming language or similar programming languages.
It should also be noted that in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures, for example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved, and it may also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, may be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The described units can also be arranged in a processor, for example, processors including a pull unit, a pre-check unit, a duplicate check unit, a storage unit, wherein the names of the units do not form a limitation to the units themselves, for example, the pull unit can also be described as a unit for pulling a data message from a preset data message set as a target data message.
The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.
Claims (12)
1, a method for reporting data, comprising:
the method comprises the steps that data messages are pulled from a preset data message set to serve as target data messages, wherein the preset data message set is used for storing the data messages reported by at least clients, and the data messages comprise data to be reported and data acquisition time;
performing pre-check on the target data message to generate a pre-check result;
in response to the fact that the pre-check result indicates that the pre-check is passed, performing data repeatability check on the data to be reported in the target data message according to the data acquisition time in the target data message to generate a data repeatability check result;
and in response to determining that the data repeatability check result indicates that the target data message does not correspond to a data message in a preset database, storing the target data message in the preset database.
2. The method of claim 1, wherein the pre-verifying the target data message, generating a pre-verification result, comprises:
matching the format of the data to be reported in the target data message with a preset reported data format to generate a format matching result;
and generating the pre-check result according to the format matching result.
3. The method of claim 1, wherein the data messages in the set of data messages further include an identification of a source of the data; and
the pre-verifying the target data message to generate a pre-verification result includes:
verifying the identifier of the data source end in the target data message according to a preset data source end verification rule to generate a source end verification result;
and generating the pre-check result based on the source side check result.
4. The method of claim 3, wherein said generating the pre-verification result based on the source-side verification result comprises:
in response to determining that the source end verification result indicates that verification is passed, acquiring a data message which is matched with the identification of the source end of the data in the target data message and has the closest data acquisition time to the current time from the preset database as a matched data message;
generating a pre-check result indicating that the pre-check passes in response to determining that the data acquisition time in the matching data message is earlier than the data acquisition time in the target data message.
5. The method of claim 1, wherein the method further comprises:
selecting a target client from the at least clients;
sending information representing the acquired reported data to the target client;
and responding to the received data message reported by the target client, and storing the data message reported by the target client into the preset data message set.
6. The method of claim 5, wherein the selecting a target client from the at least clients comprises:
and selecting the client with the reporting data time interval larger than a preset time interval from the at least clients as the target client.
7. The method of claim 1, wherein the method further comprises:
and in response to receiving the data message reported by the client meeting the active reporting condition, storing the data message reported by the client meeting the active reporting condition into the preset data message set, wherein the active reporting condition comprises at least items that the time interval from times of data reporting reaches a preset reporting period after the data to be reported locally by the client is detected to change.
8. the method of any of claims 1-7, wherein the predetermined set of data messages includes a predetermined message queue, the predetermined database includes Cyclic Redundancy Check (CRC) codes corresponding to the data messages, and the data repetition check includes a CRC check.
An apparatus for reporting data in , comprising:
the system comprises a pulling unit, a processing unit and a processing unit, wherein the pulling unit is configured to pull a data message from a preset data message set as a target data message, the preset data message set is used for storing data messages reported by at least clients, and the data messages comprise data to be reported and data acquisition time;
the pre-check unit is configured to perform pre-check on the target data message and generate a pre-check result;
the repeated checking unit is configured to respond to the fact that the pre-checking result indicates that the pre-checking is passed, and perform data repeated checking on the data to be reported in the target data message according to the data acquisition time in the target data message to generate a data repeated checking result;
an storage unit configured to store the target data message in a preset database in response to determining that the data duplication check result indicates that the target data message does not correspond to a data message in the preset database.
10, a system for reporting data, comprising:
the client is configured to actively send a data message to the server in response to the fact that the data active reporting condition is met, wherein the data message comprises data to be reported and data acquisition time;
the server side is configured to respond to the received data message actively sent by the client side, store the actively sent data message into a preset data message set, and implement the method as claimed in any of claims 1-8.
11, A server, comprising:
or more processors;
a storage device having or more programs stored thereon;
when the or more programs are executed by the or more processors, cause the or more processors to implement the method of any of claims 1-8.
12, computer readable medium, having stored thereon a computer program, wherein the program, when executed by a processor, implements the method of any of claims 1-8, wherein is defined.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910998716.8A CN110737655B (en) | 2019-10-21 | 2019-10-21 | Method and device for reporting data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910998716.8A CN110737655B (en) | 2019-10-21 | 2019-10-21 | Method and device for reporting data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110737655A true CN110737655A (en) | 2020-01-31 |
| CN110737655B CN110737655B (en) | 2021-07-30 |
Family
ID=69270204
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910998716.8A Active CN110737655B (en) | 2019-10-21 | 2019-10-21 | Method and device for reporting data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110737655B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113204574A (en) * | 2021-05-27 | 2021-08-03 | 北京深演智能科技股份有限公司 | Data processing method, data processing device, computer readable storage medium and processor |
| WO2021164171A1 (en) * | 2020-02-17 | 2021-08-26 | 平安科技（深圳）有限公司 | Method and apparatus for processing data in knowledge base, and computer device and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150331875A1 (en) * | 2014-05-16 | 2015-11-19 | Syntel, Inc. | System and method for validating integrated data recasting objects |
| CN105610836A (en) * | 2015-12-31 | 2016-05-25 | 浙江省公众信息产业有限公司 | Data transmission method and system |
| US20160306864A1 (en) * | 2015-04-14 | 2016-10-20 | Don Estes & Associates, Inc. | Method and system for data synchronization |
| CN107122355A (en) * | 2016-02-24 | 2017-09-01 | 阿里巴巴集团控股有限公司 | Data mover system and method |
| CN107809434A (en) * | 2017-11-08 | 2018-03-16 | 济南大陆机电股份有限公司 | A kind of continuous data validation verification system and method |
| CN107943808A (en) * | 2016-10-13 | 2018-04-20 | 北京京东尚科信息技术有限公司 | The method and apparatus of processing equipment reported data |
| CN108270875A (en) * | 2018-02-24 | 2018-07-10 | 量化医学研究院（深圳）有限公司 | A kind of data uploading method and front end processor |
| CN108593012A (en) * | 2018-07-19 | 2018-09-28 | 中国地质调查局水文地质环境地质调查中心 | The fast slowdown monitoring system of geological disaster |
| CN109214855A (en) * | 2018-07-11 | 2019-01-15 | 腾讯科技（北京）有限公司 | Material method of reseptance, apparatus and system |
| CN109714415A (en) * | 2018-12-26 | 2019-05-03 | 北京小米移动软件有限公司 | Data processing method and device |
-
2019
- 2019-10-21 CN CN201910998716.8A patent/CN110737655B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150331875A1 (en) * | 2014-05-16 | 2015-11-19 | Syntel, Inc. | System and method for validating integrated data recasting objects |
| US20160306864A1 (en) * | 2015-04-14 | 2016-10-20 | Don Estes & Associates, Inc. | Method and system for data synchronization |
| CN105610836A (en) * | 2015-12-31 | 2016-05-25 | 浙江省公众信息产业有限公司 | Data transmission method and system |
| CN107122355A (en) * | 2016-02-24 | 2017-09-01 | 阿里巴巴集团控股有限公司 | Data mover system and method |
| CN107943808A (en) * | 2016-10-13 | 2018-04-20 | 北京京东尚科信息技术有限公司 | The method and apparatus of processing equipment reported data |
| CN107809434A (en) * | 2017-11-08 | 2018-03-16 | 济南大陆机电股份有限公司 | A kind of continuous data validation verification system and method |
| CN108270875A (en) * | 2018-02-24 | 2018-07-10 | 量化医学研究院（深圳）有限公司 | A kind of data uploading method and front end processor |
| CN109214855A (en) * | 2018-07-11 | 2019-01-15 | 腾讯科技（北京）有限公司 | Material method of reseptance, apparatus and system |
| CN108593012A (en) * | 2018-07-19 | 2018-09-28 | 中国地质调查局水文地质环境地质调查中心 | The fast slowdown monitoring system of geological disaster |
| CN109714415A (en) * | 2018-12-26 | 2019-05-03 | 北京小米移动软件有限公司 | Data processing method and device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021164171A1 (en) * | 2020-02-17 | 2021-08-26 | 平安科技（深圳）有限公司 | Method and apparatus for processing data in knowledge base, and computer device and storage medium |
| CN113204574A (en) * | 2021-05-27 | 2021-08-03 | 北京深演智能科技股份有限公司 | Data processing method, data processing device, computer readable storage medium and processor |
| CN113204574B (en) * | 2021-05-27 | 2024-01-26 | 北京深演智能科技股份有限公司 | Data processing method, device, computer readable storage medium and processor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110737655B (en) | 2021-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109241141B (en) | Deep learning training data processing method and device | |
| CN106815254B (en) | Data processing method and device | |
| CN112765152B (en) | Method and apparatus for merging data tables | |
| CN110737655A (en) | Method and device for reporting data | |
| CN109299124B (en) | Method and apparatus for updating a model | |
| CN110347973B (en) | Method and device for generating information | |
| CN112235262A (en) | Message analysis method and device, electronic equipment and computer readable storage medium | |
| CN112215473A (en) | Distribution pressure data obtaining method and device and electronic equipment | |
| CN107291923B (en) | Information processing method and device | |
| CN113111078B (en) | Resource data processing method and device, computer equipment and storage medium | |
| US11714699B2 (en) | In-app failure intelligent data collection and analysis | |
| CN115187364A (en) | Method and device for monitoring deposit risk under bank distributed scene | |
| CN104125100A (en) | Method for monitoring dashboard in real time in communication network management | |
| CN110888811B (en) | Code coverage rate information processing method and device, electronic equipment and medium | |
| US11294704B2 (en) | Monitoring and reporting performance of online services using a monitoring service native to the online service | |
| CN110266610B (en) | Traffic identification method and device and electronic equipment | |
| CN113704486A (en) | Map data construction method and device and map data query method and device | |
| CN110309121B (en) | Log processing method and device, computer readable medium and electronic equipment | |
| CN113656239A (en) | Monitoring method and device for middleware and computer program product | |
| CN112084114A (en) | Method and apparatus for testing an interface | |
| CN110633182A (en) | System, method and apparatus for monitoring server stability | |
| CN112383448B (en) | Monitoring data processing method and device | |
| CN108037981B (en) | Accessory management method and device | |
| CN114356689A (en) | Workflow monitoring method, device and equipment | |
| CN114663073A (en) | Abnormal node discovery method and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Patentee after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Patentee before: Jingdong Digital Technology Holding Co.,Ltd. Address after: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Patentee after: Jingdong Digital Technology Holding Co.,Ltd. Address before: Room 221, 2nd floor, Block C, 18 Kechuang 11th Street, Daxing Economic and Technological Development Zone, Beijing, 100176 Patentee before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder |