CN113312367A - Yield efficiency recalculation method, apparatus, storage medium and processor - Google Patents
Yield efficiency recalculation method, apparatus, storage medium and processor Download PDFInfo
- Publication number
- CN113312367A CN113312367A CN202110712158.1A CN202110712158A CN113312367A CN 113312367 A CN113312367 A CN 113312367A CN 202110712158 A CN202110712158 A CN 202110712158A CN 113312367 A CN113312367 A CN 113312367A
- Authority
- CN
- China
- Prior art keywords
- intermediate value
- timestamp
- message
- recalculation
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000004364 calculation method Methods 0.000 claims abstract description 71
- 238000010586 diagram Methods 0.000 description 13
- 230000008569 process Effects 0.000 description 12
- 238000004590 computer program Methods 0.000 description 11
- 238000012545 processing Methods 0.000 description 8
- 230000008859 change Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000004753 textile Substances 0.000 description 1
- 238000009941 weaving Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a yield efficiency recalculation method, a yield efficiency recalculation device, a processor and a storage medium. The method comprises the following steps: when a recalculation request is acquired, acquiring a history message; assembling the history messages into corresponding intermediate value structures; sending the intermediate value corresponding to the intermediate value structure to a recalculation message queue of a stream type calculation module; updating the intermediate value through a streaming calculation module, and starting real-time calculation; and determining the corresponding real-time yield efficiency according to the updated intermediate value and the real-time calculation result. When the yield efficiency is calculated in the mode, the yield efficiency of any one piece of Internet of things equipment can be recalculated in real time, and the calculation is more convenient and simpler.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a yield efficiency recalculation method, a yield efficiency recalculation device, a storage medium and a processor.
Background
Most of the conventional textile system manufacturers are locally deployed systems, monitoring data are calculated by locally and instantly inquiring machine data, so that a plurality of risks exist, once a central computer is damaged or fails, the data are seriously lost, and the consumption of data retrieval resources is high; it is expensive to want to cloud office.
In the prior art, the shuttle adopts the technical scheme of the internet of things + sass, a single device reports acquired data to the cloud server in real time, the cloud server realizes real-time calculation of various indexes of a weaving factory by mass internet of things data based on a streaming computing platform, the data safety is improved, the data real-time performance is guaranteed, and the cloud server and the cloud factory can be realized at lower cost. Mass internet of things data for a cloud end needs to be calculated in time after the cloud end is on the device, timeliness and modification revocable of business are required to be guaranteed, and the requirement is fatal impact on a flow type calculation framework. The loss calculation has the advantage that the inflow data is calculated and output in real time, and high throughput and timeliness are guaranteed. And the modification cancellation of the service data cannot realize the callback of the data in a certain interval of a certain device.
Disclosure of Invention
An object of the embodiments of the present invention is to provide a yield-efficient recalculation method, apparatus, storage medium, and processor.
In order to achieve the above object, a first aspect of the present invention provides a yield-efficient recalculation method, comprising:
when a recalculation request is acquired, acquiring a history message;
assembling the history messages into corresponding intermediate value structures;
sending the intermediate value corresponding to the intermediate value structure to a recalculation message queue of a stream type calculation module;
updating the intermediate value through the streaming calculation module, and starting real-time calculation;
and determining the corresponding real-time yield efficiency according to the updated intermediate value and the real-time calculation result.
Optionally, the obtaining the history packet includes: and acquiring the historical message through a preset message query structure, wherein the preset message query structure is determined based on the characteristic of the hbase.
Optionally, the intermediate value structure includes at least one of a service core field shift, an axis variety, each collector, and packet batch head-to-tail data.
Optionally, updating, by the streaming module, the intermediate value comprises:
obtaining the existing intermediate value and the corresponding timestamp;
comparing a first timestamp of the existing intermediate value with a second timestamp of the intermediate value corresponding to the intermediate value structure;
when the first timestamp is earlier than the second timestamp, clearing an intermediate value corresponding to the second timestamp, and recalculating to determine a latest intermediate value;
and when the second timestamp is earlier than the first timestamp, determining that the intermediate value corresponding to the intermediate value structure is the latest intermediate value and belongs to the valid message.
Optionally, the method further comprises: acquiring a new message; and updating the intermediate value according to the new message.
A second aspect of the present invention provides a yield-efficient recalculating device comprising:
the message acquisition module is used for acquiring a historical message when a recalculation request is acquired;
the intermediate value updating module is used for assembling the historical messages into corresponding intermediate value structures; sending the intermediate value corresponding to the intermediate value structure to a recalculation message queue of a stream type calculation module; updating the intermediate value through the streaming calculation module, and starting real-time calculation;
and the efficiency calculation module is used for determining the corresponding real-time yield efficiency according to the updated intermediate value and the real-time calculation result.
A third aspect of the invention provides a machine-readable storage medium having stored thereon instructions which, when executed by a processor, cause the processor to be configured to perform the above-described yield-efficient recalculation method.
A fourth aspect of the invention provides a processor configured to perform the above yield-efficient recalculation method.
According to the yield efficiency recalculation method, when a recalculation request is obtained, the historical messages are assembled into the corresponding intermediate value structures, the intermediate values corresponding to the intermediate value structures are sent to the recalculation message queue of the stream type calculation module, the intermediate values are updated through the stream type calculation module, real-time calculation is started, the corresponding real-time yield efficiency is determined according to the updated intermediate values and the result of the real-time calculation, when the yield efficiency is calculated in the mode, the yield efficiency of any Internet of things equipment can be recalculated in real time, and the calculation is more convenient and simpler.
Additional features and advantages of embodiments of the invention will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the embodiments of the invention without limiting the embodiments of the invention. In the drawings:
FIG. 1 schematically illustrates a flow diagram of a yield-efficient recalculation method according to an embodiment of the present invention;
FIG. 2 is a block diagram schematically illustrating a structure of a yield-efficient recalculating apparatus according to an embodiment of the present invention;
fig. 3 schematically shows an internal structure diagram of a computer apparatus according to an embodiment of the present invention.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating embodiments of the invention, are given by way of illustration and explanation only, not limitation.
FIG. 1 schematically shows a flow diagram of a yield-efficient recalculation method according to an embodiment of the present invention. As shown in FIG. 1, in one embodiment of the present invention, a yield-efficient recalculation method is provided, comprising the steps of:
And 102, assembling the history messages into corresponding intermediate value structures.
And 103, sending the intermediate value corresponding to the intermediate value structure to a recalculation message queue of the streaming calculation module.
And step 104, updating the intermediate value through the streaming calculation module, and starting real-time calculation.
And 105, determining the corresponding real-time yield efficiency according to the updated intermediate value and the real-time calculation result.
In a conventional data processing flow, data is always collected and then placed in a database. When people need, the data is inquired through the database to obtain answers or perform related processing. Although this seems reasonable, the results are very compact, especially in some real-time search application environments, where offline processing like MapReduce does not solve the problem well. The flow computing mode can well analyze the large-scale flow data in real time in the constantly changing motion process, capture possibly useful information and send the result to the next computing node.
In this embodiment, a flow calculation intermediate value is specifically designed for the characteristic that real-time dimension information of the organizational business demand can be modified in a revocable manner. The computation that can be done in one step is split into 3 steps. And reserving an intermediate cache layer, and performing increment cache on the intermediate value in the calculation process. When a loom corresponding to certain Internet of things equipment needs to perform re-broadcasting calculation, a re-calculation request can be initiated, and then after the processor obtains the re-calculation request, the history message can be obtained from the database and assembled into a corresponding intermediate value structure. Specifically, in one embodiment, the intermediate value structure includes at least one of a business core field shift, an axis variety, each collector, and packet batch end data.
In one embodiment, obtaining the history message comprises: acquiring a historical message through a preset message query structure, wherein the preset message query structure is determined based on the characteristic of hbase; HBase is a distributed, column-oriented open-ended database.
Aiming at recalculation characteristics, a storage data structure convenient for quick retrieval can be designed, and a query interface service for quick retrieval is realized. The following data structure can be designed based on the characteristics of hbase, and data required by the second-counting retrieval in billions of data can be achieved by utilizing rowkey + column. Specifically, the data structure of the efficient query is as follows:
rowkey: hash (20) _ machine _ date
Column: time of data
Columnvalue: data points
The intermediate values corresponding to the intermediate value structure may then be sent to a recalculation message queue of the streaming computation module. For example, when the service system modifies the parameters of the layout, the cloth type, etc. of a certain loom, the data needs to be replayed, that is, the production efficiency of the loom on duty needs to be recalculated. After receiving the recalculation request, the processor can request message query service, uses the detail messages to collect and assemble the intermediate value required by the flow type calculation service, and pushes the intermediate value to a recalculation message queue special for real-time calculation. The intermediate value is pushed to a real-time computing service, and the real-time computing service updates the intermediate value in the memory of the real-time computing service based on the structure so as to change the intermediate value of the internal computation of the real-time computation and change the real-time computation result to be correct.
In one embodiment, updating the intermediate value by the streaming module comprises: obtaining the existing intermediate value and the corresponding timestamp; comparing the first time stamp of the existing intermediate value with the second time stamp of the intermediate value corresponding to the intermediate value structure; when the first time stamp is earlier than the second time stamp, clearing the intermediate value corresponding to the second time stamp, and recalculating to determine the latest intermediate value; and when the second timestamp is earlier than the first timestamp, determining that the intermediate value corresponding to the intermediate value structure is the latest intermediate value and belongs to the valid message.
When the product efficiency is calculated in real time through the streaming calculation, the existing intermediate value and the corresponding timestamp can be obtained first, that is, the calculated and cached intermediate value and the timestamp corresponding to the intermediate value are obtained. And comparing the first time stamp of the existing intermediate value with the second time stamp of the intermediate value corresponding to the intermediate value structure, namely, the comparison method determines which data is the latest data according to the time stamps. If the first time stamp is earlier than the second time stamp, the existing intermediate value is closer to the current time than the data of the intermediate value corresponding to the intermediate value structure, that is, the data of the existing intermediate value is more novel. The intermediate value corresponding to the second timestamp may be cleared, that is, the intermediate value corresponding to the intermediate value structure is cleared, and the intermediate value is recalculated to determine the intermediate value corresponding to the current latest data, so as to ensure the accuracy of the finally calculated product efficiency. Similarly, if the second timestamp is earlier than the first timestamp, it indicates that the intermediate value corresponding to the intermediate value structure is closer to the current time than the existing data of the intermediate value, i.e., the data of the intermediate value corresponding to the intermediate value structure is more novel. Then the intermediate value corresponding to the intermediate value structure may be determined to be the latest intermediate value and belong to the valid message.
In one embodiment, the method further comprises: acquiring a new message; and updating the intermediate value according to the new message.
In the operation process of the loom, new message data can be continuously generated, and when the new message data is subsequently acquired, the corresponding intermediate value can be calculated according to the new message data, and then the timestamp of the newly calculated intermediate value and the timestamp of the cached intermediate value are compared. The data such as batches and preset key values can be compared, the timestamp is displayed to be a middle value closer to the current time, and the middle value is determined to be an effective numerical value so as to update the middle value, and therefore the real-time yield efficiency recalculation process of real-time calculation and random machine station replay can be realized at a low cost.
For example, the user delays the business scenario such as the axis, and the real-time calculation during the shift is all wrong. At the moment, the business system calls recalculation service, the recalculation service starts recalculation logic, high-speed query service is called, and a middle value structure of the machine within a shift is obtained. Specifically, in one embodiment, the code for the intermediate value structure may be as follows:
then, the recalculation service can perform type identification according to the query result, and if the efficiency is recalculated in real time at the moment, the following recalculation messages are assembled and pushed to the real-time calculation service. For example, the message structure may be as follows:
at this time, the streaming computing service may receive the message, update the intermediate value, and trigger a real-time computation to ensure that the downstream receives a correct computation result of yield efficiency.
According to the yield efficiency recalculation method, when a recalculation request is obtained, the historical messages are assembled into the corresponding intermediate value structures, the intermediate values corresponding to the intermediate value structures are sent to the recalculation message queue of the stream type calculation module, the intermediate values are updated through the stream type calculation module, real-time calculation is started, the corresponding real-time yield efficiency is determined according to the updated intermediate values and the result of the real-time calculation, when the yield efficiency is calculated in the mode, the yield efficiency of any Internet of things equipment can be recalculated in real time, and the calculation is more convenient and simpler.
The recalculation algorithm mode is obtained by referring to real-time calculation and analysis of the loom, when the yield efficiency is calculated in real time, the head and tail information of each batch is mainly recorded, the yield efficiency of each batch is calculated according to the head and tail information, and then the yield efficiency calculation is obtained by accumulation, the algorithm can see that in the actual calculation process, the change process of the state complexity in the operation process of the loom is not concerned, only the last result grouped by batches is concerned, therefore, when the recalculation is carried out, only the head and tail information in the loom batch is required to be obtained, and then the intermediate value of the flow type calculation module is modified according to the head and tail information of the batch for calculation, so that the recalculation purpose is achieved, the algorithm skips the complex state change in the operation process of the loom, and does not need to inquire the detailed message of the complex state of the loom, the inquiry quantity of data is greatly reduced, the complexity of program design is reduced, and the recalculation program efficiency is improved, the simplest calculation is achieved.
In one embodiment, as shown in FIG. 2, there is provided a yield-efficient recalculation device, comprising:
a message obtaining module 201, configured to obtain a history message when a recalculation request is obtained;
an intermediate value updating module 202, configured to assemble the history packets into corresponding intermediate value structures; sending the intermediate value corresponding to the intermediate value structure to a recalculation message queue of a stream type calculation module; updating the intermediate value through a streaming calculation module, and starting real-time calculation;
and the efficiency calculation module 203 is used for determining the corresponding real-time yield efficiency according to the updated intermediate value and the real-time calculation result.
In an embodiment, the message obtaining module 201 is further configured to obtain the historical message through a preset message query structure, where the preset message query structure is determined based on the hbase characteristic.
In one embodiment, the intermediate value structure includes at least one of a business core field shift, a shaft variety, each collector, and packet batch end data.
In one embodiment, the intermediate value updating module is further configured to obtain an existing intermediate value and a corresponding timestamp; comparing the first time stamp of the existing intermediate value with the second time stamp of the intermediate value corresponding to the intermediate value structure; when the first timestamp is earlier than the second timestamp, clearing the second timestamp and recalculating to determine a latest intermediate value; and when the second timestamp is earlier than the first timestamp, determining that the intermediate value corresponding to the intermediate value structure is the latest intermediate value and belongs to the valid message.
In one embodiment, the intermediate value updating module is further configured to obtain a new packet; and updating the intermediate value according to the new message.
The yield efficiency recalculation device comprises a processor and a memory, wherein the message acquisition module, the intermediate value updating module, the efficiency calculation module and the like are stored in the memory as program units, and the processor executes the program modules stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the yield efficiency recalculation method is realized by adjusting kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
Embodiments of the present invention provide a storage medium having a program stored thereon, which when executed by a processor, implements the above yield efficient recalculation method.
The embodiment of the invention provides a processor, which is used for running a program, wherein the method for recalculating the yield efficiency is executed when the program runs.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 3. The computer device includes a processor a01, a network interface a02, a memory (not shown), and a database (not shown) connected by a system bus. Wherein processor a01 of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises an internal memory a03 and a non-volatile storage medium a 04. The non-volatile storage medium a04 stores an operating system B01, a computer program B02, and a database (not shown in the figure). The internal memory a03 provides an environment for the operation of the operating system B01 and the computer program B02 in the nonvolatile storage medium a 04. The database of the computer device is used for storing data such as historical messages. The network interface a02 of the computer device is used for communication with an external terminal through a network connection. The computer program B02 is executed by the processor a01 to implement a yield-efficient recalculation method.
Those skilled in the art will appreciate that the architecture shown in fig. 3 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the processor executes the program and realizes the following steps: when a recalculation request is acquired, acquiring a history message; assembling the history messages into corresponding intermediate value structures; sending the intermediate value corresponding to the intermediate value structure to a recalculation message queue of a stream type calculation module; updating the intermediate value through a streaming calculation module, and starting real-time calculation; and determining the corresponding real-time yield efficiency according to the updated intermediate value and the real-time calculation result.
In one embodiment, obtaining the history message comprises: and acquiring the historical message through a preset message query structure, wherein the preset message query structure is determined based on the characteristic of the hbase.
In one embodiment, the intermediate value structure includes at least one of a business core field shift, a shaft variety, each collector, and packet batch end data.
In one embodiment, updating the intermediate value by the streaming module comprises: obtaining the existing intermediate value and the corresponding timestamp; comparing the first time stamp of the existing intermediate value with the second time stamp of the intermediate value corresponding to the intermediate value structure; when the first time stamp is earlier than the second time stamp, clearing the intermediate value corresponding to the second time stamp, and recalculating to determine the latest intermediate value; and when the second timestamp is earlier than the first timestamp, determining that the intermediate value corresponding to the intermediate value structure is the latest intermediate value and belongs to the valid message.
In one embodiment, the method further comprises: acquiring a new message; and updating the intermediate value according to the new message.
The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device: when a recalculation request is acquired, acquiring a history message; assembling the history messages into corresponding intermediate value structures; sending the intermediate value corresponding to the intermediate value structure to a recalculation message queue of a stream type calculation module; updating the intermediate value through a streaming calculation module, and starting real-time calculation; and determining the corresponding real-time yield efficiency according to the updated intermediate value and the real-time calculation result.
In one embodiment, obtaining the history message comprises: and acquiring the historical message through a preset message query structure, wherein the preset message query structure is determined based on the characteristic of the hbase.
In one embodiment, the intermediate value structure includes at least one of a business core field shift, a shaft variety, each collector, and packet batch end data.
In one embodiment, updating the intermediate value by the streaming module comprises: obtaining the existing intermediate value and the corresponding timestamp; comparing the first time stamp of the existing intermediate value with the second time stamp of the intermediate value corresponding to the intermediate value structure; when the first time stamp is earlier than the second time stamp, clearing the intermediate value corresponding to the second time stamp, and recalculating to determine the latest intermediate value; and when the second timestamp is earlier than the first timestamp, determining that the intermediate value corresponding to the intermediate value structure is the latest intermediate value and belongs to the valid message.
In one embodiment, the method further comprises: acquiring a new message; and updating the intermediate value according to the new message.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave. It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element. The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. A method for recalculating yield efficiency, the method comprising:
when a recalculation request is acquired, acquiring a history message;
assembling the history messages into corresponding intermediate value structures;
sending the intermediate value corresponding to the intermediate value structure to a recalculation message queue of a stream type calculation module;
updating the intermediate value through the streaming calculation module, and starting real-time calculation;
and determining the corresponding real-time yield efficiency according to the updated intermediate value and the real-time calculation result.
2. The method of claim 1, wherein the obtaining the history packet comprises:
and acquiring the historical message through a preset message query structure, wherein the preset message query structure is determined based on the characteristic of the hbase.
3. The method of claim 1, wherein the intermediate value structure comprises at least one of a business core field shift, an axis variety, each collector, and packet batch end data.
4. The method of claim 1, wherein said updating, by said streaming module, said intermediate value comprises:
obtaining the existing intermediate value and the corresponding timestamp;
comparing a first timestamp of the existing intermediate value with a second timestamp of the intermediate value corresponding to the intermediate value structure;
when the first timestamp is earlier than the second timestamp, clearing an intermediate value corresponding to the second timestamp, and recalculating to determine a latest intermediate value;
and when the second timestamp is earlier than the first timestamp, determining that the intermediate value corresponding to the intermediate value structure is the latest intermediate value and belongs to the valid message.
5. The method of claim 1, further comprising:
acquiring a new message;
and updating the intermediate value according to the new message.
6. An apparatus for yield-efficient recalculation, the apparatus comprising:
the message acquisition module is used for acquiring a historical message when a recalculation request is acquired;
the intermediate value updating module is used for assembling the historical messages into corresponding intermediate value structures; sending the intermediate value corresponding to the intermediate value structure to a recalculation message queue of a stream type calculation module; updating the intermediate value through the streaming calculation module, and starting real-time calculation;
and the efficiency calculation module is used for determining the corresponding real-time yield efficiency according to the updated intermediate value and the real-time calculation result.
7. The apparatus of claim 6, wherein the intermediate value updating module is further configured to obtain an existing intermediate value and a corresponding timestamp; comparing a first timestamp of the existing intermediate value with a second timestamp of the intermediate value corresponding to the intermediate value structure; when the first timestamp is earlier than the second timestamp, clearing the second timestamp and recalculating to determine a latest intermediate value; and when the second timestamp is earlier than the first timestamp, determining that the intermediate value corresponding to the intermediate value structure is the latest intermediate value and belongs to the valid message.
8. The apparatus of claim 6, wherein the median update module is further configured to obtain a new packet; and updating the intermediate value according to the new message.
9. A machine-readable storage medium having instructions stored thereon, which when executed by a processor, cause the processor to be configured to perform the method of recalculating yield efficiency according to any of claims 1-5.
10. A processor configured to perform the yield-efficient recalculation method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110712158.1A CN113312367A (en) | 2021-06-25 | 2021-06-25 | Yield efficiency recalculation method, apparatus, storage medium and processor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110712158.1A CN113312367A (en) | 2021-06-25 | 2021-06-25 | Yield efficiency recalculation method, apparatus, storage medium and processor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113312367A true CN113312367A (en) | 2021-08-27 |
Family
ID=77380403
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110712158.1A Pending CN113312367A (en) | 2021-06-25 | 2021-06-25 | Yield efficiency recalculation method, apparatus, storage medium and processor |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113312367A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110225129A1 (en) * | 2010-03-15 | 2011-09-15 | Symantec Corporation | Method and system to scan data from a system that supports deduplication |
| US20120310934A1 (en) * | 2011-06-03 | 2012-12-06 | Thomas Peh | Historic View on Column Tables Using a History Table |
| WO2019144761A1 (en) * | 2018-01-24 | 2019-08-01 | ***股份有限公司 | Data synchronization method, distributed system and device |
| CN111046077A (en) * | 2019-10-29 | 2020-04-21 | 口碑(上海)信息技术有限公司 | Data acquisition method and device, storage medium and terminal |
| CN111077870A (en) * | 2020-01-06 | 2020-04-28 | 浙江中烟工业有限责任公司 | Intelligent OPC data real-time acquisition and monitoring system and method based on stream calculation |
-
2021
- 2021-06-25 CN CN202110712158.1A patent/CN113312367A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110225129A1 (en) * | 2010-03-15 | 2011-09-15 | Symantec Corporation | Method and system to scan data from a system that supports deduplication |
| US20120310934A1 (en) * | 2011-06-03 | 2012-12-06 | Thomas Peh | Historic View on Column Tables Using a History Table |
| WO2019144761A1 (en) * | 2018-01-24 | 2019-08-01 | ***股份有限公司 | Data synchronization method, distributed system and device |
| CN111046077A (en) * | 2019-10-29 | 2020-04-21 | 口碑(上海)信息技术有限公司 | Data acquisition method and device, storage medium and terminal |
| CN111077870A (en) * | 2020-01-06 | 2020-04-28 | 浙江中烟工业有限责任公司 | Intelligent OPC data real-time acquisition and monitoring system and method based on stream calculation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110633277A (en) | Time sequence data storage method and device, computer equipment and storage medium | |
| CN106981024B (en) | Transaction limit calculation processing system and processing method thereof | |
| CN109951323B (en) | Log analysis method and system | |
| CN111258978B (en) | Data storage method | |
| CN109408501B (en) | Position data processing method and device, server and storage medium | |
| US11500879B2 (en) | Method, device, and program product for managing index of streaming data storage system | |
| CN109918382A (en) | Data processing method, device, terminal and storage medium | |
| CN103731498B (en) | Big data real-time enquiry system load balancing method based on copy selection | |
| US12032550B2 (en) | Multi-tenant partitioning in a time-series database | |
| CN109947729B (en) | Real-time data analysis method and device | |
| CN111523004B (en) | Storage method and system for edge computing gateway data | |
| CN111752945B (en) | Time sequence database data interaction method and system based on container and hierarchical model | |
| CN116233164A (en) | Method, apparatus, storage medium and processor for collecting device data | |
| US20200293543A1 (en) | Method and apparatus for transmitting data | |
| CN115017159A (en) | Data processing method and device, storage medium and electronic equipment | |
| CN114398520A (en) | Data retrieval method, system, device, electronic equipment and storage medium | |
| CN111291054A (en) | Data processing method and device, computer equipment and storage medium | |
| CN113312367A (en) | Yield efficiency recalculation method, apparatus, storage medium and processor | |
| CN113301558A (en) | Message transmission method, device, system and storage medium | |
| CN110597830B (en) | Real-time index generation method and system, electronic equipment and storage medium | |
| CN101072156A (en) | Method and system for searching seed for P2P system | |
| CN112835896B (en) | Real-time database data hotspot balancing method, device, equipment and medium | |
| CN113360558B (en) | Data processing method, data processing device, electronic equipment and storage medium | |
| US11429626B2 (en) | Method, device, and program product for managing index of storage system | |
| CN113360576A (en) | Power grid mass data real-time processing method and device based on Flink Streaming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210827 |