CN113312526A - Network information dynamic acquisition method and device, computer equipment and storage medium - Google Patents
Network information dynamic acquisition method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113312526A CN113312526A CN202110732078.2A CN202110732078A CN113312526A CN 113312526 A CN113312526 A CN 113312526A CN 202110732078 A CN202110732078 A CN 202110732078A CN 113312526 A CN113312526 A CN 113312526A
- Authority
- CN
- China
- Prior art keywords
- network information
- window offset
- classification
- historical data
- dynamically
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000004422 calculation algorithm Methods 0.000 claims description 60
- 238000004458 analytical method Methods 0.000 claims description 51
- 238000004590 computer program Methods 0.000 claims description 27
- 238000005259 measurement Methods 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 9
- 238000012549 training Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036651 mood Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/906—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application relates to the technical field of artificial intelligence, in particular to a method and a device for dynamically acquiring network information, computer equipment and a storage medium. The method comprises the following steps: acquiring the collected current network information, and determining the classification corresponding to the current network information; inquiring window offset corresponding to the classification, wherein the window offset is obtained by analyzing historical data of the corresponding classification; and dynamically acquiring network information according to the window offset. The method can improve the accuracy of network information or lack of network information. In addition, the present application also relates to a blockchain technique, and the current network information, the window offset, and the collected network information may be stored in a blockchain node.
Description
Technical Field
The present application relates to the field of artificial intelligence technologies, and in particular, to a method and an apparatus for dynamically acquiring network information, a computer device, and a storage medium.
Background
The enterprise public opinion refers to the set of expressions of beliefs, attitudes, opinions, moods and the like held by the public to an event around the occurrence, development and change of an enterprise event in a certain social space.
At present, a method of counting once at a fixed time interval is adopted for collecting public opinion related information of enterprises on a network, namely after one message is collected, the message is collected again at a fixed time interval, but because the information is delayed, disordered or even lost on a system, the information collected at the fixed interval may have errors, and further, a decision result in a certain time interval has errors.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a network information collecting method, device, computer device and storage medium capable of accurately collecting network information.
A method for dynamically collecting network information, the method comprising:
acquiring the collected current network information, and determining the classification corresponding to the current network information;
inquiring window offset corresponding to the classification, wherein the window offset is obtained by analyzing historical data of the corresponding classification;
and dynamically acquiring network information according to the window offset.
In one embodiment, before dynamically acquiring the network information according to the window offset, the method further includes:
acquiring historical data of corresponding classification, and calculating according to the historical data to obtain the prediction quantity of the network information;
the dynamically collecting network information according to the window offset comprises the following steps:
dynamically adjusting the collection cluster scale of the collected network information according to the pre-measurement;
and dynamically acquiring network information according to the window offset through the adjusted server in the acquisition cluster scale.
In one embodiment, after the dynamically collecting the network information according to the window offset, the method further includes:
dynamically adjusting the scale of a decision cluster according to the pre-measurement;
and processing the acquired network information through the adjusted decision cluster scale to obtain early warning information.
In one embodiment, the processing the collected network information by the adjusted decision cluster scale to obtain the early warning information includes:
inquiring whether a decision algorithm of an updated plug-in mode exists;
if the updated decision algorithm of the plug-in mode exists, analyzing the acquired network data through the updated decision algorithm of the plug-in mode according to the adjusted decision cluster scale to obtain early warning information.
In one embodiment, before querying the window offset corresponding to the classification, the method further includes:
an attribution analysis algorithm that queries whether an updated plug-in pattern exists;
if the updated attribution analysis algorithm of the plug-in mode exists, grouping the historical data through the updated attribution analysis algorithm of the plug-in mode;
analyzing the grouped historical data to obtain window offset;
the determining the classification corresponding to the current network information includes:
and determining the classification corresponding to the current network information according to the updated attribution analysis algorithm.
A dynamic network information acquisition apparatus, the apparatus comprising:
the information acquisition module is used for acquiring the acquired current network information and determining the classification corresponding to the current network information;
the window offset query module is used for querying the window offset corresponding to the classification, and the window offset is obtained by analyzing the historical data of the corresponding classification;
and the acquisition module is used for dynamically acquiring the network information according to the window offset.
In one embodiment, the apparatus further comprises:
the prediction quantity calculation module is used for acquiring the historical data of the corresponding classification and calculating according to the historical data to obtain the prediction quantity of the network information;
the acquisition module comprises:
the collection cluster scale adjusting unit is used for dynamically adjusting the collection cluster scale of the collected network information according to the pre-measurement;
and the first acquisition unit is used for dynamically acquiring the network information according to the window offset through the adjusted server in the acquisition cluster scale.
In one embodiment, the apparatus further comprises:
the decision cluster scale adjusting module is used for dynamically adjusting the decision cluster scale according to the pre-measurement;
and the early warning information generation module is used for processing the acquired network information through the adjusted decision cluster scale to obtain early warning information.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the method in any of the above embodiments when executing the computer program.
A computer storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the method in any of the above embodiments.
According to the method, the device, the computer equipment and the storage medium for dynamically acquiring the network information, after the current network information is acquired, the classification corresponding to the current network information is determined, so that the window offset of the corresponding classification calculated by the background according to the historical data can be acquired according to the classification, the acquisition time can be adjusted according to the window offset, the network information is acquired according to the acquisition time, the omission of the network information cannot be caused, and the accuracy of network information acquisition can be ensured.
Drawings
FIG. 1 is a diagram illustrating an exemplary scenario for dynamic network information acquisition;
FIG. 2 is a schematic flow chart diagram illustrating a method for dynamically collecting network information in one embodiment;
FIG. 3 is a timing diagram illustrating various network information in one embodiment;
FIG. 4 is a block diagram of a method for dynamic collection of network information in one embodiment;
FIG. 5 is a block diagram of a dynamic network information acquisition device according to an embodiment;
FIG. 6 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The network information dynamic acquisition method provided by the application can be applied to the application environment shown in fig. 1. The network server 102 communicates with the information acquisition server 104 via a network. The information obtaining server 104 may collect current network information from the network server 102, and determine a classification corresponding to the current network information; inquiring window offset corresponding to the classification, wherein the window offset is obtained by analyzing historical data of the corresponding classification; and dynamically acquiring network information according to the window offset. Therefore, after the current network information is obtained, the classification corresponding to the current network information is determined, so that the window offset calculated by the background in real time according to the historical data can be obtained according to the classification, the network information is collected according to the window offset, and the accuracy of network information collection can be ensured. The network server 102 and the information obtaining server 104 may be implemented by separate servers or a server cluster composed of a plurality of servers.
In an embodiment, as shown in fig. 2, a method for dynamically collecting network information is provided, which is described by taking the method as an example applied to the server in fig. 1, and includes the following steps:
s202: and acquiring the acquired current network information, and determining the classification corresponding to the current network information.
Specifically, the current network information is information collected from the network in the current time period, which may be network information including a plurality of categories, where the current time period is within the current window, and specifically, see fig. 3, where information 1 to 4 collected from the start time to the first dynamic statistics point is the current network information. It should be noted that, when the system is just started, the first piece of information collected by the system is the current network information because there is no concept of a window.
Specifically, the classification is attribution analysis of current network information, specifically, an attribution analysis algorithm may be set in the server, and the algorithm classifies the network information into a plurality of classes to determine a delay time of each class of information, so that a time point for collecting next information of the network information may be determined according to the delay time, and a window may be generated. For example, the attribution of the network information may include an industry, an enterprise, a region, and an information source (i.e., an information provider), so that when the current network information is obtained, the industry, the enterprise, the region, and the information source of the current network information may be determined, and the classification of the current network information may be determined according to the industry, the enterprise, the region, and the information source.
S204: and inquiring window offset corresponding to the classification, wherein the window offset is obtained by analyzing the historical data of the corresponding classification.
Specifically, the window offset refers to an interval time between a time point of next collecting network information and a collecting time point of current network information. Specifically, the server background calculates and updates the window offset corresponding to each category in real time in an asynchronous manner. Specifically, the server classifies the historical data into a plurality of classes according to an attribution analysis algorithm, and then analyzes the historical data based on each class to obtain the window offset corresponding to the historical data, so that when the network information exists, the server can obtain the corresponding window offset in real time to guide the acquisition of the network information. Therefore, the statistical model with improved accuracy can be maximized while the loss of timeliness is minimized, and a dynamic balance environment is provided for rapidness and accuracy.
In practical application, the server has a first service and a second service, wherein the first service and the second service are processed asynchronously, the first service is used for collecting network information, and the second service is used for calculating window offset according to historical information and an attribution analysis algorithm. Specifically, the second service obtains a current attribution analysis algorithm, determines attributions of information acquisition time according to the current attribution analysis algorithm, classifies historical data according to the attributions, and analyzes data of each classification to obtain a window offset under the classification after the classification is completed, wherein the process of analyzing the data of each classification can be to obtain the acquisition time of the historical data of the type, calculate a time interval of adjacent acquisition times, and then calculate mathematical statistics on the obtained time interval to obtain the window offset corresponding to the classification, for example, an average value can be calculated, wherein optionally, the server can only obtain the historical data which is within a preset range from the current time to reduce the processing amount of the server and improve the calculation efficiency of the window offset, wherein the preset range can be 1 week, 1 month and the like, it is not particularly limited herein or determined according to an attribution analysis algorithm.
The attribution analysis algorithm is an algorithm for analyzing historical data, the target of the attribution analysis algorithm is window offset, the input is classification of the data, and the dimension of the classification can be obtained through analysis of an Xboost model. Thus, the data rule is fully learned, and the rule can be characterized by the delay of different data suppliers; the market response speed and the market heat of different enterprises or industries have a certain reference rule and the like in a certain period so as to ensure the accuracy of the window offset.
S206: and dynamically acquiring network information according to the window offset.
Specifically, after the server obtains the window offset, a next network information acquisition point is obtained by calculation according to a last acquisition point of the current network information and the window offset, so that all network information in the window is acquired at the next network information acquisition point. Therefore, the window offset is obtained before each acquisition, and the accuracy of the acquisition time is ensured.
According to the dynamic network information acquisition method, after the current network information is acquired, the classification corresponding to the current network information is determined, so that the window offset of the corresponding classification calculated by the background in real time according to historical data can be acquired according to the classification, the acquisition time can be adjusted according to the window offset, and the network information is acquired according to the acquisition time, so that the omission of the network information is avoided, and the accuracy of network information acquisition can be ensured.
In one embodiment, before dynamically acquiring the network information according to the window offset, the method further includes: acquiring historical data of corresponding classification, and calculating according to the historical data to obtain the prediction quantity of the network information; dynamically acquiring network information according to the window offset, comprising: dynamically adjusting the collection cluster scale of the collected network information according to the pre-measurement; and dynamically acquiring the network information according to the window offset through the adjusted server in the acquisition cluster scale.
Specifically, the predicted amount of the network information is obtained according to the corresponding classified historical data, for example, after the classified historical data appears each time, a large amount of information is generated subsequently, so that cluster expansion is required at this time, so that the network information can be collected in a next network information collection window in time and quickly.
In this embodiment, the pre-measurement of the network information is calculated according to historical data, wherein a background may be provided with a corresponding pre-measurement analysis model, after the server acquires the current network information, the background queries the historical data of the corresponding classification according to the classification of the current network information, then counts the acquisition windows of the historical data and the number of the historical data acquired in each window, and finally performs model training according to the acquisition windows of the historical data and the number of the historical data acquired in each window to obtain the pre-measurement analysis model, for example, the server first groups the historical data according to the interval time of the windows so that the acquisition of the network data in each group is performed within a period of time, for example, if the interval is greater than 2 hours, a new acquisition is performed, and then determines the acquisition window of the first network data in each acquisition, and taking other acquisition windows in the time as subsequent acquisition windows, sequencing according to the time, counting the number of network data corresponding to each acquisition window, and inputting each acquisition window sequenced according to the time and the number of the corresponding network data into the initial model for model training to obtain the predictive analysis model. Therefore, after the current network data is obtained, the classification of the current network data is determined, then the corresponding predictive measurement analysis model is determined according to the classification, and finally the predictive measurement is calculated according to the predictive measurement analysis model.
Specifically, the adjustment of the scale of the collection cluster according to the prediction amount may be determined according to a preset correspondence, for example, a correspondence table between the prediction amount and the collection cluster scale is stored in the server, the server determines the collection cluster scale by querying the correspondence table, and schedules the server cluster according to the determined collection cluster scale, so that the efficiency of subsequent collection of network information is higher, and resources of the server are not wasted.
In one embodiment, after dynamically collecting the network information according to the window offset, the method further includes: dynamically adjusting the scale of the decision cluster according to the pre-measurement; and processing the acquired network information through the adjusted decision cluster scale to obtain early warning information.
Specifically, the predicted amount of the network information is obtained according to the corresponding classified historical data, for example, after the classified historical data appears each time, a large amount of information is generated subsequently, so that decision cluster expansion needs to be performed at this time, so that only the acquired network information can be processed in a next network information acquisition window in time and quickly to obtain early warning information.
In this embodiment, the pre-measurement of the network information is calculated according to historical data, wherein a background may be provided with a corresponding pre-measurement analysis model, after the server acquires the current network information, the background queries the historical data of the corresponding classification according to the classification of the current network information, then counts the acquisition windows of the historical data and the number of the historical data acquired in each window, and finally performs model training according to the acquisition windows of the historical data and the number of the historical data acquired in each window to obtain the pre-measurement analysis model, for example, the server first groups the historical data according to the interval time of the windows so that the acquisition of the network data in each group is performed within a period of time, for example, if the interval is greater than 2 hours, a new acquisition is performed, and then determines the acquisition window of the first network data in each acquisition, and taking other acquisition windows in the time as subsequent acquisition windows, sequencing according to the time, counting the number of network data corresponding to each acquisition window, and inputting each acquisition window sequenced according to the time and the number of the corresponding network data into the initial model for model training to obtain the predictive analysis model. Therefore, after the current network data is obtained, the classification of the current network data is determined, then the corresponding predictive measurement analysis model is determined according to the classification, and finally the predictive measurement is calculated according to the predictive measurement analysis model.
Specifically, the adjustment of the scale of the decision cluster according to the prediction amount may be determined according to a preset correspondence, for example, a correspondence table between the prediction amount and the scale of the decision cluster is stored in the server, and the server determines the scale of the decision cluster by querying the correspondence table and schedules the server cluster according to the determined scale of the decision cluster, so that the processing efficiency of the subsequent acquired network information is higher, and the resource of the server is not wasted.
In the embodiment, the deployment design with automatic prejudgment and dynamic expansion can reduce the cost to the maximum extent and perfectly deal with the market hotspot.
In one embodiment, the processing the acquired network information by the adjusted decision cluster scale to obtain the early warning information includes: inquiring whether a decision algorithm of an updated plug-in mode exists; if the updated decision algorithm of the plug-in mode exists, analyzing the acquired network data through the updated decision algorithm of the plug-in mode according to the adjusted decision cluster scale to obtain early warning information.
In particular, in this embodiment, the decision algorithm is implemented in a plug-in manner, which allows the developer to quickly and flexibly adjust the logic to respond to market changes. When the decision is made each time, whether a decision algorithm of an updated plug-in mode exists or not is judged firstly, if yes, the updated algorithm is directly obtained to analyze the acquired network data, and therefore the method can be developed quickly when a new scene is generated, and the system can be used in a plug-and-play mode.
In one embodiment, before querying the window offset corresponding to the classification, the method further includes: an attribution analysis algorithm that queries whether an updated plug-in pattern exists; if the updated attribution analysis algorithm of the plug-in mode exists, grouping the historical data through the updated attribution analysis algorithm of the plug-in mode; analyzing the grouped historical data to obtain window offset; determining a classification corresponding to the current network information, including: and determining the classification corresponding to the current network information according to the updated attribution analysis algorithm.
In particular, in this embodiment, the attribution analysis algorithm is in a plug-in manner, which allows developers to quickly and flexibly adjust logic to respond to market changes. When attribution analysis is carried out each time, whether an updated attribution analysis algorithm of the plug-in mode exists or not is judged firstly, if yes, the updated algorithm is directly obtained, and classification corresponding to the current network information is determined according to the updated attribution analysis algorithm, so that the method can be rapidly developed when a new scene is generated, and the system can be plugged and used.
Specifically, as shown in fig. 4, fig. 4 is a frame diagram of a network information dynamic collection method in an embodiment, where the attribution analysis cluster performs analysis according to historical data to obtain a dynamic window offset, so that data collection can be performed through a dynamic window during collection, and after data is obtained through collection, the collected network data is analyzed through a decision statistic cluster. The information acquisition server can also be used for predicting the prediction quantity of the network information according to historical data, so that the sizes of the attribution analysis cluster and the decision statistic cluster are adjusted, and the full utilization of resources is realized.
It is emphasized that, in order to further ensure the privacy and security of the current network information and the window offset, the current network information and the window offset may also be stored in a node of a block chain.
It should be understood that, although the steps in the flowchart of fig. 2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 2 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 5, there is provided a network information dynamic collection apparatus, including: an information obtaining module 501, a window offset querying module 502 and an acquisition module 503, wherein:
an information obtaining module 501, configured to obtain the collected current network information, and determine a classification corresponding to the current network information;
a window offset query module 502, configured to query window offsets corresponding to the classifications, where the window offsets are obtained by analyzing historical data of the corresponding classifications;
the collecting module 503 is configured to dynamically collect the network information according to the window offset.
In one embodiment, the dynamic network information collecting device further includes:
the prediction quantity calculation module is used for acquiring the historical data of the corresponding classification and calculating according to the historical data to obtain the prediction quantity of the network information;
the acquisition module 503 may include:
the acquisition cluster scale adjusting unit is used for dynamically adjusting the acquisition cluster scale of the acquired network information according to the pre-measurement;
and the first acquisition unit is used for dynamically acquiring the network information according to the window offset through the adjusted server in the acquisition cluster scale.
In one embodiment, the dynamic network information collecting device further includes:
the decision cluster scale adjusting module is used for dynamically adjusting the decision cluster scale according to the prediction quantity;
and the early warning information generation module is used for processing the acquired network information through the adjusted decision cluster scale to obtain early warning information.
In one embodiment, the dynamic network information collecting device further includes:
the first query module is used for querying whether a decision algorithm of an updated plug-in mode exists;
and if the early warning information generation module is used for judging whether the updated decision algorithm of the plug-in mode exists, analyzing the acquired network data through the updated decision algorithm of the plug-in mode according to the adjusted decision cluster scale to obtain early warning information.
In one embodiment, the dynamic network information collecting device further includes:
the second query module is used for querying whether an attribution analysis algorithm of the updated plug-in mode exists;
the grouping module is used for grouping the historical data through the updated attribution analysis algorithm of the plug-in mode if the updated attribution analysis algorithm of the plug-in mode exists;
the window offset calculation module is used for analyzing the grouped historical data to obtain window offsets;
the information obtaining module 501 is further configured to determine a classification corresponding to the current network information according to the updated attribution analysis algorithm.
For specific limitations of the network information dynamic acquisition device, reference may be made to the above limitations of the network information dynamic acquisition method, which is not described herein again. The modules in the network information dynamic acquisition device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 6. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing the collected network information and the window offset. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method for dynamically collecting network information.
Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, there is provided a computer device comprising a memory storing a computer program and a processor implementing the following steps when the processor executes the computer program: acquiring the collected current network information, and determining the classification corresponding to the current network information; inquiring window offset corresponding to the classification, wherein the window offset is obtained by analyzing historical data of the corresponding classification; and dynamically acquiring network information according to the window offset.
In one embodiment, before the dynamic collection of the network information according to the window offset is implemented when the processor executes the computer program, the method further includes: acquiring historical data of corresponding classification, and calculating according to the historical data to obtain the prediction quantity of the network information; the dynamic collection of network information according to window offset, which is realized when a processor executes a computer program, includes: dynamically adjusting the collection cluster scale of the collected network information according to the pre-measurement; and dynamically acquiring the network information according to the window offset through the adjusted server in the acquisition cluster scale.
In one embodiment, after the dynamic collection of the network information according to the window offset is implemented when the processor executes the computer program, the method further includes: dynamically adjusting the scale of the decision cluster according to the pre-measurement; and processing the acquired network information through the adjusted decision cluster scale to obtain early warning information.
In one embodiment, the processing of the collected network information by the adjusted decision cluster size to obtain the early warning information, which is implemented when the processor executes the computer program, includes: inquiring whether a decision algorithm of an updated plug-in mode exists; if the updated decision algorithm of the plug-in mode exists, analyzing the acquired network data through the updated decision algorithm of the plug-in mode according to the adjusted decision cluster scale to obtain early warning information.
In one embodiment, the processor, when executing the computer program, further performs the steps of, prior to querying the window offset corresponding to the classification: an attribution analysis algorithm that queries whether an updated plug-in pattern exists; if the updated attribution analysis algorithm of the plug-in mode exists, grouping the historical data through the updated attribution analysis algorithm of the plug-in mode; analyzing the grouped historical data to obtain window offset; determining a classification corresponding to current network information implemented when the processor executes the computer program includes: and determining the classification corresponding to the current network information according to the updated attribution analysis algorithm.
In one embodiment, a computer storage medium is provided, having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of: acquiring the collected current network information, and determining the classification corresponding to the current network information; inquiring window offset corresponding to the classification, wherein the window offset is obtained by analyzing historical data of the corresponding classification; and dynamically acquiring network information according to the window offset.
In one embodiment, before the computer program is executed by a processor to dynamically collect network information according to window offset, the computer program further comprises: acquiring historical data of corresponding classification, and calculating according to the historical data to obtain the prediction quantity of the network information; the dynamic collection of network information according to window offsets, as implemented by a computer program when executed by a processor, includes: dynamically adjusting the collection cluster scale of the collected network information according to the pre-measurement; and dynamically acquiring the network information according to the window offset through the adjusted server in the acquisition cluster scale.
In one embodiment, the dynamic collection of network information based on window offsets when implemented by a processor further comprises: dynamically adjusting the scale of the decision cluster according to the pre-measurement; and processing the acquired network information through the adjusted decision cluster scale to obtain early warning information.
In one embodiment, the processing of the collected network information by the adjusted decision cluster size to obtain the early warning information, when the computer program is executed by the processor, includes: inquiring whether a decision algorithm of an updated plug-in mode exists; if the updated decision algorithm of the plug-in mode exists, analyzing the acquired network data through the updated decision algorithm of the plug-in mode according to the adjusted decision cluster scale to obtain early warning information.
In one embodiment, the computer program when executed by the processor further comprises, prior to querying the window offset corresponding to the classification: an attribution analysis algorithm that queries whether an updated plug-in pattern exists; if the updated attribution analysis algorithm of the plug-in mode exists, grouping the historical data through the updated attribution analysis algorithm of the plug-in mode; analyzing the grouped historical data to obtain window offset; determining a classification corresponding to current network information implemented when the computer program is executed by the processor, comprising: and determining the classification corresponding to the current network information according to the updated attribution analysis algorithm.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A method for dynamically collecting network information is characterized in that the method comprises the following steps:
acquiring the collected current network information, and determining the classification corresponding to the current network information;
inquiring window offset corresponding to the classification, wherein the window offset is obtained by analyzing historical data of the corresponding classification;
and dynamically acquiring network information according to the window offset.
2. The method of claim 1, wherein before dynamically collecting network information according to the window offset, further comprising:
acquiring historical data of corresponding classification, and calculating according to the historical data to obtain the prediction quantity of the network information;
the dynamically collecting network information according to the window offset comprises the following steps:
dynamically adjusting the collection cluster scale of the collected network information according to the pre-measurement;
and dynamically acquiring network information according to the window offset through the adjusted server in the acquisition cluster scale.
3. The method of claim 1, wherein after dynamically collecting network information according to the window offset, further comprising:
dynamically adjusting the scale of a decision cluster according to the pre-measurement;
and processing the acquired network information through the adjusted decision cluster scale to obtain early warning information.
4. The method of claim 3, wherein the processing the collected network information by the adjusted decision cluster size to obtain early warning information comprises:
inquiring whether a decision algorithm of an updated plug-in mode exists;
if the updated decision algorithm of the plug-in mode exists, analyzing the acquired network data through the updated decision algorithm of the plug-in mode according to the adjusted decision cluster scale to obtain early warning information.
5. The method of any of claims 1 to 4, wherein the querying the window offset corresponding to the classification is preceded by:
an attribution analysis algorithm that queries whether an updated plug-in pattern exists;
if the updated attribution analysis algorithm of the plug-in mode exists, grouping the historical data through the updated attribution analysis algorithm of the plug-in mode;
analyzing the grouped historical data to obtain window offset;
the determining the classification corresponding to the current network information includes:
and determining the classification corresponding to the current network information according to the updated attribution analysis algorithm.
6. A dynamic network information acquisition device, comprising:
the information acquisition module is used for acquiring the acquired current network information and determining the classification corresponding to the current network information;
the window offset query module is used for querying the window offset corresponding to the classification, and the window offset is obtained by analyzing the historical data of the corresponding classification;
and the acquisition module is used for dynamically acquiring the network information according to the window offset.
7. The apparatus of claim 6, further comprising:
the prediction quantity calculation module is used for acquiring the historical data of the corresponding classification and calculating according to the historical data to obtain the prediction quantity of the network information;
the acquisition module comprises:
the collection cluster scale adjusting unit is used for dynamically adjusting the collection cluster scale of the collected network information according to the pre-measurement;
and the first acquisition unit is used for dynamically acquiring the network information according to the window offset through the adjusted server in the acquisition cluster scale.
8. The apparatus of claim 6, further comprising:
the decision cluster scale adjusting module is used for dynamically adjusting the decision cluster scale according to the pre-measurement;
and the early warning information generation module is used for processing the acquired network information through the adjusted decision cluster scale to obtain early warning information.
9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 5 when executing the computer program.
10. A computer storage medium on which a computer program is stored, characterized in that the computer program, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110732078.2A CN113312526A (en) | 2021-06-29 | 2021-06-29 | Network information dynamic acquisition method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110732078.2A CN113312526A (en) | 2021-06-29 | 2021-06-29 | Network information dynamic acquisition method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113312526A true CN113312526A (en) | 2021-08-27 |
Family
ID=77381147
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110732078.2A Pending CN113312526A (en) | 2021-06-29 | 2021-06-29 | Network information dynamic acquisition method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113312526A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004178479A (en) * | 2002-11-29 | 2004-06-24 | Yokogawa Electric Corp | Data collector |
| CN102339290A (en) * | 2010-07-22 | 2012-02-01 | 北大方正集团有限公司 | Method and device for directionally acquiring webpage data information |
| CN102710441A (en) * | 2012-05-31 | 2012-10-03 | 朱峻茂 | Uniform-interface-based multi-platform information acquisition method and information acquisition system |
| CN105099739A (en) * | 2014-05-15 | 2015-11-25 | ***通信集团公司 | Plug-in type software deployment method and apparatus, and application server |
| CN106021391A (en) * | 2016-05-11 | 2016-10-12 | 广东工业大学 | Product comment information real-time collection method based on Storm |
| CN108595666A (en) * | 2018-04-28 | 2018-09-28 | 中译语通科技股份有限公司 | Dynamic calculates the method for news collection Service Source, information data processing terminal |
| CN110069692A (en) * | 2019-05-06 | 2019-07-30 | 中国船舶工业综合技术经济研究院 | A kind of resource updates time forecasting methods and system towards open source information collection |
| CN111506366A (en) * | 2020-04-17 | 2020-08-07 | 咪咕文化科技有限公司 | Plug-in calling method and device, electronic equipment and storage medium |
-
2021
- 2021-06-29 CN CN202110732078.2A patent/CN113312526A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004178479A (en) * | 2002-11-29 | 2004-06-24 | Yokogawa Electric Corp | Data collector |
| CN102339290A (en) * | 2010-07-22 | 2012-02-01 | 北大方正集团有限公司 | Method and device for directionally acquiring webpage data information |
| CN102710441A (en) * | 2012-05-31 | 2012-10-03 | 朱峻茂 | Uniform-interface-based multi-platform information acquisition method and information acquisition system |
| CN105099739A (en) * | 2014-05-15 | 2015-11-25 | ***通信集团公司 | Plug-in type software deployment method and apparatus, and application server |
| CN106021391A (en) * | 2016-05-11 | 2016-10-12 | 广东工业大学 | Product comment information real-time collection method based on Storm |
| CN108595666A (en) * | 2018-04-28 | 2018-09-28 | 中译语通科技股份有限公司 | Dynamic calculates the method for news collection Service Source, information data processing terminal |
| CN110069692A (en) * | 2019-05-06 | 2019-07-30 | 中国船舶工业综合技术经济研究院 | A kind of resource updates time forecasting methods and system towards open source information collection |
| CN111506366A (en) * | 2020-04-17 | 2020-08-07 | 咪咕文化科技有限公司 | Plug-in calling method and device, electronic equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 胡伟 等: "Nutch中网页更新预测研究与优化", 《上海师范大学学报(自然科学版)》, vol. 45, no. 04, 15 August 2016 (2016-08-15), pages 448 - 457 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108959934B (en) | Security risk assessment method, security risk assessment device, computer equipment and storage medium | |
| CN110493190B (en) | Data information processing method and device, computer equipment and storage medium | |
| CN110929036A (en) | Electric power marketing inspection management method and device, computer equipment and storage medium | |
| CN111897861A (en) | Content recommendation method and device, computer equipment and storage medium | |
| CN111741112A (en) | File downloading method, device, equipment and storage medium based on artificial intelligence | |
| CN110855477A (en) | Link log monitoring method and device, computer equipment and storage medium | |
| CN110717647A (en) | Decision flow construction method and device, computer equipment and storage medium | |
| CN113190426B (en) | Stability monitoring method for big data scoring system | |
| CN112765385A (en) | Information management method and system based on big data and Internet | |
| CN113723956A (en) | Abnormity monitoring method, device, equipment and storage medium | |
| KR20220115357A (en) | A method and apparatus for generating future demand forecast data based on attention mechanism | |
| CN110084476B (en) | Case adjustment method, device, computer equipment and storage medium | |
| CN113110961B (en) | Equipment abnormality detection method and device, computer equipment and readable storage medium | |
| CN111339500B (en) | Air pollution tracing method and device, computer equipment and storage medium | |
| CN112766536A (en) | Model training method, device and terminal for calculating road engineering labor unit price | |
| CN110598090B (en) | Interest tag generation method and device, computer equipment and storage medium | |
| CN110131852B (en) | Air conditioner data collection method and device, computer equipment and storage medium | |
| CN113312526A (en) | Network information dynamic acquisition method and device, computer equipment and storage medium | |
| CN111046240A (en) | Gateway flow statistical method, device, computer equipment and storage medium | |
| CN112860763B (en) | Real-time streaming data processing method and device, computer equipment and storage medium | |
| CN113516302B (en) | Business risk analysis method, device, equipment and storage medium | |
| CN115576973A (en) | Service deployment method, device, computer equipment and readable storage medium | |
| CN115102672A (en) | Data processing method, system, computer device and storage medium | |
| CN111400608B (en) | Data processing method and device, storage medium and electronic equipment | |
| CN111061875B (en) | Super parameter determination method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |