CN114461490B - Fortune dimension aggregation system - Google Patents
Fortune dimension aggregation system Download PDFInfo
- Publication number
- CN114461490B CN114461490B CN202111676136.0A CN202111676136A CN114461490B CN 114461490 B CN114461490 B CN 114461490B CN 202111676136 A CN202111676136 A CN 202111676136A CN 114461490 B CN114461490 B CN 114461490B
- Authority
- CN
- China
- Prior art keywords
- data
- module
- cluster
- acquisition
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Business, Economics & Management (AREA)
- Quality & Reliability (AREA)
- General Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Strategic Management (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Economics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a system for aggregating operation and data. The system comprises a nano-tube center module, an acquisition configuration module, a task management module and a data delivery module; the nano-tube center module is used for configuring credential information, project information, equipment information and cluster information required by executing operation and maintenance data acquisition tasks; the acquisition configuration module is used for configuring containerized collector information and acquisition script information required by executing the operation and data acquisition task; the task management module is used for creating a fortune dimension data acquisition task and managing the fortune dimension data acquisition task, wherein the management comprises the operations of starting, stopping, copying and deleting the fortune dimension data acquisition task; the data delivery module is used for delivering the operation and maintenance data acquired by the acquisition device into the ES cluster. The technical scheme of the invention improves the collection efficiency of the operation and maintenance data.
Description
Technical Field
The invention relates to the technical field of data aggregation, in particular to a system for aggregating operation and data.
Background
The universal operation and maintenance data aggregation is a universal aggregation method of operation and maintenance data of a cloud primary related system, which is researched aiming at the use scene taking a cloud primary technology as a bottom framework, so as to solve the aggregation difficulty of data sources of various heterogeneous systems in the industries of medical treatment, agriculture, satellite application and the like on high-performance intercommunication of daily operation and maintenance data (logs, indexes and call chains). In the industry, in order to ensure the smooth running of the software platform, three types of operation data including a log, an index and a call chain must be output in a digital software architecture to support continuous iteration of the software architecture. Aiming at the integral construction of an operation and maintenance system, an operation and maintenance developer needs to clearly determine the conditions of the basic operation environment (including a single machine and a cluster), the data source type (including a data format and a development language), the data reporting form, the operation and maintenance data processing service, the operation and maintenance data storage mechanism, the storage type selection and the like of the acquisition end, so that the access of the operation and maintenance data source can be completed once. The general scheme is that the integrated deployment is carried out through acquisition services and early warning services in different fields such as ELK, monitoring System and APM, and the operation data are stored in a non-relational database and a time sequence database through Pull means. And finally delivering the operation and maintenance data to architects or operation and maintenance personnel through software interfaces in the respective fields.
In the existing operation and maintenance data aggregation system, under two modes of a cloud native technology and a traditional IT architecture, data access is completed by singly relying on a mode of kubernetes or a virtual single machine, and a general and reliable mode for mass production cannot be solidified for an operation and maintenance system. On the one hand, under the increasing data acquisition demands, operation and maintenance developers need to solve the problems of multiple dimensions such as explosive growth of source data, difference of far-end acquisition modes, difference of far-end basic operation environments, single storage throughput bottlenecks and the like, on the other hand, data generated in the whole operation and maintenance process are split (stored in a file system, a database, a search engine and the like for example) at the data storage level, and cannot be continuous in operation and maintenance work. The problems of the two dimensions can cause difficulty in the overall operation and maintenance data development, architects and operation and maintenance personnel to manufacture upper-layer operation and maintenance data application (such as data disc watching), and further the existing software abnormality problem can not be found more conveniently. Under the prior art background, resources (manual work and machines) can be continuously input into each node for data transmission in the whole operation area to solve various problems, so that the enterprise cost is increased.
Disclosure of Invention
The invention provides an operation and maintenance data aggregation system, which improves the collection efficiency of operation and maintenance data.
The embodiment of the invention provides a system for aggregating operation and maintenance data, which comprises a nano-tube center module, an acquisition configuration module, a task management module and a data delivery module;
the nano-tube center module is used for configuring credential information, project information, equipment information and cluster information required by executing operation and maintenance data acquisition tasks;
the acquisition configuration module is used for configuring containerized collector information and acquisition script information required by executing the operation and data acquisition task;
the task management module is used for creating a fortune dimension data acquisition task and managing the fortune dimension data acquisition task, wherein the management comprises the operations of starting, stopping, copying and deleting the fortune dimension data acquisition task;
the data delivery module is used for delivering the operation and maintenance data acquired by the acquisition device into the ES cluster.
Further, the nano-tube center module comprises a certificate management sub-module, a project management sub-module, a device management sub-module and a cluster management sub-module;
the credential management submodule is used for configuring credential information required by remote host authentication or remote service authentication when the operation and maintenance data acquisition task is executed, wherein the credential information comprises SSH, TLS, mirror image warehouse keys, account numbers, passwords and keys;
the project management submodule is used for configuring project information, and the project information comprises host project information and cluster project information;
the equipment management submodule is used for configuring host information, wherein the host information comprises a host name, a host IP, a host port, an item to which a host belongs, a host type and a host certificate;
the cluster management submodule is used for configuring cluster information, wherein the cluster information comprises a cluster name, a cluster IP, a project to which a cluster belongs, a cluster architecture, a cluster service address, a cluster route and a cluster token.
Further, the acquisition configuration module comprises an acquisition sub-module and a script management sub-module;
the collector submodule is used for configuring containerized collector information required by executing a operation and maintenance data collection task, and the collector information comprises a collector name, a project to which the collector belongs, a nanotube type, a nanotube object, a collection script name, a collection path and a collection rule;
the script management submodule is used for configuring acquisition rules of acquisition scripts required by executing operation and data acquisition tasks and carrying out visual management on the acquisition scripts.
Further, configuring a data cleaning rule of the collector at the collector submodule, wherein the data cleaning rule comprises cleaning the cached operation and maintenance data at fixed time according to the configured data cleaning time, and cleaning the cached operation and maintenance data containing the text information according to the configured text information.
Further, the system comprises a fortune data management module which is used for storing collected fortune data and providing index and data management based on an index template.
Further, the system comprises a unified ES cluster storage module, wherein the unified ES cluster storage module is used for storing the collected operation and maintenance data into the ES clusters in a unified way.
Further, the system comprises an index data Restful module, wherein the index data Restful module is used for transmitting the collected operation and maintenance data to other systems through a Restful interface.
The embodiment of the invention has the following beneficial effects:
the invention provides a system for collecting operation and maintenance data, which is characterized in that a data collection mode of a containerized collector is adopted, and a collection task executed by the containerized collector is correspondingly configured in a nano tube center module, a collection configuration module, a task management module and a data delivery module of the system, so that the intrusion-free data collection mode is realized, the operation environment and the system architecture of a service system (namely a collection end) are not needed, the service system can be conveniently, quickly and uninhibited to access, and the collection efficiency of the operation and maintenance data is further improved. Meanwhile, the invention adopts the data acquisition mode of the containerized collector, and can design different operation and maintenance data aggregation systems without changing corresponding configurations according to the basic operation environment (including a single machine and a cluster), the data source type (including a data format and a development language), the data reporting form, the operation and maintenance data processing service, the operation and maintenance data storage mechanism and the difference in storage type, and only needs to change the operation and maintenance data acquisition efficiency and reduce the operation and maintenance data acquisition cost at the nanotube center module, the acquisition configuration module, the task management module and the data delivery module of the system.
Drawings
FIG. 1 is a block diagram of a system for aggregating dimension data in accordance with one embodiment of the present invention;
FIG. 2 is a flow chart of an operation and maintenance data collection system according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of the operation and data collection system according to an embodiment of the present invention before formatting;
FIG. 4 is a schematic diagram of the operation and data collected by the operation and data aggregation system after formatting according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the operation and data collection system according to an embodiment of the present invention after further formatting;
FIG. 6 is a schematic diagram of the operation and data aggregation system according to an embodiment of the present invention after further formatting.
Detailed Description
The following description of the embodiments of the present invention will be made more apparent and fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the invention are shown. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, an operation and data aggregation system provided in an embodiment of the present invention includes a nanotube center module, an acquisition configuration module, a task management module, a data delivery module, an operation and data management module, and a unified ES cluster storage module. The operation and maintenance data management module of the embodiment of the present invention corresponds to the data center storage module in fig. 1.
The nano-tube center module is used for configuring credential information, project information, equipment information and cluster information required for executing operation and maintenance data acquisition tasks. The nano tube center is a configuration center of the system and is used for managing configuration required by executing operation and maintenance data acquisition tasks of all items of the system to support basic configuration information required by operation of task instances, wherein the operation and maintenance data acquisition tasks comprise acquisition tasks aiming at a host and acquisition tasks aiming at clusters.
As one embodiment, the nanotube center module includes a credential management sub-module, a project management sub-module, a device management sub-module, and a cluster management sub-module.
The credential management submodule is used for configuring credential information required by remote host authentication or remote service authentication when the operation and maintenance data acquisition task is executed, wherein the credential information comprises SSH, TLS, mirror image warehouse keys, account numbers, passwords and keys. The credential management sub-module is used for managing the credentials required by executing the acquisition task, specifically, managing the credentials required by performing remote host authentication or remote service authentication.
The project management submodule is used for configuring project information, and the project information comprises host project information and cluster project information. Configuring the project information, which specifically comprises adding, deleting and modifying the project information; the purpose of configuring project information is to manage and maintain hosts or clusters from the dimensions of the project. An item includes multiple hosts or clusters of the item management and maintenance, or multiple hosts and clusters of the item management and maintenance.
The device management submodule is used for configuring host information, wherein the host information comprises a host name, a host IP, a host port, an item to which a host belongs, a host type and a host certificate. Specifically, the device management submodule is used for adding and importing host information needing nanotubes.
The cluster management submodule is used for configuring cluster information, wherein the cluster information comprises a cluster name, a cluster IP, a project to which a cluster belongs, a cluster architecture, a cluster service address, a cluster route and a cluster token. Specifically, the cluster management submodule is used for adding and importing cluster information needing nanotubes.
The acquisition configuration module is used for configuring containerized collector information and acquisition script information required by executing the operation data acquisition task. The acquisition configuration module is in a WEB form, so that operation and maintenance personnel can perform configuration operation in an interface form, and the operation and maintenance personnel can perform execution content configuration of corresponding acquisition tasks aiming at different types of target data sources.
As one embodiment, the acquisition configuration module includes an acquisition sub-module and a script management sub-module;
the collector submodule is used for configuring containerized collector information required by executing a operation and maintenance data collection task, and the collector information comprises a collector name, a project to which the collector belongs, a nanotube type, a nanotube object, a collection script name, a collection path and a collection rule. And configuring a data cleaning rule of the collector in the sub-module of the collector, wherein the data cleaning rule comprises cleaning the cached operation and maintenance data at fixed time according to the configured data cleaning time, and cleaning the cached operation and maintenance data containing the text information according to the configured text information.
The script management submodule is used for configuring acquisition rules of acquisition scripts required by executing operation and data acquisition tasks and carrying out visual management on the acquisition scripts. The content of the acquisition script is the acquisition rule of different containers written in advance.
The task management module is used for creating a fortune data acquisition task and managing the fortune data acquisition task, and the management comprises the operations of starting, stopping, copying and deleting the fortune data acquisition task.
The data delivery module is used for delivering the operation and maintenance data acquired by the acquisition device into the ES cluster. The ES cluster is an ES cluster of an external network. And configuring the ES cluster information of the external network in the module so as to enable the acquired operation data to be delivered to the ES cluster, wherein the configured ES cluster information of the external network comprises, but is not limited to, an address, a port, a certificate and a second index rule. Meanwhile, a default first index rule is built in the system, wherein the first index rule is an index name formed by { date [ year/month/day ] } and { time [ hour/time minute second ] } according to the project name, the equipment type, the equipment name, the acquisition type, the task name, and when a plurality of acquired data are stored in a data center, data aggregation of different layers can be performed according to the index name, including but not limited to data aggregation of project dimension and data aggregation of type dimension, and application display is performed after the scattered data are aggregated.
The operation and maintenance data management module is used for storing collected operation and maintenance data and providing index and data management based on an index template. Specifically, the operation and data management module is used for storing collected operation and data and providing index and data management based on an index template (i.e. the default first index rule). Index management provides management of index storage strategies corresponding to data pushed from a data delivery module, and comprises index field mapping rule setting, index data regular analysis rule setting and conventional monitoring of index storage, wherein the conventional monitoring comprises, but is not limited to, monitoring of the number of fragments, storage size, copy fragments, counting and running conditions. On the other hand, the data interaction mode can realize that index data management operation (the management operation comprises adding, inquiring, deleting and updating) is completed through single or combined multiple dimensions, and the dimensions comprise, but are not limited to, project names, device types, device names, collection types, task names and time intervals.
The unified ES cluster storage module is used for uniformly storing the collected operation and maintenance data into the ES clusters. The ES cluster is an intranet ES cluster. And the collected operation and maintenance data are uniformly stored in the ES cluster, so that the reliability and safety of the data are ensured.
As one embodiment, the system further comprises an index data Restful module, wherein the index data Restful module is used for transmitting the collected operation data to other systems through a Restful interface so as to enable the other systems to develop and apply the operation data for the second time.
As shown in fig. 2, the core flow of the operation and maintenance data acquisition of the system comprises an original data source for accessing the operation and maintenance data, acquisition and formatting of the operation and maintenance data through a containerized acquisition device, unified storage of the operation and maintenance data and provision of the stored operation and maintenance data to an operation and maintenance large disk. The original data source of the operation and maintenance data is unstructured log data storage generated in various services when the service system operates; the containerized collector is a universal collector container built by means of a kubernetes, docker container technology, and can realize original log data collection and structuring operation, and the collector container has at least 10 log format collection capabilities of common middleware, remote collection with various running environments and the like; the unified storage is a structured operation and maintenance data center which is used as original operation and maintenance data after two data processing stages of platform acquisition and cleaning under the clustered deployment form by means of an elastic search storage engine; the operation and maintenance large disk is an operation and maintenance data application for carrying out secondary development on structured or unstructured data in cluster storage. The operation and maintenance data acquisition process of the system is as follows:
step S001: and adding the target equipment and the credential information of the access target equipment through the nano-tube center module. For example: under the Linux system, the account number and the secret key of the target equipment are added as access credentials to remotely connect with a server through an SSH protocol.
And creating a log acquisition task of a specified server connection mode through a task management module, wherein the operation of creating the acquisition task comprises selecting a device type as a host, selecting a host name, selecting an acquisition type text log, inputting a target text file path and a target text file name, and selecting an index generation rule.
With the configuration of step S001, the system will automatically create a set of containers (i.e., create a containerized collector), and access the target log text and collect the operation and maintenance data through the created set of containers, and cache the operation and maintenance data locally. For example: under the Linux system, a container is created through a command docker run or kubecl create, the log files are firstly synchronized from a far end into the container through ssh and tail commands of the container, and then the log files in the container are collected and converted into a storage engine (namely, stored into an ES cluster) through filebean in the container. According to the embodiment of the invention, the containerized collector is adopted, so that the buffer effect on source data can be realized in the data acquisition process, and meanwhile, specific information can be intercepted, so that the data safety is ensured. Meanwhile, the timing cleaning of the cache data (for example, setting 0 point 5 minutes per day to empty the cache data) can be realized by correspondingly setting the containerized collector (namely, customizing the cache time and text content), so that the data is prevented from growing too fast and the data amount is prevented from being too large.
Step S002: formatting the operation and maintenance data collected in the step S001, specifically, using KEY to collect the operation and maintenance data: splitting the data in a VALUE mode, and delivering the formatted data to the unified ES cluster module for storage. As shown in fig. 3, a diagram of the operation and data before formatting is shown; as shown in fig. 4-6, is a schematic diagram of the transport data after formatting.
Step S003: the operation and maintenance data (including log data) stored by the unified ES cluster module is provided for external program (or interface) call in a Restful manner, namely, is provided for application of information data to a third party visual tool, wherein the third party visual tool comprises, but is not limited to, grafana and DataEase.
The embodiment of the invention adopts an intrusion-free acquisition technology, realizes the formatting and unified storage of the key value pair of the source data, and provides an external system for aggregation inquiry through a Restful interface for the acquired operation and maintenance data. Specifically, the intrusion-free operation and maintenance acquisition method of the system of the invention utilizes the container technology to pull, format, push and cache data into one or more containerized collectors and operates outside the target acquisition without modifying the acquisition target, thereby realizing intrusion-free acquisition. In order to facilitate analysis and searching of data, data formatting (such as a nginx log, an OS log, a Docker log and the like) is required after data source acquisition, and source data is formatted by a built-in formatting rule or a custom rule of an operation and maintenance system and then is formatted by a Key: value is delivered to the data center. Meanwhile, the operation and maintenance system provides interface management tools for task management, data management, resource monitoring and the like, abstract classification is carried out on complex bottom management, operation functions are simplified, and operation and maintenance management is facilitated.
The embodiment of the invention ensures the real-time acquisition of the operation and maintenance data, and designs a cache mechanism to control the data growth. And (3) designing a plurality of dimensions such as projects, types and the like, automatically indexing, and realizing the data aggregation application of later multi-dimension. The embodiment of the invention is based on the intrusion-free technology, does not need to carry out external application deployment on target equipment or clusters, simplifies the operation difficulty of operation and maintenance personnel, and greatly reduces the unstable risk of operation and maintenance data acquisition caused by the transformation of the operation environment of a service system by the operation and maintenance personnel in an intrusion-free data acquisition mode.
While the foregoing is directed to the preferred embodiments of the present invention, it will be appreciated by those skilled in the art that changes and modifications may be made without departing from the principles of the invention, such changes and modifications are also intended to be within the scope of the invention.
Those skilled in the art will appreciate that implementing all or part of the above-described embodiments may be accomplished by way of computer programs, which may be stored on a computer readable storage medium, which when executed may comprise the steps of the above-described embodiments. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or the like.
Claims (6)
1. The operation and data aggregation system is characterized by comprising a nano-tube center module, an acquisition configuration module, a task management module and a data delivery module;
the nano-tube center module is used for configuring credential information, project information, equipment information and cluster information required by executing operation and maintenance data acquisition tasks;
the nano-tube center module comprises a certificate management sub-module, a project management sub-module, a device management sub-module and a cluster management sub-module;
the credential management submodule is used for configuring credential information required by remote host authentication or remote service authentication when the operation and maintenance data acquisition task is executed, wherein the credential information comprises SSH, TLS, mirror image warehouse keys, account numbers, passwords and keys;
the project management submodule is used for configuring project information, and the project information comprises host project information and cluster project information; wherein an item includes a plurality of hosts and clusters for the item management and maintenance;
the equipment management submodule is used for configuring host information, wherein the host information comprises a host name, a host IP, a host port, an item to which a host belongs, a host type and a host certificate;
the cluster management submodule is used for configuring cluster information, wherein the cluster information comprises a cluster name, a cluster IP, a project to which a cluster belongs, a cluster architecture, a cluster service address, a cluster route and a cluster token;
the acquisition configuration module is used for configuring containerized collector information and acquisition script information required by executing the operation and data acquisition task;
the task management module is used for creating a fortune dimension data acquisition task and managing the fortune dimension data acquisition task, wherein the management comprises the operations of starting, stopping, copying and deleting the fortune dimension data acquisition task;
the data delivery module is used for delivering the operation and maintenance data acquired by the acquisition device into the ES cluster.
2. The operation and data aggregation system according to claim 1, wherein the acquisition configuration module comprises an acquisition sub-module and a script management sub-module;
the collector submodule is used for configuring containerized collector information required by executing a operation and maintenance data collection task, and the collector information comprises a collector name, a project to which the collector belongs, a nanotube type, a nanotube object, a collection script name, a collection path and a collection rule;
the script management submodule is used for configuring acquisition rules of acquisition scripts required by executing operation and data acquisition tasks and carrying out visual management on the acquisition scripts.
3. The system according to claim 2, wherein the collector submodule configures a data cleaning rule of the collector, the data cleaning rule includes cleaning the cached running data at fixed time according to the configured data cleaning time, and cleaning the cached running data containing the text information according to the configured text information.
4. A system for aggregating data of fortune dimension according to claim 3, including a fortune dimension data management module for storing collected fortune dimension data and providing index and data management based on index templates.
5. The system of claim 4, comprising a unified ES cluster storage module configured to store collected operation data into ES clusters in a unified manner.
6. A system for aggregating motion data according to any of claims 1 to 5, comprising an index data restfulization module for transmitting collected motion data to other systems via a Restful interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111676136.0A CN114461490B (en) | 2021-12-31 | 2021-12-31 | Fortune dimension aggregation system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111676136.0A CN114461490B (en) | 2021-12-31 | 2021-12-31 | Fortune dimension aggregation system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114461490A CN114461490A (en) | 2022-05-10 |
| CN114461490B true CN114461490B (en) | 2023-05-30 |
Family
ID=81407796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111676136.0A Active CN114461490B (en) | 2021-12-31 | 2021-12-31 | Fortune dimension aggregation system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114461490B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017166644A1 (en) * | 2016-03-31 | 2017-10-05 | 乐视控股（北京）有限公司 | Data acquisition method and system |
| CN110704164A (en) * | 2019-09-30 | 2020-01-17 | 珠海市新德汇信息技术有限公司 | Cloud native application platform construction method based on Kubernetes technology |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107220178B (en) * | 2017-05-26 | 2021-02-02 | 苏州浪潮智能科技有限公司 | Method for displaying diagnostic test result of server in quasi-real time |
| CN108133043B (en) * | 2018-01-12 | 2022-07-29 | 福建星瑞格软件有限公司 | Structured storage method for server running logs based on big data |
| CN109726209A (en) * | 2018-09-07 | 2019-05-07 | 网联清算有限公司 | Log aggregation method and device |
| CN109660397B (en) * | 2018-12-21 | 2022-02-15 | 北京百度网讯科技有限公司 | System, method and device for collecting logs |
| CN110851234A (en) * | 2019-10-25 | 2020-02-28 | 中信梧桐港供应链管理有限公司 | Log processing method and device based on docker container |
| CN111190875A (en) * | 2019-12-27 | 2020-05-22 | 航天信息股份有限公司 | Log aggregation method and device based on container platform |
-
2021
- 2021-12-31 CN CN202111676136.0A patent/CN114461490B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017166644A1 (en) * | 2016-03-31 | 2017-10-05 | 乐视控股（北京）有限公司 | Data acquisition method and system |
| CN110704164A (en) * | 2019-09-30 | 2020-01-17 | 珠海市新德汇信息技术有限公司 | Cloud native application platform construction method based on Kubernetes technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114461490A (en) | 2022-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11615084B1 (en) | Unified data processing across streaming and indexed data sets | |
| US11474673B1 (en) | Handling modifications in programming of an iterative message processing system | |
| US11113353B1 (en) | Visual programming for iterative message processing system | |
| US11194552B1 (en) | Assisted visual programming for iterative message processing system | |
| US11703826B1 (en) | Monitoring asset hierarchies based on asset group metrics | |
| US11886440B1 (en) | Guided creation interface for streaming data processing pipelines | |
| US11614923B2 (en) | Dual textual/graphical programming interfaces for streaming data processing pipelines | |
| US11144608B2 (en) | Triggering generation of an accelerated data model summary for a data model | |
| US11394767B2 (en) | Central repository of configuration files and two-way replication of search node configuration files | |
| US9817867B2 (en) | Dynamically processing an event using an extensible data model | |
| JP6416194B2 (en) | Scalable analytic platform for semi-structured data | |
| US20120158655A1 (en) | Non-relational function-based data publication for relational data | |
| CN106815338A (en) | A kind of real-time storage of big data, treatment and inquiry system | |
| US20220121708A1 (en) | Dynamic data enrichment | |
| US11675816B1 (en) | Grouping evens into episodes using a streaming data processor | |
| US11676072B1 (en) | Interface for incorporating user feedback into training of clustering model | |
| US11573971B1 (en) | Search and data analysis collaboration system | |
| US11892976B2 (en) | Enhanced search performance using data model summaries stored in a remote data store | |
| US11210278B1 (en) | Asset group interface driven by search-derived asset tree hierarchy | |
| CN114218218A (en) | Data processing method, device and equipment based on data warehouse and storage medium | |
| CN114461490B (en) | Fortune dimension aggregation system | |
| US11841827B2 (en) | Facilitating generation of data model summaries | |
| US11989592B1 (en) | Workload coordinator for providing state credentials to processing tasks of a data processing pipeline | |
| US11983181B1 (en) | Data stream processing instruction set previews using batch conversion | |
| CN116910062A (en) | Metadata management system based on real-time calculation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |