CN116795882A - Data acquisition method, device, computer equipment and storage medium - Google Patents
Data acquisition method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN116795882A CN116795882A CN202310734454.0A CN202310734454A CN116795882A CN 116795882 A CN116795882 A CN 116795882A CN 202310734454 A CN202310734454 A CN 202310734454A CN 116795882 A CN116795882 A CN 116795882A
- Authority
- CN
- China
- Prior art keywords
- data
- interface
- configuration
- data acquisition
- cache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000003860 storage Methods 0.000 title claims abstract description 53
- 230000001960 triggered effect Effects 0.000 claims abstract description 19
- 238000010276 construction Methods 0.000 claims abstract description 6
- 238000012545 processing Methods 0.000 claims description 30
- 230000004048 modification Effects 0.000 claims description 27
- 238000012986 modification Methods 0.000 claims description 27
- 238000012216 screening Methods 0.000 claims description 16
- 238000004140 cleaning Methods 0.000 claims description 11
- 238000011161 development Methods 0.000 abstract description 18
- 230000008569 process Effects 0.000 abstract description 14
- 238000005516 engineering process Methods 0.000 abstract description 13
- 238000004891 communication Methods 0.000 description 6
- 238000013473 artificial intelligence Methods 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000010926 purge Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000001550 time effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application belongs to the field of big data, and relates to a data acquisition method, which comprises the following steps: judging whether a data acquisition request triggered by a user is received or not; if yes, responding to a data acquisition request, and displaying a preset interface configuration page; receiving interface configuration data input by a user on an interface configuration page; the interface configuration data comprises configuration indexes selected by a user and selection condition parameters corresponding to the configuration indexes; constructing a corresponding data acquisition interface based on the interface configuration data; acquiring target data corresponding to interface configuration data from a preset cache database through a data acquisition interface; and displaying the target data. The application also provides a data acquisition device, computer equipment and a storage medium. In addition, the application also relates to a blockchain technology, and target data can be stored in the blockchain. The application realizes the automatic, rapid and efficient construction of the required data acquisition interface, effectively improves the efficiency of interface generation and simplifies the process of interface development.
Description
Technical Field
The present application relates to the field of big data technologies, and in particular, to a data acquisition method, a data acquisition device, a computer device, and a storage medium.
Background
In the current era of data-based operation, the demand for interfaces by finance and technology companies is increasing. However, the development process corresponding to the interface development mode adopted in the current finance and science company takes a long time, a common interface development needs to be performed, a data developer converts the service requirement into the data requirement, the data developer performs cleaning processing according to the service logic, after repeated communication confirmation, the application developer performs interface development, a plurality of people are involved, and through N links, the period of one interface on line is at least in units of weeks, and the time effect is far insufficient for supporting the data application. For some complex scenes, the development efficiency of the interface is lower, and the market demand of rapid change is not met.
Disclosure of Invention
The embodiment of the application aims to provide a data acquisition method, a data acquisition device, computer equipment and a storage medium, so as to solve the technical problems that the development flow corresponding to the existing interface development mode takes long time and has low development efficiency.
In order to solve the above technical problems, the embodiment of the present application provides a data acquisition method, which adopts the following technical scheme:
judging whether a data acquisition request triggered by a user is received or not;
if yes, responding to the data acquisition request, and displaying a preset interface configuration page;
receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes;
constructing a corresponding data acquisition interface based on the interface configuration data;
acquiring target data corresponding to the interface configuration data from a preset cache database through the data acquisition interface;
and displaying the target data.
Further, the step of constructing a corresponding data acquisition interface based on the interface configuration data specifically includes:
calling a preset interface platform;
acquiring a universal interface from the interface platform;
performing configuration processing on the universal interface by using the interface configuration data to generate a corresponding configuration interface;
and taking the configuration interface as the data acquisition interface.
Further, the step of acquiring, by the data acquiring interface, target data corresponding to the interface configuration data from a preset cache database specifically includes:
acquiring first data corresponding to the configuration index from the cache database through the data acquisition interface;
screening second data corresponding to the selected condition parameters from the first data based on the selected condition parameters;
and taking the second data as the target data.
Further, the step of displaying the target data specifically includes:
acquiring a preset data report template;
replacing the report dimension of the data report template by using the configuration index and the selected condition parameter to generate a corresponding initial report;
filling the target data into the corresponding position in the initial report based on the configuration index, the data corresponding relation between the selected condition parameters and the target data, and processing to obtain a target report;
and displaying the target report.
Further, the data acquisition method further includes:
acquiring the available capacity of the cache database;
judging whether the cache database accords with a preset clearing condition or not based on the available capacity;
If yes, determining data to be cleaned in the cache database;
and cleaning the data to be cleaned in the cache database.
Further, the step of determining the data to be cleaned in the cache database specifically includes:
acquiring storage record information of all cache data stored in the cache database;
acquiring the storage duration of all the cache data based on the storage record information;
screening first cache data with storage time length longer than a preset time length threshold value from all the cache data; wherein, data except the first cache data in the cache data is called second cache data;
acquiring the data use frequency of the second cache data based on the storage record information;
screening third cache data of which the data use frequency is smaller than a preset frequency threshold value from all the second cache data;
and generating the data to be cleaned based on the first cache data and the third cache data.
Further, before the step of constructing the corresponding data acquisition interface based on the interface configuration data, the method further includes:
judging whether a modification instruction for the interface configuration data triggered by the user is received or not;
If yes, responding to the modification instruction and receiving modification information input by the user on the interface configuration page;
modifying the interface configuration data based on the modification information to obtain modified target interface configuration data;
and storing the target interface configuration data, and archiving and recording the interface configuration data.
In order to solve the above technical problems, the embodiment of the present application further provides a data acquisition device, which adopts the following technical scheme:
the first judging module is used for judging whether a data acquisition request triggered by a user is received or not;
the display module is used for responding to the data acquisition request and displaying a preset interface configuration page if yes;
the receiving module is used for receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes;
the construction module is used for constructing a corresponding data acquisition interface based on the interface configuration data;
the first acquisition module is used for acquiring target data corresponding to the interface configuration data from a preset cache database through the data acquisition interface;
And the display module is used for displaying the target data.
In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical schemes:
judging whether a data acquisition request triggered by a user is received or not;
if yes, responding to the data acquisition request, and displaying a preset interface configuration page;
receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes;
constructing a corresponding data acquisition interface based on the interface configuration data;
acquiring target data corresponding to the interface configuration data from a preset cache database through the data acquisition interface;
and displaying the target data.
In order to solve the above technical problems, an embodiment of the present application further provides a computer readable storage medium, which adopts the following technical schemes:
judging whether a data acquisition request triggered by a user is received or not;
if yes, responding to the data acquisition request, and displaying a preset interface configuration page;
receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes;
Constructing a corresponding data acquisition interface based on the interface configuration data;
acquiring target data corresponding to the interface configuration data from a preset cache database through the data acquisition interface;
and displaying the target data.
Compared with the prior art, the embodiment of the application has the following main beneficial effects:
the embodiment of the application firstly judges whether a data acquisition request triggered by a user is received or not; if yes, responding to the data acquisition request, and displaying a preset interface configuration page; then receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes; then constructing a corresponding data acquisition interface based on the interface configuration data; subsequently, acquiring target data corresponding to the interface configuration data from a preset cache database through the data acquisition interface; and finally, displaying the target data. According to the embodiment of the application, the required data acquisition interface is automatically and quickly constructed by carrying out corresponding configuration processing according to the interface configuration data input by the user, so that the efficiency of interface generation is effectively improved, the process of interface development is simplified, the workload of the user is reduced, and the working experience of the user is improved. In addition, the service data is preloaded into the cache database, so that the data acquisition interface can be directly called to acquire the data from the cache database later, the real-time query load pressure can be reduced, the data query is accelerated, and the experience of a user for checking target data is effectively improved.
Drawings
In order to more clearly illustrate the solution of the present application, a brief description will be given below of the drawings required for the description of the embodiments of the present application, it being apparent that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained from these drawings without the exercise of inventive effort for a person of ordinary skill in the art.
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow chart of one embodiment of a data acquisition method according to the present application;
FIG. 3 is a schematic diagram of an embodiment of a data acquisition device according to the present application;
FIG. 4 is a schematic structural diagram of one embodiment of a computer device in accordance with the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description of the application and the claims and the description of the drawings above are intended to cover a non-exclusive inclusion. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
In order to make the person skilled in the art better understand the solution of the present application, the technical solution of the embodiment of the present application will be clearly and completely described below with reference to the accompanying drawings.
As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as a web browser application, a shopping class application, a search class application, an instant messaging tool, a mailbox client, social platform software, etc., may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video expert compression standard audio plane 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic video expert compression standard audio plane 4) players, laptop and desktop computers, and the like.
The server 105 may be a server providing various services, such as a background server providing support for pages displayed on the terminal devices 101, 102, 103.
It should be noted that, the data acquisition method provided in the embodiment of the present application is generally executed by a server/terminal device, and accordingly, the data acquisition device is generally disposed in the server/terminal device.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
With continued reference to FIG. 2, a flow chart of one embodiment of a data acquisition method according to the present application is shown. The data acquisition method comprises the following steps:
Step S201, determining whether a data acquisition request triggered by a user is received.
In this embodiment, the electronic device (e.g., the server/terminal device shown in fig. 1) on which the data acquisition method operates may acquire the data acquisition request through a wired connection manner or a wireless connection manner. It should be noted that the wireless connection may include, but is not limited to, 3G/4G/5G connection, wiFi connection, bluetooth connection, wiMAX connection, zigbee connection, UWB (ultra wideband) connection, and other now known or later developed wireless connection. The user may issue a data acquisition request through a held client. The data acquisition request is a request for configuring a corresponding data acquisition interface for data acquisition.
Step S202, if yes, responding to the data acquisition request, and displaying a preset interface configuration page.
In this embodiment, the rule configuration interface at least includes a configuration index and a data selection list corresponding to a selected condition parameter.
Step S203, receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selected condition parameters corresponding to the configuration indexes.
In this embodiment, the configuration index includes a plurality of index classes, and the selection condition parameters may include parameters such as a statistics period, a dimension, a limiting condition (such as an algorithm, a condition), etc., which are specifically used to indicate that the index is bound with the data model, which field is taken from which table, what is the aggregation logic of the fields, and which analysis dimensions are supported by the index; when the front page is called, index codes are transmitted, condition parameters are limited, and data values can be returned. The interface configuration data can be generated by storing configuration information in a self-built data set mode after a user pulls indexes or corresponding selected condition parameters. Through dynamic filtering conditions, SQL is assembled in real time according to user requirements, a data set can be quickly created by dragging, SQL is not required to be written manually, a stack of parameters such as various indexes and selection conditions are not required to be transmitted, and an access party can quickly and simply call the parameters.
Step S204, constructing a corresponding data acquisition interface based on the interface configuration data.
In this embodiment, the data acquisition interface is a constructed interface for acquiring index data, and the data acquisition interface is used for outputting the index data in a service mode, so that after a data developer cleans the index data, an application developer can output the index data without having to know the table structure too much, the table field definition and the complex service data processing logic, and the service developer can complete the interface configuration online by self-service for the application end. The specific implementation process of constructing the corresponding data acquisition interface based on the interface configuration data will be described in further detail in the following specific embodiments, which are not described herein.
Step S205, obtaining, by the data obtaining interface, target data corresponding to the interface configuration data from a preset cache database.
In this embodiment, the cache database is specifically a redis database. By preloading service data into the cache database so that the subsequent data can be directly fetched from the cache database, the real-time query load pressure can be reduced, the data query is accelerated, and the experience of a user for viewing the visual index data is effectively improved. The specific implementation process of obtaining the target data corresponding to the interface configuration data from the preset cache database through the data obtaining interface will be described in further detail in the following specific embodiments, which will not be described herein. In addition, distributed query of multiple groups of dimension combination data can be supported, so that interface timeliness is guaranteed. The data in the tables can be spliced into a unified data set and cached, so that the data can be shared conveniently, and the chimney type development work can be reduced.
Step S206, displaying the target data.
In this embodiment, the foregoing shows a specific implementation procedure of the target data, and this will be described in further detail in the following specific embodiments, which are not described herein. The data display can support application scenes such as simple index cards, index signboards and the like, can construct complex operation scenes, and finally outputs standard interface service or batch data output taking a user-defined data set ID as a dimension, and directly contacts an application-end channel.
Firstly judging whether a data acquisition request triggered by a user is received or not; if yes, responding to the data acquisition request, and displaying a preset interface configuration page; then receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes; then constructing a corresponding data acquisition interface based on the interface configuration data; subsequently, acquiring target data corresponding to the interface configuration data from a preset cache database through the data acquisition interface; and finally, displaying the target data. According to the application, the required data acquisition interface is automatically and quickly constructed by carrying out corresponding configuration processing according to the interface configuration data input by the user, so that the efficiency of interface generation is effectively improved, the process of interface development is simplified, the workload of the user is reduced, and the working experience of the user is improved. In addition, the service data is preloaded into the cache database, so that the data acquisition interface can be directly called to acquire the data from the cache database later, the real-time query load pressure can be reduced, the data query is accelerated, and the experience of a user for checking target data is effectively improved.
In some alternative implementations, step S204 includes the steps of:
and calling a preset interface platform.
In this embodiment, the interface platform is a generic interface that is built in advance and stores a plurality of built interfaces, and can perform configuration processing.
And acquiring a universal interface from the interface platform.
And carrying out configuration processing on the universal interface by using the interface configuration data to generate a corresponding configuration interface.
In this embodiment, the generic interface in the interface platform may be called, and corresponding configuration processing may be performed on the generic interface according to interface configuration data input by a user, so that a corresponding interface may be generated. Through the interface construction mode based on the universal interface and the interface configuration data, dependence of customized development on different personnel can be reduced, development links, parameter configuration and communication cost are reduced, index service is unified, data is packaged, an interface is generated rapidly, development period is shortened, and the application channel is directly reached.
And taking the configuration interface as the data acquisition interface.
The application calls the preset interface platform; then obtaining a universal interface from the interface platform; then, the interface configuration data is used for carrying out configuration processing on the universal interface to generate a corresponding configuration interface; and subsequently taking the configuration interface as the data acquisition interface. The application is based on the use of the universal interface in the interface platform, and can carry out corresponding configuration processing on the universal interface according to the interface configuration data input by a user, so as to realize automatic, rapid and efficient construction of the required data acquisition interface, effectively improve the efficiency of interface generation, reduce the workload of the user and improve the working experience of the user.
In some alternative implementations of the present embodiment, step S205 includes the steps of:
and acquiring first data corresponding to the configuration index from the cache database through the data acquisition interface.
In this embodiment, the data matched with the interface configuration data can be quickly read from the cache database by calling the data acquisition interface.
And screening second data corresponding to the selected condition parameters from the first data based on the selected condition parameters.
And taking the second data as the target data.
The application acquires the first data corresponding to the configuration index from the cache database through the data acquisition interface; then, based on the selected condition parameters, second data corresponding to the selected condition parameters are screened out from the first data; and taking the second data as the target data. The application can realize rapid acquisition of the target data matched with the interface configuration data from the cache database based on the call of the generated data acquisition interface. In addition, the service data is preloaded into the cache database so that the data can be directly fetched from the cache database through the data acquisition interface, thus the real-time query load pressure can be reduced, the data query is accelerated, and the experience of a user for viewing the visual index data is effectively improved.
In some alternative implementations, step S206 includes the steps of:
and acquiring a preset data report template.
In this embodiment, the data report template is a report template that is pre-constructed according to actual service display requirements.
And replacing the report dimension of the data report template by using the configuration index and the selected condition parameter to generate a corresponding initial report.
In this embodiment, the configuration index may be used to replace data in the report row header in the data report template, and the selection condition parameter may be used to replace data in the report column header in the data report template, so as to generate a corresponding initial report.
And filling the target data into the corresponding position in the initial report based on the configuration index and the data corresponding relation between the selected condition parameters and the target data, and processing to obtain a target report.
And displaying the target report.
In this embodiment, a visual page may be created on the current interface, and then the target report may be displayed in the visual page.
The method comprises the steps of obtaining a preset data report template; then replacing the report dimension of the data report template by using the configuration index and the selected condition parameter to generate a corresponding initial report; then filling the target data into the corresponding position in the initial report based on the configuration index, the data corresponding relation between the selected condition parameter and the target data, and processing to obtain a target report; and displaying the target report. According to the application, when the target data is acquired based on the call of the data acquisition interface, the preset data report template is subjected to data filling processing based on the target data and the interface configuration data intelligently to generate the target report and display, so that standard data arrangement and display of the target data are automatically performed by using the data report template, the user's consulting experience is improved, and the user's working efficiency is improved.
In some alternative implementations, the electronic device may further perform the steps of:
and obtaining the available capacity of the cache database.
In this embodiment, the current available capacity of the cache database may be obtained from the stored information by referring to the stored information of the storage area.
And judging whether the cache database accords with a preset clearing condition or not based on the available capacity.
In this embodiment, the above-mentioned clearing condition refers to whether the available capacity of the cache database is smaller than a preset capacity threshold. The value of the capacity threshold is not particularly limited, and may be generated according to an actual usage test result. If the current available capacity of the cache database is smaller than the capacity threshold, the current available resource space of the cache database is insufficient, and the normal operation of the data in the cache database is affected.
If yes, determining the data to be cleaned in the cache database.
In this embodiment, the above specific implementation process of determining the data to be cleaned in the cache database will be described in further detail in the following specific embodiments, which will not be described herein.
And cleaning the data to be cleaned in the cache database.
In this embodiment, a preset cleaning time period may be obtained, and when the current time reaches the cleaning time period, the process of cleaning the data to be cleaned in the cache database is performed. By performing the cleaning process of the cache data by prescribing the cleaning period, the cleaning normalization of the cache data can be improved.
The application obtains the available capacity of the cache database; then, based on the available capacity, judging whether the cache database accords with a preset clearing condition; if yes, determining data to be cleaned in the cache database; and subsequently, cleaning the data to be cleaned in the cache database. After the data storage is carried out by using the cache database, the method and the device can intelligently judge whether the cache database meets the preset clearing condition in real time, and if so, the data to be cleaned in the cache database can be intelligently cached for clearing processing in the follow-up process so as to ensure that the cache database can have sufficient available resource capacity and be beneficial to improving the stability of data operation in the cache database.
In some optional implementations of this embodiment, the determining the data to be cleaned in the cache database includes the following steps:
and acquiring storage record information of all cache data stored in the cache database.
In this embodiment, when the cache data is stored in the cache database, the storage information of the cache data, such as the storage time and the number of times of data use, is stored in statistics in synchronization.
And acquiring the storage duration of all the cache data based on the storage record information.
In this embodiment, the storage time of the cache data may be obtained from the storage record information, and then the difference between the current time and the storage time may be calculated to obtain the storage duration of the cache data.
Screening first cache data with storage time length longer than a preset time length threshold value from all the cache data; wherein, the data except the first cache data in the cache data is called second cache data.
In this embodiment, the value of the duration threshold is not specifically limited, and may be set according to actual use requirements.
And acquiring the data use frequency of the second cache data based on the storage record information.
And screening third cache data of which the data use frequency is smaller than a preset frequency threshold value from all the second cache data.
In this embodiment, the value of the data usage frequency is not specifically limited, and may be set according to actual usage requirements.
And generating the data to be cleaned based on the first cache data and the third cache data.
In this embodiment, the first cache data and the third cache data may be integrated to obtain integrated target cache data, and the target cache data may be used as the data to be cleaned.
The method comprises the steps of obtaining storage record information of all cache data stored in the cache database; then, based on the storage record information, acquiring storage time lengths of all the cache data; screening first cache data with storage time length longer than a preset time length threshold value from all the cache data; then, based on the storage record information, acquiring the data use frequency of the second cache data; subsequently screening third cache data of which the data use frequency is smaller than a preset frequency threshold value from all the second cache data; and finally, generating the data to be cleaned based on the first cache data and the third cache data. According to the method and the device for searching the storage record information of all the cache data stored in the cache database, the storage time length and the data use frequency of the cache data are analyzed and screened, so that the data which are overlong in storage time and lower in data use frequency can be rapidly and accurately determined from the cache database and used as the data to be cleaned, and the intelligence and the data accuracy of the data to be cleaned are ensured.
In some optional implementations of this embodiment, before step S204, the electronic device may further perform the following steps:
and judging whether a modification instruction for the interface configuration data triggered by the user is received or not.
In this embodiment, the modification instruction may be triggered after the user clicks a data modification button in the interface configuration page.
If yes, responding to the modification instruction, and receiving modification information input by the user on the interface configuration page.
And modifying the interface configuration data based on the modification information to obtain modified target interface configuration data.
In this embodiment, by providing the function of intelligent index configuration, the user needs to change the index configuration, and directly modify the configuration on the page without deploying a version, so as to effectively reduce the pressure of the access party and improve the working efficiency.
And storing the target interface configuration data, and archiving and recording the interface configuration data.
In this embodiment, by archiving the initial interface configuration data, the subsequent trace-back processing of the interface configuration can be performed by calling the archiving record, so that the user is not required to perform reconfiguration, and the processing intelligence of the interface configuration is improved.
The application judges whether a modification instruction for the interface configuration data triggered by the user is received or not; if yes, responding to the modification instruction and receiving modification information input by the user on the interface configuration page; then modifying the interface configuration data based on the modification information to obtain modified target interface configuration data; and storing the target interface configuration data later, and archiving and recording the interface configuration data. After the interface configuration page finishes generating the interface configuration data, the application can also carry out modification processing on the interface configuration data in the interface configuration page according to the personal requirement of the user, and the modification mode does not need the change and the edition of background code logic, thereby effectively improving the intelligence and the convenience of the configuration of the interface configuration data, greatly reducing the development work of developers and being beneficial to improving the working efficiency and the working physical examination of the user.
It is emphasized that the target data may also be stored in a blockchain node in order to further ensure the privacy and security of the target data.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, encryption algorithm and the like. The Blockchain (Blockchain), which is essentially a decentralised database, is a string of data blocks that are generated by cryptographic means in association, each data block containing a batch of information of network transactions for verifying the validity of the information (anti-counterfeiting) and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.
Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a robot technology, a biological recognition technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by computer readable instructions stored in a computer readable storage medium that, when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
With further reference to fig. 3, as an implementation of the method shown in fig. 2 described above, the present application provides an embodiment of a data acquisition apparatus, which corresponds to the method embodiment shown in fig. 2, and which is particularly applicable to various electronic devices.
As shown in fig. 3, the data acquisition device 300 according to the present embodiment includes: a loading module 301, a calling module 302, a collecting module 303, a generating module 304 and a processing module 305. Wherein:
A first determining module 301, configured to determine whether a data acquisition request triggered by a user is received;
the display module 302 is configured to respond to the data acquisition request if yes, and display a preset interface configuration page;
a receiving module 303, configured to receive interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes;
a construction module 304, configured to construct a corresponding data acquisition interface based on the interface configuration data;
a first obtaining module 305, configured to obtain, through the data obtaining interface, target data corresponding to the interface configuration data from a preset cache database;
and the display module 306 is used for displaying the target data.
In this embodiment, the operations performed by the modules or units respectively correspond to the steps of the data acquisition method in the foregoing embodiment one by one, and are not described herein again.
In some alternative implementations of the present embodiment, the building block 304 includes:
the calling sub-module is used for calling a preset interface platform;
the first acquisition sub-module is used for acquiring a universal interface from the interface platform;
The first generation sub-module is used for carrying out configuration processing on the universal interface by using the interface configuration data to generate a corresponding configuration interface;
and the first determining submodule is used for taking the configuration interface as the data acquisition interface.
In this embodiment, the operations performed by the modules or units respectively correspond to the steps of the data acquisition method in the foregoing embodiment one by one, and are not described herein again.
In some optional implementations of this embodiment, the first obtaining module 305 includes:
the second acquisition sub-module is used for acquiring first data corresponding to the configuration index from the cache database through the data acquisition interface;
the first screening submodule is used for screening second data corresponding to the selected condition parameters from the first data based on the selected condition parameters;
and the second determining submodule is used for taking the second data as the target data.
In this embodiment, the operations performed by the modules or units respectively correspond to the steps of the data acquisition method in the foregoing embodiment one by one, and are not described herein again.
In some alternative implementations of the present embodiment, the presentation module 306 includes:
The third acquisition sub-module is used for acquiring a preset data report template;
the replacing sub-module is used for replacing the report dimension of the data report template by using the configuration index and the selected condition parameter to generate a corresponding initial report;
the second generation sub-module is used for filling the target data into the corresponding position in the initial report based on the configuration index, the data corresponding relation between the selected condition parameters and the target data and processing the corresponding position to obtain a target report;
and the display sub-module is used for displaying the target report.
In this embodiment, the operations performed by the modules or units respectively correspond to the steps of the data acquisition method in the foregoing embodiment one by one, and are not described herein again.
In some optional implementations of this embodiment, the data acquisition device further includes:
the second acquisition module is used for acquiring the available capacity of the cache database;
the second judging module is used for judging whether the cache database accords with a preset clearing condition or not based on the available capacity;
the determining module is used for determining the data to be cleaned in the cache database if yes;
and the clearing module is used for clearing the data to be cleaned in the cache database.
In this embodiment, the operations performed by the modules or units respectively correspond to the steps of the data acquisition method in the foregoing embodiment one by one, and are not described herein again.
In some optional implementations of this embodiment, the purge module includes:
a fourth obtaining sub-module, configured to obtain storage record information of all cache data stored in the cache database;
a fifth obtaining sub-module, configured to obtain, based on the storage record information, storage durations of all the cache data;
the second screening sub-module is used for screening first cache data with the storage time length longer than a preset time length threshold value from all the cache data; wherein, data except the first cache data in the cache data is called second cache data;
a sixth obtaining sub-module, configured to obtain a data usage frequency of the second cache data based on the storage record information;
a third screening sub-module, configured to screen third cache data whose data usage frequency is less than a preset frequency threshold from all the second cache data;
and the third generation sub-module is used for generating the data to be cleaned based on the first cache data and the third cache data.
In this embodiment, the operations performed by the modules or units respectively correspond to the steps of the data acquisition method in the foregoing embodiment one by one, and are not described herein again.
In some optional implementations of this embodiment, the data acquisition device further includes:
the third judging module is used for judging whether a modification instruction for the interface configuration data triggered by the user is received or not;
the response module is used for responding to the modification instruction and receiving modification information input by the user on the interface configuration page if the modification instruction is yes;
the modification module is used for modifying the interface configuration data based on the modification information to obtain modified target interface configuration data;
and the storage module is used for storing the target interface configuration data and archiving and recording the interface configuration data.
In this embodiment, the operations performed by the modules or units respectively correspond to the steps of the data acquisition method in the foregoing embodiment one by one, and are not described herein again.
In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 4, fig. 4 is a basic structural block diagram of a computer device according to the present embodiment.
The computer device 4 comprises a memory 41, a processor 42, a network interface 43 communicatively connected to each other via a system bus. It should be noted that only computer device 4 having components 41-43 is shown in the figures, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculations and/or information processing in accordance with predetermined or stored instructions, the hardware of which includes, but is not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.
The computer equipment can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing equipment. The computer equipment can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.
The memory 41 includes at least one type of readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the storage 41 may be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the computer device 4. Of course, the memory 41 may also comprise both an internal memory unit of the computer device 4 and an external memory device. In this embodiment, the memory 41 is typically used to store an operating system and various application software installed on the computer device 4, such as computer readable instructions of a data acquisition method. Further, the memory 41 may be used to temporarily store various types of data that have been output or are to be output.
The processor 42 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is configured to execute computer readable instructions stored in the memory 41 or process data, such as computer readable instructions for executing the data acquisition method.
The network interface 43 may comprise a wireless network interface or a wired network interface, which network interface 43 is typically used for establishing a communication connection between the computer device 4 and other electronic devices.
Compared with the prior art, the embodiment of the application has the following main beneficial effects:
in the embodiment of the application, whether a data acquisition request triggered by a user is received is firstly judged; if yes, responding to the data acquisition request, and displaying a preset interface configuration page; then receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes; then constructing a corresponding data acquisition interface based on the interface configuration data; subsequently, acquiring target data corresponding to the interface configuration data from a preset cache database through the data acquisition interface; and finally, displaying the target data. According to the embodiment of the application, the required data acquisition interface is automatically and quickly constructed by carrying out corresponding configuration processing according to the interface configuration data input by the user, so that the efficiency of interface generation is effectively improved, the process of interface development is simplified, the workload of the user is reduced, and the working experience of the user is improved. In addition, the service data is preloaded into the cache database, so that the data acquisition interface can be directly called to acquire the data from the cache database later, the real-time query load pressure can be reduced, the data query is accelerated, and the experience of a user for checking target data is effectively improved.
The present application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the data acquisition method as described above.
Compared with the prior art, the embodiment of the application has the following main beneficial effects:
in the embodiment of the application, whether a data acquisition request triggered by a user is received is firstly judged; if yes, responding to the data acquisition request, and displaying a preset interface configuration page; then receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes; then constructing a corresponding data acquisition interface based on the interface configuration data; subsequently, acquiring target data corresponding to the interface configuration data from a preset cache database through the data acquisition interface; and finally, displaying the target data. According to the embodiment of the application, the required data acquisition interface is automatically and quickly constructed by carrying out corresponding configuration processing according to the interface configuration data input by the user, so that the efficiency of interface generation is effectively improved, the process of interface development is simplified, the workload of the user is reduced, and the working experience of the user is improved. In addition, the service data is preloaded into the cache database, so that the data acquisition interface can be directly called to acquire the data from the cache database later, the real-time query load pressure can be reduced, the data query is accelerated, and the experience of a user for checking target data is effectively improved.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present application.
It is apparent that the above-described embodiments are only some embodiments of the present application, but not all embodiments, and the preferred embodiments of the present application are shown in the drawings, which do not limit the scope of the patent claims. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a thorough and complete understanding of the present disclosure. Although the application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing description, or equivalents may be substituted for elements thereof. All equivalent structures made by the content of the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the scope of the application.
Claims (10)
1. A method of data acquisition comprising the steps of:
judging whether a data acquisition request triggered by a user is received or not;
if yes, responding to the data acquisition request, and displaying a preset interface configuration page;
receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes;
constructing a corresponding data acquisition interface based on the interface configuration data;
acquiring target data corresponding to the interface configuration data from a preset cache database through the data acquisition interface;
and displaying the target data.
2. The method according to claim 1, wherein the step of constructing a corresponding data acquisition interface based on the interface configuration data specifically includes:
calling a preset interface platform;
acquiring a universal interface from the interface platform;
performing configuration processing on the universal interface by using the interface configuration data to generate a corresponding configuration interface;
and taking the configuration interface as the data acquisition interface.
3. The method for acquiring data according to claim 1, wherein the step of acquiring, by the data acquisition interface, target data corresponding to the interface configuration data from a preset cache database specifically includes:
acquiring first data corresponding to the configuration index from the cache database through the data acquisition interface;
screening second data corresponding to the selected condition parameters from the first data based on the selected condition parameters;
and taking the second data as the target data.
4. The method according to claim 1, wherein the step of displaying the target data specifically comprises:
acquiring a preset data report template;
replacing the report dimension of the data report template by using the configuration index and the selected condition parameter to generate a corresponding initial report;
filling the target data into the corresponding position in the initial report based on the configuration index, the data corresponding relation between the selected condition parameters and the target data, and processing to obtain a target report;
and displaying the target report.
5. The data acquisition method of claim 1, wherein the data acquisition method further comprises:
acquiring the available capacity of the cache database;
judging whether the cache database accords with a preset clearing condition or not based on the available capacity;
if yes, determining data to be cleaned in the cache database;
and cleaning the data to be cleaned in the cache database.
6. The method for obtaining data according to claim 5, wherein the step of determining the data to be cleaned in the cache database specifically comprises:
acquiring storage record information of all cache data stored in the cache database;
acquiring the storage duration of all the cache data based on the storage record information;
screening first cache data with storage time length longer than a preset time length threshold value from all the cache data; wherein, data except the first cache data in the cache data is called second cache data;
acquiring the data use frequency of the second cache data based on the storage record information;
screening third cache data of which the data use frequency is smaller than a preset frequency threshold value from all the second cache data;
And generating the data to be cleaned based on the first cache data and the third cache data.
7. The data acquisition method according to claim 1, further comprising, prior to the step of constructing a corresponding data acquisition interface based on the interface configuration data:
judging whether a modification instruction for the interface configuration data triggered by the user is received or not;
if yes, responding to the modification instruction and receiving modification information input by the user on the interface configuration page;
modifying the interface configuration data based on the modification information to obtain modified target interface configuration data;
and storing the target interface configuration data, and archiving and recording the interface configuration data.
8. A data acquisition device, comprising:
the first judging module is used for judging whether a data acquisition request triggered by a user is received or not;
the display module is used for responding to the data acquisition request and displaying a preset interface configuration page if yes;
the receiving module is used for receiving interface configuration data input by the user on the interface configuration page; the interface configuration data comprises configuration indexes selected by the user and selection condition parameters corresponding to the configuration indexes;
The construction module is used for constructing a corresponding data acquisition interface based on the interface configuration data;
the first acquisition module is used for acquiring target data corresponding to the interface configuration data from a preset cache database through the data acquisition interface;
and the display module is used for displaying the target data.
9. A computer device comprising a memory having stored therein computer readable instructions which when executed by a processor implement the steps of the data acquisition method of any of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer readable instructions which when executed by a processor implement the steps of the data acquisition method according to any one of claims 1 to 7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310734454.0A CN116795882A (en) | 2023-06-19 | 2023-06-19 | Data acquisition method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310734454.0A CN116795882A (en) | 2023-06-19 | 2023-06-19 | Data acquisition method, device, computer equipment and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116795882A true CN116795882A (en) | 2023-09-22 |
Family
ID=88049118
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310734454.0A Pending CN116795882A (en) | 2023-06-19 | 2023-06-19 | Data acquisition method, device, computer equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116795882A (en) |
-
2023
- 2023-06-19 CN CN202310734454.0A patent/CN116795882A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN116661936A (en) | Page data processing method and device, computer equipment and storage medium | |
CN116450723A (en) | Data extraction method, device, computer equipment and storage medium | |
CN116383787A (en) | Page creation method, page creation device, computer equipment and storage medium | |
CN116028446A (en) | Time sequence data file management method, device, equipment and storage medium thereof | |
CN116795882A (en) | Data acquisition method, device, computer equipment and storage medium | |
CN116820443A (en) | Data analysis method, device, computer equipment and storage medium | |
CN116842011A (en) | Blood relationship analysis method, device, computer equipment and storage medium | |
CN116738084A (en) | Buried data processing method, buried data processing device, computer equipment and storage medium | |
CN117421207A (en) | Intelligent evaluation influence point test method, intelligent evaluation influence point test device, computer equipment and storage medium | |
CN116821493A (en) | Message pushing method, device, computer equipment and storage medium | |
CN116661763A (en) | Front-end and back-end development management method and device, computer equipment and storage medium | |
CN117251502A (en) | Data billboard generation method and device, computer equipment and storage medium | |
CN118035124A (en) | Use case generation method, device, computer equipment and storage medium | |
CN116932486A (en) | File generation method, device, computer equipment and storage medium | |
CN116775186A (en) | Page data processing method and device, computer equipment and storage medium | |
CN116737437A (en) | Data analysis method, device, computer equipment and storage medium | |
CN116860644A (en) | Automatic test method, device, computer equipment and storage medium | |
CN116611936A (en) | Data analysis method, device, computer equipment and storage medium | |
CN115546356A (en) | Animation generation method and device, computer equipment and storage medium | |
CN116701488A (en) | Data processing method, device, computer equipment and storage medium | |
CN117034173A (en) | Data processing method, device, computer equipment and storage medium | |
CN116643884A (en) | Data computing method, device, equipment and storage medium based on rule engine | |
CN117785832A (en) | Log processing method, device, computer equipment and storage medium | |
CN116450724A (en) | Data processing method, device, computer equipment and storage medium | |
CN116662418A (en) | Report realization method, device and equipment based on configuration and storage medium thereof |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |