CN116303608A - Data processing method and device for application service - Google Patents

Data processing method and device for application service Download PDF

Info

Publication number
CN116303608A
CN116303608A CN202310252445.8A CN202310252445A CN116303608A CN 116303608 A CN116303608 A CN 116303608A CN 202310252445 A CN202310252445 A CN 202310252445A CN 116303608 A CN116303608 A CN 116303608A
Authority
CN
China
Prior art keywords
data
storage
application service
key
stored
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310252445.8A
Other languages
Chinese (zh)
Inventor
许鹏翔
黄璐
侯园松
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
CCB Finetech Co Ltd
Original Assignee
China Construction Bank Corp
CCB Finetech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp, CCB Finetech Co Ltd filed Critical China Construction Bank Corp
Priority to CN202310252445.8A priority Critical patent/CN116303608A/en
Publication of CN116303608A publication Critical patent/CN116303608A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24552—Database cache management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a data processing method and device of application service, and relates to the technical field of big data. One embodiment of the method comprises the following steps: receiving a data storage request sent by an application service based on a distributed cache client; the data storage request comprises an application code and data to be stored, wherein the application code corresponds to an application service; determining a data type corresponding to data to be stored, inquiring a mapping relation table of the data type and a storage key, and determining the storage key corresponding to the data type; storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a storage key and an application code. According to the embodiment, the utilization rate of storage resources is improved, the data processing efficiency and accuracy are improved, and the user experience is improved.

Description

Data processing method and device for application service
Technical Field
The present invention relates to the field of big data technologies, and in particular, to a data processing method and apparatus for an application service.
Background
With the development of internet technology, in a multi-user and high-concurrency information system, due to the fact that the instantaneous flow peak value is high, great pressure is caused on a database, and the limitation of the reading efficiency of the database gradually becomes a bottleneck of the system.
In the related art, a distributed cache is used to cache data before accessing a database. The data reading rate and the system concurrency can be improved by using the distributed cache, and the database pressure is relieved, so that the flow peak clipping effect is achieved. However, at the same time, because of more application services accessed into the distributed cache, if the distributed cache sub-clusters are provided for each application service independently, larger storage resource waste can be caused, and the rationalization and utilization of the storage resource are not utilized; if the shared distributed cache cluster is provided for a plurality of application services, storage key conflict often occurs, so that subsequent data query operation is affected, the data processing efficiency and accuracy are low, and user experience is also affected.
Disclosure of Invention
In view of this, the embodiment of the invention provides a data processing method and device for application services, which can construct a storage key by combining application codes of each application service when distributed cache storage is performed, so that the problem of storage key conflict when a plurality of application services share a distributed cache cluster is avoided, the utilization rate of storage resources is improved, the data processing efficiency and accuracy are improved, and the user experience is improved.
To achieve the above object, according to an aspect of the embodiments of the present invention, there is provided a data processing method for an application service, applied to a distributed cache server, including:
receiving a data storage request sent by an application service based on a distributed cache client; the data storage request comprises an application code and data to be stored, wherein the application code corresponds to an application service;
determining a data type corresponding to data to be stored, inquiring a mapping relation table of the data type and a storage key, and determining the storage key corresponding to the data type;
storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a storage key and an application code.
Further, before the step of receiving the data storage request sent by the application service based on the distributed cache client, the method further comprises:
receiving a registration request of at least one application service, and performing registration processing on the at least one application service;
and implanting the distributed cache client in the application service with successful registration processing, and configuring the application code which is uniquely corresponding to the application service in the distributed cache client.
Further, the method further comprises a step of constructing a mapping relation table of the data types and the storage keys, and the method comprises the following steps:
Acquiring historical storage data and key value pair information corresponding to the historical storage data, and determining the corresponding relation between the data type of the historical storage data and a storage key in the key value pair information;
constructing a mapping relation table of the data types and the storage keys according to the corresponding relation; wherein each data type corresponds to a unique storage key form.
Further, the method further comprises:
receiving fault prompt information, and determining an application code corresponding to a fault storage key according to the fault storage key indicated by the fault prompt information;
and suspending the storage request of the application service corresponding to the application code.
Further, the method further comprises:
and responding to the data monitoring request of the application service, monitoring target storage data containing the target application code in the storage key according to the target application code indicated by the data monitoring request, and recording storage information and access information of the target storage data.
Further, after the step of storing the data to be stored in the distributed cache server in the form of key-value pairs, the method further includes:
and sending the data to be stored to a local database for persistence.
Further, the method further comprises:
and receiving a data query request, taking a storage key and an application code indicated in the data query request as indexes, determining query data corresponding to the index key from the distributed cache server, and feeding back the query data to the application service.
Further, if query data corresponding to the index key is not queried at the distributed cache server, the method further includes:
forwarding the data query request to a local database, and searching query data in the local database.
According to still another aspect of an embodiment of the present invention, there is provided a data processing apparatus for an application service, including:
the receiving module is used for receiving a data storage request sent by the application service based on the distributed cache client; the data storage request comprises an application code and data to be stored, wherein the application code corresponds to an application service;
the storage key determining module is used for determining the data type corresponding to the data to be stored, inquiring a mapping relation table of the data type and the storage key and determining the storage key corresponding to the data type;
the data processing module is used for storing the data to be stored to the distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a storage key and an application code.
Further, the apparatus further comprises a configuration module, before the step of receiving the data storage request sent by the application service based on the distributed cache client, the configuration module is configured to:
receiving a registration request of at least one application service, and performing registration processing on the at least one application service;
And implanting the distributed cache client in the application service with successful registration processing, and configuring the application code which is uniquely corresponding to the application service in the distributed cache client.
According to another aspect of an embodiment of the present invention, there is provided an electronic device for data processing of an application service, including:
one or more processors;
storage means for storing one or more programs,
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement a data processing method for an application service as described above.
According to still another aspect of an embodiment of the present invention, there is provided a computer-readable medium having stored thereon a computer program which, when executed by a processor, implements a data processing method of any of the application services described above.
To achieve the above object, according to still another aspect of an embodiment of the present invention, there is provided a computer program product; a computer program product according to an embodiment of the present invention includes a computer program that when executed by a processor implements a data processing method of any one of the application services provided in the above embodiment of the present invention.
One embodiment of the above invention has the following advantages or benefits: because the data storage request sent by the application receiving service based on the distributed cache client is adopted; the data storage request comprises an application code and data to be stored, wherein the application code corresponds to an application service; determining a data type corresponding to data to be stored, inquiring a mapping relation table of the data type and a storage key, and determining the storage key corresponding to the data type; storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a technical means of storage keys and application codes, so that the technical problems of low storage resource utilization rate, low data processing efficiency and accuracy and poor user experience in the related art are solved, and further the technical effects that the storage keys can be built by combining the application codes of each application service when distributed cache storage is carried out, the problem of storage key conflict when a plurality of application services share a distributed cache cluster is avoided, the utilization rate of storage resources is improved, the data processing efficiency and accuracy are improved, and the user experience is improved are achieved.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main flow of a data processing method of an application service provided according to one embodiment of the present invention;
FIG. 2 is a schematic diagram of the main flow of a data processing method of an application service according to another embodiment of the present invention;
FIG. 3 is a schematic diagram of the main flow of a data processing method of an application service according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of the main modules of a data processing apparatus for application services provided according to an embodiment of the present invention;
FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
fig. 6 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the technical scheme of the invention, the aspects of acquisition, analysis, use, transmission, storage and the like of the related user personal information all meet the requirements of related laws and regulations, are used for legal and reasonable purposes, are not shared, leaked or sold outside the aspects of legal use and the like, and are subjected to supervision and management of a supervision department. Necessary measures should be taken for the personal information of the user to prevent illegal access to such personal information data, ensure that personnel having access to the personal information data comply with the regulations of the relevant laws and regulations, and ensure the personal information of the user. Once these user personal information data are no longer needed, the risk should be minimized by limiting or even prohibiting the data collection and/or deletion.
User privacy is protected, when applicable, by de-identifying the data, including in some related applications, such as by removing a particular identifier (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at a city level rather than at a specific address level), controlling how the data is stored, and/or other methods.
FIG. 1 is a schematic diagram of the main flow of a data processing method of an application service provided according to one embodiment of the present invention; as shown in fig. 1, the data processing method for an application service provided by the embodiment of the present invention mainly includes:
step S101, receiving a data storage request sent by an application service based on a distributed cache client; the data storage request comprises application codes corresponding to the application service and data to be stored.
Specifically, according to an embodiment of the present invention, before the step of receiving the data storage request sent by the application service based on the distributed cache client, the method further includes:
receiving a registration request of at least one application service, and performing registration processing on the at least one application service;
and implanting the distributed cache client in the application service with successful registration processing, and configuring the application code which is uniquely corresponding to the application service in the distributed cache client.
By the arrangement, a unique application code is set for each application service, so that the application code is added to the keys in the key value pair when the data of the application service are stored in the key value pair later. According to a specific implementation of the embodiment of the present invention, the application code may be used as a prefix or suffix to the key in the key value pair.
Step S102, determining a data type corresponding to the data to be stored, inquiring a mapping relation table of the data type and the storage key, and determining the storage key corresponding to the data type.
Through the arrangement, the storage keys corresponding to the data to be stored can be rapidly determined by combining the mapping relation table, so that the storage efficiency is improved, and meanwhile, the conflict of the storage keys is avoided.
Further, according to an embodiment of the present invention, the method further includes a step of constructing a mapping relationship table of data types and storage keys, including:
acquiring historical storage data and key value pair information corresponding to the historical storage data, and determining the corresponding relation between the data type of the historical storage data and a storage key in the key value pair information;
constructing a mapping relation table of the data types and the storage keys according to the corresponding relation; wherein each data type corresponds to a unique storage key form.
Through the setting, corresponding storage key forms can be set for each data type according to the historical storage habit of the user. It should be noted that the above setting is not limited to this application, and the storage key format corresponding to each data type may be configured according to the setting requirement of the user.
According to a specific implementation manner of the embodiment of the present invention, in the mapping relationship table, if the data type is a user name, the corresponding storage key is generally a userName; if the data type is a client number, the corresponding storage key is typically customerNo. After the shared distributed cache server-side is in butt joint with a plurality of application services, if the data stored by each application service adopts the key setting mode in the mapping relation table, the situation of key conflict often occurs. Thus, a combination of the storage key and the application code is used as a key, and the data to be stored is used as a key value. The application code app_id may be used as a key prefix (the combined key form is app id userName, appID _customerno, etc.) or as a key suffix (the combined key form is userName appID, customerNo _appid, etc.).
Step S103, storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a storage key and an application code.
By the arrangement, when the data to be stored is stored in the shared distributed cache server, the application code can be combined with the storage key to serve as a key in the key value pair, and the data to be stored serves as a key value. Through the arrangement, the problem of storage key conflict when a plurality of application services share the distributed cache cluster is avoided, meanwhile, when a subsequent user inquires data, the data processing efficiency and accuracy are guaranteed, and the user experience is improved.
Further, according to an embodiment of the present invention, the method further includes:
receiving fault prompt information, and determining an application code corresponding to a fault storage key according to the fault storage key indicated by the fault prompt information;
and suspending the storage request of the application service corresponding to the application code.
When the distributed cache server side stores data, the keys in the key value pair comprise application codes, so that when a fault prompt is received, corresponding application service can be determined according to the application codes, a storage request of the application service is stopped, and the influence on storage and query service of the distributed cache server side is avoided. After the fault is removed, the storage request of the application service can be continuously received, and the corresponding storage service is carried out.
Preferably, according to an embodiment of the present invention, the method further includes:
and responding to the data monitoring request of the application service, monitoring target storage data containing the target application code in the storage key according to the target application code indicated by the data monitoring request, and recording storage information and access information of the target storage data.
Through the arrangement, unified and subdivision management of the distributed cache server to the accessed application service is realized, and further, the monitoring and maintenance can be performed in a targeted manner, the application scene of data processing is expanded, and the maintenance cost is reduced.
Illustratively, according to an embodiment of the present invention, after the step of storing the data to be stored in the distributed cache server in the form of key-value pairs, the method further includes:
and sending the data to be stored to a local database for persistence.
Through the secondary storage arrangement, the data reading efficiency and the system concurrency of the data are improved, the pressure of a database is relieved, and meanwhile, the stability of the stored data is effectively guaranteed.
Optionally, according to an embodiment of the present invention, the method further includes:
and receiving a data query request, taking a storage key and an application code indicated in the data query request as indexes, determining query data corresponding to the index key from the distributed cache server, and feeding back the query data to the application service.
Through the arrangement, the storage keys and the application codes are used as indexes, so that the data query efficiency is further improved, and the user experience is improved.
Further, according to an embodiment of the present invention, if query data corresponding to the index key is not queried at the distributed cache server, the method further includes:
forwarding the data query request to a local database, and searching query data in the local database.
After receiving the data query request, querying the distributed cache server, and if not, querying the local database. Through the arrangement, on one hand, the data query efficiency is improved, and on the other hand, the accuracy of the data query is ensured.
According to the technical scheme of the embodiment of the invention, the data storage request sent by the application service based on the distributed cache client is received; the data storage request comprises an application code and data to be stored, wherein the application code corresponds to an application service; determining a data type corresponding to data to be stored, inquiring a mapping relation table of the data type and a storage key, and determining the storage key corresponding to the data type; storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a technical means of storage keys and application codes, so that the technical problems of low storage resource utilization rate, low data processing efficiency and accuracy and poor user experience in the related art are solved, and further the technical effects that the storage keys can be built by combining the application codes of each application service when distributed cache storage is carried out, the problem of storage key conflict when a plurality of application services share a distributed cache cluster is avoided, the utilization rate of storage resources is improved, the data processing efficiency and accuracy are improved, and the user experience is improved are achieved.
FIG. 2 is a schematic diagram of the main flow of a data processing method of an application service according to another embodiment of the present invention; as shown in fig. 2, the data processing method for an application service provided by the embodiment of the present invention mainly includes:
step S201, receiving a registration request of at least one application service, and performing registration processing on the at least one application service; and implanting the distributed cache client in the application service with successful registration processing, and configuring the application code which is uniquely corresponding to the application service in the distributed cache client.
By the arrangement, a unique application code is set for each application service, so that the application code is added to the keys in the key value pair when the data of the application service are stored in the key value pair later. According to a specific implementation of the embodiment of the present invention, the application code may be used as a prefix or suffix to the key in the key value pair.
Step S202, acquiring historical storage data and key value pair information corresponding to the historical storage data, and determining the corresponding relation between the data type of the historical storage data and a storage key in the key value pair information; constructing a mapping relation table of the data types and the storage keys according to the corresponding relation; wherein each data type corresponds to a unique storage key form.
Through the setting, corresponding storage key forms can be set for each data type according to the historical storage habit of the user. It should be noted that the above setting is not limited to this application, and the storage key format corresponding to each data type may be configured according to the setting requirement of the user.
Step S203, receiving a data storage request sent by an application service based on a distributed cache client; the data storage request comprises application codes corresponding to the application service and data to be stored.
Step S204, determining the data type corresponding to the data to be stored, inquiring a mapping relation table of the data type and the storage key, and determining the storage key corresponding to the data type.
Through the arrangement, the storage keys corresponding to the data to be stored can be rapidly determined by combining the mapping relation table, so that the storage efficiency is improved, and meanwhile, the conflict of the storage keys is avoided.
Step S205, storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a storage key and an application code.
By the arrangement, when the data to be stored is stored in the shared distributed cache server, the application code can be combined with the storage key to serve as a key in the key value pair, and the data to be stored serves as a key value. Through the arrangement, the problem of storage key conflict when a plurality of application services share the distributed cache cluster is avoided, meanwhile, when a subsequent user inquires data, the data processing efficiency and accuracy are guaranteed, and the user experience is improved.
Further, according to an embodiment of the present invention, the method further includes:
receiving fault prompt information, and determining an application code corresponding to a fault storage key according to the fault storage key indicated by the fault prompt information;
and suspending the storage request of the application service corresponding to the application code.
When the distributed cache server side stores data, the keys in the key value pair comprise application codes, so that when a fault prompt is received, corresponding application service can be determined according to the application codes, a storage request of the application service is stopped, and the influence on storage and query service of the distributed cache server side is avoided. After the fault is removed, the storage request of the application service can be continuously received, and the corresponding storage service is carried out.
Preferably, according to an embodiment of the present invention, the method further includes:
and responding to the data monitoring request of the application service, monitoring target storage data containing the target application code in the storage key according to the target application code indicated by the data monitoring request, and recording storage information and access information of the target storage data.
Through the arrangement, unified and subdivision management of the distributed cache server to the accessed application service is realized, and further, the monitoring and maintenance can be performed in a targeted manner, the application scene of data processing is expanded, and the maintenance cost is reduced.
According to the technical scheme of the embodiment of the invention, the data storage request sent by the application service based on the distributed cache client is received; the data storage request comprises an application code and data to be stored, wherein the application code corresponds to an application service; determining a data type corresponding to data to be stored, inquiring a mapping relation table of the data type and a storage key, and determining the storage key corresponding to the data type; storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a technical means of storage keys and application codes, so that the technical problems of low storage resource utilization rate, low data processing efficiency and accuracy and poor user experience in the related art are solved, and further the technical effects that the storage keys can be built by combining the application codes of each application service when distributed cache storage is carried out, the problem of storage key conflict when a plurality of application services share a distributed cache cluster is avoided, the utilization rate of storage resources is improved, the data processing efficiency and accuracy are improved, and the user experience is improved are achieved.
FIG. 3 is a schematic diagram of the main flow of a data processing method of an application service according to another embodiment of the present invention; as shown in fig. 3, the data processing method for an application service provided by the embodiment of the present invention mainly includes:
Step S301, acquiring historical storage data and key value pair information corresponding to the historical storage data, and determining the corresponding relation between the data type of the historical storage data and a storage key in the key value pair information; constructing a mapping relation table of the data types and the storage keys according to the corresponding relation; wherein each data type corresponds to a unique storage key form.
Through the setting, corresponding storage key forms can be set for each data type according to the historical storage habit of the user. It should be noted that the above setting is not limited to this application, and the storage key format corresponding to each data type may be configured according to the setting requirement of the user.
Step S302, receiving a registration request of at least one application service, and performing registration processing on the at least one application service; and implanting the distributed cache client in the application service with successful registration processing, and configuring the application code which is uniquely corresponding to the application service in the distributed cache client.
By the arrangement, a unique application code is set for each application service, so that the application code is added to the keys in the key value pair when the data of the application service are stored in the key value pair later. According to a specific implementation of the embodiment of the present invention, the application code may be used as a prefix or suffix to the key in the key value pair.
Step S303, receiving a data storage request sent by an application service based on a distributed cache client; the data storage request comprises application codes corresponding to the application service and data to be stored.
Step S304, determining the data type corresponding to the data to be stored, inquiring the mapping relation table of the data type and the storage key, and determining the storage key corresponding to the data type.
Through the arrangement, the storage keys corresponding to the data to be stored can be rapidly determined by combining the mapping relation table, so that the storage efficiency is improved, and meanwhile, the conflict of the storage keys is avoided.
Step S305, storing the data to be stored to the distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a storage key and an application code.
By the arrangement, when the data to be stored is stored in the shared distributed cache server, the application code can be combined with the storage key to serve as a key in the key value pair, and the data to be stored serves as a key value. Through the arrangement, the problem of storage key conflict when a plurality of application services share the distributed cache cluster is avoided, meanwhile, when a subsequent user inquires data, the data processing efficiency and accuracy are guaranteed, and the user experience is improved.
And step S306, sending the data to be stored to a local database for persistence.
Through the secondary storage arrangement, the data reading efficiency and the system concurrency of the data are improved, the pressure of a database is relieved, and meanwhile, the stability of the stored data is effectively guaranteed.
Step S307, a data query request is received, the storage key and the application code indicated in the data query request are used as indexes, query data corresponding to the index key are sequentially determined from the distributed cache server side and the local database, and the query data are fed back to the application service.
After receiving the data query request, querying the distributed cache server, and if not, querying the local database. Through the arrangement, on one hand, the data query efficiency is improved, and on the other hand, the accuracy of the data query is ensured.
According to the technical scheme of the embodiment of the invention, the data storage request sent by the application service based on the distributed cache client is received; the data storage request comprises an application code and data to be stored, wherein the application code corresponds to an application service; determining a data type corresponding to data to be stored, inquiring a mapping relation table of the data type and a storage key, and determining the storage key corresponding to the data type; storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a technical means of storage keys and application codes, so that the technical problems of low storage resource utilization rate, low data processing efficiency and accuracy and poor user experience in the related art are solved, and further the technical effects that the storage keys can be built by combining the application codes of each application service when distributed cache storage is carried out, the problem of storage key conflict when a plurality of application services share a distributed cache cluster is avoided, the utilization rate of storage resources is improved, the data processing efficiency and accuracy are improved, and the user experience is improved are achieved.
FIG. 4 is a schematic diagram of the main modules of a data processing apparatus for application services provided according to an embodiment of the present invention; as shown in fig. 4, a data processing apparatus 400 for an application service according to an embodiment of the present invention mainly includes:
a receiving module 401, configured to receive a data storage request sent by an application service based on a distributed cache client; the data storage request comprises application codes corresponding to the application service and data to be stored.
Specifically, according to an embodiment of the present invention, the data processing apparatus 400 of an application service further includes a configuration module, before the step of receiving a data storage request sent by the application service based on the distributed cache client, the configuration module is configured to:
receiving a registration request of at least one application service, and performing registration processing on the at least one application service;
and implanting the distributed cache client in the application service with successful registration processing, and configuring the application code which is uniquely corresponding to the application service in the distributed cache client.
By the arrangement, a unique application code is set for each application service, so that the application code is added to the keys in the key value pair when the data of the application service are stored in the key value pair later. According to a specific implementation of the embodiment of the present invention, the application code may be used as a prefix or suffix to the key in the key value pair.
The storage key determining module 402 is configured to determine a data type corresponding to the data to be stored, query a mapping relationship table of the data type and the storage key, and determine the storage key corresponding to the data type.
Through the arrangement, the storage keys corresponding to the data to be stored can be rapidly determined by combining the mapping relation table, so that the storage efficiency is improved, and meanwhile, the conflict of the storage keys is avoided.
Further, according to an embodiment of the present invention, the data processing apparatus 400 of an application service further includes a mapping relationship table construction module, configured to:
acquiring historical storage data and key value pair information corresponding to the historical storage data, and determining the corresponding relation between the data type of the historical storage data and a storage key in the key value pair information;
constructing a mapping relation table of the data types and the storage keys according to the corresponding relation; wherein each data type corresponds to a unique storage key form.
Through the setting, corresponding storage key forms can be set for each data type according to the historical storage habit of the user. It should be noted that the above setting is not limited to this application, and the storage key format corresponding to each data type may be configured according to the setting requirement of the user.
The data processing module 403 is configured to store data to be stored in a distributed cache server in a key value pair form; the key value pair is data to be stored, and the key comprises a storage key and an application code.
By the arrangement, when the data to be stored is stored in the shared distributed cache server, the application code can be combined with the storage key to serve as a key in the key value pair, and the data to be stored serves as a key value. Through the arrangement, the problem of storage key conflict when a plurality of application services share the distributed cache cluster is avoided, meanwhile, when a subsequent user inquires data, the data processing efficiency and accuracy are guaranteed, and the user experience is improved.
Further, according to an embodiment of the present invention, the data processing apparatus 400 of the application service further includes a fault processing module, configured to:
receiving fault prompt information, and determining an application code corresponding to a fault storage key according to the fault storage key indicated by the fault prompt information;
and suspending the storage request of the application service corresponding to the application code.
When the distributed cache server side stores data, the keys in the key value pair comprise application codes, so that when a fault prompt is received, corresponding application service can be determined according to the application codes, a storage request of the application service is stopped, and the influence on storage and query service of the distributed cache server side is avoided. After the fault is removed, the storage request of the application service can be continuously received, and the corresponding storage service is carried out.
Preferably, according to an embodiment of the present invention, the data processing apparatus 400 of the application service further includes a data monitoring module, configured to:
and responding to the data monitoring request of the application service, monitoring target storage data containing the target application code in the storage key according to the target application code indicated by the data monitoring request, and recording storage information and access information of the target storage data.
Through the arrangement, unified and subdivision management of the distributed cache server to the accessed application service is realized, and further, the monitoring and maintenance can be performed in a targeted manner, the application scene of data processing is expanded, and the maintenance cost is reduced.
Illustratively, according to an embodiment of the present invention, the data processing apparatus 400 of the application service further includes a persistence processing module, after the step of storing the data to be stored in the form of the key-value pair to the distributed cache server, the persistence processing module is configured to:
and sending the data to be stored to a local database for persistence.
Through the secondary storage arrangement, the data reading efficiency and the system concurrency of the data are improved, the pressure of a database is relieved, and meanwhile, the stability of the stored data is effectively guaranteed.
Optionally, according to an embodiment of the present invention, the data processing apparatus 400 of the application service further includes a query module, configured to:
and receiving a data query request, taking a storage key and an application code indicated in the data query request as indexes, determining query data corresponding to the index key from the distributed cache server, and feeding back the query data to the application service.
Through the arrangement, the storage keys and the application codes are used as indexes, so that the data query efficiency is further improved, and the user experience is improved.
Further, according to an embodiment of the present invention, if query data corresponding to the index key is not queried at the distributed cache server, the query module is further configured to:
forwarding the data query request to a local database, and searching query data in the local database.
After receiving the data query request, querying the distributed cache server, and if not, querying the local database. Through the arrangement, on one hand, the data query efficiency is improved, and on the other hand, the accuracy of the data query is ensured.
According to the technical scheme of the embodiment of the invention, the data storage request sent by the application service based on the distributed cache client is received; the data storage request comprises an application code and data to be stored, wherein the application code corresponds to an application service; determining a data type corresponding to data to be stored, inquiring a mapping relation table of the data type and a storage key, and determining the storage key corresponding to the data type; storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a technical means of storage keys and application codes, so that the technical problems of low storage resource utilization rate, low data processing efficiency and accuracy and poor user experience in the related art are solved, and further the technical effects that the storage keys can be built by combining the application codes of each application service when distributed cache storage is carried out, the problem of storage key conflict when a plurality of application services share a distributed cache cluster is avoided, the utilization rate of storage resources is improved, the data processing efficiency and accuracy are improved, and the user experience is improved are achieved.
Fig. 5 shows an exemplary system architecture 500 of a data processing method of an application service or a data processing apparatus of an application service to which an embodiment of the invention may be applied.
As shown in fig. 5, the system architecture 500 may include terminal devices 501, 502, 503, a network 504, and a server 505. The network 504 is used as a medium to provide communication links between the terminal devices 501, 502, 503 and the server 505. The network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 505 via the network 504 using the terminal devices 501, 502, 503 to receive or send messages or the like. Various communication client applications may be installed on the terminal devices 501, 502, 503, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 501, 502, 503 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 505 may be a server providing various services, such as a server (for example only) that utilizes (performs data processing on) the terminal devices 501, 502, 503 for the user. The server may perform processing such as analysis on the received data such as the data storage request, and feedback the processing result (for example, a storage key corresponding to the data type—only an example) to the terminal device.
It should be noted that, the data processing method of the application service provided by the embodiment of the present invention is generally executed by the server 505, and accordingly, the data processing device of the application service is generally disposed in the server 505.
It should be understood that the number of terminal devices, networks and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 6, there is illustrated a schematic diagram of a computer system 600 suitable for use in implementing a terminal device or server in accordance with an embodiment of the present invention. The terminal device or server shown in fig. 6 is only an example, and should not impose any limitation on the functions and scope of use of the embodiments of the present invention.
As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, mouse, etc.; an output portion 607 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The drive 610 is also connected to the I/O interface 605 as needed. Removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on drive 610 so that a computer program read therefrom is installed as needed into storage section 608.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 609, and/or installed from the removable medium 611. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 601.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The computer program product of an embodiment of the present invention comprises a computer program which, when executed by a processor, implements a data processing method of any one of the application services provided by the above-described embodiments of the present invention.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes a receiving module, a storage key determination module, and a data processing module. Wherein the names of the modules do not constitute a limitation on the module itself in some cases, for example, the receiving module may also be described as "for receiving data storage requests sent by application services based on distributed cache clients"; the data storage request comprises an application code corresponding to the application service and a module for data to be stored.
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include: receiving a data storage request sent by an application service based on a distributed cache client; the data storage request comprises an application code and data to be stored, wherein the application code corresponds to an application service; determining a data type corresponding to data to be stored, inquiring a mapping relation table of the data type and a storage key, and determining the storage key corresponding to the data type; storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a storage key and an application code.
According to the technical scheme of the embodiment of the invention, the data storage request sent by the application service based on the distributed cache client is received; the data storage request comprises an application code and data to be stored, wherein the application code corresponds to an application service; determining a data type corresponding to data to be stored, inquiring a mapping relation table of the data type and a storage key, and determining the storage key corresponding to the data type; storing the data to be stored to a distributed cache server in the form of key value pairs; the key value pair is data to be stored, and the key comprises a technical means of storage keys and application codes, so that the technical problems of low storage resource utilization rate, low data processing efficiency and accuracy and poor user experience in the related art are solved, and further the technical effects that the storage keys can be built by combining the application codes of each application service when distributed cache storage is carried out, the problem of storage key conflict when a plurality of application services share a distributed cache cluster is avoided, the utilization rate of storage resources is improved, the data processing efficiency and accuracy are improved, and the user experience is improved are achieved.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (13)

1. The data processing method of the application service is characterized by being applied to a distributed cache server and comprising the following steps:
receiving a data storage request sent by an application service based on a distributed cache client; the data storage request comprises an application code and data to be stored, wherein the application code and the data to be stored correspond to the application service;
determining a data type corresponding to the data to be stored, inquiring a mapping relation table of the data type and a storage key, and determining the storage key corresponding to the data type;
storing the data to be stored to the distributed cache server in the form of key value pairs; and the key value pair is used for storing the data to be stored, and the key comprises the storage key and the application code.
2. The method for processing data of an application service according to claim 1, wherein before the step of receiving a data storage request sent by an application service based on a distributed cache client, the method further comprises:
receiving a registration request of at least one application service, and performing registration processing on the at least one application service;
and implanting a distributed cache client in the application service with successful registration processing, and configuring an application code which is uniquely corresponding to the application service in the distributed cache client.
3. The method for processing data of an application service according to claim 1, further comprising a step of constructing a mapping relation table of data types and storage keys, comprising:
acquiring historical storage data and key value pair information corresponding to the historical storage data, and determining the corresponding relation between the data type of the historical storage data and a storage key in the key value pair information;
constructing a mapping relation table of the data types and the storage keys according to the corresponding relation; wherein each data type corresponds to a unique storage key form.
4. The method for processing data of an application service according to claim 1, wherein the method further comprises:
receiving fault prompt information, and determining an application code corresponding to a fault storage key according to the fault storage key indicated by the fault prompt information;
and stopping the storage request of the application service corresponding to the application code.
5. The method for processing data of an application service according to claim 1, wherein the method further comprises:
and responding to a data monitoring request of the application service, monitoring target storage data containing the target application code in a storage key according to the target application code indicated by the data monitoring request, and recording storage information and access information of the target storage data.
6. The method for processing data of an application service according to claim 1, wherein after the step of storing the data to be stored in the form of key-value pairs to the distributed cache server, the method further comprises:
and sending the data to be stored to a local database for persistence processing.
7. The method for processing data of an application service according to claim 6, further comprising:
and receiving a data query request, taking a storage key and an application code indicated in the data query request as indexes, determining query data corresponding to the index key from the distributed cache server, and feeding back the query data to the application service.
8. The method for processing data of an application service according to claim 7, wherein if query data corresponding to the index key is not queried at the distributed cache server, the method further comprises:
and forwarding the data query request to a local database, and searching the query data in the local database.
9. A data processing apparatus for an application service, comprising:
the receiving module is used for receiving a data storage request sent by the application service based on the distributed cache client; the data storage request comprises an application code and data to be stored, wherein the application code and the data to be stored correspond to the application service;
The storage key determining module is used for determining the data type corresponding to the data to be stored, inquiring a mapping relation table of the data type and the storage key and determining the storage key corresponding to the data type;
the data processing module is used for storing the data to be stored to the distributed cache server in the form of key value pairs; and the key value pair is used for storing the data to be stored, and the key comprises the storage key and the application code.
10. The application service data processing apparatus according to claim 9, further comprising a configuration module for, prior to the step of receiving a data storage request by the application service based on the distributed cache client,:
receiving a registration request of at least one application service, and performing registration processing on the at least one application service;
and implanting a distributed cache client in the application service with successful registration processing, and configuring an application code which is uniquely corresponding to the application service in the distributed cache client.
11. An electronic device for data processing of an application service, comprising:
one or more processors;
Storage means for storing one or more programs,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-8.
12. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-8.
13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-8.
CN202310252445.8A 2023-03-16 2023-03-16 Data processing method and device for application service Pending CN116303608A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310252445.8A CN116303608A (en) 2023-03-16 2023-03-16 Data processing method and device for application service

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310252445.8A CN116303608A (en) 2023-03-16 2023-03-16 Data processing method and device for application service

Publications (1)

Publication Number Publication Date
CN116303608A true CN116303608A (en) 2023-06-23

Family

ID=86784681

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310252445.8A Pending CN116303608A (en) 2023-03-16 2023-03-16 Data processing method and device for application service

Country Status (1)

Country Link
CN (1) CN116303608A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116701413A (en) * 2023-08-08 2023-09-05 北京久其金建科技有限公司 Main data processing method and device
CN116701563A (en) * 2023-08-08 2023-09-05 佳瑛科技有限公司 Unstructured data processing method and related device based on distributed server

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116701413A (en) * 2023-08-08 2023-09-05 北京久其金建科技有限公司 Main data processing method and device
CN116701563A (en) * 2023-08-08 2023-09-05 佳瑛科技有限公司 Unstructured data processing method and related device based on distributed server

Similar Documents

Publication Publication Date Title
CN110019350B (en) Data query method and device based on configuration information
CN116303608A (en) Data processing method and device for application service
CN111460129B (en) Method, device, electronic equipment and storage medium for generating identification
CN110321252B (en) Skill service resource scheduling method and device
CN109428926B (en) Method and device for scheduling task nodes
CN113641706B (en) Data query method and device
CN109683942B (en) Script management method, script management device, script management medium and electronic equipment
CN111831503B (en) Monitoring method based on monitoring agent and monitoring agent device
CN116775167A (en) Service processing method, device, electronic equipment and computer readable medium
CN112948138A (en) Method and device for processing message
CN111984686A (en) Data processing method and device
CN113765871B (en) Method and device for managing fort machine
CN116861397A (en) Request processing method, device, electronic equipment and computer readable medium
CN112783914B (en) Method and device for optimizing sentences
CN109213815B (en) Method, device, server terminal and readable medium for controlling execution times
CN113742617A (en) Cache updating method and device
CN113127416A (en) Data query method and device
CN112559001A (en) Method and device for updating application
CN114301778B (en) Access control method and device
CN117478535B (en) Log storage method and device
CN110262756B (en) Method and device for caching data
CN116450622B (en) Method, apparatus, device and computer readable medium for data warehouse entry
CN114490128A (en) Data storage method and device under remote multi-active architecture
CN113452739A (en) Message processing method and device
CN114091044A (en) System authority management method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination