WO2017190580A1 - Method and device for accessing database - Google Patents

Method and device for accessing database Download PDF

Info

Publication number
WO2017190580A1
WO2017190580A1 PCT/CN2017/080290 CN2017080290W WO2017190580A1 WO 2017190580 A1 WO2017190580 A1 WO 2017190580A1 CN 2017080290 W CN2017080290 W CN 2017080290W WO 2017190580 A1 WO2017190580 A1 WO 2017190580A1
Authority
WO
WIPO (PCT)
Prior art keywords
request
log
database
data
local
Prior art date
Application number
PCT/CN2017/080290
Other languages
French (fr)
Chinese (zh)
Inventor
宋昭
Original Assignee
北京奇虎科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京奇虎科技有限公司 filed Critical 北京奇虎科技有限公司
Publication of WO2017190580A1 publication Critical patent/WO2017190580A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2471—Distributed queries

Definitions

  • the present invention relates to the field of computer technology, and in particular, to a method and apparatus for accessing a database.
  • the database is a technology that adapts to such requirements. Simply put, a database is a collection of data that is preserved in a way that can be persisted and manipulated.
  • the LSM Tree database represented by leveldb is taken as an example.
  • the database will look for the read operation from the local client after receiving the read operation request from the user client. , then return a response message to the user client.
  • Such a reading method has a low read performance and cannot satisfy the situation where a large amount of hot data is required. Therefore, this technical problem needs to be solved urgently.
  • the present invention has been made in order to provide a method and corresponding apparatus for accessing a database that overcomes the above problems or at least partially solves the above problems.
  • a method of accessing a database comprising: receiving a request from a client to access a database; performing type analysis on the request to determine whether the request is a data write request; When the request is a data write request, the corresponding write operation of the request is recorded to the local log, and the corresponding data of the request is written to the cache layer of the database.
  • an apparatus for accessing a database comprising: a request receiving module adapted to receive a request from a client to access a database; and a request analysis module adapted to perform type analysis on the request to Determining whether the request is a data write request; and the first write module is adapted to record the write operation corresponding to the request to the local when the request analysis module determines that the request is a write request for data Log and write the corresponding data of the request to the cache layer of the database.
  • a computer program comprising computer readable code, when said computer readable code is run on a computing device, causing said computing device to perform access to a database as described above Methods.
  • a type analysis is performed on the request to determine if the request is a data write request.
  • the corresponding write operation of the data write request is recorded to the local log, and the corresponding data of the data write request is written into the cache layer of the database. It can be seen that the embodiment of the present invention directly writes the data corresponding to the data write request to the cache layer of the database, and returns a response message to the client, which can improve the efficiency of subsequent reading of the data.
  • the embodiment of the present invention records the corresponding write operation of the data write request to the local log, so that the local log can be used to asynchronously write the corresponding data of the write request into the database, thereby ensuring the persistence and integrity of the data in the database.
  • FIG. 1 is a flow chart showing a method of accessing a database according to an embodiment of the present invention
  • FIG. 2 is a flow chart showing a method of accessing a database according to another embodiment of the present invention.
  • FIG. 3 is a schematic structural diagram of an apparatus for accessing a database according to an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of an apparatus for accessing a database according to another embodiment of the present invention.
  • Figure 5 is a schematic illustration of a computing setup for performing a method of accessing a database in accordance with the present invention Prepared block diagram
  • Fig. 6 schematically shows a storage unit for holding or carrying program code implementing a method of accessing a database according to the present invention.
  • FIG. 1 shows a flow diagram of a method of accessing a database in accordance with one embodiment of the present invention. As shown in FIG. 1, the method may at least include step S102, step S104, and step S106:
  • Step S102 receiving a request from a client to access a database
  • Step S104 performing type analysis on the request to determine whether the request is a data write request
  • Step S106 when the request is a data write request, the corresponding write operation of the request is recorded to the local log, and the corresponding data of the request is written into the cache layer of the database.
  • the local log refers to the log file stored by the database on the hard disk of the local physical machine, which is used to record the operation of the database being accessed.
  • a type analysis is performed on the request to determine if the request is a data write request.
  • the corresponding write operation of the data write request is recorded to the local log, and the corresponding data of the data write request is written into the cache layer of the database. Therefore, the embodiment of the present invention directly writes the data corresponding to the data write request to the cache layer of the database, and returns a response message to the client without writing the data into the database, thereby improving subsequent reading of the data. effectiveness.
  • the embodiment of the present invention records the corresponding write operation of the data write request to the local log, so that the local log can be used to asynchronously write the corresponding data of the write request into the database, thereby ensuring the persistence and integrity of the data in the database.
  • the security authentication policy can be set to be based on which IP address the client is not allowed to query the database; except for a certain IP address, the client cannot delete, modify the data in the database, and so on.
  • the statement accessing the database may be parsed from the request, and the type of operation of the statement is analyzed to determine whether the request is data.
  • Write request The types here are such as add, delete, modify, query, etc.
  • the statement is a statement to increase the operation, the request is determined to be a data write request; when the statement is a delete operation statement, the request is determined to be a data write request.
  • the statement is a statement to modify the operation, it is determined that the request is a data write request; when the statement is a statement of the query operation, the request is determined to be a data read request, and so on.
  • the cache layer mentioned in the above step S106 may be a cache layer newly created outside the database. Since the database itself does not have a cache layer, the embodiment of the present invention chooses to create a new cache layer outside the database for temporarily storing data. For example, after the corresponding write operation of the write request is recorded to the local log, the corresponding data of the write request is written into the newly created cache layer, and a response message is returned to the client, thereby improving subsequent reading of the data. effectiveness.
  • the cache layer of the corresponding storage capacity may be selected according to the service requirement, and the type of the cache layer may be, for example, a distributed cache system memcached or a content-based redis.
  • the above step S106 records the corresponding write operation of the request to the local log, and writes the corresponding data of the request to the cache layer of the database.
  • the first thread can be invoked, and the first thread is used to record the corresponding write operation to the local log and the corresponding data is requested to be written to the cache layer. That is to say, the first thread writes the corresponding write operation to the local log, and then writes the corresponding data to the cache layer, and then returns a response message indicating that the database write has been completed to the client.
  • the embodiment of the present invention can use the local log to asynchronously write the corresponding data of the write request to the database, that is, call the second thread and scan the local log by using the second thread.
  • the second thread can scan the local log in real time, or can set a specific scanning period (such as 0.5 seconds or 1 second, etc.) to scan the local log.
  • the embodiment of the present invention when determining the log increment, provides a An optional solution is to use the second thread to scan the local log to determine whether there is an unmarked log record in the local log, and if there is an unmarked log record, determine that the unmarked log record is a log increment. . Further, after the corresponding data of the log increment is written into the database, the corresponding log record of the log increment is marked in the local log to indicate that the corresponding data of the log increment has been written into the database, and the repeated writing can be avoided. The problem. For example, after the corresponding data of the log increment is written into the database, the corresponding log record of the log increment can be marked in the local log as "operated" or "recorded".
  • the marked log records in the local log may be deleted, for example, the local log is set, and the marked log records may be deleted every predetermined time.
  • the local log can be divided according to the size. When the log records in a certain segmentation area are all marked, the log records in the segmentation area can be deleted.
  • the local log can be utilized after the database is restarted next time. Data recovery for the database to achieve the purpose of fault repair. Specifically, after the database fails to restart, the scan thread is enabled, and the scan thread is used to scan the local log to determine whether there is an unmarked log record in the local log, and if so, the unmarked log record is written correspondingly. Enter the database and mark unmarked log records in the local log.
  • the database works normally, and new data is also written in the cache layer (here, the first thread mentioned above may be used to record the corresponding write operation to the local log and request the corresponding data to be written.
  • the first thread mentioned above may be used to record the corresponding write operation to the local log and request the corresponding data to be written.
  • the scan thread here has the same effect as the second thread mentioned above, and the scan thread is asynchronous with the first thread.
  • the data since the data is cached at the cache layer, when the request for accessing the database from the client is a data read request, it may be directly in the cache layer. Find the corresponding data of the read request, and return the found data to the client, thereby reducing the read access to the database and improving the efficiency of data reading.
  • the embodiment of the present invention may read the request. Forwarding to the database, looking up the corresponding data of the read request in the database, returning the corresponding data of the read request to the client, and buffering the corresponding data of the read request in the cache layer.
  • the cache layer of the corresponding storage capacity is selected according to the service requirement, and a cache layer is newly created for the database, and the type of the cache layer may be a distributed cache system memcached or a memory-based redis.
  • step S202 is a flow chart showing a method of accessing a database according to another embodiment of the present invention. As shown in FIG. 2, the method may at least include step S202, step S204, step S206, step S208, and step S210.
  • Step S202 a new cache layer is created for the database, and a request for accessing the database from the client is received.
  • Step S204 performing type analysis on the request to determine whether the request is a data write request, and if yes, proceeding to step S206; if not, proceeding to step S208.
  • Step S206 the first thread is called, and the first thread is used to record the corresponding write operation to the local log and the corresponding data is requested to be written into the cache layer.
  • the local log refers to the log file stored by the database on the hard disk of the local physical machine, which is used to record the operation of the database being accessed.
  • Step S208 When the request for accessing the database from the client is a data read request, the data corresponding to the read request is searched in the cache layer, and the found data is returned to the client.
  • the embodiment of the present invention may forward the read request to the database.
  • the database searches for the corresponding data of the read request, returns the corresponding data of the read request to the client, and caches the corresponding data of the read request in the cache layer.
  • Step S210 invoking the second thread and scanning the local log by using the second thread to determine whether there is a log increment in the local log, where the second thread is asynchronous with the first thread, if the local log is found If there is a log increment, the corresponding data of the log increment is written to the database.
  • the second thread may be used to scan the local log to determine whether there is an unmarked log record in the local log. If there is an unmarked log record, it is determined that the unmarked log record is a log increment. . Further, after the corresponding data of the log increment is written into the database, the corresponding log record of the log increment is marked in the local log to indicate that the corresponding data of the log increment has been written into the database, and the repeated writing can be avoided. The problem. For example, after the corresponding data of the log increment is written into the database, the corresponding log record of the log increment can be marked in the local log as "operated" or "recorded". In addition, to save local storage space, you can delete the log records that have been marked in the local log.
  • the database can be performed by using the local log after the database is restarted next time. Data recovery to achieve the purpose of fault repair. Specifically, after the database fails to restart, the scan thread is enabled, and the scan thread is used to scan the local log to determine whether there is an unmarked log record in the local log, and if so, the unmarked log record is written correspondingly. Enter the database and mark unmarked log records in the local log.
  • the database works normally, and new data is also written in the cache layer (here, the first thread mentioned above may be used to record the corresponding write operation to the local log and request the corresponding data to be written.
  • the first thread mentioned above may be used to record the corresponding write operation to the local log and request the corresponding data to be written.
  • the scan thread here has the same effect as the second thread mentioned above, and the scan thread is asynchronous with the first thread.
  • the embodiment of the present invention further provides an apparatus for accessing a database.
  • FIG. 3 is a block diagram showing the structure of an apparatus for accessing a database according to an embodiment of the present invention.
  • the apparatus 300 can include at least a request receiving module 310, a request analyzing module 320, and a first writing module 330.
  • the request receiving module 310 is adapted to receive a request from the client to access the database
  • the request analysis module 320 is coupled to the request receiving module 310 and is adapted to perform type analysis on the request to determine whether the request is a data write request;
  • the first write module 330 is coupled to the request analysis module 320, and is adapted to, when the request analysis module determines that the request is a write request for data, record the corresponding write operation to the local log, and write the corresponding data request. Enter the cache layer of the database.
  • the apparatus shown in FIG. 3 may further include:
  • the reading module 340 is coupled to the request analysis module 320 and the first write module 330, and is adapted to look up the request corresponding data in the cache layer when the request analysis module 320 determines that the request is a read request for data, and is to search The data returned is returned to the client.
  • the first writing module 330 is further adapted to:
  • the first thread is called, and the first thread is used to record the corresponding write operation to the local log and the corresponding data is requested to be written to the cache layer.
  • the apparatus shown in FIG. 3 may further include:
  • the second write module 350 is coupled to the first write module 330, and is adapted to invoke the second thread and scan the local log by using the second thread to determine whether there is a log increment in the local log, where the second thread and the first thread Asynchronous; and if a log increment is found in the local log, the corresponding data of the log increment is written to the database.
  • the second writing module 350 is further adapted to:
  • the apparatus shown in FIG. 3 may further include:
  • the marking module 360 is coupled to the second writing module 350, and is adapted to mark the corresponding log record of the log increment in the local log after the second writing module 350 writes the corresponding data of the log increment into the database. .
  • the apparatus shown in FIG. 3 may further include:
  • a deletion module 370 coupled to the tagging module 360, is adapted to delete the marked log records in the local log.
  • the apparatus shown in FIG. 3 may further include:
  • the fault processing module 380 is coupled to the first writing module 330 and the marking module 360, and is adapted to After the database fails to restart, scan the local log to determine if there are unmarked log records in the local log;
  • the corresponding data of the unmarked log records is written to the database, and the unmarked log records are marked in the local log.
  • the reading module 340 is further adapted to:
  • the request analysis module determines that the request is a read request for data, if the corresponding data is not found in the cache layer, the request is forwarded to the database;
  • the cache layer is a distributed cache system memcached.
  • the embodiment of the present invention can achieve the following beneficial effects:
  • a type analysis is performed on the request to determine if the request is a data write request.
  • the corresponding write operation of the data write request is recorded to the local log, and the corresponding data of the data write request is written into the cache layer of the database. It can be seen that the embodiment of the present invention directly writes the data corresponding to the data write request to the cache layer of the database, and returns a response message to the client, which can improve the efficiency of subsequent reading of the data.
  • the embodiment of the present invention records the corresponding write operation of the data write request to the local log, so that the local log can be used to asynchronously write the corresponding data of the write request into the database, thereby ensuring the persistence and integrity of the data in the database.
  • modules in the devices of the embodiments can be adaptively changed and placed in one or more devices different from the embodiment.
  • the modules or units or components of the embodiments may be combined into one module or unit or component, and further they may be divided into a plurality of sub-modules or sub-units or sub-components.
  • any combination of the features disclosed in the specification, including the accompanying claims, the abstract and the drawings, and any methods so disclosed, or All processes or units of the device are combined.
  • Each feature disclosed in this specification (including the accompanying claims, the abstract and the drawings) may be replaced by alternative features that provide the same, equivalent or similar purpose.
  • the various component embodiments of the present invention may be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof.
  • a microprocessor or digital signal processor may be used in practice to implement some or all of the functionality of some or all of the components of the device for accessing a database in accordance with embodiments of the present invention.
  • the invention can also be implemented as a device or device program (e.g., a computer program and a computer program product) for performing some or all of the methods described herein.
  • a program implementing the invention may be stored on a computer readable medium or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
  • Figure 5 illustrates a computing device that can implement a method of accessing a database in accordance with the present invention.
  • the computing device conventionally includes a processor 510 and a computer program product or computer readable medium in the form of a memory 520.
  • the memory 520 may be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), an EPROM, a hard disk, or a ROM.
  • Memory 520 has a storage space 530 that stores program code 531 for performing any of the method steps described above.
  • the storage space 530 storing the program code can be stored separately for implementing the above
  • the program code can be read from or written to one or more computer program products.
  • Such computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks.
  • Such a computer program product is typically a portable or fixed storage unit such as that shown in FIG.
  • the storage unit may have storage segments, storage spaces, and the like that are similarly arranged to memory 520 in the computing device of FIG.
  • the program code can be compressed, for example, in an appropriate form.
  • the storage unit stores computer readable code 531' for performing the method steps of the present invention, ie, code that can be read by a processor, such as 510, when the code is run by a computing device, causing the computing device Perform the various steps in the method described above.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A method and device for accessing a database. The method comprises: receiving a request from a client to access a database (S102); analyzing the type of request so as to determine whether the request is a data writing request (S104); and when the request is the data writing request, recording a writing operation corresponding to the request to a local log, and writing data corresponding to the request to a buffer layer of the database (S106). By means of the method, the efficiency of subsequent data reading can be improved.

Description

访问数据库的方法及装置Method and device for accessing database 技术领域Technical field
本发明涉及计算机技术领域，特别是一种访问数据库的方法及装置。The present invention relates to the field of computer technology, and in particular, to a method and apparatus for accessing a database.
背景技术Background technique
随着社会的发展，大量的数据需要存储起来，对存储的数据，根据不同用户的需求应用一定的数据访问方法进行处理，数据库就是适应这种需求的一种技术。简单来说，数据库是数据以一种能够持久保存，并且可以***作的方式来保存的数据集合。With the development of society, a large amount of data needs to be stored, and the stored data is processed according to the needs of different users by applying certain data access methods. The database is a technology that adapts to such requirements. Simply put, a database is a collection of data that is preserved in a way that can be persisted and manipulated.
在实际应用中，用户可以对数据库中的数据进行读写操作，如增加、删除、修改、查询等操作。相关技术中，以leveldb为代表的LSM Tree数据库为例，当用户对数据库中的热数据进行读取操作时，数据库接收到用户客户端发出的读取操作请求后，会从本地文件中去查找，之后向用户客户端返回响应消息。这样的读取方式，其读性能较低，满足不了需要大量请求热数据的情况。因而，亟待解决这一技术问题。In practical applications, users can perform read and write operations on data in the database, such as adding, deleting, modifying, and querying operations. In the related art, the LSM Tree database represented by leveldb is taken as an example. When the user reads the hot data in the database, the database will look for the read operation from the local client after receiving the read operation request from the user client. , then return a response message to the user client. Such a reading method has a low read performance and cannot satisfy the situation where a large amount of hot data is required. Therefore, this technical problem needs to be solved urgently.
发明内容Summary of the invention
鉴于上述问题，提出了本发明以便提供一种克服上述问题或者至少部分地解决上述问题的访问数据库的方法及相应的装置。In view of the above problems, the present invention has been made in order to provide a method and corresponding apparatus for accessing a database that overcomes the above problems or at least partially solves the above problems.
依据本发明的一方面，提供了一种访问数据库的方法，包括：接收来自客户端的访问数据库的请求；对所述请求进行类型分析，以确定所述请求是否为数据写入请求；以及当所述请求为数据写入请求时，将所述请求相应的写入操作记录至本地日志，并将所述请求相应的数据写入所述数据库的缓存层。According to an aspect of the present invention, a method of accessing a database is provided, comprising: receiving a request from a client to access a database; performing type analysis on the request to determine whether the request is a data write request; When the request is a data write request, the corresponding write operation of the request is recorded to the local log, and the corresponding data of the request is written to the cache layer of the database.
依据本发明的另一方面，还提供了一种访问数据库的装置，包括：请求接收模块，适于接收来自客户端的访问数据库的请求；请求分析模块，适于对所述请求进行类型分析，以确定所述请求是否为数据写入请求；以及第一写入模块，适于当所述请求分析模块确定所述请求为数据的写入请求时，将所述请求相应的写入操作记录至本地日志，并将所述请求相应的数据写入所述数据库的缓存层。 According to another aspect of the present invention, there is provided an apparatus for accessing a database, comprising: a request receiving module adapted to receive a request from a client to access a database; and a request analysis module adapted to perform type analysis on the request to Determining whether the request is a data write request; and the first write module is adapted to record the write operation corresponding to the request to the local when the request analysis module determines that the request is a write request for data Log and write the corresponding data of the request to the cache layer of the database.
依据本发明的又一方面，还提供了一种计算机程序，其包括计算机可读代码，当所述计算机可读代码在计算设备上运行时，导致所述计算设备执行如上文所述的访问数据库的方法。According to still another aspect of the present invention, there is also provided a computer program comprising computer readable code, when said computer readable code is run on a computing device, causing said computing device to perform access to a database as described above Methods.
依据本发明的再一方面，还提供了一种计算机可读介质，其中存储了如上文所述的计算机程序。According to still another aspect of the present invention, there is also provided a computer readable medium storing a computer program as described above.
在本发明实施例中，当接收到来自客户端的访问数据库的请求时，对该请求进行类型分析以确定该请求是否为数据写入请求。当确定该请求为数据写入请求时，将该数据写入请求相应的写入操作记录至本地日志，并将该数据写入请求相应的数据写入数据库的缓存层。由此可知，本发明实施例直接将数据写入请求相应的数据写入数据库的缓存层，并向客户端返回响应消息，其能够提升后续对所述数据读取的效率。并且，本发明实施例将数据写入请求相应的写入操作记录至本地日志，从而可以利用本地日志异步地将写入请求相应的数据写入数据库，保证数据库中数据的持久化和完整性。另外，在数据库出现故障重新启动后，也可以利用本地日志对数据库进行数据恢复，实现故障修复的目的。In an embodiment of the invention, when a request from a client to access a database is received, a type analysis is performed on the request to determine if the request is a data write request. When it is determined that the request is a data write request, the corresponding write operation of the data write request is recorded to the local log, and the corresponding data of the data write request is written into the cache layer of the database. It can be seen that the embodiment of the present invention directly writes the data corresponding to the data write request to the cache layer of the database, and returns a response message to the client, which can improve the efficiency of subsequent reading of the data. Moreover, the embodiment of the present invention records the corresponding write operation of the data write request to the local log, so that the local log can be used to asynchronously write the corresponding data of the write request into the database, thereby ensuring the persistence and integrity of the data in the database. In addition, after the database fails to restart, you can also use the local log to recover data from the database to achieve the purpose of fault repair.
上述说明仅是本发明技术方案的概述，为了能够更清楚了解本发明的技术手段，而可依照说明书的内容予以实施，并且为了让本发明的上述和其它目的、特征和优点能够更明显易懂，以下特举本发明的具体实施方式。The above description is only an overview of the technical solutions of the present invention, and the above-described and other objects, features and advantages of the present invention can be more clearly understood. Specific embodiments of the invention are set forth below.
附图说明DRAWINGS
通过阅读下文优选实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的，而并不认为是对本发明的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：Various other advantages and benefits will become apparent to those skilled in the art from a The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting. Throughout the drawings, the same reference numerals are used to refer to the same parts. In the drawing:
图1示出了根据本发明一个实施例的访问数据库的方法的流程示意图；1 is a flow chart showing a method of accessing a database according to an embodiment of the present invention;
图2示出了根据本发明另一个实施例的访问数据库的方法的流程示意图；2 is a flow chart showing a method of accessing a database according to another embodiment of the present invention;
图3示出了根据本发明一个实施例的访问数据库的装置的结构示意图；FIG. 3 is a schematic structural diagram of an apparatus for accessing a database according to an embodiment of the present invention; FIG.
图4示出了根据本发明另一个实施例的访问数据库的装置的结构示意图；FIG. 4 is a schematic structural diagram of an apparatus for accessing a database according to another embodiment of the present invention; FIG.
图5示意性地示出了用于执行根据本发明的访问数据库的方法的计算设 备的框图；以及Figure 5 is a schematic illustration of a computing setup for performing a method of accessing a database in accordance with the present invention Prepared block diagram;
图6示意性地示出了用于保持或者携带实现根据本发明的访问数据库的方法的程序代码的存储单元。Fig. 6 schematically shows a storage unit for holding or carrying program code implementing a method of accessing a database according to the present invention.
具体实施方式detailed description
下面结合附图和具体的实施方式对本发明作进一步的描述。The invention is further described below in conjunction with the drawings and specific embodiments.
为解决上述技术问题，本发明实施例提供了一种访问数据库的方法。图1示出了根据本发明一个实施例的访问数据库的方法的流程示意图。如图1所示，该方法至少可以包括步骤S102、步骤S104以及步骤S106：To solve the above technical problem, an embodiment of the present invention provides a method for accessing a database. FIG. 1 shows a flow diagram of a method of accessing a database in accordance with one embodiment of the present invention. As shown in FIG. 1, the method may at least include step S102, step S104, and step S106:
步骤S102，接收来自客户端的访问数据库的请求；Step S102, receiving a request from a client to access a database;
步骤S104，对该请求进行类型分析，以确定该请求是否为数据写入请求；Step S104, performing type analysis on the request to determine whether the request is a data write request;
步骤S106，当该请求为数据写入请求时，将该请求相应的写入操作记录至本地日志，并将该请求相应的数据写入数据库的缓存层。Step S106, when the request is a data write request, the corresponding write operation of the request is recorded to the local log, and the corresponding data of the request is written into the cache layer of the database.
在该步骤中，本地日志是指数据库在本地物理机上硬盘存储的日志文件，它用来记录数据库被访问的操作。In this step, the local log refers to the log file stored by the database on the hard disk of the local physical machine, which is used to record the operation of the database being accessed.
在本发明实施例中，当接收到来自客户端的访问数据库的请求时，对该请求进行类型分析以确定该请求是否为数据写入请求。当确定该请求为数据写入请求时，将该数据写入请求相应的写入操作记录至本地日志，并将该数据写入请求相应的数据写入数据库的缓存层。由此可知，本发明实施例直接将数据写入请求相应的数据写入数据库的缓存层，并向客户端返回响应消息，而无需将数据写入数据库，能够提高后续对所述数据读取的效率。并且，本发明实施例将数据写入请求相应的写入操作记录至本地日志，从而可以利用本地日志异步地将写入请求相应的数据写入数据库，保证数据库中数据的持久化和完整性。另外，在数据库出现故障重新启动后，也可以利用本地日志对数据库进行数据恢复，实现故障修复的目的。In an embodiment of the invention, when a request from a client to access a database is received, a type analysis is performed on the request to determine if the request is a data write request. When it is determined that the request is a data write request, the corresponding write operation of the data write request is recorded to the local log, and the corresponding data of the data write request is written into the cache layer of the database. Therefore, the embodiment of the present invention directly writes the data corresponding to the data write request to the cache layer of the database, and returns a response message to the client without writing the data into the database, thereby improving subsequent reading of the data. effectiveness. Moreover, the embodiment of the present invention records the corresponding write operation of the data write request to the local log, so that the local log can be used to asynchronously write the corresponding data of the write request into the database, thereby ensuring the persistence and integrity of the data in the database. In addition, after the database fails to restart, you can also use the local log to recover data from the database to achieve the purpose of fault repair.
在本发明的可选实施例中，为了提高数据库访问的安全性，在上文步骤S102接收到来自客户端的访问数据库的请求后，可以对该请求的合法性进行验证，在验证通过后进行后续的步骤S104以及步骤S106，若验证不通过，则进行拦截处理，并发出告警提示。这里的合法性是根据预先设置的安全认证策略来度量的，安全认证策略可以包括权限的合法性、请求的有效性或者 攻击检查等，本发明不限于此。例如，安全认证策略可以设置为根据哪个IP地址的客户端对数据库的查询是不允许的；除了某个IP地址外的客户端是不能删除、修改数据库中的数据，等等策略。In an optional embodiment of the present invention, in order to improve the security of the database access, after receiving the request for accessing the database from the client in step S102 above, the validity of the request may be verified, and the verification is performed after the verification is passed. In step S104 and step S106, if the verification fails, the interception process is performed, and an alert is issued. The legality here is measured according to a preset security authentication policy, which may include the validity of the permission, the validity of the request, or The attack check or the like is not limited to the present invention. For example, the security authentication policy can be set to be based on which IP address the client is not allowed to query the database; except for a certain IP address, the client cannot delete, modify the data in the database, and so on.
在本发明的可选实施例中，在步骤S104对该请求进行类型分析时，可以从该请求中解析出访问数据库的语句，分析该语句是哪种类型的操作，从而确定该请求是否为数据写入请求。这里的类型如增加、删除、修改、查询等，当语句为增加操作的语句时，则确定该请求为数据写入请求；当语句为删除操作的语句时，则确定该请求为数据写入请求；当语句为修改操作的语句时，则确定该请求为数据写入请求；当语句为查询操作的语句时，则确定该请求为数据读取请求，等等。In an optional embodiment of the present invention, when the request is type-checked in step S104, the statement accessing the database may be parsed from the request, and the type of operation of the statement is analyzed to determine whether the request is data. Write request. The types here are such as add, delete, modify, query, etc. When the statement is a statement to increase the operation, the request is determined to be a data write request; when the statement is a delete operation statement, the request is determined to be a data write request. When the statement is a statement to modify the operation, it is determined that the request is a data write request; when the statement is a statement of the query operation, the request is determined to be a data read request, and so on.
在本发明的可选实施例中，上文步骤S106中提及的缓存层可以是在数据库之外新建的缓存层。由于数据库本身不具备缓存层，本发明实施例选择在数据库之外新建缓存层用于临时存储数据。例如，在将写入请求相应的写入操作记录至本地日志后，将写入请求相应的数据写入新建的缓存层，并向客户端返回响应消息，从而提高后续对所述数据读取的效率。这里，可以根据业务需求选取相应存储容量的缓存层，缓存层的类型可以如分布式的高速缓存***memcached或者基于内容存的redis等。In an optional embodiment of the present invention, the cache layer mentioned in the above step S106 may be a cache layer newly created outside the database. Since the database itself does not have a cache layer, the embodiment of the present invention chooses to create a new cache layer outside the database for temporarily storing data. For example, after the corresponding write operation of the write request is recorded to the local log, the corresponding data of the write request is written into the newly created cache layer, and a response message is returned to the client, thereby improving subsequent reading of the data. effectiveness. Here, the cache layer of the corresponding storage capacity may be selected according to the service requirement, and the type of the cache layer may be, for example, a distributed cache system memcached or a content-based redis.
在本发明的可选实施例中，上文步骤S106将该请求相应的写入操作记录至本地日志，并将该请求相应的数据写入数据库的缓存层，本发明实施例提供了一种可选的方案，在该方案中，可以调用第一线程，并利用第一线程将请求相应的写入操作记录至本地日志并将请求相应的数据写入缓存层。也就是说，由第一线程将请求相应的写入操作记录至本地日志，随后将请求相应的数据写入缓存层，之后向客户端返回表示已完成数据库写入的响应消息。In an optional embodiment of the present invention, the above step S106 records the corresponding write operation of the request to the local log, and writes the corresponding data of the request to the cache layer of the database. In the solution, the first thread can be invoked, and the first thread is used to record the corresponding write operation to the local log and the corresponding data is requested to be written to the cache layer. That is to say, the first thread writes the corresponding write operation to the local log, and then writes the corresponding data to the cache layer, and then returns a response message indicating that the database write has been completed to the client.
进一步地，为了保证数据库中数据的持久化和完整性，本发明实施例可以利用本地日志异步地将写入请求相应的数据写入数据库，即，调用第二线程并利用第二线程扫描本地日志以确定本地日志中是否存在日志增量，其中第二线程与第一线程异步，若发现本地日志中存在日志增量，则将日志增量相应的数据写入数据库中。这里，第二线程可以实时地对本地日志进行扫描，也可以设定指定的扫描周期(如0.5秒或1秒等)对本地日志进行扫描。Further, in order to ensure the persistence and integrity of the data in the database, the embodiment of the present invention can use the local log to asynchronously write the corresponding data of the write request to the database, that is, call the second thread and scan the local log by using the second thread. To determine whether there is a log increment in the local log, where the second thread is asynchronous with the first thread, and if there is a log increment in the local log, the corresponding data of the log increment is written into the database. Here, the second thread can scan the local log in real time, or can set a specific scanning period (such as 0.5 seconds or 1 second, etc.) to scan the local log.
在本发明的可选实施例中，在确定日志增量时，本发明实施例提供了一 种可选的方案，即，利用第二线程扫描本地日志，确定本地日志中是否存在未被标记的日志记录，若存在未被标记的日志记录，则确定未被标记的日志记录为日志增量。进一步地，将日志增量相应的数据写入数据库中之后，在本地日志中对日志增量相应的日志记录进行标记，以表示日志增量相应的数据已写入数据库中，可以避免重复写入的问题。例如，将日志增量相应的数据写入数据库中之后，可以在本地日志中标记日志增量相应的日志记录为“已操作”或“已记录”等。In an optional embodiment of the present invention, when determining the log increment, the embodiment of the present invention provides a An optional solution is to use the second thread to scan the local log to determine whether there is an unmarked log record in the local log, and if there is an unmarked log record, determine that the unmarked log record is a log increment. . Further, after the corresponding data of the log increment is written into the database, the corresponding log record of the log increment is marked in the local log to indicate that the corresponding data of the log increment has been written into the database, and the repeated writing can be avoided. The problem. For example, after the corresponding data of the log increment is written into the database, the corresponding log record of the log increment can be marked in the local log as "operated" or "recorded".
在本发明的可选实施例中，为了节省本地存储空间，可以删除本地日志中已被标记的日志记录，例如，对本地日志进行设置，每隔预定时间可以删除已被标记的日志记录。另外，本地日志可以按照大小切分，当某个切分区域中的日志记录已全部被标记，则可以删除该切分区域中的日志记录。In an optional embodiment of the present invention, in order to save local storage space, the marked log records in the local log may be deleted, for example, the local log is set, and the marked log records may be deleted every predetermined time. In addition, the local log can be divided according to the size. When the log records in a certain segmentation area are all marked, the log records in the segmentation area can be deleted.
在本发明的可选实施例中，如果因为宕机或其他因素导致数据库出现故障，使得新写在缓存层中的数据还未在数据库中持久化，则在下次重启数据库后，可以利用本地日志对数据库进行数据恢复，实现故障修复的目的。具体地，在数据库出现故障重新启动后，启用扫描线程，并利用扫描线程扫描本地日志，确定本地日志中是否存在未被标记的日志记录，若是，则将未被标记的日志记录相应的数据写入数据库，并在本地日志中对未被标记的日志记录进行标记。例如，新写在缓存层中的数据“b＝2”未在数据库中持久化(“b＝2”相应的写入操作已记录至本地日志)，数据库突发故障，在数据库重新启动后，启用扫描线程，并利用扫描线程扫描本地日志，发现本地日志中存在未被标记的日志记录，即“b＝2”相应的写入操作，则将“b＝2”写入数据库，并在本地日志中对未被标记的日志记录(即“b＝2”相应的写入操作)进行标记，如标记为“已操作”或“已记录”等。In an alternative embodiment of the present invention, if the database fails due to downtime or other factors, so that the data newly written in the cache layer has not been persisted in the database, the local log can be utilized after the database is restarted next time. Data recovery for the database to achieve the purpose of fault repair. Specifically, after the database fails to restart, the scan thread is enabled, and the scan thread is used to scan the local log to determine whether there is an unmarked log record in the local log, and if so, the unmarked log record is written correspondingly. Enter the database and mark unmarked log records in the local log. For example, the data "b=2" newly written in the cache layer is not persisted in the database ("b=2" corresponding write operation has been logged to the local log), the database is in a burst, after the database is restarted, Enable the scan thread and scan the local log with the scan thread. If there is an unmarked log record in the local log, that is, the corresponding write operation of "b=2", write "b=2" to the database and localize it. The unmarked log records (that is, the corresponding write operations of "b=2") are marked in the log, such as marked as "operated" or "recorded".
进一步地，在数据恢复后数据库正常工作，缓存层中也有新的数据写入(这里，可以采用前文提及的第一线程将请求相应的写入操作记录至本地日志并将请求相应的数据写入缓存层)，此时可以继续利用扫描线程扫描本地日志，以确定本地日志中是否存在日志增量，若发现本地日志中存在日志增量，则将日志增量相应的数据写入数据库中。这里的扫描线程与前文提及的第二线程作用相同，扫描线程与第一线程异步。Further, after the data is restored, the database works normally, and new data is also written in the cache layer (here, the first thread mentioned above may be used to record the corresponding write operation to the local log and request the corresponding data to be written. Into the cache layer, you can continue to use the scan thread to scan the local log to determine whether there is a log increment in the local log. If there is a log increment in the local log, the corresponding data of the log increment is written into the database. The scan thread here has the same effect as the second thread mentioned above, and the scan thread is asynchronous with the first thread.
在本发明的可选实施例中，由于在缓存层对数据进行了缓存，因而当来自客户端的访问数据库的请求为数据的读取请求时，则可以直接在缓存层中 查找该读取请求相应的数据，并将查找到的数据返回给客户端，从而可以减少对数据库的读取访问，提高数据读取的效率。In an optional embodiment of the present invention, since the data is cached at the cache layer, when the request for accessing the database from the client is a data read request, it may be directly in the cache layer. Find the corresponding data of the read request, and return the found data to the client, thereby reducing the read access to the database and improving the efficiency of data reading.
在本发明的可选实施例中，如果数据库故障或其他因素导致缓存层数据部分或全部丢失，使得在缓存层中未查找到读取请求相应的数据，则本发明实施例可以将读取请求转发至数据库，在数据库中查找读取请求相应的数据，将查找到的读取请求相应的数据返回给客户端，并在缓存层中缓存读取请求相应的数据。In an optional embodiment of the present invention, if a database failure or other factors cause part or all of the cache layer data to be lost, so that the corresponding data of the read request is not found in the cache layer, the embodiment of the present invention may read the request. Forwarding to the database, looking up the corresponding data of the read request in the database, returning the corresponding data of the read request to the client, and buffering the corresponding data of the read request in the cache layer.
下面将通过一具体实施例详细介绍本发明的访问数据库的方法的实现过程。在该实施例中，根据业务需求选取相应存储容量的缓存层，为数据库新建缓存层，缓存层的类型可以如分布式的高速缓存***memcached或者基于内存的redis等。The implementation process of the method for accessing the database of the present invention will be described in detail below through a specific embodiment. In this embodiment, the cache layer of the corresponding storage capacity is selected according to the service requirement, and a cache layer is newly created for the database, and the type of the cache layer may be a distributed cache system memcached or a memory-based redis.
图2示出了根据本发明另一个实施例的访问数据库的方法的流程示意图。如图2所示，该方法至少可以包括步骤S202、步骤S204、步骤S206、步骤S208以及步骤S210。2 is a flow chart showing a method of accessing a database according to another embodiment of the present invention. As shown in FIG. 2, the method may at least include step S202, step S204, step S206, step S208, and step S210.
步骤S202，为数据库新建缓存层，接收来自客户端的访问数据库的请求。Step S202, a new cache layer is created for the database, and a request for accessing the database from the client is received.
步骤S204，对该请求进行类型分析，确定该请求是否为数据写入请求，若是，则继续执行步骤S206；若否，则继续执行步骤S208。Step S204, performing type analysis on the request to determine whether the request is a data write request, and if yes, proceeding to step S206; if not, proceeding to step S208.
步骤S206，调用第一线程，并利用第一线程将请求相应的写入操作记录至本地日志并将请求相应的数据写入缓存层。Step S206, the first thread is called, and the first thread is used to record the corresponding write operation to the local log and the corresponding data is requested to be written into the cache layer.
在该步骤中，本地日志是指数据库在本地物理机上硬盘存储的日志文件，它用来记录数据库被访问的操作。In this step, the local log refers to the log file stored by the database on the hard disk of the local physical machine, which is used to record the operation of the database being accessed.
步骤S208，当来自客户端的访问数据库的请求为数据的读取请求时，在缓存层中查找该读取请求相应的数据，并将查找到的数据返回给客户端。Step S208: When the request for accessing the database from the client is a data read request, the data corresponding to the read request is searched in the cache layer, and the found data is returned to the client.
在该步骤中，如果数据库故障或其他因素导致缓存层数据部分或全部丢失，使得在缓存层中未查找到读取请求相应的数据，则本发明实施例可以将读取请求转发至数据库，在数据库中查找读取请求相应的数据，将查找到的读取请求相应的数据返回给客户端，并在缓存层中缓存读取请求相应的数据。In this step, if the database failure or other factors cause the cache layer data to be partially or completely lost, so that the corresponding data of the read request is not found in the cache layer, the embodiment of the present invention may forward the read request to the database. The database searches for the corresponding data of the read request, returns the corresponding data of the read request to the client, and caches the corresponding data of the read request in the cache layer.
步骤S210，调用第二线程并利用第二线程扫描本地日志以确定本地日志中是否存在日志增量，其中第二线程与第一线程异步，若发现本地日志中 存在日志增量，则将日志增量相应的数据写入数据库中。Step S210, invoking the second thread and scanning the local log by using the second thread to determine whether there is a log increment in the local log, where the second thread is asynchronous with the first thread, if the local log is found If there is a log increment, the corresponding data of the log increment is written to the database.
在确定日志增量时，可以利用第二线程扫描本地日志，确定本地日志中是否存在未被标记的日志记录，若存在未被标记的日志记录，则确定未被标记的日志记录为日志增量。进一步地，将日志增量相应的数据写入数据库中之后，在本地日志中对日志增量相应的日志记录进行标记，以表示日志增量相应的数据已写入数据库中，可以避免重复写入的问题。例如，将日志增量相应的数据写入数据库中之后，可以在本地日志中标记日志增量相应的日志记录为“已操作”或“已记录”等。另外，为了节省本地存储空间，可以删除本地日志中已被标记的日志记录。When determining the log increment, the second thread may be used to scan the local log to determine whether there is an unmarked log record in the local log. If there is an unmarked log record, it is determined that the unmarked log record is a log increment. . Further, after the corresponding data of the log increment is written into the database, the corresponding log record of the log increment is marked in the local log to indicate that the corresponding data of the log increment has been written into the database, and the repeated writing can be avoided. The problem. For example, after the corresponding data of the log increment is written into the database, the corresponding log record of the log increment can be marked in the local log as "operated" or "recorded". In addition, to save local storage space, you can delete the log records that have been marked in the local log.
在本发明实施例中，如果因为宕机或其他因素导致数据库出现故障，使得新写在缓存层中的数据还未在数据库中持久化，则在下次重启数据库后，可以利用本地日志对数据库进行数据恢复，实现故障修复的目的。具体地，在数据库出现故障重新启动后，启用扫描线程，并利用扫描线程扫描本地日志，确定本地日志中是否存在未被标记的日志记录，若是，则将未被标记的日志记录相应的数据写入数据库，并在本地日志中对未被标记的日志记录进行标记。In the embodiment of the present invention, if the database fails due to downtime or other factors, so that the data newly written in the cache layer has not been persisted in the database, the database can be performed by using the local log after the database is restarted next time. Data recovery to achieve the purpose of fault repair. Specifically, after the database fails to restart, the scan thread is enabled, and the scan thread is used to scan the local log to determine whether there is an unmarked log record in the local log, and if so, the unmarked log record is written correspondingly. Enter the database and mark unmarked log records in the local log.
进一步地，在数据恢复后数据库正常工作，缓存层中也有新的数据写入(这里，可以采用前文提及的第一线程将请求相应的写入操作记录至本地日志并将请求相应的数据写入缓存层)，此时可以继续利用扫描线程扫描本地日志，以确定本地日志中是否存在日志增量，若发现本地日志中存在日志增量，则将日志增量相应的数据写入数据库中。这里的扫描线程与前文提及的第二线程作用相同，扫描线程与第一线程异步。Further, after the data is restored, the database works normally, and new data is also written in the cache layer (here, the first thread mentioned above may be used to record the corresponding write operation to the local log and request the corresponding data to be written. Into the cache layer, you can continue to use the scan thread to scan the local log to determine whether there is a log increment in the local log. If there is a log increment in the local log, the corresponding data of the log increment is written into the database. The scan thread here has the same effect as the second thread mentioned above, and the scan thread is asynchronous with the first thread.
需要说明的是，在实际应用中，上述所有可选实施方式可以采用结合的方式任意组合，形成本发明的可选实施例，在此不再一一赘述。It should be noted that, in an actual application, all the foregoing optional embodiments may be combined in any combination to form an optional embodiment of the present invention, and details are not described herein again.
基于上文各个实施例提供的访问数据库的方法，基于同一发明构思，本发明实施例还提供了一种访问数据库的装置。Based on the method for accessing a database provided by the foregoing embodiments, the embodiment of the present invention further provides an apparatus for accessing a database.
图3示出了根据本发明一个实施例的访问数据库的装置的结构示意图。如图3所示，该装置300至少可以包括请求接收模块310、请求分析模块320以及第一写入模块330。FIG. 3 is a block diagram showing the structure of an apparatus for accessing a database according to an embodiment of the present invention. As shown in FIG. 3, the apparatus 300 can include at least a request receiving module 310, a request analyzing module 320, and a first writing module 330.
现介绍本发明实施例的访问数据库的装置300的各组成或器件的功能以及各部分间的连接关系： The functions of the components or devices of the device 300 for accessing the database and the connection relationship between the components of the embodiment of the present invention will now be described:
请求接收模块310，适于接收来自客户端的访问数据库的请求；The request receiving module 310 is adapted to receive a request from the client to access the database;
请求分析模块320，与请求接收模块310相耦合，适于对请求进行类型分析，以确定请求是否为数据写入请求；以及The request analysis module 320 is coupled to the request receiving module 310 and is adapted to perform type analysis on the request to determine whether the request is a data write request;
第一写入模块330，与请求分析模块320相耦合，适于当请求分析模块确定请求为数据的写入请求时，将请求相应的写入操作记录至本地日志，并将请求相应的数据写入数据库的缓存层。The first write module 330 is coupled to the request analysis module 320, and is adapted to, when the request analysis module determines that the request is a write request for data, record the corresponding write operation to the local log, and write the corresponding data request. Enter the cache layer of the database.
在本发明一实施例中，如图4所示，图3展示的装置还可以包括：In an embodiment of the present invention, as shown in FIG. 4, the apparatus shown in FIG. 3 may further include:
读取模块340，与请求分析模块320、第一写入模块330相耦合，适于当请求分析模块320确定请求为数据的读取请求时，在缓存层中查找请求相应的数据，并将查找到的数据返回给客户端。The reading module 340 is coupled to the request analysis module 320 and the first write module 330, and is adapted to look up the request corresponding data in the cache layer when the request analysis module 320 determines that the request is a read request for data, and is to search The data returned is returned to the client.
在本发明一实施例中，第一写入模块330还适于：In an embodiment of the invention, the first writing module 330 is further adapted to:
调用第一线程，并利用第一线程将请求相应的写入操作记录至本地日志并将请求相应的数据写入缓存层。The first thread is called, and the first thread is used to record the corresponding write operation to the local log and the corresponding data is requested to be written to the cache layer.
在本发明一实施例中，如图4所示，图3展示的装置还可以包括：In an embodiment of the present invention, as shown in FIG. 4, the apparatus shown in FIG. 3 may further include:
第二写入模块350，与第一写入模块330相耦合，适于调用第二线程并利用第二线程扫描本地日志以确定本地日志中是否存在日志增量，其中第二线程与第一线程异步；以及若发现本地日志中存在日志增量，则将日志增量相应的数据写入数据库中。The second write module 350 is coupled to the first write module 330, and is adapted to invoke the second thread and scan the local log by using the second thread to determine whether there is a log increment in the local log, where the second thread and the first thread Asynchronous; and if a log increment is found in the local log, the corresponding data of the log increment is written to the database.
在本发明一实施例中，第二写入模块350还适于：In an embodiment of the invention, the second writing module 350 is further adapted to:
利用第二线程扫描本地日志，确定本地日志中是否存在未被标记的日志记录；以及Scan the local log with a second thread to determine if there are unmarked log records in the local log;
若是，则确定未被标记的日志记录为日志增量。If so, it is determined that the unmarked log record is a log increment.
在本发明一实施例中，如图4所示，图3展示的装置还可以包括：In an embodiment of the present invention, as shown in FIG. 4, the apparatus shown in FIG. 3 may further include:
标记模块360，与第二写入模块350相耦合，适于在第二写入模块350将日志增量相应的数据写入数据库中之后，在本地日志中对日志增量相应的日志记录进行标记。The marking module 360 is coupled to the second writing module 350, and is adapted to mark the corresponding log record of the log increment in the local log after the second writing module 350 writes the corresponding data of the log increment into the database. .
在本发明一实施例中，如图4所示，图3展示的装置还可以包括：In an embodiment of the present invention, as shown in FIG. 4, the apparatus shown in FIG. 3 may further include:
删除模块370，与标记模块360相耦合，适于删除本地日志中已被标记的日志记录。A deletion module 370, coupled to the tagging module 360, is adapted to delete the marked log records in the local log.
在本发明一实施例中，如图4所示，图3展示的装置还可以包括：In an embodiment of the present invention, as shown in FIG. 4, the apparatus shown in FIG. 3 may further include:
故障处理模块380，与第一写入模块330、标记模块360相耦合，适于 在数据库出现故障重新启动后，扫描本地日志，确定本地日志中是否存在未被标记的日志记录；以及The fault processing module 380 is coupled to the first writing module 330 and the marking module 360, and is adapted to After the database fails to restart, scan the local log to determine if there are unmarked log records in the local log;
若是，则将未被标记的日志记录相应的数据写入数据库，并在本地日志中对未被标记的日志记录进行标记。If so, the corresponding data of the unmarked log records is written to the database, and the unmarked log records are marked in the local log.
在本发明一实施例中，读取模块340还适于：In an embodiment of the invention, the reading module 340 is further adapted to:
当请求分析模块确定请求为数据的读取请求时，若在缓存层中未查找到请求相应的数据，则将请求转发至数据库；When the request analysis module determines that the request is a read request for data, if the corresponding data is not found in the cache layer, the request is forwarded to the database;
在数据库中查找请求相应的数据，将查找到的请求相应的数据返回给客户端，并在缓存层中缓存请求相应的数据。Find the corresponding data in the database, return the corresponding data of the found request to the client, and cache the corresponding data in the cache layer.
在本发明一实施例中，缓存层为分布式的高速缓存***memcached。In an embodiment of the invention, the cache layer is a distributed cache system memcached.
根据上述任意一个优选实施例或多个优选实施例的组合，本发明实施例能够达到如下有益效果：According to any one of the preferred embodiments or the combination of the preferred embodiments, the embodiment of the present invention can achieve the following beneficial effects:
在本发明实施例中，当接收到来自客户端的访问数据库的请求时，对该请求进行类型分析以确定该请求是否为数据写入请求。当确定该请求为数据写入请求时，将该数据写入请求相应的写入操作记录至本地日志，并将该数据写入请求相应的数据写入数据库的缓存层。由此可知，本发明实施例直接将数据写入请求相应的数据写入数据库的缓存层，并向客户端返回响应消息，其能够提高后续对所述数据读取的效率。并且，本发明实施例将数据写入请求相应的写入操作记录至本地日志，从而可以利用本地日志异步地将写入请求相应的数据写入数据库，保证数据库中数据的持久化和完整性。另外，在数据库出现故障重新启动后，也可以利用本地日志对数据库进行数据恢复，实现故障修复的目的。In an embodiment of the invention, when a request from a client to access a database is received, a type analysis is performed on the request to determine if the request is a data write request. When it is determined that the request is a data write request, the corresponding write operation of the data write request is recorded to the local log, and the corresponding data of the data write request is written into the cache layer of the database. It can be seen that the embodiment of the present invention directly writes the data corresponding to the data write request to the cache layer of the database, and returns a response message to the client, which can improve the efficiency of subsequent reading of the data. Moreover, the embodiment of the present invention records the corresponding write operation of the data write request to the local log, so that the local log can be used to asynchronously write the corresponding data of the write request into the database, thereby ensuring the persistence and integrity of the data in the database. In addition, after the database fails to restart, you can also use the local log to recover data from the database to achieve the purpose of fault repair.
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。In the description provided herein, numerous specific details are set forth. However, it is understood that the embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of the description.
类似地，应当理解，为了精简本公开并帮助理解各个发明方面中的一个或多个，在上面对本发明的示例性实施例的描述中，本发明的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释成反映如下意图：即所要求保护的本发明要求比在每个权利要求中所明确记载的特征更多的特征。更确切地说，如下面的权利要求书所反映的那样，发明方面在于少于前面公开的单个实施例的所有特征。因此，遵循 具体实施方式的权利要求书由此明确地并入该具体实施方式，其中每个权利要求本身都作为本发明的单独实施例。Similarly, the various features of the invention are sometimes grouped together into a single embodiment, in the above description of the exemplary embodiments of the invention, Figure, or a description of it. However, the method disclosed is not to be interpreted as reflecting the intention that the claimed invention requires more features than those recited in the claims. Rather, as the following claims reflect, inventive aspects reside in less than all features of the single embodiments disclosed herein. Therefore, follow The claims of the detailed description are hereby expressly incorporated into the specific embodiments, and each of the claims
本领域那些技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且把它们设置在与该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或组件组合成一个模块或单元或组件，以及此外可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过程或者单元中的至少一些是相互排斥之外，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。Those skilled in the art will appreciate that the modules in the devices of the embodiments can be adaptively changed and placed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and further they may be divided into a plurality of sub-modules or sub-units or sub-components. In addition to such features and/or at least some of the processes or units being mutually exclusive, any combination of the features disclosed in the specification, including the accompanying claims, the abstract and the drawings, and any methods so disclosed, or All processes or units of the device are combined. Each feature disclosed in this specification (including the accompanying claims, the abstract and the drawings) may be replaced by alternative features that provide the same, equivalent or similar purpose.
此外，本领域的技术人员能够理解，尽管在此所述的一些实施例包括其它实施例中所包括的某些特征而不是其它特征，但是不同实施例的特征的组合意味着处于本发明的范围之内并且形成不同的实施例。例如，在下面的权利要求书中，所要求保护的实施例的任意之一都可以以任意的组合方式来使用。In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features that are included in other embodiments and not in other features, combinations of features of different embodiments are intended to be within the scope of the present invention. Different embodiments are formed and formed. For example, in the following claims, any one of the claimed embodiments can be used in any combination.
本发明的各个部件实施例可以以硬件实现，或者以在一个或者多个处理器上运行的软件模块实现，或者以它们的组合实现。本领域的技术人员应当理解，可以在实践中使用微处理器或者数字信号处理器(DSP)来实现根据本发明实施例的访问数据库的装置中的一些或者全部部件的一些或者全部功能。本发明还可以实现为用于执行这里所描述的方法的一部分或者全部的设备或者装置程序(例如，计算机程序和计算机程序产品)。这样的实现本发明的程序可以存储在计算机可读介质上，或者可以具有一个或者多个信号的形式。这样的信号可以从因特网网站上下载得到，或者在载体信号上提供，或者以任何其他形式提供。The various component embodiments of the present invention may be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functionality of some or all of the components of the device for accessing a database in accordance with embodiments of the present invention. The invention can also be implemented as a device or device program (e.g., a computer program and a computer program product) for performing some or all of the methods described herein. Such a program implementing the invention may be stored on a computer readable medium or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
例如，图5示出了可以实现根据本发明的访问数据库的方法的计算设备。该计算设备传统上包括处理器510和以存储器520形式的计算机程序产品或者计算机可读介质。存储器520可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。存储器520具有存储用于执行上述方法中的任何方法步骤的程序代码531的存储空间530。例如，存储程序代码的存储空间530可以存储分别用于实现上面的方 法中的各种步骤的各个程序代码531。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。这些计算机程序产品包括诸如硬盘，紧致盘(CD)、存储卡或者软盘之类的程序代码载体。这样的计算机程序产品通常为例如图6所示的便携式或者固定存储单元。该存储单元可以具有与图5的计算设备中的存储器520类似布置的存储段、存储空间等。程序代码可以例如以适当形式进行压缩。通常，存储单元存储用于执行本发明的方法步骤的计算机可读代码531’，即可以由例如510之类的处理器读取的代码，当这些代码当由计算设备运行时，导致该计算设备执行上面所描述的方法中的各个步骤。For example, Figure 5 illustrates a computing device that can implement a method of accessing a database in accordance with the present invention. The computing device conventionally includes a processor 510 and a computer program product or computer readable medium in the form of a memory 520. The memory 520 may be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), an EPROM, a hard disk, or a ROM. Memory 520 has a storage space 530 that stores program code 531 for performing any of the method steps described above. For example, the storage space 530 storing the program code can be stored separately for implementing the above Each program code 531 of various steps in the method. The program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such a computer program product is typically a portable or fixed storage unit such as that shown in FIG. The storage unit may have storage segments, storage spaces, and the like that are similarly arranged to memory 520 in the computing device of FIG. The program code can be compressed, for example, in an appropriate form. Typically, the storage unit stores computer readable code 531' for performing the method steps of the present invention, ie, code that can be read by a processor, such as 510, when the code is run by a computing device, causing the computing device Perform the various steps in the method described above.
本文中所称的“一个实施例”、“实施例”或者“一个或者多个实施例”意味着，结合实施例描述的特定特征、结构或者特性包括在本发明的至少一个实施例中。此外，请注意，这里“在一个实施例中”的词语例子不一定全指同一个实施例。"an embodiment," or "an embodiment," or "an embodiment," In addition, it is noted that the phrase "in one embodiment" is not necessarily referring to the same embodiment.
应该注意的是上述实施例对本发明进行说明而不是对本发明进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本发明可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。It is to be noted that the above-described embodiments are illustrative of the invention and are not intended to be limiting, and that the invention may be devised without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as a limitation. The word "comprising" does not exclude the presence of the elements or steps that are not recited in the claims. The word "a" or "an" The invention can be implemented by means of hardware comprising several distinct elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means can be embodied by the same hardware item. The use of the words first, second, and third does not indicate any order. These words can be interpreted as names.
此外，还应当注意，本说明书中使用的语言主要是为了可读性和教导的目的而选择的，而不是为了解释或者限定本发明的主题而选择的。因此，在不偏离所附权利要求书的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。对于本发明的范围，对本发明所做的公开是说明性的，而非限制性的，本发明的范围由所附权利要求书限定。 In addition, it should be noted that the language used in the specification has been selected for the purpose of readability and teaching, and is not intended to be construed or limited. Therefore, many modifications and changes will be apparent to those skilled in the art without departing from the scope of the invention. The disclosure of the present invention is intended to be illustrative, and not restrictive, and the scope of the invention is defined by the appended claims.

Claims (22)

  1. 一种访问数据库的方法，包括：A method of accessing a database, including:
    接收来自客户端的访问数据库的请求；Receiving a request from the client to access the database;
    对所述请求进行类型分析，以确定所述请求是否为数据写入请求；以及Performing a type analysis on the request to determine if the request is a data write request;
    当所述请求为数据写入请求时，将所述请求相应的写入操作记录至本地日志，并将所述请求相应的数据写入所述数据库的缓存层。When the request is a data write request, the corresponding write operation of the request is recorded to a local log, and the corresponding data of the request is written to a cache layer of the database.
  2. 根据权利要求1所述的方法，还包括：The method of claim 1 further comprising:
    当所述请求为数据的读取请求时，在所述缓存层中查找所述请求相应的数据，并将查找到的所述数据返回给所述客户端。When the request is a read request of data, the corresponding data of the request is searched in the cache layer, and the found data is returned to the client.
  3. 根据权利要求1或2所述的方法，其中，所述将所述请求相应的写入操作记录至本地日志，并将所述请求相应的数据写入所述数据库的缓存层的步骤进一步包括：The method according to claim 1 or 2, wherein the step of recording the corresponding write operation of the request to the local log and writing the corresponding data of the request to the cache layer of the database further comprises:
    调用第一线程，并利用所述第一线程将所述请求相应的写入操作记录至所述本地日志并将所述请求相应的数据写入所述缓存层。The first thread is invoked, and the first thread is used to record the corresponding write operation of the request to the local log and the corresponding data of the request is written to the cache layer.
  4. 根据权利要求1-3中任一项所述的方法，还包括：A method according to any one of claims 1 to 3, further comprising:
    调用第二线程并利用所述第二线程扫描所述本地日志以确定所述本地日志中是否存在日志增量，其中所述第二线程与所述第一线程异步；以及Retrieving a second thread and scanning the local log with the second thread to determine if there is a log increment in the local log, wherein the second thread is asynchronous with the first thread;
    若发现所述本地日志中存在日志增量，则将所述日志增量相应的数据写入所述数据库中。If the log increment is found in the local log, the corresponding data of the log increment is written into the database.
  5. 根据权利要求1-4中任一项所述的方法，其中，所述利用所述第二线程扫描所述本地日志以确定所述本地日志中是否存在日志增量的步骤进一步包括：The method according to any one of claims 1 to 4, wherein the step of scanning the local log with the second thread to determine whether there is a log increment in the local log further comprises:
    利用所述第二线程扫描所述本地日志，确定所述本地日志中是否存在未被标记的日志记录；以及Scanning the local log by the second thread to determine whether there is an unmarked log record in the local log;
    若是，则确定所述未被标记的日志记录为所述日志增量。If so, it is determined that the unmarked log record is the log increment.
  6. 根据权利要求1-5中任一项所述的方法，其中，所述将所述日志增量相应的数据写入所述数据库中之后，所述方法还包括：The method according to any one of claims 1 to 5, wherein after the data corresponding to the log increment is written into the database, the method further comprises:
    在所述本地日志中对所述日志增量相应的日志记录进行标记。The corresponding log record of the log increment is marked in the local log.
  7. 根据权利要求1-6中任一项所述的方法，还包括：The method of any of claims 1-6, further comprising:
    删除所述本地日志中已被标记的日志记录。 Delete the marked log records in the local log.
  8. 根据权利要求1-7中任一项所述的方法，还包括：The method of any of claims 1-7, further comprising:
    在所述数据库出现故障重新启动后，扫描所述本地日志，确定所述本地日志中是否存在未被标记的日志记录；以及After the database fails to restart, scanning the local log to determine whether there is an unmarked log record in the local log;
    若是，则将所述未被标记的日志记录相应的数据写入所述数据库，并在所述本地日志中对所述未被标记的日志记录进行标记。If so, the unmarked log record corresponding data is written to the database, and the unmarked log record is marked in the local log.
  9. 根据权利要求1-8中任一项所述的方法，还包括：The method of any of claims 1-8, further comprising:
    当所述请求为数据的读取请求时，若在所述缓存层中未查找到所述请求相应的数据，则将所述请求转发至所述数据库；以及And when the request is a read request of data, if the corresponding data of the request is not found in the cache layer, forwarding the request to the database;
    在所述数据库中查找所述请求相应的数据，将查找到的所述请求相应的数据返回给所述客户端，并在所述缓存层中缓存所述请求相应的数据。Searching, in the database, the corresponding data of the request, returning the searched corresponding data of the request to the client, and buffering the corresponding data in the cache layer.
  10. 根据权利要求1-9中任一项所述的方法，其中，所述缓存层为分布式的高速缓存***memcached。The method of any of claims 1-9, wherein the cache layer is a distributed cache system memcached.
  11. 一种访问数据库的装置，包括：A device for accessing a database, comprising:
    请求接收模块，适于接收来自客户端的访问数据库的请求；a request receiving module adapted to receive a request from a client to access a database;
    请求分析模块，适于对所述请求进行类型分析，以确定所述请求是否为数据写入请求；以及Requesting an analysis module adapted to perform a type analysis on the request to determine if the request is a data write request;
    第一写入模块，适于当所述请求分析模块确定所述请求为数据的写入请求时，将所述请求相应的写入操作记录至本地日志，并将所述请求相应的数据写入所述数据库的缓存层。a first write module, configured to: when the request analysis module determines that the request is a write request for data, record a write operation corresponding to the request to a local log, and write corresponding data of the request The cache layer of the database.
  12. 根据权利要求11所述的装置，还包括：The apparatus of claim 11 further comprising:
    读取模块，适于当所述请求分析模块确定所述请求为数据的读取请求时，在所述缓存层中查找所述请求相应的数据，并将查找到的所述数据返回给所述客户端。a reading module, configured to: when the request analysis module determines that the request is a read request for data, search for data corresponding to the request in the cache layer, and return the found data to the Client.
  13. 根据权利要求11或12所述的装置，其中，所述第一写入模块还适于：The apparatus according to claim 11 or 12, wherein the first writing module is further adapted to:
    调用第一线程，并利用所述第一线程将所述请求相应的写入操作记录至所述本地日志并将所述请求相应的数据写入所述缓存层。The first thread is invoked, and the first thread is used to record the corresponding write operation of the request to the local log and the corresponding data of the request is written to the cache layer.
  14. 根据权利要求11-13中任一项所述的装置，还包括：The apparatus of any of claims 11-13, further comprising:
    第二写入模块，适于调用第二线程并利用所述第二线程扫描所述本地日志以确定所述本地日志中是否存在日志增量，其中所述第二线程与所述第一线程异步；以及若发现所述本地日志中存在日志增量，则将所述日志增量相 应的数据写入所述数据库中。a second write module, adapted to invoke the second thread and scan the local log by the second thread to determine whether there is a log increment in the local log, wherein the second thread is asynchronous with the first thread And if there is a log increment in the local log, the log increment is The data should be written to the database.
  15. 根据权利要求11-14中任一项所述的装置，其中，所述第二写入模块还适于：The apparatus of any of claims 11-14, wherein the second writing module is further adapted to:
    利用所述第二线程扫描所述本地日志，确定所述本地日志中是否存在未被标记的日志记录；以及Scanning the local log by the second thread to determine whether there is an unmarked log record in the local log;
    若是，则确定所述未被标记的日志记录为所述日志增量。If so, it is determined that the unmarked log record is the log increment.
  16. 根据权利要求11-15中任一项所述的装置，还包括：The apparatus of any of claims 11-15, further comprising:
    标记模块，适于在所述第二写入模块将所述日志增量相应的数据写入所述数据库中之后，在所述本地日志中对所述日志增量相应的日志记录进行标记。And a marking module, configured to mark, in the local log, a corresponding log record of the log increment after the second writing module writes the corresponding data of the log increment into the database.
  17. 根据权利要求11-16中任一项所述的装置，还包括：The apparatus of any of claims 11-16, further comprising:
    删除模块，适于删除所述本地日志中已被标记的日志记录。The delete module is adapted to delete the marked log records in the local log.
  18. 根据权利要求11-17中任一项所述的装置，还包括：The apparatus of any one of claims 11-17, further comprising:
    故障处理模块，适于在所述数据库出现故障重新启动后，扫描所述本地日志，确定所述本地日志中是否存在未被标记的日志记录；以及a fault processing module, configured to scan the local log after the database fails to restart, and determine whether there is an unmarked log record in the local log;
    若是，则将所述未被标记的日志记录相应的数据写入所述数据库，并在所述本地日志中对所述未被标记的日志记录进行标记。If so, the unmarked log record corresponding data is written to the database, and the unmarked log record is marked in the local log.
  19. 根据权利要求11-18中任一项所述的装置，其中，所述读取模块还适于：The apparatus of any of claims 11-18, wherein the reading module is further adapted to:
    当所述请求分析模块确定所述请求为数据的读取请求时，若在所述缓存层中未查找到所述请求相应的数据，则将所述请求转发至所述数据库；以及When the request analysis module determines that the request is a read request for data, if the request corresponding data is not found in the cache layer, forwarding the request to the database;
    在所述数据库中查找所述请求相应的数据，将查找到的所述请求相应的数据返回给所述客户端，并在所述缓存层中缓存所述请求相应的数据。Searching, in the database, the corresponding data of the request, returning the searched corresponding data of the request to the client, and buffering the corresponding data in the cache layer.
  20. 根据权利要求11-19中任一项所述的装置，其中，所述缓存层为分布式的高速缓存***memcached。The apparatus of any of claims 11-19, wherein the cache layer is a distributed cache system memcached.
  21. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在计算设备上运行时，导致所述计算设备执行根据权利要求1-10中任一项所述的访问数据库的方法。A computer program comprising computer readable code, when the computer readable code is run on a computing device, causing the computing device to perform the method of accessing a database according to any one of claims 1-10.
  22. 一种计算机可读介质，其中存储了如权利要求21所述的计算机程序。 A computer readable medium storing the computer program of claim 21.
PCT/CN2017/080290 2016-05-06 2017-04-12 Method and device for accessing database WO2017190580A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610299223.1 2016-05-06
CN201610299223.1A CN106021335A (en) 2016-05-06 2016-05-06 A database accessing method and device

Publications (1)

Publication Number Publication Date
WO2017190580A1 true WO2017190580A1 (en) 2017-11-09

Family

ID=57082231

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/080290 WO2017190580A1 (en) 2016-05-06 2017-04-12 Method and device for accessing database

Country Status (2)

Country Link
CN (1) CN106021335A (en)
WO (1) WO2017190580A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107895039A (en) * 2017-11-29 2018-04-10 华中科技大学 A kind of construction method of campus network Verification System log database
CN108829730A (en) * 2018-05-11 2018-11-16 中国平安人寿保险股份有限公司 The method, apparatus and its readable storage medium storing program for executing that a kind of pair of daily record data is stored
CN111008233A (en) * 2019-11-24 2020-04-14 浪潮电子信息产业股份有限公司 Method, device and equipment for accessing KV database
CN112231274A (en) * 2020-10-16 2021-01-15 京东数字科技控股股份有限公司 Log summarizing method and device, electronic equipment and storage medium

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106021335A (en) * 2016-05-06 2016-10-12 北京奇虎科技有限公司 A database accessing method and device
CN106713450A (en) * 2016-12-21 2017-05-24 广州优视网络科技有限公司 Downloading acceleration method and apparatus based on read-write separation mode
CN108733704B (en) * 2017-04-20 2020-11-20 北京京东尚科信息技术有限公司 Multi-database data processing method and device, storage medium and electronic equipment
CN109213432B (en) * 2017-07-04 2021-10-26 华为技术有限公司 Storage device for writing data using log structured merge tree and method thereof
CN107341267A (en) * 2017-07-24 2017-11-10 郑州云海信息技术有限公司 A kind of distributed file system access method and platform
CN110019132A (en) * 2017-12-20 2019-07-16 北京潘达互娱科技有限公司 Thumb up date storage method and equipment
CN110019345A (en) * 2017-12-28 2019-07-16 北京京东尚科信息技术有限公司 Data processing method, device, system and medium
CN110908838B (en) * 2019-11-19 2022-09-02 杭州安恒信息技术股份有限公司 Data processing method and device, electronic equipment and storage medium
CN114489480A (en) * 2021-12-23 2022-05-13 深圳市世强元件网络有限公司 Method and system for high-concurrency data storage

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103810056A (en) * 2012-11-06 2014-05-21 南京壹进制信息技术有限公司 Method for obtaining consistent backup on Linux platform
CN104317926A (en) * 2014-10-31 2015-01-28 北京思特奇信息技术股份有限公司 Persistent data storage method, persistent data query method, persistent data storage device and persistent data query system
CN104573128A (en) * 2014-10-28 2015-04-29 北京国双科技有限公司 Business data processing method, a business data processing device and server
CN104834724A (en) * 2015-05-12 2015-08-12 广东睿江科技有限公司 Method and device for synchronizing database
CN106021335A (en) * 2016-05-06 2016-10-12 北京奇虎科技有限公司 A database accessing method and device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1858738B (en) * 2006-02-15 2010-08-25 华为技术有限公司 Method and device for access data bank

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103810056A (en) * 2012-11-06 2014-05-21 南京壹进制信息技术有限公司 Method for obtaining consistent backup on Linux platform
CN104573128A (en) * 2014-10-28 2015-04-29 北京国双科技有限公司 Business data processing method, a business data processing device and server
CN104317926A (en) * 2014-10-31 2015-01-28 北京思特奇信息技术股份有限公司 Persistent data storage method, persistent data query method, persistent data storage device and persistent data query system
CN104834724A (en) * 2015-05-12 2015-08-12 广东睿江科技有限公司 Method and device for synchronizing database
CN106021335A (en) * 2016-05-06 2016-10-12 北京奇虎科技有限公司 A database accessing method and device

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107895039A (en) * 2017-11-29 2018-04-10 华中科技大学 A kind of construction method of campus network Verification System log database
CN108829730A (en) * 2018-05-11 2018-11-16 中国平安人寿保险股份有限公司 The method, apparatus and its readable storage medium storing program for executing that a kind of pair of daily record data is stored
CN111008233A (en) * 2019-11-24 2020-04-14 浪潮电子信息产业股份有限公司 Method, device and equipment for accessing KV database
CN112231274A (en) * 2020-10-16 2021-01-15 京东数字科技控股股份有限公司 Log summarizing method and device, electronic equipment and storage medium
CN112231274B (en) * 2020-10-16 2024-04-05 京东科技控股股份有限公司 Log summarizing method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN106021335A (en) 2016-10-12

Similar Documents

Publication Publication Date Title
WO2017190580A1 (en) Method and device for accessing database
US9875183B2 (en) Method and apparatus for content derived data placement in memory
CN106302595B (en) Method and equipment for carrying out health check on server
KR20090026296A (en) Predictive data-loader
JP2015511347A5 (en)
CN107807797B (en) Data writing method and device and server
TW201220197A (en) for improving the safety and reliability of data storage in a virtual machine based on cloud calculation and distributed storage environment
JPWO2015166540A1 (en) Storage apparatus, data processing method thereof, and storage system
WO2017084557A1 (en) File scanning method and device
WO2015149628A1 (en) Dns cache information processing method, device and system
WO2017097048A1 (en) Data searching method and apparatus
CN110888837B (en) Object storage small file merging method and device
TW201415216A (en) System and method for retrieving test logs
CN104239353B (en) WEB classification control and log audit method
US20150378745A1 (en) Data processing method during boot procedure of smart device and smart device
US10049113B2 (en) File scanning method and apparatus
WO2016107306A1 (en) Message subscription method, processing node device and message bus
CN112363995A (en) Incremental data comparison method and device based on log analysis and electronic equipment
CN110990365A (en) Data synchronization method, device, server and storage medium
WO2016095687A1 (en) Virtualisation security detection method and system
WO2021226822A1 (en) Log write method and apparatus, electronic device, and storage medium
CN106612283B (en) Method and device for identifying source of downloaded file
CN112860507A (en) Method and device for controlling sampling rate of distributed link tracking system
CN111382179A (en) Data processing method and device and electronic equipment
US20120089849A1 (en) Cookie management system and method

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17792407

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17792407

Country of ref document: EP

Kind code of ref document: A1