CN114036164A - High-concurrency transaction locking method and system combining optimistic lock and pessimistic lock - Google Patents

High-concurrency transaction locking method and system combining optimistic lock and pessimistic lock Download PDF

Info

Publication number
CN114036164A
CN114036164A CN202111322477.8A CN202111322477A CN114036164A CN 114036164 A CN114036164 A CN 114036164A CN 202111322477 A CN202111322477 A CN 202111322477A CN 114036164 A CN114036164 A CN 114036164A
Authority
CN
China
Prior art keywords
distributed
lock
pessimistic
pessimistic lock
transaction request
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111322477.8A
Other languages
Chinese (zh)
Inventor
陈达
张平
殷钊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Construction Bank Corp
Original Assignee
China Construction Bank Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Construction Bank Corp filed Critical China Construction Bank Corp
Priority to CN202111322477.8A priority Critical patent/CN114036164A/en
Publication of CN114036164A publication Critical patent/CN114036164A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2308Concurrency control
    • G06F16/2336Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
    • G06F16/2343Locking methods, e.g. distributed locking or locking implementation details
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to a high concurrent transaction locking method and system of a combination of an optimistic lock and a pessimistic lock, which comprises the steps of establishing a distributed pessimistic lock configuration file, judging the starting state of the distributed pessimistic lock according to the configuration file, adding the distributed pessimistic lock to a transaction request timely according to the checking state, additionally adding a distributed pessimistic lock on the basis of the optimistic lock of a database, dynamically adjusting the locking time, and carrying out logic control on the pessimistic lock before business logic access, thereby avoiding the waste of resources, reducing the pressure on components, realizing better performance and reducing the consumption of a non-functional system; through the combination of optimistic lock and pessimistic lock, the advantages of the optimistic lock and the pessimistic lock are fully exerted, the defects of the optimistic lock and the pessimistic lock are avoided, and the stability and the usability of the system are improved while the data consistency is ensured.

Description

High-concurrency transaction locking method and system combining optimistic lock and pessimistic lock
Technical Field
The invention relates to the technical field of distributed data processing and automatic program design, in particular to a high-concurrency transaction locking method and system combining optimistic locks and pessimistic locks.
Background
In order to process high-concurrency transactions, most B2C systems adopt a distributed system to relieve the processing pressure of a machine, a database operation layer usually adopts a database sub-table, different data are used for carrying out routing distinction on transactions according to different partition keys (shardingkeys) and a hash algorithm, and different data are respectively corresponding to different databases and different database tables, so that the problem that the pressure of a single database is too high when the data are excessively concentrated is solved. In addition, because of the limited processing speed of the database, dirty data problems may result if the amount of concurrency is too large for the same record, and thus optimistic locking of the database is often used to prevent the dirty data problems. However, a new problem is introduced, when the consumption transaction concurrency is particularly large, the transaction success rate (the number of successful transactions divided by the total number of transactions in a period of time) is reduced, that is, the transaction availability of the system is reduced, and there is no waiting mechanism for the client to continue processing the request when the system is idle, but an error is reported directly, so that the experience of the client is often reduced.
The pessimistic lock has strong exclusivity and has conservative attitude for the data modified by the outside world, so that the data is always in a locked state in the processing process, and the lock is not released until the data is modified. Compared with a pessimistic lock, the optimistic lock adopts a more optimistic and loose locking mechanism, only checks and judges data when a transaction modifies the data, and does not lock the data and resources all the time. At present, when processing high concurrent maintenance transactions, the use of optimistic lock on the database operation level is one of the common ways to control and maintain the consistency of transactions, and this way uses the updated version number for comparison. Most of the currently adopted service frameworks or other technical frameworks generally use a section-oriented programming technology to uniformly process some specific naming methods as a transaction in order to ensure the consistency of maintenance operations when processing maintenance transactions, when processing a request record a maintained by a client, the version number of the record is increased by one, the system does not submit a task, and at the same time, another maintenance request may also maintain the record a. The version number control method checks whether the version number to be maintained is the same as the existing version number in the database before maintenance, if so, the maintenance is allowed, otherwise, the exception is thrown, and the problem of dirty data submitted by the database at high concurrency is controlled by the method. In addition, for the high-concurrency system of the single system, locking is performed by using a multithreading mode, for example, locking is performed by using a mode in common multithreading such as synchronous lock, semaphore, variable of special domain, blocking queue and the like, so that the problem of high concurrency of the single system can be solved.
In addition, other pessimistic lock schemes are also employed in other companies or industries when dealing with high concurrency. For example, if a semaphore is set on the database operation level to perform locking, the locking mode increases the reading and writing times of the database; distributed locking via other open source frameworks such as zookeeper or redission are also available, which tend to be too dependent on the framework for flexibility. Other industry pessimistic locking schemes, such as the high concurrency system of the monolithic system, have a limitation in using the multithreading tool to lock, that is, the locking scheme is only convenient for the monolithic system, but cannot be used for the distributed system.
In another solution, a transaction failure retry mechanism is adopted, and in the processing logic after the database transaction is rolled back and the transaction is returned, the judgment on the return exception is added to try to re-execute all processing logic of the transaction. The method has the advantages that a third-party middleware is not required to be introduced, most transaction processing efficiency is not influenced, the defects that the whole transaction processing flow becomes complicated, the transaction chain becomes long, and if the transaction concurrency degree becomes large, excessive waste of system resources is easily caused.
If the locking of the database layer is adopted, for some specific application scenarios, such as financial transactions, due to the fact that other components or third parties need to be called, business logic is often already entered and various verification and remote processing operations are already carried out. Therefore, if pessimistic lock control is performed at the database level, a great deal of stress is applied to the databases of the own component system and the other component system, and even when the amount of conflict is too large, database crash occurs, so that the method is not a good way. Other pessimistic lock schemes, such as other open-source frameworks zookeeper and reddison, cannot adapt to the verification process of highly concurrent transactions because the parameters cannot be well adjusted and dynamic flexibility is lost.
Disclosure of Invention
In order to solve the defects of the prior art, the invention provides a high concurrent transaction locking method and system combining an optimistic lock and a pessimistic lock, wherein a distributed pessimistic lock is additionally added on the basis of the optimistic lock of a database, the locking time is dynamically adjusted, the logic control of the pessimistic lock is carried out before business logic access, the waste of resources is avoided, the pressure on components is reduced, the better performance can be realized, and the non-functional system consumption is less; through the combination of optimistic lock and pessimistic lock, the advantages of the optimistic lock and the pessimistic lock are fully exerted, the defects of the optimistic lock and the pessimistic lock are avoided, and the stability and the usability of the system are improved while the data consistency is ensured.
In order to achieve the above purpose, the technical scheme adopted by the invention comprises the following steps:
a high concurrent transaction locking method combining an optimistic lock and a pessimistic lock is characterized by comprising the following steps:
s1, establishing a distributed pessimistic lock configuration file, wherein the distributed pessimistic lock configuration file at least comprises a distributed pessimistic lock master switch and distributed pessimistic lock flag bits corresponding to each transaction code;
s2, obtaining a transaction request, wherein the transaction request at least comprises a transaction code and a client number corresponding to the transaction;
s3, checking the current distributed pessimistic lock master switch state; when the state of the distributed pessimistic lock master switch is closed, adding the distributed pessimistic lock to the transaction request, and marking the transaction request as pending;
s4, when the total switch state of the distributed pessimistic lock is open, the flag bit state of the distributed pessimistic lock of the transaction code corresponding to the transaction request is continuously checked; when the flag bit state of the distributed pessimistic lock is not enabled, the distributed pessimistic lock is not added to the transaction request, and the transaction request is marked to be processed;
s5, when the flag bit state of the distributed pessimistic lock is enabled, continuing to perform flow control judgment on the current distributed cache, and when the distributed cache is judged to have a fault, not adding the distributed pessimistic lock to the transaction request and marking the transaction request as pending;
s6, when the distributed cache is judged to have no fault, executing the operation of adding the distributed pessimistic lock to the transaction request;
s7, checking whether the distributed pessimistic lock is added successfully; when the distributed pessimistic lock is successfully added, the transaction request is marked as locked; when the addition of the distributed pessimistic lock is unsuccessful, marking the transaction request as pending;
s8, the marked transaction request continues to process transaction service processing logic; when the transaction request is marked as locked, executing pessimistic lock processing logic on the transaction request; when the transaction request is marked as pending, an optimistic lock is added to the transaction request and optimistic lock processing logic is executed.
Further, the method further comprises:
and S9, when the feedback processing of the transaction request marked as locked is completed, unlocking the distributed pessimistic lock added to the transaction request.
Further, the flow control judgment on the current distributed cache includes:
and verifying whether the failure frequency of continuous access to the distributed cache exceeds a preset threshold value, and judging that the distributed cache has a fault when the failure frequency of continuous access to the distributed cache exceeds the preset threshold value.
Further, the step S6 further includes:
when the distributed cache is judged to have faults, one transaction request is selected as a probing request, and the operation of adding a distributed pessimistic lock is executed on the probing request;
clearing the number of failures to continuously access the distributed cache when the addition of the distributed pessimistic lock to the probe request is successful.
Further, the performing the operation of adding the distributed pessimistic lock to the transaction request comprises:
and acquiring a client number corresponding to the transaction request, calling a put method of the distributed cache to add a distributed pessimistic lock to the client number corresponding to the transaction request, and setting the survival time of the distributed pessimistic lock.
Further, the checking whether the addition of the distributed pessimistic lock is successful comprises:
and verifying whether the put method of the distributed cache is successfully called, wherein the distributed pessimistic lock is successfully added when the calling is successful, and otherwise, the adding is not successful.
Further, the step S7 further includes:
when the addition of the distributed pessimistic lock is unsuccessful, re-executing the operation of adding the distributed pessimistic lock to the transaction request according to a preset waiting interval and the maximum retry number;
when the distributed pessimistic lock is successfully added within the maximum retry number, the transaction request is marked as locked; when the distributed pessimistic lock is still added unsuccessfully beyond the maximum number of retries, the transaction request is marked as pending.
The invention also relates to a high concurrent transaction locking system combining optimistic locks and pessimistic locks, which is characterized by comprising the following steps:
the configuration file management module is used for establishing a distributed pessimistic lock configuration file;
the distributed pessimistic lock state control module is used for checking the current distributed pessimistic lock master switch state and the distributed pessimistic lock flag bit state of the transaction code corresponding to the transaction request;
the distributed cache verification module is used for carrying out flow control judgment on the current distributed cache;
and the distributed pessimistic lock adding module is used for executing the operation of adding the distributed pessimistic lock to the transaction request and checking whether the addition of the distributed pessimistic lock is successful.
The invention also relates to a computer-readable storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the above-mentioned method.
The invention also relates to an electronic device, characterized in that it comprises a processor and a memory;
the memory is used for storing a distributed pessimistic lock configuration file and a transaction request;
the processor is used for executing the method by calling the distributed pessimistic lock configuration file and the transaction request.
The invention has the beneficial effects that:
by adopting the high concurrent transaction locking method and system combining the optimistic lock and the pessimistic lock, on the basis of the optimistic lock in the database, various kinds of verification are carried out through the configuration file, a distributed pessimistic lock is additionally added, the locking time is dynamically adjusted, the pessimistic lock is logically controlled before business logic is admitted, the waste of resources is avoided, the pressure on components is reduced, the more excellent performance can be realized, and the non-functional system consumption is less; through the combination of optimistic lock and pessimistic lock, the advantages of the optimistic lock and the pessimistic lock are fully exerted, the defects of the optimistic lock and the pessimistic lock are avoided, and the stability and the usability of the system are improved while the data consistency is ensured.
Drawings
FIG. 1 is a flow chart of a method for locking a high concurrent transaction by combining an optimistic lock and a pessimistic lock according to the present invention.
FIG. 2 is a schematic structural diagram of a high-concurrency transaction locking system combining an optimistic lock and a pessimistic lock according to the present invention.
Detailed Description
For a clearer understanding of the contents of the present invention, reference will be made to the accompanying drawings and examples.
For the database part, the technical scheme of the invention adopts an optimistic locking mode to lock the records of the same partition key. Because the processing speed of the database is slow, when the system is in a concurrent condition, two modification requests are likely to arrive at the database at the same time, at this time, the records need to be updated or modified in sequence aiming at the two modification requests, so a version number can be designed for the records of the database, the numerical value of the version number is increased by one when the records are modified each time, the version number is checked each time the modification or update request is submitted, and if the version number is not the maximum (latest) in the database, the update request is not the latest data in the database, and the request is rejected; if the version number at the moment is checked to be the maximum (latest) version number in the database, the latest version number in the database indicates that the update request is updated, and the update is successful. Through the optimistic lock of the database, the system can control the problem of dirty data generated during concurrency, so that the problem of inconsistent data of the database layer is solved, but only through the optimistic lock of the database layer, under the scene of large concurrency, the failure rate of the system is higher, because the optimistic lock rejects the submission request which is not the latest record according to the version number, the consistency of submitted data is guaranteed, and the experience of a client is not good.
On the basis of optimistic lock of the database, the invention also adds a distributed pessimistic lock and dynamically adjusts the locking time. For the transaction service application system, a pre-processing chain can be preferably designed, and processing logic required to be processed before business logic is dynamically configured through a configuration file. For example, the processing chain may be designed as a java-like object, a transaction processing chain is added to the processing stack file, and the distributed pessimistic lock control processing stack corresponding to the transaction is configured in the processing stack. And after the transaction request reaches the application system, acquiring the distributed pessimistic lock through the preprocessing logic of the transaction, and executing the pessimistic lock processing logic. The pessimistic lock is in a preprocessing chain before the business logic processing is authorized, and after the pessimistic lock is successfully acquired, the subsequent processing is carried out; if a pessimistic lock is not acquired or the wait time and retry number are exceeded, the subsequent processing can still be performed without using the pessimistic lock.
Specifically, the first aspect of the present invention relates to a method for locking a high concurrent transaction of a combination of optimistic locks and pessimistic locks, which has a step flow shown in fig. 1, and includes:
s1, establishing a distributed pessimistic lock configuration file, wherein the distributed pessimistic lock configuration file at least comprises a distributed pessimistic lock master switch and distributed pessimistic lock flag bits corresponding to each transaction code. And determining whether to execute the distributed pessimistic lock locking processing logic according to the distributed pessimistic lock master switch and the distributed pessimistic lock flag bit of the transaction code in the configuration file, wherein the pessimistic lock locking processing logic can be executed only under the condition that the master switch is opened and the flag bit is enabled, and the configuration parameters can be dynamically validated. A typical profile and parameters may include:
relevant attribute parameters based on redis distributed locks
Properties named envvariables
Pessimistic lock maximum latency:
pessimisticLockMaxWaitSecond=1000
pessimistic lock maximum timeout:
pessimisticLockMaxExpireSecond=3000
# pessimistic Lock master switch:
pessimisticLockSwitch=False
# distributed pessimistic lock flag bit:
pessimisticLockOdaSwitch=True
# RedisMaster name:
redisMasterName=mymaster
pessimistic lock maximum number of consecutive exceptions:
pessimisticLockMaxContinueFailNum=10
pessimistic lock maximum attempt to acquire the number of strokes:
pessimisticLockMaxAttemptNum=20
# pessimistic lock control transaction code List- -switch settings are made for the transaction for which lock control is desired:
pessimisticLockService.A02XXXXX=True
pessimisticLockService.A02XXXXX=True
pessimisticLockService.XXXXX=True
AP deployment cluster machine redis configuration case:
properties, named redis:
by18cisauap1001.srvInfoList=${redis.ip}
by18cisauap1002.srvInfoList=${redis.ip}
s2, obtaining a transaction request, wherein the transaction request at least comprises a transaction code and a client number corresponding to the transaction.
S3, checking the current distributed pessimistic lock master switch state; when the total switch state of the distributed pessimistic lock is closed, the distributed pessimistic lock is not added to the transaction request, and the transaction request is marked to be processed. The transaction request marked as pending is a request for not executing the addition logic of the distributed pessimistic lock, but can still normally execute the business processing logic of the optimistic lock.
S4, when the total switch state of the distributed pessimistic lock is open, the flag bit state of the distributed pessimistic lock of the transaction code corresponding to the transaction request is continuously checked; when the flag bit state of the distributed pessimistic lock is not enabled, the distributed pessimistic lock is not added to the transaction request, and the transaction request is marked to be processed.
And S5, when the flag bit state of the distributed pessimistic lock is enabled, continuing to perform flow control judgment on the current distributed cache, including verifying whether the failure frequency of continuous access to the distributed cache exceeds a preset threshold, when the failure frequency of continuous access to the distributed cache exceeds the preset threshold, judging that the distributed cache has a fault, not adding the distributed pessimistic lock to the transaction request, and marking the transaction request as pending. If the distributed cache fails, the access to the distributed cache can be dynamically isolated and recovered by means of dynamic flow control, and the problem that the overall transaction processing time is prolonged due to the failure of the distributed cache is solved. The distributed cache is preferably accessed by a redis connection. When the access of the redis connection fails and the access of the distributed cache fails abnormally in a large area, the flow control switch needs to be opened, the distributed locking and unlocking mechanism is dynamically isolated, the corresponding distributed cache access logic is eliminated, the processing logic of the transaction is directly executed, and the phenomena that the transaction server fails, the failure spreads, the whole response time of the transaction is prolonged and the like due to the failure of the distributed cache are prevented.
S6, when the distributed cache is judged to have no fault, executing the operation of adding the distributed pessimistic lock to the transaction request, wherein the operation comprises the steps of obtaining the client number corresponding to the transaction request, calling a put method of the distributed cache to add the distributed pessimistic lock to the client number corresponding to the transaction request, and setting the survival time of the distributed pessimistic lock; when the distributed cache is judged to have faults, one transaction request is selected as a probing request, and the operation of adding a distributed pessimistic lock is executed on the probing request; clearing the number of failures to continuously access the distributed cache when the addition of the distributed pessimistic lock to the probe request is successful. Dynamic isolation can be performed on the distributed cache by adopting a continuous failure strategy, for example, the number of failures in continuous access to the distributed cache exceeds 20, 2000 requests for accessing the distributed cache next are isolated, and after the 2000 requests, locking is attempted by using the distributed cache; or other strategies can be adopted, such as that the exception is caused more than 10 times when the access to the distributed lock is attempted, and the fusing processing occurs, namely that the transaction request acquired later does not attempt to acquire the distributed pessimistic lock any more, but a trial transaction is selected to attempt to add the pessimistic lock, and whether the distributed cache is recovered to be normal or not is checked and confirmed. The distributed cache can ensure that only one thread can operate successfully for the same client's request to add a pessimistic lock.
S7, checking whether the distributed pessimistic lock is added successfully, particularly checking whether the put method of the distributed cache is invoked successfully; when the distributed pessimistic lock is successfully added, the transaction request is marked as locked, for the successfully locked transaction request, the transaction request can be marked as locked by adopting a mode of setting a locking success flag bit in an internal data exchange area, corresponding monitoring data is updated, and the processing logic of the transaction is executed; when the addition of the distributed pessimistic lock is unsuccessful, re-executing the operation of adding the distributed pessimistic lock to the transaction request according to a preset waiting interval and the maximum retry number; when the distributed pessimistic lock is successfully added within the maximum retry number, the transaction request is marked as locked; when the distributed pessimistic lock is still unsuccessfully added after the maximum retry number, the transaction request is marked to be processed, if the locking fails, whether corresponding distributed cache access exception is thrown or not can be preferably judged, if the distributed cache access exception is thrown, exception capture is carried out, monitoring data are updated, and then subsequent transaction processing logic is executed. Preferably, the preset waiting interval (the waiting time of the thread) may be set to be unequal, for example, 200ms is waited for the first time, 150ms is waited for the second time, and 100ms is waited for the third time, so that the probability of the phenomenon of 'starvation' of the waiting thread can be reduced. In the process of accessing the distributed cache, corresponding timeout control such as connection timeout, read timeout control and the like is well performed, and routing work of a plurality of distributed cache clusters is well performed.
S8, the marked transaction request continues to process transaction service processing logic; when the transaction request is marked as locked, executing pessimistic lock processing logic on the transaction request; when the transaction request is marked as pending, an optimistic lock is added to the transaction request and optimistic lock processing logic is executed.
And S9, when the feedback processing of the transaction request marked as locked is completed, unlocking the distributed pessimistic lock added to the transaction request. The post-transaction processing distributed pessimistic lock unlocking processing flow is a conventional technical means, if a transaction request is locked in the pre-transaction processing, corresponding unlocking operation is required no matter the transaction is successful or failed, namely, corresponding client number cache data is deleted.
The server of the distributed cache preferably adopts a master-slave mode, one master server and n slave servers, wherein the master server is responsible for receiving corresponding read-write requests, and when the master server is down, one of the slave servers takes over the work of the master and receives the corresponding read-write requests.
Different languages have different access control classes for the redis, and the technical scheme of the invention is not particularly limited. However, it may be preferable to encapsulate the connection object and the data collection object of the redis, so as to implement a redisaccesor interface or class, which provides the following features: 1) a method of providing an access method putifnotexit (key, value) to redis, the method aiming at inserting a key-value pair into redis, returning 0 if the formulated key already exists without operating; otherwise the specified key is inserted into redis and returns 1; 2) providing methods such as place and the like, and updating the key-value into redis; 3) and providing a remove _ data method, deleting the key-value from the redis, returning 1 if the deletion is successful, and returning 0 if the deletion is failed or abnormal.
The invention realizes the effect of dynamic attempt acquisition for the addition of the distributed pessimistic lock, and can specifically execute the following steps:
1. inserting a key-value pair by a putIfNotExist (key, value) method, and trying to acquire a pessimistic lock, and setting a pessimistic lock acquisition flag bit in the swap area if the acquisition is successful;
2. under the condition of failure acquisition, calculating the system time, and calculating the reference time of acquiring the pessimistic lock every time, wherein the method and the standard for calculating the reference time are different from person to person, the sleep time of each time is approximately shorter and shorter, and the thread is allowed to sleep for the calculated reference time;
3. the number of attempts to acquire a pessimistic lock and the total wait time are calculated, and the loop is skipped if the number of attempts to acquire the lock exceeds a preset maximum number, or the total wait time exceeds a preset wait time. The success logic continues to be executed if the acquisition is successful in the course of the loop.
For transaction requests of the same customer number, the transaction requests often have the same key value, in the concurrent transaction thread, a queue queuing mode is not adopted to sequentially obtain distributed pessimistic locks according to a first-come-first-serve principle, new and old transaction requests request locking requests of the same time point, the probability of obtaining the distributed locks is the same, if the transaction requests of the same customer are large in concurrency, a phenomenon of 'starvation' occurs in part of threads, and the time for obtaining the locks is prolonged. If the queuing mechanism is referred to, the distributed locks are acquired in turn, and the lock access mechanism can be realized by referring to a queue cache data structure of the distributed cache, but the technical complexity and the dependency on the distributed cache are higher, and according to the existing use data, the concurrency of transaction requests of the same client is not large enough to be tolerated, and the queuing mechanism is not completely necessary.
The overall high availability of the distributed pessimistic lock is improved by adopting a distributed cache multi-cluster multi-active mode, and the processing efficiency is ensured. Assuming that there are two sets of distributed cache clusters providing services to the outside simultaneously, the client numbers may be uniformly distributed in the two sets of clusters based on the basket number corresponding to the transaction request client number, the first set of clusters stores a client number list with the basket number% 2 ═ 0 corresponding to the client number, and the second set of clusters stores a client number list with the basket number% 2 ═ 1 corresponding to the client number. And when the distributed lock accesses the distributed cache, accessing the corresponding distributed cache cluster according to the distribution rule of the data.
When one set of distributed cache clusters fails, all access requests can be transferred to the other set of clusters by adopting manual intervention or an event notification mode. In the process of migrating the distributed cache cluster, data stored in the cache is temporary data, so that data migration work is not needed.
In another aspect, the present invention further relates to a high concurrent transaction locking system combining optimistic lock and pessimistic lock, whose structure is shown in fig. 2, including:
the configuration file management module is used for establishing a distributed pessimistic lock configuration file;
the distributed pessimistic lock state control module is used for checking the current distributed pessimistic lock master switch state and the distributed pessimistic lock flag bit state of the transaction code corresponding to the transaction request;
the distributed cache verification module is used for carrying out flow control judgment on the current distributed cache;
and the distributed pessimistic lock adding module is used for executing the operation of adding the distributed pessimistic lock to the transaction request and checking whether the addition of the distributed pessimistic lock is successful.
By using this system, the above-described arithmetic processing method can be executed and a corresponding technical effect can be achieved.
Embodiments of the present invention also provide a computer-readable storage medium capable of implementing all the steps of the method in the above embodiments, the computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements all the steps of the method in the above embodiments.
Embodiments of the present invention further provide an electronic device for executing the method, as an implementation apparatus of the method, the electronic device at least has a processor and a memory, and particularly, the memory stores data and related computer programs, such as a distributed pessimistic lock configuration file and a transaction request, which are required for executing the method, and all steps of implementing the method are executed by invoking the data and the programs in the memory by the processor, and a corresponding technical effect is obtained.
Preferably, the electronic device may comprise a bus architecture, which may include any number of interconnected buses and bridges linking together various circuits including one or more processors and memory. The bus may also link various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the receiver and transmitter. The receiver and transmitter may be the same element, i.e., a transceiver, providing a means for communicating with various other systems over a transmission medium. The processor is responsible for managing the bus and general processing, while the memory may be used for storing data used by the processor in performing operations.
Additionally, the electronic device may further include a communication module, an input unit, an audio processor, a display, a power source, and the like. The processor (or controller, operational controls) employed may include a microprocessor or other processor device and/or logic device that receives input and controls the operation of various components of the electronic device; the memory may be one or more of a buffer, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices, and may store the above-mentioned related data information, and may also store a program for executing the related information, and the processor may execute the program stored in the memory to realize information storage or processing, etc.; the input unit is used for providing input to the processor, and can be a key or a touch input device; the power supply is used for supplying power to the electronic equipment; the display is used for displaying display objects such as images and characters, and may be an LCD display, for example. The communication module is a transmitter/receiver that transmits and receives signals via an antenna. The communication module (transmitter/receiver) is coupled to the processor to provide an input signal and receive an output signal, which may be the same as in the case of a conventional mobile communication terminal. Based on different communication technologies, a plurality of communication modules, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be disposed in the same electronic device. The communication module (transmitter/receiver) is also coupled to a speaker and a microphone via an audio processor to provide audio output via the speaker and receive audio input from the microphone to implement the usual telecommunication functions. The audio processor may include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor is also coupled to the central processor, so that recording on the local machine can be realized through the microphone, and sound stored on the local machine can be played through the loudspeaker.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a system for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. A high concurrent transaction locking method combining an optimistic lock and a pessimistic lock is characterized by comprising the following steps:
s1, establishing a distributed pessimistic lock configuration file, wherein the distributed pessimistic lock configuration file at least comprises a distributed pessimistic lock master switch and distributed pessimistic lock flag bits corresponding to each transaction code;
s2, obtaining a transaction request, wherein the transaction request at least comprises a transaction code and a client number corresponding to the transaction;
s3, checking the current distributed pessimistic lock master switch state; when the state of the distributed pessimistic lock master switch is closed, adding the distributed pessimistic lock to the transaction request, and marking the transaction request as pending;
s4, when the total switch state of the distributed pessimistic lock is open, the flag bit state of the distributed pessimistic lock of the transaction code corresponding to the transaction request is continuously checked; when the flag bit state of the distributed pessimistic lock is not enabled, the distributed pessimistic lock is not added to the transaction request, and the transaction request is marked to be processed;
s5, when the flag bit state of the distributed pessimistic lock is enabled, continuing to perform flow control judgment on the current distributed cache, and when the distributed cache is judged to have a fault, not adding the distributed pessimistic lock to the transaction request and marking the transaction request as pending;
s6, when the distributed cache is judged to have no fault, executing the operation of adding the distributed pessimistic lock to the transaction request;
s7, checking whether the distributed pessimistic lock is added successfully; when the distributed pessimistic lock is successfully added, the transaction request is marked as locked; when the addition of the distributed pessimistic lock is unsuccessful, marking the transaction request as pending;
s8, the marked transaction request continues to process transaction service processing logic; when the transaction request is marked as locked, executing pessimistic lock processing logic on the transaction request; when the transaction request is marked as pending, an optimistic lock is added to the transaction request and optimistic lock processing logic is executed.
2. The method of claim 1, wherein the method further comprises:
and S9, when the feedback processing of the transaction request marked as locked is completed, unlocking the distributed pessimistic lock added to the transaction request.
3. The method of claim 1, wherein the determining the current distributed cache for flow control comprises:
and verifying whether the failure frequency of continuous access to the distributed cache exceeds a preset threshold value, and judging that the distributed cache has a fault when the failure frequency of continuous access to the distributed cache exceeds the preset threshold value.
4. The method of claim 3, wherein the step S6 further comprises:
when the distributed cache is judged to have faults, one transaction request is selected as a probing request, and the operation of adding a distributed pessimistic lock is executed on the probing request;
clearing the number of failures to continuously access the distributed cache when the addition of the distributed pessimistic lock to the probe request is successful.
5. The method of claim 1, wherein the performing the operation of adding a distributed pessimistic lock to the transaction request comprises:
and acquiring a client number corresponding to the transaction request, calling a put method of the distributed cache to add a distributed pessimistic lock to the client number corresponding to the transaction request, and setting the survival time of the distributed pessimistic lock.
6. The method of claim 5, wherein the checking whether the addition of the distributed pessimistic lock is successful comprises:
and verifying whether the put method of the distributed cache is successfully called, wherein the distributed pessimistic lock is successfully added when the calling is successful, and otherwise, the adding is not successful.
7. The method of claim 6, wherein the step S7 further comprises:
when the addition of the distributed pessimistic lock is unsuccessful, re-executing the operation of adding the distributed pessimistic lock to the transaction request according to a preset waiting interval and the maximum retry number;
when the distributed pessimistic lock is successfully added within the maximum retry number, the transaction request is marked as locked; when the distributed pessimistic lock is still added unsuccessfully beyond the maximum number of retries, the transaction request is marked as pending.
8. A high concurrent transaction locking system combining optimistic and pessimistic locks, comprising:
the configuration file management module is used for establishing a distributed pessimistic lock configuration file;
the distributed pessimistic lock state control module is used for checking the current distributed pessimistic lock master switch state and the distributed pessimistic lock flag bit state of the transaction code corresponding to the transaction request;
the distributed cache verification module is used for carrying out flow control judgment on the current distributed cache;
and the distributed pessimistic lock adding module is used for executing the operation of adding the distributed pessimistic lock to the transaction request and checking whether the addition of the distributed pessimistic lock is successful.
9. A computer-readable storage medium, characterized in that the storage medium has stored thereon a computer program which, when being executed by a processor, carries out the method of any one of claims 1 to 7.
10. An electronic device comprising a processor and a memory;
the memory is used for storing a distributed pessimistic lock configuration file and a transaction request;
the processor to perform the method of any of claims 1 to 7 by invoking a distributed pessimistic lock profile and a transaction request.
CN202111322477.8A 2021-11-09 2021-11-09 High-concurrency transaction locking method and system combining optimistic lock and pessimistic lock Pending CN114036164A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111322477.8A CN114036164A (en) 2021-11-09 2021-11-09 High-concurrency transaction locking method and system combining optimistic lock and pessimistic lock

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111322477.8A CN114036164A (en) 2021-11-09 2021-11-09 High-concurrency transaction locking method and system combining optimistic lock and pessimistic lock

Publications (1)

Publication Number Publication Date
CN114036164A true CN114036164A (en) 2022-02-11

Family

ID=80137019

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111322477.8A Pending CN114036164A (en) 2021-11-09 2021-11-09 High-concurrency transaction locking method and system combining optimistic lock and pessimistic lock

Country Status (1)

Country Link
CN (1) CN114036164A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115168341A (en) * 2022-06-17 2022-10-11 北京结慧科技有限公司 Service processing method, system, medium and equipment

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115168341A (en) * 2022-06-17 2022-10-11 北京结慧科技有限公司 Service processing method, system, medium and equipment

Similar Documents

Publication Publication Date Title
US11321303B2 (en) Conflict resolution for multi-master distributed databases
JP7142152B2 (en) Transaction processing method, device, equipment and computer program
US11899684B2 (en) System and method for maintaining a master replica for reads and writes in a data store
CN111338766B (en) Transaction processing method and device, computer equipment and storage medium
US10360113B2 (en) Transaction recovery in a transaction processing computer system employing multiple transaction managers
CN107077382B (en) System and method for transaction recovery in a multi-tenant application server environment
WO2018103318A1 (en) Distributed transaction handling method and system
US9116862B1 (en) System and method for data replication using a single master failover protocol
US9753954B2 (en) Data node fencing in a distributed file system
US8510334B2 (en) Lock manager on disk
US8676760B2 (en) Maintaining data integrity in data servers across data centers
US20100023564A1 (en) Synchronous replication for fault tolerance
CN108932338B (en) Data updating method, device, equipment and medium
CN104793988A (en) Cross-database distributed transaction implementation method and device
US20190220361A1 (en) Monitoring Containers In A Distributed Computing System
CN110597910A (en) Remote data synchronization method, device and system
US8843581B2 (en) Live object pattern for use with a distributed cache
WO2021103499A1 (en) Multi-active data center-based traffic switching method and device
EP4213038A1 (en) Data processing method and apparatus based on distributed storage, device, and medium
CN112039970B (en) Distributed business lock service method, server, system and storage medium
CN110413687B (en) Distributed transaction fault processing method and related equipment based on node interaction verification
CN114925084B (en) Distributed transaction processing method, system, equipment and readable storage medium
WO2022111188A1 (en) Transaction processing method, system, apparatus, device, storage medium, and program product
US11449241B2 (en) Customizable lock management for distributed resources
US7890468B2 (en) Rollback support in distributed data management systems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination