CN112948498A

CN112948498A - Method and device for generating global identification of distributed system

Info

Publication number: CN112948498A
Application number: CN202110342577.0A
Authority: CN
Inventors: 朱晓康; 张勇辉; 曹文伟; 王黎君
Original assignee: China Construction Bank Corp
Current assignee: China Construction Bank Corp
Priority date: 2021-03-30
Filing date: 2021-03-30
Publication date: 2021-06-11

Abstract

The invention discloses a method and a device for generating a global identifier of a distributed system, and relates to the technical field of computers. A specific implementation mode of the method comprises the steps of monitoring an application program starting event, sending a data request to a database, reading an initial number section of the database to obtain a global identification interval based on a preset step length, caching the global identification interval to a memory and updating the initial number section of the database; receiving a data request of an application program, calling a preset accumulation model, and generating a global identifier corresponding to the request based on a global identifier interval in a memory. Therefore, the embodiment of the invention can solve the problem of poor accuracy of the existing distributed ID generation algorithm.

Description

Method and device for generating global identification of distributed system

Technical Field

The invention relates to the technical field of computers, in particular to a method and a device for generating a global identifier of a distributed system.

Background

At present, large-scale business systems such as an accounting system, an e-commerce system and a logistics system are all realized based on a distributed technical architecture, and due to the fact that data volume is large, a traditional single-base single table and single application cannot support data access and storage of the data volume. Therefore, under the design of a distributed system, the application and the database need to be split, the micro services are dependent on each other, the data are isolated from each other, and the different micro services are interacted in an interface mode, so that the application and the database can be expanded and contracted as required. In each micro service system and in each sub-database and sub-table, a plurality of service data need a global ID for identification, and the global ID is used as a key field of data record, and the uniqueness, high availability and high performance of the data record need to be ensured. The global ID is a unique ID of a full link of a plurality of service scenarios (such as orders, payment orders, logistics orders, pipelining, and the like) in the distributed system, and is used for uniquely identifying a certain record or data.

In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art:

snowfly is a distributed ID generation algorithm with Twitter open source, and the result is a long type ID. The snowfall algorithm is completely time-dependent, and if a clock callback occurs, repeated IDs are generated, so that ID collision is caused.

Disclosure of Invention

In view of this, embodiments of the present invention provide a method and an apparatus for generating a global identifier of a distributed system, which can solve the problem of poor accuracy of an existing distributed ID generation algorithm.

In order to achieve the above object, according to an aspect of the embodiments of the present invention, a method for generating a global identifier of a distributed system is provided, including monitoring an application program start event, sending a data request to a database, reading an initial number segment of the database to obtain a global identifier interval based on a preset step size, caching the global identifier interval to a memory, and updating the initial number segment of the database; receiving a data request of an application program, calling a preset accumulation model, and generating a global identifier corresponding to the request based on a global identifier interval in a memory.

Optionally, after receiving the data request of the application program, the method includes:

judging whether an unallocated global identifier exists in a global identifier interval in a memory;

if so, calling a preset accumulation model, and generating a global identifier corresponding to the request based on a global identifier interval in the memory;

if not, sending a data request to the database to read the initial number section of the database so as to obtain a global identification interval based on a preset step length, caching the global identification interval to the memory and updating the initial number section of the database.

Optionally, comprising:

and if the concurrent application program starting events are monitored, triggering the optimistic lock control program.

Optionally, comprising:

monitoring a plurality of application program starting events which are sent concurrently, and triggering an optimistic lock control program realized by the version number in the database field; and updating the version number while updating the initial number section of the database.

Optionally, comprising:

an interface is set in rpc or http mode, and then a data request is sent to the database through the interface.

Optionally, comprising:

the accumulation model is used for randomly generating an accumulation number or acquiring a preset accumulation number.

In addition, the invention also provides a device for generating the global identifier of the distributed system, which comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for monitoring the starting event of the application program, sending a data request to the database, reading the initial number section of the database to obtain a global identifier interval based on a preset step length, caching the global identifier interval to the memory and updating the initial number section of the database; and the processing module is used for receiving a data request of an application program, calling a preset accumulation model and generating a global identifier corresponding to the request based on a global identifier interval in the memory.

Optionally, after the processing module receives a data request of the application program, the processing module includes:

Optionally, the obtaining module is further configured to:

an interface is set in an rpc or http mode, and a data request sent to the database is received through the interface.

Optionally, comprising:

One embodiment of the above invention has the following advantages or benefits: the method can send a data request to the database by monitoring an application program starting event, read the initial number section of the database to obtain a global identification interval based on a preset step length, cache the global identification interval to the memory and update the initial number section of the database; receiving a data request of an application program, calling a preset accumulation model, and generating a global identifier corresponding to the request based on a global identifier interval in a memory. Therefore, the invention realizes a globally unique and high-performance ID generation method, does not depend on a system clock, and avoids the problem of ID conflict; moreover, the system is independently deployed in a micro-service mode, and can provide services to the outside through rpc or an http interface; meanwhile, the application can be expanded and contracted as required, and high availability and high performance are supported.

Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.

Drawings

The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:

fig. 1 is a schematic diagram of a main flow of a method for generating a global identifier of a distributed system according to a first embodiment of the present invention;

FIG. 2 is a diagram illustrating relationships between applications, memory, and databases, according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a main flow of a method for generating a global identifier of a distributed system according to a second embodiment of the present invention;

fig. 4 is a schematic diagram of the main modules of the generation apparatus of the global identifier of the distributed system according to the embodiment of the present invention;

FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;

fig. 6 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 is a schematic diagram of a main flow of a method for generating a global identifier of a distributed system according to a first embodiment of the present invention, and as shown in fig. 1, the method for generating a global identifier of a distributed system includes:

step S101, monitoring an application program starting event, sending a data request to a database, reading a database starting number section to obtain a global identification interval based on a preset step length, caching the global identification interval to an internal memory and updating the database starting number section.

Step S102, receiving a data request of an application program, calling a preset accumulation model, and generating a global identification corresponding to the request based on a global identification interval in a memory.

In an embodiment, the database fields may include a primary key, a start number segment, and a step size. Wherein, the step length can be adjusted and configured according to the service requirement. After the cache number segments in the initial number segment memory are used up, a data request needs to be sent to the database to obtain more number segments, and the initial number segment of the database needs to be updated at the same time. The specific implementation process comprises the following steps: judging whether an unallocated global identifier exists in a global identifier interval in a memory; if so, calling a preset accumulation model, and generating a global identifier corresponding to the request based on a global identifier interval in the memory; if not, sending a data request to the database to read the initial number section of the database so as to obtain a global identification interval based on a preset step length, caching the global identification interval to the memory and updating the initial number section of the database.

In order to ensure high availability of the database, a multi-master mode can be adopted, and each master node uses different number segments to stagger the number segments. As shown in fig. 2, the database DB and the application AP are divided into two parts. The DB is mainly used for storing the number segments and the step length which are available at present, and the AP is used for acquiring the step length and generating a global identification ID based on the number segments in the global identification interval in the memory. The number segment intervals acquired by different APs are not repeated. After the AP1 is started, the initial number segment in the database is read, the global identification ID interval in the step range is obtained, assuming that the step is 1000, the ID interval range is 1-1000, and the initial number segment in the database is updated to be the next number segment for the next time to obtain or use by other APs.

After the number segment in the current global identifier interval is used up (one step is accumulated), the AP1 re-reads the current start number segment in the database, updates the global identifier ID interval range in the memory, and starts to generate a new global identifier. The manner of acquiring the global identification ID interval by each AP is the same. If the second application AP2 is started at this time, since the AP1 has updated the initial number segment in the database to be the next number segment 1001, the range of the global ID interval acquired by the current application is 1001-2000, and the number segment in the database is updated to be the next number segment at the same time. Therefore, the global identification intervals used on each AP are independent and separated, a plurality of APs provide services to the outside, cluster deployment is realized through load balancing, and high-reliability services are provided. Because the number segment acquisition in the whole global identification interval is completed by calculation in the memory, the database is operated once only after the number segment is used, the pressure of the database is greatly reduced, the ID generation efficiency and performance are ensured, and the global ID generation under high concurrency can be met. Even if the database service is suddenly down, the normal operation of the AP service can be maintained for a period of time.

In some embodiments, the optimistic lock control procedure is triggered when multiple concurrent application start events are detected. The specific implementation process comprises the following steps:

monitoring a plurality of application program starting events which are sent concurrently, and triggering an optimistic lock control program realized by the version number in the database field; and updating the version number while updating the initial number section of the database. That is, a data request is sent to the database to obtain the version number included in the database field, and the version number is used for data lock control. When a plurality of application programs AP simultaneously acquire the global identification of the distributed system, the problem of concurrency can be caused. Meanwhile, when the global identification ID of the distributed system is acquired, attention needs to be paid to control of transactions, the number segment after acquisition failure should not be rolled back, and otherwise, the risk of repetition is also caused.

It should be noted that after the number segments of the cache in the memory are used up, a data request needs to be sent to the database to obtain more number segments, and the initial number segment and the version number of the database need to be updated at the same time.

As another embodiment, a data request of an application program is received, a preset accumulation model is called, and a global identifier corresponding to the request is generated based on a global identifier interval in a memory. The accumulation model is used for randomly generating an accumulation number or acquiring a preset accumulation number. For example: each time the application AP1 receives a request, the global identifier of the last request in the memory is accumulated by 1 or by a random number to obtain the global identifier corresponding to the request.

As still other embodiments, the interface is set up in rpc or http mode, and the data request is sent to the database through the interface. That is to say, the interface is exposed to the caller in rpc or http mode, and can be directly accessed through rpc-client or http, which is convenient and easy to use.

In conclusion, the invention adopts a memory + db mode in performance and micro-service deployment, so that the time consumption of basic service is low, the performance is high, and meanwhile, the database is not stressed greatly. In the aspect of availability, the application and the database are guaranteed to be highly available as far as possible, the capacity expansion and contraction machine can be used as required, and the robustness of the service is improved.

Fig. 3 is a schematic main flow diagram of a method for generating a global identifier of a distributed system according to a second embodiment of the present invention, and as shown in fig. 3, the method for generating a global identifier of a distributed system may include:

step S301, monitoring a plurality of application program starting events, and triggering the optimistic lock control program realized by the version number in the database field.

Step S302, a data request is sent to the database through rpc or an interface in an http mode.

Step S303, reading the initial number segment of the database to obtain a global identifier interval based on a preset step size, caching the global identifier interval to the memory, and updating the initial number segment of the database.

In step S304, a data request of an application is received.

Step S305, determining whether the global flag interval in the memory has an unallocated global flag, if so, executing step S306, otherwise, executing step S307.

Step S306, a preset accumulation model is called, and a global identifier corresponding to the request is generated based on the global identifier interval in the memory.

Step S307, sending a data request to the database to read the initial number segment of the database, so as to obtain a global identifier interval based on a preset step size, caching the global identifier interval to the memory, updating the initial number segment of the database, and returning to step S305.

Therefore, the invention provides high-performance and high-availability basic service on the basis of ensuring global uniqueness, adopts the mode that the database number segment of the memory cache controls the global number segment, provides a distributed service interface, and has high and stable processing efficiency. And meanwhile, a plurality of access modes are provided, so that the caller can conveniently access.

Fig. 4 is a schematic diagram of main modules of a device for generating a global identifier of a distributed system according to an embodiment of the present invention, and as shown in fig. 4, the device for generating a global identifier of a distributed system includes an obtaining module 401 and a processing module 402. The obtaining module 401 monitors an application program starting event, sends a data request to the database, reads an initial number segment of the database to obtain a global identification interval based on a preset step length, caches the global identification interval to the memory, and updates the initial number segment of the database; the processing module 402 receives a data request of an application program, calls a preset accumulation model, and generates a global identifier corresponding to the request based on a global identifier interval in a memory.

In some embodiments, after the processing module 402 receives the data request of the application program, the method includes:

In some embodiments, the obtaining module 401 is further configured to:

In some embodiments, the method comprises:

It should be noted that, the method for generating the global identifier of the distributed system and the device for generating the global identifier of the distributed system according to the present invention have a corresponding relationship in the specific implementation content, and therefore, the repeated content is not described again.

Fig. 5 shows an exemplary system architecture 500 to which the method for generating a distributed system global identity or the apparatus for generating a distributed system global identity according to the embodiments of the present invention may be applied.

As shown in fig. 5, the system architecture 500 may include

terminal devices

501, 502, 503, a network 504, and a server 505. The network 504 serves to provide a medium for communication links between the

terminal devices

501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

501, 502, 503 to interact with a server 505 over a network 504 to receive or send messages or the like. The

terminal devices

501, 502, 503 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).

The

terminal devices

501, 502, 503 may be various electronic devices that generate screens with distributed system global identification and support web browsing, including but not limited to smart phones, tablets, laptop portable computers, desktop computers, and the like.

The server 505 may be a server providing various services, such as a background management server (for example only) providing support for shopping websites browsed by users using the

terminal devices

501, 502, 503. The backend management server may analyze and perform other processing on the received data such as the product information query request, and feed back a processing result (for example, target push information, product information — just an example) to the terminal device.

It should be noted that the method for generating the global identifier of the distributed system provided by the embodiment of the present invention is generally executed by the server 505, and accordingly, the computing device is generally disposed in the server 505.

It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Referring now to FIG. 6, a block diagram of a computer system 600 suitable for use with a terminal device implementing an embodiment of the invention is shown. The terminal device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data necessary for the operation of the computer system 600 are also stored. The CPU601, ROM602, and RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output section 607 including a signal such as a Cathode Ray Tube (CRT), a generator (LCD) of a liquid crystal distributed system global identification, and the like, and a speaker and the like; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.

In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 601.

It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present invention may be implemented by software or hardware. The described modules may also be provided in a processor, which may be described as: a processor includes an acquisition module and a processing module. Wherein the names of the modules do not in some cases constitute a limitation of the module itself.

As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs, and when the one or more programs are executed by the device, the device comprises a monitoring unit, a data request sending unit, a cache unit and a data updating unit, wherein the monitoring unit monitors an application program starting event, sends the data request to the database, reads a database starting number segment to obtain a global identification interval based on a preset step size, and caches the global identification interval to the memory and updates the database starting number segment; receiving a data request of an application program, calling a preset accumulation model, and generating a global identifier corresponding to the request based on a global identifier interval in a memory.

According to the technical scheme of the embodiment of the invention, the problem of poor accuracy of the existing distributed ID generation algorithm can be solved.

The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating a global identifier of a distributed system is characterized by comprising the following steps:

monitoring an application program starting event, sending a data request to a database, reading an initial number section of the database to obtain a global identification interval based on a preset step length, caching the global identification interval to a memory and updating the initial number section of the database;

receiving a data request of an application program, calling a preset accumulation model, and generating a global identifier corresponding to the request based on a global identifier interval in a memory.

2. The method of claim 1, wherein receiving the data request of the application program comprises:

3. The method of claim 1, comprising:

4. The method of claim 2, comprising:

5. The method of claim 1, comprising:

6. The method according to any one of claims 1 to 5, comprising:

7. An apparatus for generating a global identifier of a distributed system, comprising:

the acquisition module is used for monitoring an application program starting event, sending a data request to the database, reading an initial number section of the database to obtain a global identification interval based on a preset step length, caching the global identification interval to the internal memory and updating the initial number section of the database;

and the processing module is used for receiving a data request of an application program, calling a preset accumulation model and generating a global identifier corresponding to the request based on a global identifier interval in the memory.

8. The apparatus of claim 7, wherein the processing module, after receiving the data request of the application program, comprises:

9. The apparatus of claim 7, wherein the obtaining module is further configured to:

10. The apparatus of claim 9, wherein the obtaining module is further configured to:

11. The apparatus of claim 7, wherein the obtaining module is further configured to:

12. The apparatus according to any one of claims 7-11, comprising:

13. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-6.

14. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-6.