CN114880404A - Group model quasi-delay synchronization method of rail transit distributed database - Google Patents
Group model quasi-delay synchronization method of rail transit distributed database Download PDFInfo
- Publication number
- CN114880404A CN114880404A CN202210820496.1A CN202210820496A CN114880404A CN 114880404 A CN114880404 A CN 114880404A CN 202210820496 A CN202210820496 A CN 202210820496A CN 114880404 A CN114880404 A CN 114880404A
- Authority
- CN
- China
- Prior art keywords
- synchronization
- record
- library
- target
- station
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 230000001360 synchronised effect Effects 0.000 claims abstract description 25
- 238000012986 modification Methods 0.000 claims abstract description 18
- 230000004048 modification Effects 0.000 claims abstract description 18
- 230000006870 function Effects 0.000 claims abstract description 8
- 101150060512 SPATA6 gene Proteins 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000003780 insertion Methods 0.000 claims description 2
- 230000037431 insertion Effects 0.000 claims description 2
- 230000008859 change Effects 0.000 abstract description 4
- 238000012544 monitoring process Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Tourism & Hospitality (AREA)
- General Physics & Mathematics (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Human Resources & Organizations (AREA)
- General Business, Economics & Management (AREA)
- General Health & Medical Sciences (AREA)
- Marketing (AREA)
- Educational Administration (AREA)
- Computing Systems (AREA)
- Development Economics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a group model quasi-delay synchronization method of a rail transit distributed database, which comprises the following steps: generating modification time, abstract time and abstract identification functions; whether synchronization is executed or not is determined locally according to whether a source IP is present or not, a target central library and a station non-local redundant library are synchronized, whether synchronization is executed or not is determined locally according to whether the source IP is present or not, and meanwhile, a station redundant library and a central non-local redundant library corresponding to synchronization are determined according to a target station; and after the synchronization is successful, re-reading the target record digests in the table, summing to obtain a synchronization digest, comparing whether the target synchronization digests in the synchronization object are consistent or not, generating a synchronization result and recording the successful synchronization record. According to the invention, the content comparison is converted into the abstract comparison, so that the identification efficiency of the change is improved; the invention is very suitable for the full life cycle characteristic of the rail transit field; the invention only writes one target database at a time, and the consistency of the model and the usability of the system are enhanced.
Description
Technical Field
The invention relates to a rail transit comprehensive automation system, in particular to a group model quasi-delay synchronization method of a rail transit distributed database.
Background
In the field of rail transit automation, the most important data is the equipment-related model, and the basic classification mode is the station. The number of rail transit stations is large, the No. 6 line of the Chengdu subway has 56 stations, for example, comprehensive monitoring is taken as an example, a standard requirement system provides a three-level control center, stations and sites, and when the center is off-line, the stations have complete equipment models and can independently run. This presents a significant challenge for synchronizing up to 114 databases. In the future, the comprehensive monitoring will upgrade the network level monitoring, the number of databases needing model synchronization will increase dramatically, and the risk will increase sharply.
The conventional method comprises the following steps: 1. the method has the advantages that the multiple target databases are written simultaneously, when the target databases are offline, synchronous records are cached locally, the synchronous records are huge, a synchronous program cannot load cache files easily, and meanwhile, due to the fact that a verification mechanism is not arranged, real-time performance is guaranteed, consistency is poor, potential safety hazards exist, and risks are increased rapidly along with scale expansion; 2. the method has the advantages that the central modeling is adopted, the database is periodically copied, the process is complex, the database needs to be emptied and then synchronized, the problem that the emptied data cannot be recovered exists, and greater potential safety hazards can be caused; 3. by adopting commercial synchronization tools such as RANK and the like or a synchronization technology or an open source component based on a Binlog mechanism, the method is only suitable for synchronization of a small amount of databases, is not suitable for synchronization of distributed databases, and is particularly not suitable for application scenarios of unequal station databases in a rail transit center. In detail, as shown in fig. 2, the central repository has all the station models, and the station only has the own station model.
Disclosure of Invention
The purpose of the invention is as follows: the invention aims to provide a group model quasi-delay synchronization method of a track traffic distributed database, so that the problem of data inconsistency in a multi-database mode is solved.
The technical scheme is as follows: the invention relates to a group model quasi-delay synchronization method of a rail transit distributed database, which comprises the following steps of:
(1) according to the property of the local database and the property of the target database, when the local database is a central database and the target database is a station, acquiring the target station, and setting the target database to be a null value under other conditions;
(2) reading the database-table synchronization record, reading the last successful synchronization time and the abstract, and executing the step (2.1) if the record is empty; if the record is not empty, executing the step (2.2);
(2.1) reading all records of which the source _ db is a local machine and the attribute is a target station (optional, required when a central synchronization station is used) in the table, if the records are empty, recording the synchronization success, generating a default hash code, if the records are not empty, calculating the source of the target library-table as the table hash code of the table, if the records are not empty, executing the step (2), and if the records are not identical, executing the step (6);
(2.2) reading records of record _ changed _ time > = synchronized _ time of all records of which the source _ db is a local machine and the attribute is a target station (optional, required when a central synchronization station is needed) in the table, if the records are empty, recording the synchronization successfully, and keeping the hashcode unchanged; if the record is not empty, executing the step (2);
(3) judging that the record _ HASH _ time is empty or the record _ changed _ time > = record _ HASH _ time, generating modification time, summary time and summary identification functions, wherein the modification time is generated as a record updating and processing identification, and the summary identification is generated by a non-keyword sequence HASH;
(4) judging whether the target library record exists or not, wherein no direct insertion record exists; record existence, comparison of record hashcodes for consistency, skipping synchronization, updating for inconsistency,
(5) recalculating the hash code of the library;
(6) generating a synchronization result and recording a synchronization record, recording the hashcode of the library successfully in synchronization, and recording the log if the synchronization is unsuccessful;
(7) when the library is damaged, manually importing the redundant library, and if the local library is the main library, generating a synchronous record according to the step (2.1) without synchronous operation; if the library is a spare library, no influence is caused.
A group model quasi-delay synchronization system of a rail transit distributed database comprises the following modules:
a record synchronization module: generating modification time, abstract time and abstract identification; the generation modification time is record updating and processing identification, and the summary identification is generated by non-keyword sequence HASH;
station synchronization module: whether synchronization is executed or not is locally determined according to whether a source IP is local or not, a target central library and a station non-local redundant library are synchronized, a station synchronization module in the center can locally determine whether synchronization is executed or not according to whether the source IP is local or not, and meanwhile a corresponding station redundant library and a central non-local redundant library are synchronized according to the determination of a target station;
a table synchronization module: after the station synchronization module successfully synchronizes, re-reading the target record digests in the table, summing the target record digests to obtain a synchronization digest, comparing whether the target synchronization digests in the synchronization object are consistent or not, and generating a synchronization result;
a synchronous result recording module: recording a synchronization result generated by the table synchronization module; and functions of a database, a data table and a station synchronization cursor are provided so as to finely identify which are the modifications after synchronization.
A computer storage medium on which a computer program is stored which, when being executed by a processor, implements a group model quasi-delay synchronization method of a rail transit distributed database as described above.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the group model quasi-delay synchronization method of a track traffic distributed database as described above when executing the computer program.
Has the advantages that: compared with the prior art, the invention has the following advantages:
1. according to the invention, the database records and tables are subjected to an abstract representation method, and content comparison is converted into abstract comparison, so that the identification efficiency of changes is greatly improved;
2. the invention is very suitable for the full life cycle characteristics of the rail transit field, namely the characteristics of large change, low dependence on backup and low synchronization requirement in the early construction, and the characteristics of small change or almost no change, strong dependence on backup and high synchronization requirement in the later operation and maintenance;
3. due to the delayed synchronization, the invention only writes one target database at a time, and the writing efficiency, the model consistency and the system availability are enhanced.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a diagram of an application environment and synchronization effect.
Detailed Description
The technical scheme of the invention is further explained by combining the attached drawings.
As shown in fig. 1, a group model quasi-delay synchronization method for a distributed rail transit database includes the following steps:
(1) the record synchronization module has generation modification time, summary time and summary identification, as shown in table 1, the generation modification time is record updating and processing identification, and the summary identification is generated by non-keyword sequential HASH. The source IP and the generation modification time are generated by the database writer, generated and modified along with the record. The abstract time and the abstract identification are generated regularly by a synchronous management program, the synchronous management program runs independently, the model service program is simpler and lighter due to the separation of service and management, and the reliability and stability of the service program are improved.
(2) The station synchronization module located in the station can determine whether to execute synchronization or not according to whether the source IP is local or not, synchronize the target central library and the station non-local redundant library, and the station synchronization module located in the center can determine whether to execute synchronization or not according to whether the source IP is local or not, and simultaneously determine to synchronize the corresponding station redundant library and the central non-local redundant library according to the target station. And the unique source IP is used as a system modification synchronization source, so that the unique read-write of the unique database by the service module is realized, a synchronization task is initiated from the unique database, and the synchronization of other databases is not influenced. Stations are used as group identifiers, the basic classification mode of rail transit is met, and the unequal application scene of a station database in a rail transit center is well met.
(3) The table synchronization module re-reads the target record digests in the table after the station synchronization module succeeds in synchronization to obtain a synchronization digest, compares whether the target synchronization digests in the synchronization objects are consistent or not to generate a synchronization result, and records the successful synchronization record through the synchronization result recording module, as shown in table 2.
(4) The synchronous recording module is used for recording a synchronous result and also providing functions of a database, a data table and a station synchronous cursor so as to finely identify the synchronous modification.
TABLE 1 expansion columns and examples for data tables such as model-related devices, point tables, linkage tables, and the like
Table 2 synchronization results table and examples
A group model quasi-delay synchronization method of a rail transit distributed database comprises the following modules:
a record synchronization module: generating modification time, abstract time and abstract identification; the generation modification time is record updating and processing identification, and the summary identification is generated by non-keyword sequence HASH;
station synchronization module: whether synchronization is executed or not is determined locally according to whether a source IP is present or not, a target central library and a station non-local redundant library are synchronized, a station synchronization module in the center determines whether synchronization is executed or not locally according to whether the source IP is present or not, and meanwhile, a station redundant library and a central non-local redundant library corresponding to the station redundant library are determined to be synchronized according to a target station;
a table synchronization module: after the station synchronization module successfully synchronizes, re-reading the target record digests in the table, summing the target record digests to obtain a synchronization digest, comparing whether the target synchronization digests in the synchronization object are consistent or not, and generating a synchronization result;
a synchronous result recording module: recording a synchronization result generated by the table synchronization module; and functions of a database, a data table and a station synchronization cursor are provided so as to finely identify which are the modifications after synchronization.
Claims (4)
1. A group model quasi-delay synchronization method of a rail transit distributed database is characterized by comprising the following steps:
(1) according to the property of the local database and the property of the target database, when the local database is a central database and the target database is a station, acquiring the target station, and setting the target database to be a null value under other conditions;
(2) reading the database-table synchronization record, reading the last successful synchronization time and the abstract, and executing the step (2.1) if the record is empty; if the record is not empty, executing the step (2.2);
(2.1) reading all records of which the source _ db is a local machine and the attribute is a target station in the table, if the records are null, successfully synchronizing the records, generating a default hashcode, if the records are not null, calculating the table hashcode of which the target library-table source is the local library, if the records are not null, executing the step (2), and if the records are consistent, executing the step (6);
(2.2) reading records of record _ changed _ time > = synced _ time of all records of which the source _ db is a local machine and the attribute is a target station in the table, and if the records are null, recording that the synchronization is successful and the hashcode is unchanged; if the record is not empty, executing the step (2);
(3) judging that the record _ HASH _ time is empty or the record _ changed _ time > = record _ HASH _ time, generating modification time, summary time and summary identification functions, wherein the modification time is generated as a record updating and processing identification, and the summary identification is generated by a non-keyword sequence HASH;
(4) judging whether the target library record exists or not, wherein no direct insertion record exists; record existence, comparison of record hashcodes for consistency, skipping synchronization, updating for inconsistency,
(5) recalculating the hash code of the library;
(6) generating a synchronization result and recording a synchronization record, recording the hashcode of the library successfully in synchronization, and recording the log if the synchronization is unsuccessful;
(7) when the library is damaged, manually importing the redundant library, and if the local library is the main library, generating a synchronous record according to the step (2.1) without synchronous operation; if the library is a spare library, no influence is caused.
2. A group model quasi-delay synchronization system of a rail transit distributed database, which adopts the group model quasi-delay synchronization method of the rail transit distributed database according to claim 1, and is characterized by comprising the following modules:
a record synchronization module: the method has three functions of generating modification time, abstract time and abstract identification; generating modification time as record whether needs updating and processing identification, wherein the summary identification is generated by non-keyword sequence HASH;
station synchronization module: whether synchronization is executed or not is determined locally according to whether a source IP is present or not, a target central library and a station non-local redundant library are synchronized, a station synchronization module in the center determines whether synchronization is executed or not locally according to whether the source IP is present or not, and meanwhile, a station redundant library and a central non-local redundant library corresponding to the station redundant library are determined to be synchronized according to a target station;
a table synchronization module: after the station synchronization module successfully synchronizes, re-reading the target record digests in the table, summing the target record digests to obtain a synchronization digest, comparing whether the target synchronization digests in the synchronization object are consistent or not, and generating a synchronization result;
a synchronous result recording module: recording a synchronization result generated by the table synchronization module; the functions of a database, a data table and a station synchronization cursor are provided, and the records which are modified after synchronization are finely identified.
3. A computer storage medium on which a computer program is stored which, when being executed by a processor, implements a group model quasi-delay synchronization method of a rail transit distributed database as claimed in claim 1.
4. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements a group model quasi-delay synchronization method of a rail transit distributed database as claimed in claim 1 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210820496.1A CN114880404A (en) | 2022-07-13 | 2022-07-13 | Group model quasi-delay synchronization method of rail transit distributed database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210820496.1A CN114880404A (en) | 2022-07-13 | 2022-07-13 | Group model quasi-delay synchronization method of rail transit distributed database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114880404A true CN114880404A (en) | 2022-08-09 |
Family
ID=82683048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210820496.1A Pending CN114880404A (en) | 2022-07-13 | 2022-07-13 | Group model quasi-delay synchronization method of rail transit distributed database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114880404A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116708481A (en) * | 2023-08-07 | 2023-09-05 | 北京全路通信信号研究设计院集团有限公司 | Railway service system data synchronization method and system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090119277A1 (en) * | 2007-11-07 | 2009-05-07 | Richard Dean Dettinger | Differentiation of field attributes as value constraining versus record set constraining |
| CN107590207A (en) * | 2017-08-24 | 2018-01-16 | 阿里巴巴集团控股有限公司 | Method of data synchronization and device, electronic equipment |
| CN112307119A (en) * | 2020-10-27 | 2021-02-02 | 广州市网星信息技术有限公司 | Data synchronization method, device, equipment and storage medium |
| CN114051036A (en) * | 2021-09-23 | 2022-02-15 | 通号城市轨道交通技术有限公司 | Data synchronization method, device and equipment for rail transit signal system and storage medium |
-
2022
- 2022-07-13 CN CN202210820496.1A patent/CN114880404A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090119277A1 (en) * | 2007-11-07 | 2009-05-07 | Richard Dean Dettinger | Differentiation of field attributes as value constraining versus record set constraining |
| CN107590207A (en) * | 2017-08-24 | 2018-01-16 | 阿里巴巴集团控股有限公司 | Method of data synchronization and device, electronic equipment |
| CN112307119A (en) * | 2020-10-27 | 2021-02-02 | 广州市网星信息技术有限公司 | Data synchronization method, device, equipment and storage medium |
| CN114051036A (en) * | 2021-09-23 | 2022-02-15 | 通号城市轨道交通技术有限公司 | Data synchronization method, device and equipment for rail transit signal system and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116708481A (en) * | 2023-08-07 | 2023-09-05 | 北京全路通信信号研究设计院集团有限公司 | Railway service system data synchronization method and system |
| CN116708481B (en) * | 2023-08-07 | 2023-10-27 | 北京全路通信信号研究设计院集团有限公司 | Railway service system data synchronization method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7308456B2 (en) | Method and apparatus for building one or more indexes on data concurrent with manipulation of data | |
| US7801846B2 (en) | Generating log sequence identifiers to apply a transaction to a storage system | |
| CN111367886A (en) | Method and device for data migration in database | |
| CN110555770B (en) | Block chain world state checking and recovering method based on incremental hash | |
| US11176110B2 (en) | Data updating method and device for a distributed database system | |
| US20220019739A1 (en) | Item Recall Method and System, Electronic Device and Readable Storage Medium | |
| CN110543446B (en) | Block chain direct filing method based on snapshot | |
| CN102541691B (en) | Log check point recovery method applied to memory data base OLTP (online transaction processing) | |
| CN109213432B (en) | Storage device for writing data using log structured merge tree and method thereof | |
| CN111651519A (en) | Data synchronization method, data synchronization device, electronic device, and storage medium | |
| CN110413631B (en) | Data query method and device | |
| CN109947801A (en) | Database in phase system, method and device | |
| CN114880404A (en) | Group model quasi-delay synchronization method of rail transit distributed database | |
| CN103207916A (en) | Metadata processing method and device | |
| CN110543485B (en) | Block chain reservation filing method based on snapshot | |
| CN114756408A (en) | Metadata backup recovery method and device, electronic equipment and storage medium | |
| CN111338574B (en) | Data self-repairing method, device, medium and electronic equipment based on block chain | |
| CN113902531A (en) | Automatic generation system, method, electronic device and storage medium for bid document | |
| CN112860376A (en) | Snapshot chain making method and device, electronic equipment and storage medium | |
| CN116756247B (en) | Data restoration method, device, computer equipment and storage medium | |
| CN116414902B (en) | Quick data source access method | |
| CN112732331B (en) | Code history record recovery method, system and medium | |
| CN111984470B (en) | Storage cluster system fault recovery automatic detection method and device | |
| CN112685431B (en) | Asynchronous caching method, device, system, electronic equipment and storage medium | |
| CN117171129A (en) | Data processing method, device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220809 |
|
| RJ01 | Rejection of invention patent application after publication |