CN115543585B

CN115543585B - Enterprise number card data synchronization method, server and storage medium

Info

Publication number: CN115543585B
Application number: CN202211495727.2A
Authority: CN
Inventors: 王亮; 林锋; 董芸; 付朝印; 刘广达
Original assignee: Beijing Capitek Co ltd
Current assignee: Beijing Capitek Co ltd
Priority date: 2022-11-28
Filing date: 2022-11-28
Publication date: 2023-08-22
Anticipated expiration: 2042-11-28
Also published as: CN115543585A

Abstract

The application discloses an enterprise number card data synchronization method, a server and a storage medium, belonging to the field of computer data processing, comprising the following steps: step 1: grouping the enterprise number cards according to the enterprise identification numbers, and dividing the number card data in each group of enterprise number cards into a plurality of task fragments; step 2: the task service instance acquires task fragments to be processed in a timing polling mode, and step 3 is carried out after judging that the processing conditions are met, wherein the data synchronization task of each group is carried out in a serial synchronization mode; step 3: sequentially acquiring the validity period of the number card data in the task fragment, and judging whether synchronization is needed; if the synchronization is not needed, continuing to acquire new number card data; if synchronization is required, step 4 is executed: and (3) calling a plurality of operator core network interfaces to acquire data for synchronizing the number card data, and continuing to perform the step (3) after the synchronization task is completed until the number card data in the task fragments are all completed. The method can achieve the effects of parallel processing and load balancing.

Description

Enterprise number card data synchronization method, server and storage medium

Technical Field

The application belongs to the field of mobile network data processing, and particularly relates to an enterprise number card data synchronization method, a server and a storage medium.

Background

In the field of computer data management and data synchronization technology of mobile networks, different data contents determine the manner in which data is stored, managed and synchronized. The traditional data synchronization mode is generally based on a C/S or B/S architecture, and the deployment client side sequentially pulls data from the server side, so that the mode has the advantages of simplicity and convenience, but also has the defects of flexibility, high efficiency, stability and the like.

Currently, private networks in the 5G industry rapidly develop, and enterprise clients also provide urgent requirements for self-management and self-service for the private networks in the 5G industry, wherein the private network enterprise client self-management system in the 5G industry provides visual, controllable means for the private networks in the 5G industry for the enterprise clients. In the process of researching and developing a private network enterprise client self-management system in the 5G industry, the problem of enterprise number card data synchronization is faced, the method needs to create a client, start a timing task, periodically collect the data of enterprise number cards of data platforms such as a mobile network operator core network, a group data center, a local data center, an Internet of things platform and the like, but the method encounters a plurality of practical problems in the process of docking results, such as failure in calling in certain cases of an interface provided by a core network side, limited calling times or time intervals of the interface provided by the core network side, untimely enterprise number card data synchronization and the like, and meanwhile, the practical problems of enterprise number card data synchronization in the mobile network are effectively solved by taking future expansion, high-availability use scenes in different scenes into consideration, and a method with reliable design performance, high stability and high efficiency is needed.

Disclosure of Invention

In order to solve the problems that in the prior art, when enterprise number card data in a mobile network are synchronized, a traditional timing data acquisition mode is executed at one time, a midway abnormal or access limited task is terminated, the data acquisition in the current period is failed, the data is permanently lost, distributed multi-instance parallel processing and load balancing cannot be supported, high availability cannot be realized and the like.

The application solves the problems by the following scheme:

according to a first aspect of the present application, there is provided an enterprise number card data synchronization method, comprising the steps of:

step 1: grouping the enterprise number cards according to the enterprise identification numbers, dividing the number card data in each group of enterprise number cards into a plurality of task fragments, wherein each task fragment comprises a number card data synchronization task of a plurality of enterprises;

step 2: the task service instance acquires task fragments to be processed in a timing polling mode, and step 3 is carried out after judging that the processing conditions are met, wherein the data synchronization task of each group is carried out in a serial synchronization mode;

step 3: sequentially acquiring the number card data in the task fragments, checking the validity period of the number card data, and judging whether synchronization is needed;

if the synchronization is not needed, continuing to re-execute the step 3 to acquire new number card data;

if the synchronization is needed, executing the step 4;

step 4: and (3) calling a plurality of operator core network interfaces to acquire data for synchronizing the number card data, and continuing to perform the step (3) after the synchronization task is completed until the number card data in the task fragments are all completed.

Preferably, in step 2, the task service instance adopts a hash consistency algorithm to acquire task slices, so as to realize a timing polling mode.

Preferably, the specific method of the hash consistency algorithm is as follows:

2-32 points are clockwise formed into a circle to form a hash ring;

mapping the task service instance and the task fragment to a hash ring;

starting from the position of the tile on the hash ring, the first service instance encountered in the clockwise direction is the task service instance that handles the tile.

Preferably, in step 2, the judgment that the processing condition is satisfied is specifically: judging whether other task service instances exist in the current task partition, if not, meeting the processing conditions.

Preferably, in step 3, if synchronization is required, the last progress of the current card data synchronization task is first obtained, if the progress is completed, a new synchronization task is started, and if the last progress is not completed, the last synchronization task is continued.

Preferably, the starting of the new synchronization task is specifically:

it is preferred to determine if the dependent data is conditional,

if the condition is not met, the polling processing is exited;

if the condition is met, the dependent data is acquired, whether the dependent data is updated or not is judged, and the task data is updated according to the change condition of the dependent data and then the synchronization task is carried out.

Preferably, the progress of the number card data synchronization task is stored in the database every time, and if the previous progress is not completed, the progress of the synchronization task is read through the database and is continued.

Preferably, in step 4, when the calling of the interfaces of the several operators fails or the calling is limited, the currently processed synchronous task progress data information needs to be saved, and the polling process is exited.

According to a second aspect of the present application, there is provided a server comprising: a memory and at least one processor;

the memory stores a computer program, and the at least one processor executes the computer program stored in the memory to implement the method for synchronizing data of the enterprise number card according to any one of the above technical solutions.

According to a third aspect of the present application, there is provided a computer-readable storage medium having stored therein a computer program which, when executed, implements the enterprise card data synchronization method of any one of the above technical solutions.

Compared with the traditional timing data acquisition mode mentioned in the background art, the method has the advantages that the timing is performed once, the midway abnormal or access limited task is terminated, the current period data acquisition is failed, the data is permanently lost, the distributed multi-instance parallel processing and load balancing cannot be supported, and the high availability cannot be realized, so that the method has the following remarkable advantages and effects:

1. the distributed multi-service instance parallel collaborative processing task is supported by adopting task segmentation according to enterprises, so that high availability and load balancing can be realized, and the service instance dynamic expansion and contraction capacity is supported.

2. The task segmentation processing can be realized by adopting a timing polling mechanism, the task progress is gradually advanced, and the fault tolerance capability is strong when the task is abnormal or limited.

3. The task data period can be dynamically adjusted without restarting the service instance, and the configuration is supported to be dynamically effective.

Drawings

In order to more clearly illustrate the embodiments of the application or the prior art solutions, the drawings which are used in the description of the embodiments or the prior art will be briefly described below, it being obvious that the drawings in the description below are only some of the embodiments described in the present application, and that other drawings can be obtained according to these drawings without inventive faculty for a person skilled in the art.

FIG. 1 is a flowchart of a method for synchronizing data of an enterprise number card according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of a server according to an embodiment of the application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

As shown in fig. 1, in one embodiment of the present application, a method for synchronizing data of an enterprise number card in a mobile network includes the following steps:

s110: grouping the enterprise number cards according to the enterprise identification numbers, dividing the number card data in each group of enterprise number cards into a plurality of task fragments, wherein each task fragment comprises a number card data synchronization task of a plurality of enterprises;

in this step, a synchronous task is divided into different task slices, which are completed by different task service instances.

S120: the task service instance acquires task fragments to be processed in a timing polling mode, and S130 is carried out after judging that the processing conditions are met, wherein the data synchronization task of each group is carried out in serial synchronization;

in this step, a server in which a task service instance is a distributed running timed task service is referred to as a task service instance, and a plurality of task service instances cooperatively complete the same timed task.

The timing task will process the synchronous task in a polling mode, for example, the task processing encounters interface limitation or other abnormality to cause interruption, and the task processing can be continued when polling is performed next time, so that the problem of the traditional timing data acquisition mode in the prior art is solved, namely: the method is characterized by comprising the steps of executing the task at one time, stopping the midway exception or access limited task, failing to acquire the data in the current period, and permanently deleting the data.

The time interval between two polls is called a task polling period, and the initial default value of the polling period can be set to 5 minutes and can be adjusted through a configuration file.

In this embodiment, the judgment that the processing condition is satisfied is specifically: judging whether other task service instances exist in the current task partition, if not, meeting the processing conditions.

In an embodiment of the application, the task service instance adopts a hash consistency algorithm to acquire the task fragments, thereby realizing a timing polling mode. The method comprises the following steps:

clockwise forming 2-32 points into a circular hash ring, wherein the point right above the circular ring represents 0, the first point on the right side of the 0 point represents 1, and so on, 2, 3, 4, 5, 6 … … until 2-32-1, that is, the first point on the left side of the 0 point represents 2-32-1;

mapping the task service instance onto the hash ring through a formula hash (task service instance IP)% 2A & lt-2 & gt to determine the position of the task service instance in the hash ring;

mapping task fragments to a hash ring through a formula hash (task fragment identification)% 2-32;

The task fragments can enable the task service instances to cooperatively process the tasks in parallel, and the task fragments are distributed by using a hash consistency distribution algorithm, so that only a small number of task fragments can be influenced when the task service instances dynamically expand and contract.

S130: sequentially acquiring the number card data in the task fragments, and checking the validity period of the number card data; and the data acquired by each timing task has different validity periods, and the timing tasks do not need to be newly synchronized when the data is in the validity periods. The time interval between the two new syncs is called a data sync period, and the data sync period of the task can be configured by the database.

Then, S131: judging whether synchronization is needed, checking the data validity period, and judging whether the task of the current data period is finished;

if the synchronization is not needed, the step S130 is continuously executed again to acquire new number card data;

if synchronization is required, S140 is executed;

s140: and calling a plurality of operator core network interfaces to acquire data for synchronizing the number card data, and continuing to perform S130 after the synchronization task is completed until the number card data in the task fragments are all completed.

In the step, firstly, the last progress of the current card data synchronization task is required to be obtained, if the progress is completed, a new synchronization task is started, and if the last progress is not completed, the last synchronization task is continued, wherein the synchronization tasks are as follows: calling a plurality of operator core network interfaces to acquire data for synchronizing the number card data. And the progress of the number card data synchronization task is stored in the database every time, and if the previous progress is not completed, the progress of the synchronization task is read through the database and is continued.

In an embodiment of the present application, the starting of a new synchronization task is specifically:

it is preferred to determine if the dependent data is conditional,

if the condition is not met, the polling processing is exited;

The dependent data in the step refers to that the input of some tasks can relate to the output of other tasks, so that before the tasks are performed, whether the dependent tasks are finished or not needs to be judged, and whether the required data have conditions or not needs to be judged, thereby improving the expandability of the method.

In the step, when the calling of a plurality of operator core network interfaces fails or is limited, the currently processed synchronous task progress data information is required to be stored, and the polling process is exited.

In an embodiment of the present application, the enterprise list, that is, the enterprise card data, the validity period of the card data, the synchronization task progress data, the dependency data, the number of failures of the synchronization task, etc. are all stored in the database, and in the process of executing the above steps, the database is accessed at any time to complete the synchronization task.

Compared with the traditional timing data acquisition mode mentioned in the prior art, the method has the advantages that the timing is performed once, the midway abnormal or access limited task is terminated, the current period data acquisition is failed, the data is permanently lost, the distributed multi-instance parallel processing and load balancing cannot be supported, the high availability cannot be realized, and the method has the following remarkable advantages and effects:

As shown in fig. 2, in one embodiment of the present application, there is provided a server including: a memory 201 and at least one processor 202;

the memory 201 stores a computer program, and the at least one processor 202 executes the computer program stored in the memory 201 to implement the enterprise card data synchronization method of any one of the above methods.

According to a third aspect of the present application, there is provided a computer readable storage medium having a computer program stored therein, the computer program when executed implementing the method of synchronizing enterprise card data of any of the above methods.

It should be noted that the foregoing detailed description is exemplary and is intended to provide further explanation of the application. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present application. As used herein, the singular is intended to include the plural unless the context clearly indicates otherwise. Furthermore, it will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, steps, operations, devices, components, and/or groups thereof.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or otherwise described herein.

Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those elements but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Spatially relative terms, such as "above … …," "above … …," "upper surface at … …," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial location relative to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as "above" or "over" other devices or structures would then be oriented "below" or "beneath" the other devices or structures. Thus, the exemplary term "above … …" may include both orientations of "above … …" and "below … …". The device may also be positioned in other different ways, such as rotated 90 degrees or at other orientations, and the spatially relative descriptors used herein interpreted accordingly.

In the above detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, like numerals typically identify like components unless context indicates otherwise. The illustrated embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein.

The above is only a preferred embodiment of the present application, and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. The enterprise number card data synchronization method is characterized by comprising the following steps:

step 2: the task service instance acquires task fragments to be processed in a timing polling mode, and step 3 is carried out after judging that the processing conditions are met, wherein the data synchronization task of each group is carried out in a serial synchronization mode; the task service instance adopts a hash consistency algorithm to acquire task fragments, so that a timing polling mode is realized; after the data synchronization task is interrupted, continuing to carry out the next polling; the specific method of the hash consistency algorithm comprises the following steps:

2-32 points are clockwise formed into a circle to form a hash ring;

mapping the task service instance and the task fragment to a hash ring;

starting from the position of the fragment on the hash ring, the first service instance encountered in the clockwise direction is the task service instance that handles the fragment;

if the synchronization is needed, executing the step 4; if the synchronization is needed, firstly acquiring the last progress of the current card data synchronization task, if the progress is completed, starting a new synchronization task, and if the last progress is not completed, continuing the last synchronization task; the progress of the number card data synchronization task is stored in a database, and the progress of the synchronization task is read through the database; the method for starting the new synchronous task specifically comprises the following steps:

it is preferred to determine if the dependent data is conditional,

if the condition is not met, the polling processing is exited;

if the condition is met, acquiring the dependent data, judging whether the dependent data is updated or not, and updating task data according to the change condition of the dependent data and then carrying out a synchronous task;

2. The method for synchronizing data of an enterprise card as claimed in claim 1, wherein in step 2, the judgment that the processing condition is satisfied is specifically: judging whether other task service instances exist in the current task partition, if not, meeting the processing conditions.

3. The method for synchronizing data of an enterprise number card according to claim 1, wherein the progress of the task for synchronizing the number card data is stored in the database each time, and if the previous progress is not completed, the progress of the task for synchronizing is read and continued through the database.

4. The method for synchronizing data of an enterprise card according to claim 1, wherein in step 4, when the calling of the interfaces of the several operators' core networks fails or the calling is limited, the currently processed progress data information of the synchronized task is saved, and the polling process is exited.

5. A server, comprising: a memory and at least one processor;

the memory stores a computer program, and the at least one processor executes the computer program stored by the memory to implement the enterprise card data synchronization method of any one of claims 1 to 4.

6. A computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, the computer program when executed implementing the enterprise card data synchronization method of any one of claims 1 to 4.