Detailed description of the invention
Fig. 2 and Fig. 3 have listed the framework of a long-distance interactive system based on cloud computing node. It comprises terminal device, web front-end, security gateway. Described terminal device can be PC, and any computing capabilitys such as mobile terminal or TV set-top box are competent decoding SD or the H.264 equipment of video flowing of high definition enough. Described security gateway comprises one or more service nodes, and first terminal device accesses the front end of the system of cloud computing, by obtaining the authority that connects a certain service node in security gateway after checking; Equipment uses this authority to be connected with specific service node in cloud computing system subsequently.
By one specifically by hardware, software, or it realizes above-mentioned functions in conjunction with the platform of realizing or environment (being called for short " microenvironment " below), taking windows operating system as example, can be virtual its use to user interactive system of other windows application program, thereby make the computing node of single operation windows operating system can serve multiple users simultaneously. With respect to other schemes, microenvironment has following advantage:
Microenvironment makes multiple users utilize the resource (including the software resource of single operating itself) of computing node to become possibility simultaneously.
The application program being performed is with almost primary mode access hardware resource (particularly graphic process unit resource).
Microenvironment does not change the binary code of performed application program.
In the present invention, in service node, a microenvironment is the application service that user starts appointment, described service can be including application program all services, the interface of application program and audio frequency output are converted to Audio and Video stream by microenvironment, and a very little additional information stream of (optionally) data volume passes to terminal by internet. User receives after above-mentioned Audio and Video stream and (optionally) additional information stream on use terminal device, by the interface hardware (keyboard of equipment, mouse, remote controller, game console etc.) send and control the microenvironment of data to calculation services node, this is controlled the user interactive data that data receive as moved application program by microenvironment. Thereby complete a mutual process.
The realization of microenvironment is at operating system level, between application program and system API or network, those skilled in the art is to be understood that, although used " microenvironment " this term here, but should be by its extend geographic solution a functional structure body can being realized by hardware, software, an or processing unit. Consider an application program arbitrarily, for computer on hardware interactive device carry out alternately, the interface accessing hardware driving that application program must provide by operating system, and then control hardware. (as Fig. 4 left) microenvironment comprises a function address monitoring modular, for detection of the function address variable of application program in internal memory, an and function address redirection module, the function address variable of storing for changing the internal memory of application program, and an address preservation module, for preserving the function address being again directed; And comprise multiple interfaces, for realization and computer hardware and windows application programming interfaces between communicate by letter, for example program interface, picture is played up, the input of audio frequency output and control information (keyboard, the information such as mouse).
By changing the function address variable of storing in the internal memory of application program, make application program be redirected to the access of operating system interface interface that microenvironment provides and no longer contact (as right-hand in Fig. 4) with hardware layer or the hardware driving of actual PC system. For application program, microenvironment provides interface and the respondent behavior identical with operating system. Therefore in microenvironment, carry out and can't affect its normal workflow. Microenvironment has obtained application program and user's interaction data, comprises program interface, and picture is played up, the input of audio frequency output and control information (keyboard, the information such as mouse). Microenvironment can realize in cloud computing system, and be connected with mixed-media network modules mixed-media with video/audio compression module, thereby by the interface of application program with play up data transaction and become video/audio stream, be sent to other terminal (user of cloud computing service) on internet through mixed-media network modules mixed-media. Equally, the customer interaction information of terminal arrives after microenvironment by internet, the customer interaction information that has replaced the interactive device on former cause computing node to send conveys to application program, final realization by internet in the upper object of carrying out interactive application of remote computer (cloud computing service node).
Fig. 5 has shown the embodiment that the another kind of microenvironment is general. Taking windows operating system as example, the realization mechanism of microenvironment is in conjunction with the distinctive dynamic link library of windows system (DLL) mode of operation and process space mechanism independently. In operating system, microenvironment can further comprise multiple processes, and described process is connected with client, carries out specific request by client-requested, and described request can be the data of application program or transmission or reception. Microenvironment is written into internal memory by above-mentioned request subsequently, before the actual execution of above-mentioned request, microenvironment utilizes the working method of dynamic link library that redirection module (with the form of DLL) is loaded to the process space into application program, and in the memory headroom of program process, revises the loaded run time version of application program. Or in order to save the resource of PC hardware, microenvironment directly utilizes the redirected hardware module of self to revise the function address variable comprising in above-mentioned request, to change the funcall entry position of operating system application programming interfaces API of function. After application program is carried out, when routine call operating system API, the function address being modified is jumped to the function address that redirection module is stored by redirection module. Redirection module is connected by specific Communication tunnel with the microenvironment main body of moving in other process, and interactive content when program is carried out outputs in microenvironment process, finally sends to client by network after treatment.
Fig. 6 has shown typical workflow when system API is called after amendment in the memory headroom of program process: by amendment internal storage data, the machine instruction that API entrance function starts is most modified to a jump instruction, and the interface function entrance that provides of redirection module that microenvironment loads is provided. Thereby first application program is redirected in microenvironment redirection module to the use of system API. All parameters when the interface function of microenvironment has received application call system API, after this can be used one of following manner to obtain the interactive information of application program:
1) parameter of importing into when application call system API has just comprised required interactive information, for example location to mouse pointer, the path parameter of file access etc.
2) microenvironment is used the parameter obtaining, and replaces the required system API calling of its script of application call. As shown in Figure 5, in microenvironment, can first carry out the instruction that is modified the internal memory of part in former API, from former API, internal memory starts to carry out until api function finishes after being modified part again, complete the invoked procedure of an API, and in the result data returning, obtain the interactive information of application program, such as coloured program interface picture etc. from API Calls. These information microenvironment interface functions finish, and give back before application program, can first be output to the derivation module of microenvironment.
3) application program is often relevant with hardware device to the use of system API, and specific hardware (or hardware driving) can be realized and is directly connected with derivation module by microenvironment. By the parameter of amendment API Calls, microenvironment can be handed to this by application program to the access of certain hardware device and derive specific hardware of being connected of module (or driving) and carry out, thus the information that the program that is applied is exported from hardware device. The derivation of for example audio-frequency information. Microenvironment also has more effect by application program to the redirected function of hardware access, will set forth in the back.
The importing module workflow of microenvironment and derive module class seemingly: microenvironment interface function obtains from importing the information that needs module, and parameter or the return of value etc. imported into during by application call API pass to application program.
With further reference to accompanying drawing 3, in microenvironment, the mutual all information that comprise are processed separately, and finally by different information flows, be sent to client-side program, in this embodiment, microenvironment includes but not limited to lower module: sound is derived module, interface is derived module, other information derivation modules and input and imported module. Taking PC game as example, it is video flowing output that the picture of game is derived module converts by interface, and it is audio stream output that background music and effect sound sound are derived module converts. In addition in order to realize the function of user interactions, some extra information such as in application program position and the pattern etc. of cursor also by information derive module from one independently channel of information flow export. The interactive information receiving at subscription client is sent to input and imports module, and is converted to the content such as mouse and Keyboard Message of the actual acquisition of application program.
Further, the interface that microenvironment redirection module comprises sequence of operations system call, includes but not limited to Types Below:
Graphical display interface, comprises WindowsGDI and DirectX. In recent years, the PC game of the overwhelming majority utilizes DirectX interface to play up pictures whole in its implementation.
Audio frequency play interface, for example DirectSound or be windows media device interface (MultimediaDeviceAPI).
Input/output interface, comprises DirectInput and keyboard/mouse message interface.
Storage and application context interface, comprise file access API, registry access API, environmental variance and kernel objects etc. The collision problem that can avoid resource to use when these interfaces that microenvironment is redirected are carried out multiple identical application programs under same operating system. The all right record cell of microenvironment for recording and retaining these contextual informations, thereby is prepared from current context information and executing for application program next time.
Because can replacing application program, microenvironment carries out calling of system API. Therefore in the time that application program need to be used hardware resource, microenvironment does not need to simulate all Hardware Response Delay. In actual system, in microenvironment, each request of deriving module application programs is being carried out after necessary processing, conventionally all application program can be come the directed time original position of the access of hardware resource (by actual system function interface). In fact application program is compared with original working method in microenvironment, only has more extra operation required in two step jump instructions and microenvironment, and this compares the expense with much less with Intel Virtualization Technology, can think and be in close proximity to primary working method.
For example, although Windows operating system can support many group hardware resources to work, the CPU of multi-core and many GPU simultaneously. But it is upper that operating system is still confined to CPU to the Automatic dispatching of resource, and can't Automatic dispatching and the computing capability of distributing other hardware (particularly GPU). Even for example multiple GPU of computing node deploy, general application program is often only used in operating system the equipment that is designated as " main GPU " to carry out playing up of 3D picture. Even if application program can have access to the information of whole GPU equipment and can select certain GPU equipment by user's instruction, in the time moving multiple application program on computing node, for each user, it is also impossible selecting suitable GPU equipment.
Utilize microenvironment interface derive module to DirectX interface be redirected, microenvironment can be the GPU resource that certain specific application assigned is suitable at multiple GPU equipment rooms of multiple computing nodes (or service node). select the current GPU equipment with enough idle computing capabilitys, and give different resource occupation to these GPU equipment and complete a specific calculation. for this reason, in microenvironment, further comprise a logging modle, for recording the service condition of current all application programs of carrying out therein to GPU resource, further, it also comprises a supervision module, for monitoring the duty of GPU, to be the next applicable GPU equipment of application assigned of carrying out, that is to say, the GPU of each computing node (or service node) may be assigned with and participate in the calculating irrelevant with this computing node, an embodiment of described supervision module is based on reading the system interface of standard and driving the real-time parameter of interface to monitor its duty.
The redirected output that sound is derived module application programs audio frequency is implemented in hardware driving aspect. Microenvironment before intervention the form with hardware driving realize multiple (virtual) audio frequency and play up and playback equipment. After application program is realized by virtual unit calling of audio frequency play interface, boil down to audio stream outputs to long-distance user's computer by internet. Under normal circumstances, when multiple application programs are moved simultaneously, the output of the audio frequency of each program is by mixed and be output in the audio frequency apparatus of system simultaneously. On the other hand, microenvironment can be one of them of these virtual audio device of application assigned of wherein each execution, makes the audio frequency output of different application be become independently audio stream and the voice-grade channel by different microenvironments transmits.
Outside picture and audio content, application program for user correctly mutual required out of Memory also to derive module by out of Memory collected and be sent to client by accessory channel. For example, in current application program, the physical location of mouse pointer need to be sent to client, to show exactly its position and out of Memory on client screen.
Mouse pointer is the most common element in interactively windows program. As seen from Figure 7, when mutual with long-range application program, any the interactive operation of mouse pointer is all needed to obtain by Internet Transmission, after response, could show back subscription client. This means that user is in the time of client rolling mouse pointer, will observe one and the suitable delay of network delay (TTL). We observe, and user is highstrung to the delay of rolling mouse pointer; But relatively,, in the time clicking on screen element-specific, user's the tolerance to postponing is much bigger. Therefore, in microenvironment, directly the information of cursor is not merged in program interface and transmitted as video stream. But transmit the information of mouse pointer and directly draw out in client by supplemental stream. In the time of user's rolling mouse pointer, first client terminal local will respond the event that mouse moves and repaint mouse pointer, and this event is sent to the application program of long-range execution by control information simultaneously. Noticeable delay when this time strategy has been eliminated rolling mouse pointer. The movement that the delay that network delay causes occurs over just mouse pointer causes program interface to occur in the process of specific response. And as aforementioned, this delay can be tolerated by most users.
Specify the screen resolution that specific remote application may have relatively high expectations and client screen possibly cannot meet this requirement, this is particularly common on the equipment such as mobile terminal. In the case, and the interface Zoom module of deriving module coupling the output interface of application program can be narrowed down to the acceptable scope of client screen and be output as video flowing. After interface is reduced, on interface, original Partial Elements, particularly word content etc., may lose its readability owing to dwindling. Now, an extra application program will start in microenvironment, the interface of this routine access interface exporter, the picture (keeping its original size) of ad-hoc location on program interface (conventionally to be a little square region centered by mouse pointer position pointed) is marked separately, utilize video encoder to be converted into a less video flowing, be transferred to client (as Fig. 7) by supplemental stream, not at normal video flowing part coding; Normal video flowing can be for the whole image of encoding. This video flowing after client decoding, reappear module by its with remote application on identical resolution ratio show, and be superimposed on shown (reduced) the program interface picture of client screen. User can wait operation by moving of mouse pointer, from then on the amplification picture of stack, reads the picture element that dwindles rear None-identified. Client is reappeared module the part marking is separately shown by former resolution ratio, and the described picture that presents is zoomed to the resolution ratio that the display device that uses with this client adapts and shows, the described part marking separately shows as for the superiors.
The coding module that the Audio and Video of microenvironment is derived module and low delay is coupled. The required computing of coding module can utilize CPU or GPU on calculation services node to complete, or uses special coded hardware to process. The graphical information of Application Program Interface forms each frame of outputting video streams by certain time interval. At a time, Application Program Interface figure is converted to the scramble time of corresponding frame in video flowing and is not more than 10 milliseconds; On user's terminal device, this frame of video is also not more than 10 milliseconds by video decoder decodes time of being presented on terminal screen. Under normal circumstances, we expect that Internet Transmission time delay (TTL) from service system to user terminal is in 40-50 millisecond left and right. On the other hand, the frame per second per second of video flowing is limited between 25-30, therefore user with the viewed delay of interbehavior of remote application at most 100 milliseconds of left and right. For general application program, and even mutual PC game is all an acceptable scope for this.
Fig. 8 has shown video encoding module, the streamline strategy (nethermost strategy in figure) that decoder module and transport module are taked in order to reduce coding-decoding delay. The picture of program is actual is split into multiple parts, each part is coded separately into an independent video streaming to client, after being decoded by decoder module, then by reappearing module by the picture amalgamation of cutting apart and showing, the picture of wherein said independent division is with higher bandwidth for transmission. GPU carries out compressed encoding by changing program interface and is converted to video flowing, and the process of decoding at Terminal Server Client. Coding-decode procedure is quasi real time, its postpone transmit in network with data the delay causing compare be a small amount of. In order to reduce the delay of Code And Decode, Application Program Interface is actual in cataloged procedure is divided into multiple parts, and every part encodes independently-transmits-decodes, the treatment step of each several part is with the concurrent execution of pipeline system.
Be that single video flowing is compared (Fig. 8 top strategy) by whole picture coding, although lose, can effectively utilize the parallel ability of computing node in compression ratio, reduce coding delay (strategy in the middle of Fig. 8). But the decoder module of client, due to the restriction of client computing capability, does not often have parallel decoding ability, therefore can not significantly reduce decoding delay. Also can see, in this two kinds of strategies above, the computing capability of equipment is idle in transmitting procedure simultaneously. And in streamline strategy, the process of Code And Decode and the process of transmission are carried out alternately between different pictures piece, when a picture block transmission therein, be encoded module and decoder module of the computing capability of equipment is used for other picture block to process, in fact will encode-transmit-three steps of decoding partly walk abreast, thereby have reduced total mutual delay.
H.264 video decode module on the video encoding module of microenvironment and corresponding client device is used or similar standard. In Video coding, must comprise the image frame of two types. One of them (I frame) do not rely on any other image frame in the time of decoding, the information of some frame being received before the decoding of another kind of (P frame) depends on. Single I frame need to be larger than P frame byte number, the distribution of I frame in video flowing is uncertain, the time interval between the appearance of every two I frames is between 1-10 second. Video flowing between two I frames is made up of the less P frame of byte number.
Within the specific moment, may be suitable difficulty for the node of cloud computing system provides what carry out miscellaneous application program. One of distinct issues are that the binary system run time version of preserving each application program may need a large amount of stored bits units. Particularly, concerning PC game, the binary system run time version of the PC game in the present age often exceedes 4G byte (1 byte number that standard DVD can carry). If it is upper the binary code of program to be stored in respectively to the local storage (hard disk) of each computing node, need to provide for each computing node the storage capacity of several TB bytes. The cost of this storage system of giving and maintenance all bring considerable difficulty. A scheme of saving the memory space of computing node is that the binary code of program is kept on single memory device, i.e. program storage array as shown in Figure 3, and each node in the time of performing a programme by the code data on this equipment of network connected reference at a high speed.
Can this Concurrency Access can only for example, in file system Network Based (SAMBA) aspect and can not for example, realize network disk system (iSCSI) is upper. But to a lot of be only the application program that operation design on local storage at a high speed (for example nearly all PC game), program execution to user interactions before need from storage system, to read in advance a large amount of data in internal memory. In theory, in the time transmitting data with the ideal velocity of gigabit (Gigabit) network between computing node and memory node, the reading rate (~ 100MB/s) of data can match with conventional local storage (hard disk, about 70MB/s). If stand-by period that the reading of data causes and on local storage quite when (even slightly long), this wait has been familiar with by user and has been accepted, and will can not reduce user's experience. But, these data are usually deposited with the form of small documents, and the conventional NFS based on SAMBA agreement be not suitable for the concurrent task that reads of large amount of small documents under windows, result causes actual valid data reading speed to be often less than 10MB/s, before program execution, be several times and even ten several times on local storage with executory data read waiting time, seriously reduced user's experience.
By microenvironment, we have optimized NFS under the SAMBA agreement data reading performance using redundancy in the time that application program is accessed, and do not need to develop in addition other NFS that difficulty more and stability leave a question open. In the system of Fig. 3, in cloud computing system, all service nodes are connected with single storage array. As shown in Figure 9, under normal circumstances, application program is directly by the network file position (dotted arrow) on SAMBA protocol access storage array. And in microenvironment, the actual buffer memory that is arranged in local memory device that is redirected to of the access of application program. Cache contents is managed by microenvironment. In the time that computing node runs application first, by another, the agreement based on rSync scheme (we are called rSync_i) reads the network file on storage array to microenvironment. In the time reading, multiple concurrent file access requests are merged into single TCP request and are conducted interviews according to directory location, make the reading speed of file reality be approached the extreme value that network environment allows.
Multiple computing nodes are the application data of jointly accessing on same memory node, and the excessive data that each user is produced when the executive utility stores on single memory node. Importing and exporting and transmitting by microenvironment of data realizes. When microenvironment has been used specific strategy to carry out accelerating application to carry out from memory node the speed of reading out data. Multiple computing nodes are formed a node array by system, and multiple node arrays are shared a memory node (array) and read the data of performed application data and recording user generation. User can be by one in the multiple computing node arrays of the data selection such as network delay and bandwidth. Multiple memory nodes have formed distributed memory system that can phase mutually synchronization.
Subscription client is accepted discernible any user interactive and is sent in microenvironment by control information. But the input filtering module of microenvironment will be differentiated all user's operations and only allow specific operation can send in the middle of the application program being performed. This judgement is situation based on current application program interface and dynamically changing. For example, in the time that current cursor (can access and carry out the window of the file of the local store path of computing node) at the interface that is judged as " danger ", input filtering module can stop long-distance user to carry out may be to computing node and the adventurous operation of whole service system.
Derive the video/audio stream of module output and the data of supplemental stream are stored in microenvironment output buffer module as shown in Figure 4 from interface and sound. Therefore, the user's that the information such as the interface of performed application program not only can requested execution terminal is seen, also can output in one or more users' that participate in onlooker's identity terminal by output buffer module and transport module. Connect the microenvironment of same computing node as onlooker's user, flow in corresponding microenvironment but cannot send control information. Onlooker user, except watching the action with the user of program interaction, can also send specific interactive information to cloud computing system by other passage. As shown in figure 10, a specific program can start in this microenvironment, receives and gather other and watch user's exchange of information. Interface is derived module a video flowing is merged into in the interface of these exchange of information and application program, thereby these interactive information are distributed to again to user's (comprising interactive user and onlooker user) of current all these microenvironments of connection, and are may be displayed in all users' terminal. For example, in the microenvironment at a user place of carrying out PC game, onlooker can input comment in real time with written form, and these comments can send to all other onlooker and game user.
The video flowing that beholder's user and the mutual user of execution access is synchronous (except the impact causing due to different network delays between user). But the video decode module of user terminal generally can not have the frame (B frame) of reference to decode and correct display of video streams since one. In output buffer module, can preserve all videos frame data in a period of time of having exported to interactive user, wherein must comprise at least one I frame without reference. As shown in figure 11, when a beholder user is in the time connecting microenvironment at first, output buffer module by preserve from sending to beholder user when the nearest I frame of output frame of forward direction interactive user, until while having next I frame to occur, this frame will see that beholder user and interactive user export simultaneously. The time interval occurring due to two I frames may reach 10 seconds, and the terminal that this strategy has significantly reduced beholder user is the delay when there is video from successful connection.
Particular application that is called as " displaying wall " may operate in one independently in microenvironment, as shown in figure 12. The intrasystem service managerZ-HU of displaying wall access computation is obtained the application program of moving in other microenvironment, and is connected in these microenvironments as beholder user. Show the wall program all video flowings that get of decoding, and these video flowings are reorganized in a single interface with the size of preview graph. This interface is encoded as new video flowing and sends to user terminal screen. User can select to show certain preview video stream within the walls, and other users in flowing corresponding microenvironment as a beholder user with preview video exchange.
The Audio and Video stream of microenvironment is sending to before network and can both be merged into the multimedia content flows (as shown in Figure 4) of specific format by the agreement module of reforming. Such multimedia content flows can further be arrived at the client terminal program with special realization by internet. For example, the multimedia content flows generating from reformation module can be by with RTMP(RealTimeMessagingProtocol) mode of the agreement AdobeShockwaveFlash platform that carries and be widely used presents. Thereby user allows to use cloud service system on the basis of not moving special client-side program. For example, this makes any user only need to have the terminal (all PC, net book, WebOS system and most mobile terminal etc.) that can move modern Web browser and can be used as beholder user's access.
Use AdobeShockwaveFlash platform can receive the supplemental stream that microenvironment is sent, and the operation on flash platform sends to input as control information stream and imports module using user, thereby realize a complete user terminal and be not only beholder. But based on the security limitations of Flash platform, while carrying out remote application, some required interbehavior can not directly be realized. For example portion keyboard function. A typical situation is, in the PC of for example first person of some application program game, to allow mouse to do unlimited movement (corresponding to the continuous rotation at visual angle) to a specific direction. But the mouse action that the client terminal of being realized by Flash can only receive in a limited rectangular extent, when mouse moves while causing pointer movement to the edge of rectangle to a direction, follow-up movement cannot be detected again.
Figure 13 has shown this problem solution: the rectangular area that receives mouse pointer mobile message is split into two parts, and in normal region, the information that mouse moves is normally sent to long-range application program. In the time that mouse pointer moves in the buffering edge region of rectangular edges scope, on receiving once before mouse move, last mobile information can be sent to remote application at certain intervals repeatedly, continues mobile information thereby allow user to use mouse to send to specific direction. In the time of buffering edge region, specific pattern will be presented near mouse pointer focal position, to point out special mobile behavior of user to occur.
In the service system shown in Fig. 3, service managerZ-HU is responsible for controlling the situation of the application program of carrying out in whole microenvironments on all service nodes in cloud system, and itself and customer data base are associated. Use a performed typical workflow of the user of cloud computing remote interaction procedure service as shown in figure 14. User's the terminal program first front end of access system (conventionally with Web form) obtains by security gateway and the authority (1) that is connected a certain microenvironment; Front end and service managerZ-HU alternately taking on security gateway as user terminal is opened an outlet, and according to the information of customer data base, corresponding microenvironment is set. In the time that user terminal connects this microenvironment, the list of the application program that user can carry out in microenvironment is passed through database information specified (2) by service managerZ-HU; Carry out in the process of specific application program user, relevant information (contents of program of for example carrying out, time of implementation, the data that user generates etc.) is passed service managerZ-HU and sends back customer data base (3); Finally, complete when user's executive utility, user leaves microenvironment and informs the information that service managerZ-HU exits. and required information in microenvironment before collecting is synchronized to customer data base by service managerZ-HU, for user accesses prepare (4) next time.
In the application of remote interaction procedure service, network environment is experienced and has been caused the most directly restriction user. Experience in order to obtain enough users, user needs the network of a low delay and high bandwidth to the link of computing node. Conventionally, the transmission delay of network should be lower than 50 milliseconds. In addition, test shows that, under a rational level of resolution (being conventionally assumed to be 1024x768), the video flowing of transmission reaches 25-30FPS and keeps enough readability to require the bandwidth of minimum 2Mbps. This means when a cloud service system is served a large number of users at the same time, to grid, outlet will cause huge pressure to the flow of network.
Such cloud service system requires a distributed deployment scheme so that the network bandwidth that effectively utilizes differentiation is to improve efficiency of transmission and to reduce cost. As shown in figure 15, distributed calculation services node array is deployed in multiple positions in all parts of the country, the network egress of its served user terminals single line network that normally campus network is lower. The array of multiple deployment is connected with the distributed storage array on access backbone network by special line. All these distributed systems are all with high-quality but limited bandwidth is connected to central database. User's terminal is after access web front-end, and first from connecting multiple calculation services nodes and selecting connective best one, then the microenvironment that connects certain node is wherein with executive utility service. The subscriber data file producing in user and the application program interaction direct-connected storage array of service node that will write direct. After user interactions finishes, service managerZ-HU will send request, in a specific time-domain, (conventionally be less than 12 hours), and the data file that user interactions produces will be synchronized in whole storage arrays. A small amount of core data is write customer data base in real time.
This distributed schemes based on following some consider: 1. the network egress at single calculation services node array place cannot bear bandwidth load that numerous users bring and provide the JA(junction ambient) that meets the demands (bandwidth of minimum 2Mbps and postpone lower than 50ms) for all users; 2. user may produce quite a large amount of user data in the time carrying out an interactive program. For example, in the time that user carries out the game of a physical culture racing tip, can record a video and generate the data file that reaches hundreds of MB some scene in game. Single storage array cannot be in real time or the data that quasi real time complete magnanimity with whole distributed array of services synchronous, still, and limited array service node array to make real-time data be synchronously possible; 3. due to the restriction of geographical conditions, user's possible translational speed in network, by lower than completing synchronous time threshold between storage array. For example, suppose to have to be distributed in the regional storage array in Beijing and two, Guangzhou, user store into user data in the storage array of In Guangzhou Area can be in the longest 12 hours and the array of Beijing area to complete data synchronous. And in most cases, a user's (we suppose its corresponding natural person's individuality) who finishes service in In Guangzhou Area can not apply new service in existing communications and transportation level in 12 hours in Beijing area. This real time threshold makes above-mentioned distributed schemes become possibility.