CN102355489A - Method, system, server and client side for displaying and operating 3D (Three-Dimensional) game - Google Patents
Method, system, server and client side for displaying and operating 3D (Three-Dimensional) game Download PDFInfo
- Publication number
- CN102355489A CN102355489A CN2011102350716A CN201110235071A CN102355489A CN 102355489 A CN102355489 A CN 102355489A CN 2011102350716 A CN2011102350716 A CN 2011102350716A CN 201110235071 A CN201110235071 A CN 201110235071A CN 102355489 A CN102355489 A CN 102355489A
- Authority
- CN
- China
- Prior art keywords
- client
- server end
- equipment
- input
- keyboard
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention provides a method, system, server and client side for remotely displaying and operating a 3D (Three-Dimensional) game. The method comprises the following steps of: 1, acquiring image data of a DirectX-based 3D game display image of a server side by a server-side application layer display device in real time; 2, compressing the acquired image data of the 3D game display image by the server and transmitting the compressed image data to the client side; 3, receiving, decoding and displaying the compressed image data by the client side, receiving an operation instruction of a user by input equipment and passing the operation instruction back to a server-side virtual input device at the server side; and 4, receiving the operation instruction passed back by the client side by the server-side virtual input device in real time, and converting the operation instruction into operation information capable of being acquired by a DirectX game program. According to the method and system provided by the invention, the image data of the 3D game are acquired in real time in a user-layer code injection manner on the premise of not modifying the original game; and the user input passed back by the client side is provided to the game in the user-layer code injection manner on the premise of not modifying the original game.
Description
One, technical field:
The present invention relates to the computer network remote control field, relate in particular to a kind of method, system and server of playing that show through network with operation 3D.
Two, background technology:
Computer 3D recreation is to use the space multistory computing technique to realize the recreation of operation; Generally speaking; The 3D recreation has higher requirement to the arithmetic speed and the memory size of video card than other recreation, if hardware can not reach requirement, will move during recreation slowly or even deadlock.In other words, the 3D recreation is very high for the configuration requirement of computer hardware.
On the other hand; Come the pattern of long-range use, operational computations machine just obtaining more and more general application through network; Especially along with the development of cloud computing pattern; People tend to that computational resource, storage resources, software resource are concentrated on data center and unify operation and management; And the general user comes long-range these resources of visit through network, carries out software and uses and operation.
But; For the 3D recreation; There is following problem in this mode of operation: at first; Under the remote application pattern; Software (i.e. recreation) is actually and operates in server end;, image shows that the user's of client reception simultaneously input sends it to server end with complete operation and being transferred back to client.And the 3D recreation needs the real-time speed display image with per second tens frames, and the realtime graphic that transmits these volumes like this through network is unpractical; Secondly, need the real-time operation with client to pass server end back with the control recreation, this also is unpractical under current pattern.
Prior art related to the present invention has remote desktop technology and RemoteFX technology.
1. remote desktop technology.
The remote desktop that is built in the Windows system (after the NT 4.0) has used RDP (Remote Desktop Protocol) agreement, makes the user and to use the target desktop from Telnet on other the computer, visit.RDP agreement Microsoft the earliest is used for the access protocal of the Terminal Service (Terminal Service) on the Windows Server; Realized the multi-user mode on the Windows Server; Make the user that the various application on the long-range use server can be installed under any condition of application in this locality.This technology is that runnable interface on the target machine is transferred on the actual manipulator screen of user, and with keyboard, a series of peripheral hardware input such as mouse is transferred to target machine, realizes mutual.But the RDP agreement is not supported long-range 3D recreation, and in other words, remote activation 3D recreation can't move under this pattern.
Also comprise the ICA agreement that Citrix develops with the similar RDP of RDP, generally, ICA protocol transmission efficient will be higher than the RDP agreement, but it also can't support the Long-distance Control of 3D recreation.
2.RemoteFX technology.
RemoteFX is the desktop virtual technology that Microsoft increases in Windows 7/2008 R2 SP1, makes the user when using remote desktop to carry out games application or figure creation, can obtain and local desktop similar effects.The core of RemoteFX is two aspects: the one, adopt the video card Intel Virtualization Technology at server end, and drive a plurality of video card equipment that fictionalize through system kernel, each long-range connection desktop uses a virtual video card; The 2nd, through virtual video card, show passing to the client decompress(ion) after the video data compression.So just can support long-range 3D gaming display.RemoteFX has relatively high expectations for system hardware and software, at first must be that the server-side processes device must be supported SLAT-enabled processor characteristic more than windows server 2008 SP1 reached, and realize at the inner nuclear layer of system.
Three, summary of the invention:
Have above-mentioned technical problem for solving in the prior art, the present invention provides a kind of and shows and the method and system of operating the 3D recreation through network.This system is by the server of installation, running game, and the client of demonstration and user's operate game links to each other with network and constitutes.The user can be through client (being not limited to PC); The image real-time data that the 3D that obtains to move on the remote server plays; After this locality shows; The input operation that receives the user is transferred to remote server; After this 3D recreation response; Generate new view data and show that this process is gone round and begun again, thereby carries out long-range game control.
According to a first aspect of the invention, a kind of through the method that network shows and operation 3D plays, this method may further comprise the steps:
Step 1: server end application layer display unit is obtained the view data based on the 3D gaming display image of DirectX of server end in real time;
Step 2: the view data that server compresses the view data of the 3D gaming display image that obtains and will compress is sent to client;
Step 3: client receives the view data and the decoding and displaying of above-mentioned compression, receives user's operational order and the server end virtual input device that server end is given in passback through input equipment;
Step 4: the server end virtual input device receives the operational order of client passback in real time, is converted into the obtainable operation information of DirectX games.
Generation and demonstration based on the view data of the 3D gaming display image of DirectX in the wherein said step 1 comprise the steps:
Step 1.1) DirectX running environment initialization;
Step 1.2) the 3D games are created the server end application layer display unit of a DirectX;
Step 1.3) is directed against the render process that above-mentioned display unit is carried out 3D gaming display image, goes round and begins again, until withdrawing from recreation.
Wherein above-mentioned steps 1.3) comprising:
Step 1.3.1) empties the current content of this display unit;
Step 1.3.2) notify this display unit to begin to play up;
Step 1.3.3) on this display unit, plays up said image;
Step 1.3.4) notify this display unit render process to finish;
Step 1.3.5) shows this image.
Wherein said step 1.3.5) is to inject the default behavior of revising the DirectX call function, makes its current image date of obtaining this display unit, and it is copied in the internal memory, to accomplish obtaining of 3D gaming display image through server end client layer code.
The view data of obtaining in the wherein said step 1 is the continuous images data, uses video compression algorithm that the said view data of obtaining is compressed.
Wherein the said input equipment in the step 3 comprises: mouse.
Wherein the said input equipment in the step 3 comprises: keyboard.
Wherein step 4 is to inject with the DirectInput interface function of revising DirectX through server end client layer code to realize that the 3D recreation receives the input of input equipment in real time, thus operation 3D recreation; Wherein said DirectInput interface function is that the DirectX recreation is used to collect the interface function of operation input, and it is through the data of DirectInputDevice object accesses input equipment, and said DirectInputDevice object is represented input equipment.
Wherein said DirectInputDevice object is a keyboard, and then the workflow of DirectInputDevice (keyboard) is following:
Step 4.1) creates a keyboard equipment;
Step 4.2) being set is the array return key disc status value of passing through one 256 byte;
Step 4.3) sets collaboration mode;
Step 4.4) activation is to the access rights of this keyboard equipment;
Step 4.5) obtain the operation information of this keyboard equipment,
Wherein, step 4.5) be a process that circulation is obtained, make the 3D recreation receive the keyboard equipment input in real time, thereby accomplish control.
Through one of any obtaining step 4.5 of following dual mode) in the operation information of keyboard equipment:
Direct mode: the state that promptly obtains the current time keyboard equipment; Keyboard equipment is regarded as equipment with 256 different keys; The state of each button comprises down state or non-down state; And the state of each button is all usually represented by a unit of one 256 byte arrays of correspondence; The non-down state of 0 expression; 0x80 representes down state; Inject the behavior of revising the corresponding interface function of keyboard equipment through server end client layer code; Making it is not the input that obtains the local physical keyboard of server end, but the key-press status behavior of client is returned;
Buffering method: return the last continuous situation of change that obtains the input keyboard equipment afterwards of keyboard equipment; Inject the behavior of revising the corresponding interface function of keyboard equipment through server end client layer code; Making it is not the input that obtains the local physical keyboard of server end, but the key-press status behavior of client is returned.
Wherein said client layer code be infused in that interface function that the 3D games use calls and the actual function that realizes between insert code, to intercept and capture the call parameters information of program.
It is that a kind of transparent code inserts that wherein said client layer code injects, and it carries out the function parcel after application program is loaded into internal memory again, directly in internal memory, rewrites the code of correlation function, and does not change the size of execute file.
According to a further aspect in the invention, a kind of through the system that network shows and operation 3D plays, this system comprises:
Server end application layer display unit is obtained the view data based on the 3D gaming display image of DirectX of server end in real time;
Server end video compression device, the view data that the view data of the 3D gaming display image that obtains is compressed and will be compressed is sent to client;
Client video playback and user operate the passback device, receive the view data and the decoding and displaying of above-mentioned compression, and the server end virtual input device is given in the operational order and the passback that receive the user through input equipment;
The server end virtual input device receives the operational order that client returns in real time, is converted into the obtainable operation information of DirectX games.
Wherein said server end application layer display unit is used and is comprised like lower device and generate and show the view data based on the 3D gaming display image of DirectX:
Device is used for the initialization of DirectX running environment;
Device is used for the server end application layer display unit that the 3D games are created a DirectX;
Device is used for being directed against the render process that above-mentioned display unit is carried out 3D gaming display image, goes round and begins again, until withdrawing from recreation.
The wherein above-mentioned render process that is used for carrying out to above-mentioned display unit 3D gaming display image is gone round and begun again, and comprises until the device that withdraws from recreation:
Install, be used to empty the current content of display unit;
Device is used to notify this display unit to begin to play up;
Device is used on this display unit, playing up said image;
Device is used to notify this display unit render process to finish;
Above-mentioned display unit is used to show this image.
Be used to wherein show that the display unit of this image is to inject the default behavior of revising the DirectX call function through server end client layer code; Make it obtain the current image date of this display unit; And it is copied in the internal memory, to accomplish obtaining of 3D gaming display image.
The view data that wherein said server end application layer display unit is obtained is the continuous images data, uses video compression algorithm that the said view data of obtaining is compressed.
Wherein said input equipment comprises: mouse.
Wherein said input equipment comprises: keyboard.
Wherein the server end virtual input device is to inject with the DirectInput interface function of revising DirectX through server end client layer code to realize that the 3D recreation receives the input of input equipment in real time, thus operation 3D recreation; Wherein said DirectInput interface function is that the DirectX recreation is used to collect the interface function of operation input, and it is through the data of DirectInputDevice object accesses input equipment, and said DirectInputDevice object is represented input equipment.
Wherein said DirectInputDevice object is a keyboard, and then DirectInputDevice (keyboard) comprises as follows:
Device is used to create a keyboard equipment;
Device is used for said keyboard equipment is set at the array return key disc status value through one 256 byte;
Device is used to set collaboration mode;
Device is used to activate the access rights to this keyboard equipment;
Install, be used to obtain the operation information of this keyboard equipment,
Wherein, operation information is obtained in the device circulation that is used to obtain the operation information of this keyboard equipment, makes the 3D recreation receive the keyboard equipment input in real time, thereby accomplishes control.
Through one of any operation information that obtains keyboard equipment of following dual mode:
Direct mode: the state that promptly obtains the current time keyboard equipment; Keyboard equipment is regarded as equipment with 256 different keys; The state of each button comprises down state or non-down state; And the state of each button is all usually represented by a unit of one 256 byte arrays of correspondence; The non-down state of general 0 expression; 0x80 representes down state; Inject the behavior of revising the corresponding interface function of keyboard equipment through the client layer code; Making it is not the input that obtains the local physical keyboard of server end, but the key-press status behavior of client is returned;
Buffering method: return the last continuous situation of change that obtains the input keyboard equipment afterwards of keyboard equipment; Inject the behavior of revising the corresponding interface function of keyboard equipment through server end client layer code; Making it is not the input that obtains the local physical keyboard of server end, but the key-press status behavior of client is returned.
Wherein said client layer code be infused in that interface function that the 3D games use calls and the actual function that realizes between insert code, to intercept and capture the call parameters information of program.
It is a kind of transparent code insertion technology that wherein said client layer code injects, and it carries out the function parcel after application program is loaded into internal memory again, directly in internal memory, rewrites the code of correlation function, and does not change the size of execute file.
According to third aspect present invention, a kind of server is provided, be used for showing recreation with operation 3D through network, this server is connected with client through network, and said server comprises:
Server end application layer display unit is obtained the view data based on the 3D gaming display image of DirectX of server end in real time;
Server end video compression device, the view data that the view data of the 3D gaming display image that obtains is compressed and will be compressed is sent to client;
The server end virtual input device receives the operational order that client returns in real time, is converted into the obtainable operation information of DirectX games,
The operational order of wherein said client passback is view data and the decoding and displaying that client receives above-mentioned compression, and gives this server end virtual input device through input equipment reception user's operational order and passback.
Wherein inject the default behavior of revising the DirectX call function, make its view data of obtaining 3D gaming display image, and it is copied in the internal memory, to accomplish obtaining of 3D gaming display image through server end client layer code
Wherein this server further comprises DirectX display device interfaces and DirectInput equipment interface.
Wherein the server end virtual input device is to inject with the DirectInput interface function of revising DirectX through server end client layer code to realize that the 3D recreation receives the input of input equipment in real time, thus operation 3D recreation; Wherein said DirectInput interface function is that the DirectX recreation is used to collect the interface function of operation input, and it is through the data of DirectInputDevice object accesses input equipment, and said DirectInputDevice object is represented input equipment.
Wherein said client layer code be infused in that interface function that the 3D games use calls and the actual function that realizes between insert code, to intercept and capture the call parameters information of program.
It is a kind of transparent code insertion technology that wherein said client layer code injects, and it carries out the function parcel after application program is loaded into internal memory again, directly in internal memory, rewrites the code of correlation function, and does not change the size of execute file.
According to a forth aspect of the invention, a kind of client is provided, is used for showing the recreation with operation 3D through network, this client is connected through the server of network with operation 3D as claimed in claim 25 recreation, and said client comprises:
Client video playback and user operate the passback device, and reception server sends the 3D game image data of the compression of client to, and decoding and displaying, and server is given in the operational order and the passback that receive the user through input equipment.
The beneficial effect that technical scheme of the present invention is brought:
The first, can be on the client device of not supporting large-scale 3D recreation (except portable set, also can comprise PC and notebook computer that the video card ability is more weak) support this type of recreation;
The second, can on the client device of originally not supporting existing network game, (especially portable set) support network game to control, expanded audient's scope greatly;
The 3rd, the issue and the operation platform of unified all kinds of PC recreation are provided, the user can be with on time/in due order/monthly payment charging etc. flexibly form on various kinds of equipment, control various types of games.
Further; The game mode that proposes meets " software is service mode (Software as Service) " this issue of following main flow software and use pattern; And be beneficial to protection Games Software copyright---adopt the server centered operational mode; Client need not installed recreation; Can help to prevent copy right piracy, help the enforcement of the intellectual property protection policy of China.
Four, description of drawings:
Fig. 1 shows the block schematic diagram of system according to the invention;
Fig. 2 shows the method flow diagram according to the inventive method;
Fig. 3 shows according to the 3D rendering of the inventive method and plays up flow chart;
Fig. 4 shows the flow chart according to the client passback keyboard equipment operation information of the inventive method.
Fig. 5 shows the long-range demonstration and the overall operation flow chart of controlling the 3D recreation according to the inventive method.
Five, embodiment:
Such as shown in Figure 1, the core of system of the present invention comprises following four parts: server end application layer display unit and virtual input device, and server end video compression device, client video playback and user operate the passback device.Equally, as shown in Figure 2, also comprise four steps, i.e. step 1 according to the core of the inventive method: server end application layer display unit is obtained the view data based on the 3D gaming display image of DirectX of server end in real time; Step 2: the view data that server compresses the view data of the 3D gaming display image that obtains and will compress is sent to client; Step 3: client receives the view data and the decoding and displaying of above-mentioned compression, receives user's operational order and the server end virtual input device that server end is given in passback through input equipment; Step 4: the server end virtual input device receives the operational order of client passback in real time, is converted into the obtainable operation information of DirectX games.Next, specifically describe concrete implementation of the present invention with reference to figure 1-4:
1) server end application layer display unit is obtained the view data based on the 3D gaming display image of DirectX of server end in real time.
The gaming display picture of the acquisition server end that server end application layer display unit is real-time (under the prerequisite of not revising game identification code).We adopt the virtualized method of application layer to implement.
General, the 3D development of games depends on the multimedia DLL (dynamic link library) that system provides, and the main flow development interface under the windows platform is exactly the DirectX and the OPENGL of Microsoft at present.But DirectX occupies absolute predominance at present---most Windows3D recreation of exploitation later in 2002 all are DirectX.Therefore this embodiment mainly is to be directed against DirectX, but also can be used for OPENGL.
The game image of DirectX generates with procedure for displaying following:
Step 1) initialization of DirectX running environment;
Step 2) games create the server end application layer display unit of a DirectX, are example (down with) with DirectX 9, and relevant establishment equipment calls is---
HRESULT?IDirect3D9::CreateDevice(UINT?adapter，D3DDEVTYPE?deviceType，HWND?focusWindow，DWORDbehaviourFlags，D3DPRESENT_PARAMETERS*presentationParameters，IDirect3DDevice9**device)；
Step 3) be directed against the render process that this equipment begins 3D rendering, go round and begin again, withdraw from until recreation.
Step 3.1) empty the current content of display device;
Step 3.2) notify this equipment to begin to play up;
Step 3.3) rendering image on this equipment;
Step 3.4) notify this equipment render process to finish;
Step 3.5) show this image;
Key step related to the present invention is Step 3.5, and this is to call through Direct X:
HRESULT IDirect3DDevice9::SCPresent (IDirect3DSwapChain9*pSwapChain, const RECT*src, const RECT*dest, HWND hWnd, LPVOID unused, DWORD dwFlags) realizes.
The present invention injects through client layer code method for implanting (concrete code method for implanting " 5) client layer code " joint sets forth); Revised the default behavior of this function; Make it can obtain the current image date of this display device; And it is copied in the internal memory, concrete implementation method is following:
● obtain this display device, the DirectX of use calls and is
IDirect3DSwapChain9::GetDevice(IDirect3DDevice9**ppDevice)；
● obtain the attribute of this equipment epigraph, comprise size etc.The DirectX that uses calls and is
IDirect3DDevice9::GetBackBuffer(UINT?iSwapChain，UINT?BackBuffer，D3DBACKBUFFER_TYPEType，IDirect3DSurface9**ppBackBuffer)；
With IDirect3DSurface9::GetDesc (D3DSURFACE_DESC*pDesc);
● create a new Surface then, the view data of this equipment is copied on this surface, and pin this Surface, and view data is copied in the internal memory;
IDirect3DDevice9::CreateOffscreenPlainSurface (UINT Width, UINT Height, D3DFORMAT Format, D3DPOOL Pool, IDirect3DSurface9**ppSurface, HANDLE*pSharedHandle); New Surface of // establishment
IDirect3DDevice9::GetRenderTargetData (IDirect3DSurface9*pRenderTarget, IDirect3DSurface9*pDestSurface); // view data of this equipment is copied on this surface
IDirect3DSurface9::LockRect (D3DLOCKED_RECT*pLockedRect, const RECT*pRect, DWORD Flags); This Surface of // pinning
One of return value of LockRect, pLockedRect is the pointer of picture material in the sensing display device, it can be copied to internal memory, thereby accomplish the acquisition process of this gaming display picture.
2) server view data that the view data of the 3D gaming display image that obtains is compressed and will be compressed is sent to client
Through 1) view data that obtains is the BMP form, the present invention is sent to client through it is compressed with the video data after the compression, accomplishes to show.Because the intercepting and capturing of view data are carried out continuously 1), the continuous images data have therefore in fact just been obtained.
Concrete video compression algorithm can adopt MPEG4 or H264 (but being not limited to), and this is mature technology on the one hand.
3) client receives the view data and the decoding and displaying of above-mentioned compression
Client software receives decoding and displaying behind the video data, and its operation principle is identical with general video jukebox software.Difference is that client software also must be accepted the operation of user for game video, and it is passed to the virtual unit layer of server end.This generic operation is divided into two classifications:
● keyboard: mainly be the button operation that receives the user, send it to server end.Promptly if client intercept and capture this locality certain button state variation (such as from normal condition to being pressed, perhaps discharge from pressing to) just it need be passed to server end;
● pointing device, like mouse: receive user's move operation and button operation, send server end to.Mobile message comprises the displacement of pointer at level and vertical direction.
4) the server end virtual input device receives the operational order of client passback in real time, is converted into the obtainable operation information of DirectX games
After receiving the operation back information of client, server end need be converted into the obtainable operation information of DirectX games.Here mainly through client layer code method for implanting; Having revised the interface that the behavior of DirectInput interface among the DirectX-DirectX recreation is used to receive the operation input is the DirectInput interface, and it is through the data of DirectInputDevice object accesses each input equipment.A DirectInputDevice object is represented an input equipment, like mouse, keyboard or joystick.With the keyboard is example, and the present invention has revised the certain methods of DirectInputDevice object, reaches the purpose that the information of controlling that client is sent passes to recreation.
The workflow of DirectInputDevice (keyboard) is following:
Step 1) keyboard equipment of establishment;
Step 2) be set the array return key disc status value that is through one 256 byte;
Step 3) the setting collaboration mode;
Step 4) activate access rights to equipment;
Step 5) obtain operation of equipment information;
Step 5) be a process that circulation is obtained, make the receiving equipment that recreation can be real-time import, thereby accomplish control.Key step related to the present invention also is this step.
Can come the input of acquisition equipment through dual mode---
● direct mode: the state that promptly obtains the current time keyboard.Keyboard is regarded as the equipment with 256 different keys here, the state of each button (" pressing " be " non-pressing " perhaps) is all usually represented by each unit of one 256 byte arrays of correspondence, general 0 expression " non-pressing ", and 0x80 representes " pressing ".The corresponding interface function is IDirectInputDevice8::GetDeviceState (DWORD cbData, LPVOID lpvData).The present invention injects the behavior of having revised this function through code, and making it is not the input that obtains the local physical keyboard of server end, but the key-press status behavior of client is returned.
● buffering method: after returning the input of last acquisition keyboard, the continuous situation of change of keyboard.As, obtaining for twice between the calling of keyboard input, the user presses space bar, the space bar of upspringing, press " x " key, " x " key of upspringing again.So through " direct mode ", what return is the state of keyboard behind " x " key of upspringing, and also is " x " and the pressing/upspring this process and can't return of space bar; And under this pattern, then can return the sequence of operations state, what comprise above-mentioned " x " and space bar presses/upsprings process.
The present invention injects the behavior of having revised this function through code, and making it is not the input that obtains the local physical keyboard of server end, but the key-press status behavior of client is returned.
5) the client layer code injects
The essence that the client layer code injects is exactly between interface interchange that games are used and the actual function that realizes, to insert code, so that the information such as call parameters of the program of intercepting and capturing.
The present invention has adopted transparent code insertion technology, and it carries out the function parcel after application program is loaded into internal memory again, directly in internal memory, rewrites the code of correlation function, can not change the size of execute file like this, and effect is the same.This technology can need not to revise source code the user, need not to recompilate to connect, and reaches and realizes required function under the prerequisite of not replacing any dynamic link libraries.
Concrete method is following---and figure below has shown the composition of execute file under the Windows; Comprise file header; The PE head; Text segment comprises program code; Data segment comprises the input chained list of initialization data, used dynamic link libraries and system call, output (Exported) function table and debug symbol.
For revising the Windows run time version, the present invention has created a new section between debug symbol and output function table---and contain copy and a new function input table of original PE head, revised original PE head simultaneously and made it to point to new input table.Be the code that inserts at last.
Insert new input table two purposes are arranged: at first it has kept original input table so that the execute file that will revise is changeed back when needed.Secondly, new input table can comprise that new dynamic link libraries is in order to accomplish required additional functionality.When target program called the relevant interface of DirectX, in fact inquiry was the new function input table that inserts, thereby has called the insertion code, thereby has changed the default behavior of relevant interface like this.
The concrete function that is wrapped is:
●IDirect3DDevice9::SCPresent。Obtain the current image date of this display device, and it is copied in the internal memory;
●IDirectInputDevice8::GetDeviceState，IDirectInputDevice8::GetDeviceData。The key-press status behavior of client is returned.
Specific embodiment and performance test
The inventor has installed Direct X 3D recreation on a Windows XP PC server, comprise Need For Speed, crysis etc., and what use at present is Direct X 9.0C interface.
Client is then used a windows XP notebook, interconnects with 100 m ethernet between the two.
At server of the present invention of server operation, it mainly carries out following function, can accept the connection of client simultaneously, starts local recreation of installing, injecting codes simultaneously according to request; The function of injecting codes is as the above, promptly accomplishes the obtaining in real time of 3D gaming display picture, Compression and Transmission, and the back delivery operations of client is passed to recreation through virtual input device.Overall operational process as shown in Figure 5.
With above-mentioned RDP, RemoteFX compares, and characteristics of the present invention are:
1. support the long-range demonstration and the control of 3D recreation;
2. the application layer at server end realizes;
3. lower to the requirement of server end software and hardware;
4. the server side operation system visit of supporting is wider, comprises operating systems such as Windows XP/VISTA/2007/Server.
Contrast as shown in the table:
The beneficial effect that technical scheme of the present invention is brought:
The first, can be on the client device of not supporting large-scale 3D recreation (except portable set, also can comprise PC and notebook computer that the video card ability is more weak) support this type of recreation;
The second, can on the client device of originally not supporting existing network game, (especially portable set) support network game to control, expanded audient's scope greatly;
The 3rd, the issue and the operation platform of unified all kinds of PC recreation are provided, the user can be with on time/in due order/monthly payment charging etc. flexibly form on various kinds of equipment, control various types of games.
Further; The game mode that proposes meets " software is service mode (Software as Service) " this issue of following main flow software and use pattern; And be beneficial to protection Games Software copyright---adopt the server centered operational mode; Client need not installed recreation; Can help to prevent copy right piracy, help the enforcement of the intellectual property protection policy of China.
Those skilled in the art should understand that above description of the invention only is used to illustrate some embodiments of the present invention with diagram, rather than is used for limiting scope of the present invention.More than description and illustrated various modification should comprise within the scope of the invention.
Claims (31)
1. one kind is passed through the method that network shows and operation 3D plays, and this method may further comprise the steps:
Step 1: server end application layer display unit is obtained the view data based on the 3D gaming display image of DirectX of server end in real time;
Step 2: the view data that server compresses the view data of the 3D gaming display image that obtains and will compress is sent to client;
Step 3: client receives the view data and the decoding and displaying of above-mentioned compression, receives user's operational order and the server end virtual input device that server end is given in passback through input equipment;
Step 4: the server end virtual input device receives the operational order of client passback in real time, is converted into the obtainable operation information of DirectX games.
2. the method for claim 1, generation and demonstration based on the view data of the 3D gaming display image of DirectX in the wherein said step 1 comprise the steps:
Step 1.1) DirectX running environment initialization;
Step 1.2) the 3D games are created the server end application layer display unit of a DirectX;
Step 1.3) is directed against the render process that above-mentioned display unit is carried out 3D gaming display image, goes round and begins again, until withdrawing from recreation.
3. method as claimed in claim 2, wherein above-mentioned steps 1.3) comprising:
Step 1.3.1) empties the current content of above-mentioned display unit;
Step 1.3.2) notify this display unit to begin to play up;
Step 1.3.3) on this display unit, plays up said image;
Step 1.3.4) notify this display unit render process to finish;
Step 1.3.5) shows this image.
4. be to inject the default behavior of revising the DirectX call function the wherein said step 1.3.5 of method as claimed in claim 3) through server end client layer code; Make it obtain the current image date of this display unit; And it is copied in the internal memory, to accomplish obtaining of 3D gaming display image.
5. the method for claim 1, the view data of obtaining in the wherein said step 1 is the continuous images data, uses video compression algorithm that the said view data of obtaining is compressed.
6. the method for claim 1, wherein the said input equipment in the step 3 comprises: mouse.
7. the method for claim 1, wherein the said input equipment in the step 3 comprises: keyboard.
8. the method for claim 1, wherein step 4 is to inject with the DirectInput interface function of revising DirectX through server end client layer code to realize that the 3D recreation receives the input of input equipment in real time, thus operation 3D recreation; Wherein said DirectInput interface function is that the DirectX recreation is used to collect the interface function of operation input, and it is through the data of DirectInputDevice object accesses input equipment, and said DirectInputDevice object is represented input equipment.
9. method as claimed in claim 8, wherein said DirectInputDevice object is a keyboard, then the workflow of DirectInputDevice (keyboard) is following:
Step 4.1) creates a keyboard equipment;
Step 4.2) being set is the array return key disc status value of passing through one 256 byte;
Step 4.3) sets collaboration mode;
Step 4.4) activation is to the access rights of this keyboard equipment;
Step 4.5) obtain the operation information of this keyboard equipment,
Wherein, step 4.5) be a process that circulation is obtained, make the 3D recreation receive the keyboard equipment input in real time, thereby accomplish control.
10. method as claimed in claim 9 is through one of any obtaining step 4.5 of following dual mode) in the operation information of keyboard equipment:
Direct mode: the state that promptly obtains the current time keyboard equipment; Keyboard equipment is regarded as equipment with 256 different keys; The state of each button comprises down state or non-down state; And the state of each button is all usually represented by a unit of one 256 byte arrays of correspondence; The non-down state of 0 expression; 0x80 representes down state; Inject the behavior of revising the corresponding interface function of keyboard equipment through server end client layer code; Making it is not the input that obtains the local physical keyboard of server end, but the key-press status behavior of client is returned;
Buffering method: return the last continuous situation of change that obtains the input keyboard equipment afterwards of keyboard equipment; Inject the behavior of revising the corresponding interface function of keyboard equipment through server end client layer code; Making it is not the input that obtains the local physical keyboard of service end, but the key-press status behavior of client is returned.
11. like claim 4,8 or 10 arbitrary described methods, wherein said client layer code be infused in that interface function that the 3D games use calls and the actual function that realizes between insert code, to intercept and capture the call parameters information of program.
12. like claim 4,8 or 10 arbitrary described methods; It is that a kind of transparent code inserts that wherein said client layer code injects; It carries out the function parcel after application program is loaded into internal memory again, directly in internal memory, rewrites the code of correlation function, and does not change the size of execute file.
13. a system that shows and operate the 3D recreation through network, this system comprises:
Server end application layer display unit is obtained the view data based on the 3D gaming display image of DirectX of server end in real time;
Server end video compression device, the view data that the view data of the 3D gaming display image that obtains is compressed and will be compressed is sent to client;
Client video playback and user operate the passback device, receive the view data and the decoding and displaying of above-mentioned compression, and the server end virtual input device is given in the operational order and the passback that receive the user through input equipment;
The server end virtual input device receives the operational order that client returns in real time, is converted into the obtainable operation information of DirectX games.
14. system as claimed in claim 13, wherein said server end application layer display unit use and comprise like lower device and generate and show the view data based on the 3D gaming display image of DirectX:
Device is used for the initialization of DirectX running environment;
Device is used for the server end application layer display unit that the 3D games are created a DirectX;
Device is used for being directed against the render process that above-mentioned display unit is carried out 3D gaming display image, goes round and begins again, until withdrawing from recreation.
15. system as claimed in claim 14, the wherein above-mentioned render process that is used for carrying out to above-mentioned display unit 3D gaming display image is gone round and begun again, and comprises until the device that withdraws from recreation:
Install, be used to empty the current content of above-mentioned display unit;
Device is used to notify this display unit to begin to play up;
Device is used on this display unit, playing up said image;
Device is used to notify this display unit render process to finish;
Above-mentioned 3D display unit is used to show this image.
16. system as claimed in claim 15 is used to wherein show that the display unit of this image is to inject the default behavior of revising the DirectX call function through server end client layer code; Make it obtain the current image date of this display unit; And it is copied in the internal memory, to accomplish obtaining of 3D gaming display image.
17. system as claimed in claim 13, the view data that wherein said server end application layer display unit is obtained is the continuous images data, uses video compression algorithm that the said view data of obtaining is compressed.
18. system as claimed in claim 13, wherein said input equipment comprises: mouse.
19. system as claimed in claim 13, wherein said input equipment comprises: keyboard.
20. system as claimed in claim 19; Wherein the server end virtual input device is to inject with the DirectInput interface function of revising DirectX through server end client layer code to realize that the 3D recreation receives the input of input equipment in real time, thus operation 3D recreation; Wherein said DirectInput interface function is that the DirectX recreation is used to collect the interface function of operation input, and it is through the data of DirectInputDevice object accesses input equipment, and said DirectInputDevice object is represented input equipment.
21. system as claimed in claim 20, wherein said DirectInputDevice object is a keyboard, and then DirectInputDevice (keyboard) comprises as follows:
Device is used to create a keyboard equipment;
Device is used for said keyboard equipment is set at the array return key disc status value through one 256 byte;
Device is used to set collaboration mode;
Device is used to activate the access rights to this keyboard equipment;
Install, be used to obtain the operation information of this keyboard equipment,
Wherein, operation information is obtained in the device circulation that is used to obtain the operation information of this keyboard equipment, makes the 3D recreation receive the keyboard equipment input in real time, thereby accomplishes control.
22. system as claimed in claim 21, through one of any operation information that obtains keyboard equipment of following dual mode:
Direct mode: the state that promptly obtains the current time keyboard equipment; Keyboard equipment is regarded as equipment with 256 different keys; The state of each button comprises down state or non-down state; And the state of each button is all usually represented by a unit of one 256 byte arrays of correspondence; The non-down state of general 0 expression; 0x80 representes down state; Inject the behavior of revising the corresponding interface function of keyboard equipment through the client layer code; Making it is not the input that obtains the local physical keyboard of server end, but the key-press status behavior of client is returned;
Buffering method: return the last continuous situation of change that obtains the input keyboard equipment afterwards of keyboard equipment; Inject the behavior of revising the corresponding interface function of keyboard equipment through server end client layer code; Making it is not the input that obtains the local physical keyboard of service end, but the key-press status behavior of client is returned.
23. like claim 14,20 or 22 arbitrary described systems, wherein said client layer code be infused in that interface function that the 3D games use calls and the actual function that realizes between insert code, to intercept and capture the call parameters information of program.
24. like claim 14,20 or 22 arbitrary described systems; It is a kind of transparent code insertion technology that wherein said client layer code injects; It carries out the function parcel after application program is loaded into internal memory again, directly in internal memory, rewrites the code of correlation function, and does not change the size of execute file.
25. a server is used for showing the recreation with operation 3D through network that this server is connected with client through network, said server comprises:
Server end application layer display unit is obtained the view data based on the 3D gaming display image of DirectX of server end in real time;
Server end video compression device, the view data that the view data of the 3D gaming display image that obtains is compressed and will be compressed is sent to client;
The server end virtual input device receives the operational order that client returns in real time, is converted into the obtainable operation information of DirectX games,
The operational order of wherein said client passback is view data and the decoding and displaying that client receives above-mentioned compression, and gives this server end virtual input device through input equipment reception user's operational order and passback.
26. server as claimed in claim 25; Wherein inject the default behavior of revising the DirectX call function through server end client layer code; Make its view data of obtaining 3D gaming display image, and it is copied in the internal memory, to accomplish obtaining of 3D gaming display image
27. server as claimed in claim 25, wherein this server further comprises DirectX display device interfaces and DirectInput equipment interface.
28. server as claimed in claim 27; Wherein the server end virtual input device is to inject with the DirectInput interface function of revising DirectX through server end client layer code to realize that the 3D recreation receives the input of input equipment in real time, thus operation 3D recreation; Wherein said DirectInput interface function is that the DirectX recreation is used to collect the interface function of operation input, and it is through the data of DirectInputDevice object accesses input equipment, and said DirectInputDevice object is represented input equipment.
29. like claim 26 or 28 described servers, wherein said client layer code be infused in that interface function that the 3D games use calls and the actual function that realizes between insert code, to intercept and capture the call parameters information of program.
30. like claim 26 or 28 described servers; It is a kind of transparent code insertion technology that wherein said client layer code injects; It carries out the function parcel after application program is loaded into internal memory again, directly in internal memory, rewrites the code of correlation function, and does not change the size of execute file.
31. a client is used for showing the recreation with operation 3D through network that this client is connected through the server of network with operation 3D as claimed in claim 25 recreation, said client comprises:
Client video playback and user operate the passback device, and reception server sends the 3D game image data of the compression of client to, and decoding and displaying, and server is given in the operational order and the passback that receive the user through input equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110235071.6A CN102355489B (en) | 2011-08-17 | 2011-08-17 | Method, system, server and client side for displaying and operating 3D (Three-Dimensional) game |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201110235071.6A CN102355489B (en) | 2011-08-17 | 2011-08-17 | Method, system, server and client side for displaying and operating 3D (Three-Dimensional) game |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102355489A true CN102355489A (en) | 2012-02-15 |
| CN102355489B CN102355489B (en) | 2015-03-04 |
Family
ID=45578983
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201110235071.6A Expired - Fee Related CN102355489B (en) | 2011-08-17 | 2011-08-17 | Method, system, server and client side for displaying and operating 3D (Three-Dimensional) game |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102355489B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103513862A (en) * | 2012-06-29 | 2014-01-15 | 联想(北京)有限公司 | Method, local device and remote device for processing information |
| CN103577661A (en) * | 2012-07-19 | 2014-02-12 | 中兴通讯股份有限公司 | Method and device for obtaining information |
| CN104768023A (en) * | 2014-01-02 | 2015-07-08 | 优必达公司 | System and method for delivering graphics over network |
| CN106375343A (en) * | 2016-10-31 | 2017-02-01 | 上海斐讯数据通信技术有限公司 | Client, server, interactive communication system and communication ways |
| CN107174825A (en) * | 2017-04-28 | 2017-09-19 | 苏州蜗牛数字科技股份有限公司 | A kind of remote image method for reconstructing and system based on model |
| CN107609065A (en) * | 2017-08-30 | 2018-01-19 | 武汉斗鱼网络科技有限公司 | One kind game classification method, apparatus and electronic equipment |
| CN109104626A (en) * | 2017-06-20 | 2018-12-28 | 武汉斗鱼网络科技有限公司 | The method and apparatus of image is filtered in a kind of network direct broadcasting |
| CN110314374A (en) * | 2018-03-30 | 2019-10-11 | 武汉斗鱼网络科技有限公司 | A kind of game processing method, device, electronic equipment and medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109218402A (en) * | 2018-08-02 | 2019-01-15 | 胡雄风 | Hold cloud integrative solution |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1743043A (en) * | 2005-06-19 | 2006-03-08 | 珠海市西山居软件有限公司 | Network game system and method for realizing same |
| US20070087829A1 (en) * | 2005-10-14 | 2007-04-19 | Derek Liu | Multi-player game architecture |
| CN102054121A (en) * | 2009-11-04 | 2011-05-11 | 沈阳隆惠科技有限公司 | Method for building 3D (three-dimensional) panoramic live-action online game platform |
-
2011
- 2011-08-17 CN CN201110235071.6A patent/CN102355489B/en not_active Expired - Fee Related
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1743043A (en) * | 2005-06-19 | 2006-03-08 | 珠海市西山居软件有限公司 | Network game system and method for realizing same |
| US20070087829A1 (en) * | 2005-10-14 | 2007-04-19 | Derek Liu | Multi-player game architecture |
| CN102054121A (en) * | 2009-11-04 | 2011-05-11 | 沈阳隆惠科技有限公司 | Method for building 3D (three-dimensional) panoramic live-action online game platform |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103513862A (en) * | 2012-06-29 | 2014-01-15 | 联想(北京)有限公司 | Method, local device and remote device for processing information |
| CN103513862B (en) * | 2012-06-29 | 2016-08-17 | 联想(北京)有限公司 | Information processing method, local device and remote equipment |
| CN103577661A (en) * | 2012-07-19 | 2014-02-12 | 中兴通讯股份有限公司 | Method and device for obtaining information |
| CN104768023A (en) * | 2014-01-02 | 2015-07-08 | 优必达公司 | System and method for delivering graphics over network |
| CN104768023B (en) * | 2014-01-02 | 2018-03-23 | 优必达公司 | The system and method for media is transmitted by network |
| CN106375343A (en) * | 2016-10-31 | 2017-02-01 | 上海斐讯数据通信技术有限公司 | Client, server, interactive communication system and communication ways |
| CN107174825A (en) * | 2017-04-28 | 2017-09-19 | 苏州蜗牛数字科技股份有限公司 | A kind of remote image method for reconstructing and system based on model |
| CN109104626A (en) * | 2017-06-20 | 2018-12-28 | 武汉斗鱼网络科技有限公司 | The method and apparatus of image is filtered in a kind of network direct broadcasting |
| CN109104626B (en) * | 2017-06-20 | 2021-11-09 | 武汉斗鱼网络科技有限公司 | Method and equipment for filtering images in live webcast |
| CN107609065A (en) * | 2017-08-30 | 2018-01-19 | 武汉斗鱼网络科技有限公司 | One kind game classification method, apparatus and electronic equipment |
| CN110314374A (en) * | 2018-03-30 | 2019-10-11 | 武汉斗鱼网络科技有限公司 | A kind of game processing method, device, electronic equipment and medium |
| CN110314374B (en) * | 2018-03-30 | 2024-02-02 | 广州天梯网络科技有限公司 | Game processing method, game processing device, electronic equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102355489B (en) | 2015-03-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102355489A (en) | Method, system, server and client side for displaying and operating 3D (Three-Dimensional) game | |
| US7830388B1 (en) | Methods and apparatus of sharing graphics data of multiple instances of interactive application | |
| Boos et al. | Flashback: Immersive virtual reality on mobile devices via rendering memoization | |
| Humphreys et al. | Distributed rendering for scalable displays | |
| CN102378976B (en) | The compression of images of multiple processor is used to accelerate | |
| CN102411786A (en) | Low complexity method for motion compensation of dwt based systems | |
| CN102707986A (en) | Shared memory between child and parent partitions | |
| US20130210522A1 (en) | Data center architecture for remote graphics rendering | |
| CN102447901B (en) | Adaptive grid generation for improved caching and image classification | |
| EP2654911A2 (en) | Load balancing between general purpose processors and graphics processors | |
| EP4181068A1 (en) | A method and system for interactive graphics streaming | |
| CN102254292A (en) | Remote 3D instruction rendering system and method | |
| CN106797398B (en) | For providing the method and system of virtual desktop serve to client | |
| CN108762934B (en) | Remote graphic transmission system and method and cloud server | |
| CN110968395B (en) | Method for processing rendering instruction in simulator and mobile terminal | |
| CN112316433B (en) | Game picture rendering method, device, server and storage medium | |
| WO2024037110A1 (en) | Data processing method and apparatus, device, and medium | |
| CN108073350B (en) | Object storage system and method for cloud rendering | |
| TW201407536A (en) | Server based graphics processing techniques | |
| CN113209632A (en) | Cloud game processing method, device, equipment and storage medium | |
| Eisert et al. | Low delay streaming of computer graphics | |
| Shi et al. | SHARC: A scalable 3D graphics virtual appliance delivery framework in cloud | |
| Eilemann et al. | Equalizer 2.0–convergence of a parallel rendering framework | |
| EP4143688A1 (en) | System and method for efficient multi-gpu execution of kernels by region based dependencies | |
| Park et al. | Mobile collaborative medical display system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20150304 Termination date: 20210817 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |