WO2011072049A2 - Systèmes et procédés destinés à une présentation distante côté client d'un flux multimédia - Google Patents
Systèmes et procédés destinés à une présentation distante côté client d'un flux multimédia Download PDFInfo
- Publication number
- WO2011072049A2 WO2011072049A2 PCT/US2010/059526 US2010059526W WO2011072049A2 WO 2011072049 A2 WO2011072049 A2 WO 2011072049A2 US 2010059526 W US2010059526 W US 2010059526W WO 2011072049 A2 WO2011072049 A2 WO 2011072049A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- computing device
- multimedia data
- multimedia
- application
- executing
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
- H04L65/612—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for unicast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/765—Media network packet handling intermediate
Definitions
- This application generally relates to remotely providing data.
- this application relates to remotely providing multimedia data generated by a multimedia device.
- the methods and systems described herein are embodiments of methods and systems for remotely providing multimedia data.
- a user of a client computer connected to a multimedia device may wish to use an application remotely executing on a remote server, to display or otherwise modify output generated by the multimedia device.
- Access to the application in many instances, can be provided via a thin client protocol.
- the application can receive the multimedia output from the client computer via a virtual channel over which the client computer can transmit the multimedia output using the thin client protocol. Transmitting multimedia data from one computer to another can be taxing on both systems because multimedia data can require a great deal of storage resources, computing (e.g. CPU) resources, memory resources and bandwidth resources.
- a method for remotely presenting multimedia data generated by a multimedia device connected to a second computing machine includes streaming multimedia data from a virtual channel established between a first computing machine and a second computing machine, where the multimedia data is generated by a multimedia device that communicates with a second computing machine.
- the multimedia data is streamed by a proxy streaming module that executes on the first computing machine and that forwards the multimedia data to a presentation application executing on the first computing machine.
- a first remoting application that executes on the first computing machine can capture the application output generated by the presentation application during the presentation of the multimedia data.
- the first remoting application can also transmit the application output to the second computing machine which can then display the application output in an application output window.
- the proxy streaming module can stream the multimedia data in response to receiving a request from a user of the second computing machine to execute the presentation application.
- a second remoting application that executes on the second computing machine can transmit the multimedia data from the second computing machine to the first computing machine over the virtual channel.
- a streaming module executing on the second computing machine streams the multimedia data from a multimedia device interface, and then streams the multimedia data to the remoting application executing on the second computing machine.
- Other embodiments include an encoder that executes on the second computing machine to encode the multimedia data before transmitting the multimedia data to the first computing machine. Still other embodiments include an encryption agent that executes on the second computing machine to encrypt the multimedia data before transmitting the multimedia data to the first computing machine.
- the remoting application in some embodiments, encapsulates the multimedia data in a protocol wrapper before transmitting the multimedia data to the first computing machine.
- a decoder executing on the first computing machine can decode the multimedia data before forwarding the multimedia data to the presentation application.
- the decoder in some embodiments, can include a decryption agent.
- the remoting application can un-encapsulate the multimedia data before forwarding the multimedia data to the presentation application.
- the presentation application can, in some embodiments, store the multimedia data in a storage repository.
- a system for remotely presenting multimedia data generated by a multimedia device connected to a second computing machine can include a first computing machine and a second computing machine communicating with the first computing machine over a virtual channel established between the first computing machine and the second computing machine.
- the system can further include a multimedia device communicating with the second computing machine, and a proxy streaming module executing on the first computing machine to stream multimedia data from the virtual channel.
- the multimedia data in some embodiments, is generated by the multimedia device, and forwarded by the proxy streaming module to a presentation application executing on the first computing machine.
- a remoting application executing on the first computing machine can capture application output generated by the presentation application during presentation of the multimedia data, and transmit the application output to the second computing machine.
- the second computing machine can display the application output in an application output window.
- the second computing machine can directly, or locally, display the multimedia data captured by the local multimedia device in an application output window.
- the present application relates to a method for displaying on a local computing device, by an application executing on a remote computing device, multimedia data generated by a multimedia device connected to the local computing device.
- the remote computing device may be in communication with the local computing device via a virtual channel.
- a proxy streaming module executing on a remote computing device may stream multimedia data received from a local computing device via a virtual channel.
- the multimedia data may be generated by a multimedia device associated with the local computing device.
- the proxy streaming module may forward the multimedia data to a presentation application executing on the remote computing device, the presentation application generating multimedia application output.
- a remoting application executing on the remote computing device may capture the generated multimedia application output.
- the remoting application executing on the remote computing device may transmit the generated multimedia application output to the local computing device for display in an application output window on the local computing device.
- multimedia data is streamed responsive to receiving a request to execute the presentation application from a user of the local computing device.
- a local streaming module executing on the local computing device transmits the multimedia data to the remote computing device via the virtual channel.
- a local streaming module executing on the local computing device streams the multimedia data from a multimedia device interface.
- the local streaming module executing on the local computing device may stream the multimedia data to a local remoting application executing on the local computing device.
- an encoder executing on the local computing device encodes the multimedia data before transmitting the multimedia data to the remote computing device.
- a decoder executing on the remote computing device decodes the multimedia data before forwarding the multimedia data to the presentation application.
- a local remoting application on the local computing device constructs a packet to include the multimedia data in a protocol wrapper before transmitting the packet comprising the multimedia data to the remote computing device.
- the remoting application executing on the remote computing device deconstructs the packet to extract the multimedia data from the packet before forwarding the multimedia data to the presentation application.
- an encoder executing on the local computing device encrypts the multimedia data before transmitting the multimedia data to the remote computing device.
- a decoder executing on the remote computing device decrypts the multimedia data before forwarding the multimedia data to the presentation application.
- the presentation application stores the multimedia data in a storage repository.
- the present disclosure relates to a system for displaying on a local computing device, by an application executing on a remote computing device.
- Multimedia data may be generated by a multimedia device connected to the local computing device.
- the remote computing device may be in communication with the local computing device via a virtual channel.
- the system may include a local computing device.
- the system may further include a multimedia device communicating with the local computing device, the multimedia device generating multimedia data.
- the system may also include a remote computing device hosting the executing of a proxy streaming module to stream multimedia data received from the local computing device via a virtual channel.
- the remote computing device may forward the multimedia data to a presentation application executing on the remote computing device.
- the presentation application may generate multimedia application output.
- the remoting application may capture the generated multimedia application output and transmit the multimedia application output to the local computing device for displaying in an application output window of the local computing device.
- the proxy streaming module streams the multimedia data responsive to receiving a request from a user of the local computing device to execute the presentation application.
- a local remoting application executing on the local computing device transmits the multimedia data to the remote computing device over the virtual channel.
- a streaming module executing on the local computing device streams the multimedia data from a multimedia interface to the remote application executing on the local computing device.
- an encoder executing on the local computing device encodes the multimedia data before transmitting the multimedia data to the remote computing device.
- a decoder executing on the remote computing device decodes the multimedia data before forwarding the multimedia data to the presentation application.
- the local remoting application executing on the local computing device constructs a packet to include the multimedia data in a protocol wrapper before transmitting the packet comprising the multimedia data to the remote computing device.
- the remoting application executing on the remote computing device deconstructs the packet by extracting the multimedia data from the packet before forwarding the multimedia data to the presentation application.
- the presentation application stores the multimedia data in a storage repository.
- the present disclosure relates to a method for displaying on a client device, by an application executing on a remote server, multimedia data generated by a multimedia device connected to the client device.
- One or more filters of the client device may transform multimedia data to conform to a protocol of a virtual channel established between the client device and a remote server.
- a streaming module executing on the client device may forward compressed multimedia data to the remote server via the virtual channel.
- a proxy streaming module executing on the remote server may stream multimedia data received from the client device via the virtual channel.
- the proxy streaming module may forward multimedia data to a presentation application executing on the remote server.
- a remoting application executing on the remote server may capture application output generated by the presentation application during presentation of the multimedia data.
- the remoting application executing on the remote computing device may transmit the application output to the local computing device.
- the client device may display the application output in an application output window.
- the present disclosure relates to methods and systems for displaying on multiple client devices, by one or more applications executing on one or more remote servers, multimedia data generated by a multimedia device connected to each of the remote client devices.
- two or more clients may wish to communicate with each other via audio/video conferencing using their locally connected web cameras by processing the data from these cameras on one or more remote applications executing on one or more remote servers.
- a first multimedia data is generated by a multimedia device connected to a first client computing device.
- Second multimedia data may be generated by a second multimedia device connected to a second client computing device.
- the first and the second client computing devices may each be in communication with a remote server via one or more virtual channels.
- a first proxy streaming module executing on the remote server may stream multimedia data received from a first client computing device via a first virtual channel.
- a second proxy streaming module executing on the remote server may stream multimedia data received from a second client computing device via a second virtual channel.
- a single proxy streaming module streams the multimedia data from both the first and the second client computing devices.
- the one or more proxy streaming modules may forward the multimedia data received from the first and the second client via the one or more virtual channels to a presentation application executing on the remote computing device.
- the presentation application may generate first multimedia application output for the first multimedia data and the second multimedia data application output for the second multimedia data.
- multiple presentation applications generate the first and the second multimedia application outputs.
- a first remoting application executing on the remote computing device may capture the generated first multimedia application output for the first client.
- a second remoting application executing on the remote computing device may capture the generated second multimedia application output for the second client.
- the first remoting application executing on the remote computing device may transmit the generated first multimedia application output to the first client.
- the second remoting application executing on the remote computing device may transmit the generated second multimedia application output to the second client.
- the first client may display in an application output window on the first client the generated first multimedia application output.
- the second client may display in an application output window on the second client the generated second multimedia application output.
- the first and the second clients may each receive and display the first and the second generated multimedia application outputs.
- the first client may receive from the first and the second remoting application both the first and the second generated multimedia application outputs.
- the second client may receive both the first and the second generated multimedia application outputs.
- the first client displays in one or more local application output windows the generated first multimedia application output and the second multimedia application output.
- the second client may also display in one or more local application output windows the generated first multimedia application output and the second multimedia application output.
- the first client and the second client may thus implement audio/video conferencing while processing their audio/video data generated by their locally connected multimedia capture devices on one or more applications executing on one or more remote servers.
- the remote server includes one or more filters or functions enabling the operation of the presentation application with the multimedia data from the remote clients.
- the one or more filters may comprise or utilize any of the proxy streaming modules and remoting applications to ensure that the presentation application treats the multimedia data from each of the remote clients as multimedia data generated by a locally connected multimedia capture device.
- the one or more filters may cause the presentation application to register the one or more filters as a multimedia capture device, such as a web camera, locally connected to the remote client or the server.
- the presentation application which may be designed to operate with locally connected multimedia capture devices only may continue to operate with a multimedia capture device connected to a remote client without any modifications to the presentation application.
- the presentation application may treat and communicate with the one or more filters in a manner in which it communicates with a locally connected multimedia capture device.
- the client may include one or more filters or functions enabling the client to receive the multimedia data generated by a locally connected multimedia capture device and transmit the received multimedia data to the remote server.
- the one or more filters may also enable the client to receive the generated multimedia application output from the remote server and display the received generated multimedia application output locally.
- the one or more filters may satisfy all the requirements of the presentation application executing on the remote server.
- the one or more filters on the client and the server side may enable the client and the server to process, modify and/or edit the multimedia data received from the locally stored multimedia capture device in a manner in which it can be received, processed and transmitted by the presentation application executing on the remote server.
- the present disclosure relates to a method for displaying on a local computing device, by an application executing on a remote computing device, multimedia data generated by a multimedia device connected to the local computing device.
- the remote computing device may be in communication with the local computing device via a virtual channel.
- a local computing device may stream multimedia data to a remote computing device via a virtual channel.
- the multimedia data may be generated by a multimedia device associated with the local computing device and comprising input to a presentation application executing on the remote computing device.
- Application output generated may be received from the remote computing device by the presentation application responsive to the transmitted multimedia data.
- the received application output may include the multimedia data generated by the multimedia device and a second stream of multimedia data generated by a third computing device.
- the multimedia data is streamed responsive to receiving a request to execute the presentation application from a user of the local computing device.
- a local streaming module executing on the local computing device transmits the multimedia data to the remote computing device via the virtual channel.
- a local streaming module executing on the local computing device may stream the multimedia data from a multimedia device interface.
- the local streaming module executing on the local computing device may stream the multimedia data to a local remoting application executing on the local computing device.
- an encoder executing on the local computing device encodes the multimedia data before transmitting the multimedia data to the remote computing device.
- a local remoting application on the local computing device constructs a packet to include the multimedia data in a protocol wrapper before transmitting the packet comprising the multimedia data to the remote computing device.
- an encoder executing on the local computing device encrypts the multimedia data before transmitting the multimedia data to the remote computing device.
- the present disclosure relates to a local computing device for use in a system in which multimedia data generated by a multimedia device connected to the local computing device is displayed on the local computing device by an application executing on a remote computing device.
- the local computing device may comprise a multimedia interface acquiring multimedia data from a multimedia device associated with the local computing device.
- the multimedia data may comprise input to a presentation application executing on a remote computing device.
- the local computing device may also comprise a streaming module transmitting the acquired multimedia data to the remote computing device via a virtual channel.
- the local computing device may further comprise a local agent receiving, from the remote computing device, application output generated by the presentation application responsive to the transmitted multimedia data.
- the received application output may include the multimedia data generated by the multimedia device and a second stream of multimedia data generated by a third computing device.
- the streaming module may stream the multimedia data responsive to receiving a request from a user of the local computing device to execute the presentation application.
- the streaming module may execute on the local computing device and streams the multimedia data directly from the multimedia interface.
- the encoder executing on the local computing device encodes the acquired multimedia data before transmission to the remote computing device.
- a packet engine executes on the local computing device and constructs a packet including the acquired multimedia data in a protocol wrapper before transmitting the packet comprising the multimedia data to the remote computing device.
- the present disclosure relates to a local computing device for use in a system in which multimedia data generated by a multimedia device connected to the local computing device is displayed on the local computing device by an application executing on a remote computing device.
- the local computing device may comprise a means for streaming, by a local computing device, multimedia data to a remote computing device via a virtual channel, the multimedia data generated by a multimedia device associated with the local computing device and comprising input to a presentation application executing on the remote computing device.
- the local computing device may also include a means for receiving, from the remote computing device, application output generated by the presentation application responsive to the transmitted multimedia data, the received application output including the multimedia data generated by the multimedia device and a second stream of multimedia data generated by a third computing device.
- FIG. 1 A depicts embodiments of network environments that provide remote access to computing devices that can execute application programs.
- FIG. IB and FIG. 1C depict embodiments of computing devices.
- FIG. ID illustrates a block diagram depicting an embodiment of a server farm.
- FIGs. 2A-2C illustrate block diagrams of embodiments of a virtualization environment.
- FIGs. 3A-3B illustrate block diagrams depicting embodiments of a system for remoting multimedia data for display within a remote application.
- FIG. 4 illustrates a flow diagram depicting an embodiment of a process for obtaining and remoting multimedia data.
- FIG. 5A illustrates a flow diagram depicting an embodiment of a method for obtaining and transmitting multimedia data.
- FIG. 5B illustrates a flow diagram depicting an embodiment of a method for retrieving and displaying multimedia data.
- FIG. 5C illustrates a block diagram of an embodiment of a system for delivering support for desktop audio/video conferencing applications on remote network devices.
- FIG. 5D illustrates a block diagram of an embodiment of one or more filters to enable a multimedia application on a remote server to process audio/video data generated by a capture device connected to a client machine.
- FIG. 5E illustrates a block diagram of an embodiment of an ICA engine processing multimedia data on a client.
- FIG. 5F illustrates a block diagram of an embodiment of an application space on a remote hosting server, processing multimedia data received from a remote client.
- FIG. 5G illustrates an embodiment of a protocol sequence for communicating multimedia data via a virtual channel between a client and a hosting server.
- Section A describes a network environment and computing environment which may be useful for practicing embodiments described herein;
- Section B describes embodiments of systems and methods for virtualizing a computing environment
- Section C describes embodiments of systems and methods for remotely presenting a multimedia data stream.
- FIG. 1A illustrates one embodiment of a computing environment 101 that includes one or more client machines 102A-102N (generally referred to herein as “client machine(s) 102") that are in communication with one or more servers 106A-106N (generally referred to herein as “server(s) 106"). Installed in between the client machine(s) 102 and server(s) 106 is a network 104.
- client machine(s) 102 generally referred to herein as “client machine(s) 102”
- servers 106A-106N generally referred to herein as “server(s) 106”
- Installed in between the client machine(s) 102 and server(s) 106 is a network 104.
- the computing environment 101 can include an appliance installed between the server(s) 106 and client machine(s) 102. This appliance can manage client/server connections, and in some cases can load balance client connections amongst a plurality of backend servers.
- the client machine(s) 102 can in some embodiment be referred to as a single client machine 102 or a single group of client machines 102, while server(s) 106 may be referred to as a single server 106 or a single group of servers 106.
- a single client machine 102 communicates with more than one server 106, while in another embodiment a single server 106 communicates with more than one client machine 102.
- a single client machine 102 communicates with a single server 106.
- a client machine 102 can, in some embodiments, be referenced by any one of the following terms: client machine(s) 102; client(s); client computer(s); client device(s); client computing device(s); local machine; remote machine; client node(s); endpoint(s); endpoint node(s); a second machine; a first machine; or a third machine.
- the server 106 in some embodiments, may be referenced by any one of the following terms: server(s), local machine; remote machine; server farm(s), host computing device(s), a first machine(s); or a second machine(s).
- the client machine 102 can be a virtual machine 102C.
- the virtual machine 102C can be any virtual machine, while in some embodiments the virtual machine 102C can be any virtual machine managed by a hypervisor developed by XenSolutions, Citrix Systems, IBM, VMware, or any other hypervisor. In other embodiments, the virtual machine 102C can be managed by any hypervisor, while in still other embodiments, the virtual machine 102C can be managed by a hypervisor executing on a server 106 or a hypervisor executing on a client 102.
- the client machine 102 can in some embodiments execute, operate or otherwise provide an application that can be any one of the following: software; a program; executable instructions; a virtual machine; a hypervisor; a web browser; a web-based client; a client-server application; a thin-client; an ActiveX control; a Java applet; software related to voice over internet protocol (VoIP) communications like a soft IP telephone; an application for streaming video and/or audio; an application for facilitating real-time-data communications; a HTTP client; a FTP client; an Oscar client; a Telnet client; or any other set of executable instructions.
- VoIP voice over internet protocol
- Still other embodiments include a client device 102 that displays application output generated by an application remotely executing on a server 106 or other remotely located machine.
- the client device 102 can display the application output in an application window, a browser, or other output window.
- the application is a desktop, while in other embodiments the application is an application that generates a desktop.
- the server 106 executes a remote presentation client or other client or program that uses a thin-client or remote-display protocol to capture display output generated by an application executing on a server 106 and transmits the application display output to a remote client 102.
- the thin-client or remote-display protocol can be any one of the following protocols: the Independent Computing Architecture (ICA) protocol manufactured by Citrix Systems, Inc. of Ft. Lauderdale, Florida; or the Remote Desktop Protocol (RDP) manufactured by the Microsoft Corporation of Redmond, Washington.
- ICA Independent Computing Architecture
- RDP Remote Desktop Protocol
- the computing environment 101 can include more than one server 106A- 106N such that the servers 106A-106N are logically grouped together into a server farm 106.
- the server farm 106 can include servers 106 that are geographically dispersed and logically grouped together in a server farm 106, or servers 106 that are located proximate to each other and logically grouped together in a server farm 106.
- Geographically dispersed servers 106A-106N within a server farm 106 can, in some embodiments, communicate using a WAN, MAN, or LAN, where different geographic regions can be characterized as: different continents; different regions of a continent; different countries; different states; different cities; different campuses; different rooms; or any combination of the preceding geographical locations.
- the server farm 106 may be administered as a single entity, while in other embodiments the server farm 106 can include multiple server farms 106.
- a server farm 106 can include servers 106 that execute a substantially similar type of operating system platform (e.g., WINDOWS NT, manufactured by Microsoft Corp. of Redmond, Washington, UNIX, LINUX, or SNOW LEOPARD.)
- the server farm 106 can include a first group of servers 106 that execute a first type of operating system platform, and a second group of servers 106 that execute a second type of operating system platform.
- the server farm 106 in other embodiments, can include servers 106 that execute different types of operating system platforms.
- the server 106 can be any server type.
- the server 106 can be any of the following server types: a file server; an application server; a web server; a proxy server; an appliance; a network appliance; a gateway; an application gateway; a gateway server; a virtualization server; a deployment server; a SSL VPN server; a firewall; a web server; an application server or a master application server; a server 106 executing an active directory; or a server 106 executing an application acceleration program that provides firewall functionality, application functionality, or load balancing functionality.
- a server 106 may be a RADIUS server that includes a remote authentication dial-in user service.
- the server 106 can be an appliance manufactured by any one of the following manufacturers: the Citrix Application Networking Group; Silver Peak Systems, Inc; Riverbed Technology, Inc.; F5 Networks, Inc.; or Juniper Networks, Inc.
- Some embodiments include a first server 106A that receives requests from a client machine 102, forwards the request to a second server 106B, and responds to the request generated by the client machine 102 with a response from the second server 106B.
- the first server 106A can acquire an enumeration of applications available to the client machine 102 and well as address information associated with an application server 106 hosting an application identified within the enumeration of applications.
- the first server 106A can then present a response to the client's request using a web interface, and communicate directly with the client 102 to provide the client 102 with access to an identified application.
- the server 106 can, in some embodiments, execute any one of the following applications: a thin-client application using a thin-client protocol to transmit application display data to a client; a remote display presentation application; any portion of the CITRIX ACCESS SUITE by Citrix Systems, Inc. like the METAFRAME or CITRIX PRESENTATION SERVER; MICROSOFT
- server 106 that is an application server such as: an email server that provides email services such as MICROSOFT EXCHANGE manufactured by the Microsoft Corporation; a web or Internet server; a desktop sharing server; a collaboration server; or any other type of application server. Still other embodiments include a server 106 that executes any one of the following types of hosted servers
- GOTOMEETING provided by Citrix Online Division, Inc.
- WEB EX provided by WebEx, Inc. of Santa Clara, California
- Microsoft Office LIVE MEETING provided by Microsoft Corporation.
- Client machines 102 can, in some embodiments, be a client node that seeks access to resources provided by a server 106.
- the server 106 may provide clients 102 or client nodes with access to hosted resources.
- the server 106 functions as a master node such that it communicates with one or more clients 102 or servers 106.
- the master node can identify and provide address information associated with a server 106 hosting a requested application, to one or more clients 102 or servers 106.
- the master node can be a server farm 106, a client 102, a cluster of client nodes 102, or an appliance.
- One or more clients 102 and/or one or more servers 106 can transmit data over a network 104 installed between machines and appliances within the computing environment 101.
- the network 104 can comprise one or more sub-networks, and can be installed between any combination of the clients 102, servers 106, computing machines and appliances included within the computing environment 101.
- the network 104 can be: a local-area network (LAN); a metropolitan area network (MAN); a wide area network (WAN); a primary network 104 comprised of multiple sub-networks 104 located between the client machines 102 and the servers 106; a primary public network 104 with a private sub-network 104; a primary private network 104 with a public sub-network 104; or a primary private network 104 with a private sub-network 104.
- LAN local-area network
- MAN metropolitan area network
- WAN wide area network
- a primary network 104 comprised of multiple sub-networks 104 located between the client machines 102 and the servers 106
- a primary public network 104 with a private sub-network 104 a primary private network 104 with a public sub-network 104
- a primary private network 104 with a private sub-network 104 or a primary private network 104 with a private sub-network 104.
- Still further embodiments include a network 104 that can be any of the following network types: a point to point network; a broadcast network; a telecommunications network; a data communication network; a computer network; an ATM (Asynchronous Transfer Mode) network; a SONET (Synchronous Optical Network) network; a SDH (Synchronous Digital Hierarchy) network; a wireless network; a wireline network; or a network 104 that includes a wireless link where the wireless link can be an infrared channel or satellite band.
- a network 104 can be any of the following network types: a point to point network; a broadcast network; a telecommunications network; a data communication network; a computer network; an ATM (Asynchronous Transfer Mode) network; a SONET (Synchronous Optical Network) network; a SDH (Synchronous Digital Hierarchy) network; a wireless network; a wireline network; or a network 104 that includes a wireless link where the wireless link can be
- the network topology of the network 104 can differ within different embodiments, possible network topologies include: a bus network topology; a star network topology; a ring network topology; a repeater-based network topology; or a tiered-star network topology. Additional embodiments may include a network 104 of mobile telephone networks that use a protocol to communicate among mobile devices, where the protocol can be any one of the following: AMPS; TDMA;
- CDMA Code Division Multiple Access
- GSM Global System for Mobile communications
- GPRS UMTS any other protocol able to transmit data among mobile devices.
- FIG. 1 B Illustrated in Figure 1 B is an embodiment of a computing device 100, where the client machine 102 and server 106 illustrated in Figure 1A can be deployed as and/or executed on any embodiment of the computing device 100 illustrated and described herein.
- a system bus 150 that communicates with the following components: a central processing unit 121 ; a main memory 122; storage memory 128; an input/output (I/O) controller 123; display devices 124A-124N; an installation device 1 16; and a network interface 1 18.
- the storage memory 128 includes: an operating system, software routines, and a client agent 120.
- the I/O controller 123 in some embodiments, is further connected to a key board 126, and a pointing device 127. Other embodiments may include an I/O controller 123 connected to more than one input/output device 13 OA- 13 ON.
- Figure 1C illustrates one embodiment of a computing device 100, where the client machine 102 and server 106 illustrated in Figure 1 A can be deployed as and/or executed on any embodiment of the computing device 100 illustrated and described herein.
- a system bus 150 that communicates with the following components: a bridge 170, and a first I/O device 130A.
- the bridge 170 is in further communication with the main central processing unit 121, where the central processing unit 121 can further communicate with a second I/O device 130B, a main memory 122, and a cache memory 140.
- I/O ports I/O ports, a memory port 103, and a main processor.
- Embodiments of the computing machine 100 can include a central processing unit 121 characterized by any one of the following component configurations: logic circuits that respond to and process instructions fetched from the main memory unit 122; a microprocessor unit, such as: those manufactured by Intel Corporation; those manufactured by Motorola Corporation; those manufactured by Transmeta Corporation of Santa Clara, California; the RS/6000 processor such as those manufactured by International Business Machines; a processor such as those manufactured by Advanced Micro Devices; or any other combination of logic circuits.
- a central processing unit 121 characterized by any one of the following component configurations: logic circuits that respond to and process instructions fetched from the main memory unit 122; a microprocessor unit, such as: those manufactured by Intel Corporation; those manufactured by Motorola Corporation; those manufactured by Transmeta Corporation of Santa Clara, California; the RS/6000 processor such as those manufactured by International Business Machines; a processor such as those manufactured by Advanced Micro Devices; or any other combination of logic circuits.
- central processing unit 122 may include any combination of the following: a microprocessor, a microcontroller, a central processing unit with a single processing core, a central processing unit with two processing cores, or a central processing unit with more than one processing core.
- One embodiment of the computing machine 100 includes a central processing unit 121 that communicates with cache memory 140 via a secondary bus also known as a backside bus, while another embodiment of the computing machine 100 includes a central processing unit 121 that communicates with cache memory via the system bus 150.
- the local system bus 150 can, in some embodiments, also be used by the central processing unit to communicate with more than one type of I/O device 130A-130N.
- the local system bus 150 can be any one of the following types of buses: a VESA VL bus; an ISA bus; an EISA bus; a MicroChannel Architecture (MCA) bus; a PCI bus; a PCI-X bus; a PCI-Express bus; or a NuBus.
- computing machine 100 examples include an I/O device 13 OA- 13 ON that is a video display 124 that communicates with the central processing unit 121. Still other versions of the computing machine 100 include a processor 121 connected to an I/O device 130A-130N via any one of the following connections: HyperTransport, Rapid I/O, or InfiniBand. Further embodiments of the computing machine 100 include a processor 121 that communicates with one I/O device 130A using a local interconnect bus and a second I/O device 130B using a direct connection.
- the computing device 100 includes a main memory unit 122 and cache memory 140.
- the cache memory 140 can be any memory type, and in some embodiments can be any one of the following types of memory: SRAM; BSRAM; or EDRAM.
- cache memory 140 and a main memory unit 122 that can be any one of the following types of memory: Static random access memory (SRAM), Burst SRAM or SynchBurst SRAM (BSRAM); Dynamic random access memory (DRAM); Fast Page Mode DRAM (FPM DRAM); Enhanced DRAM (EDRAM), Extended Data Output RAM (EDO RAM); Extended Data Output DRAM (EDO DRAM); Burst Extended Data Output DRAM (BEDO DRAM); Enhanced DRAM (EDRAM); synchronous DRAM (SDRAM); JEDEC SRAM; PC 100 SDRAM; Double Data Rate SDRAM (DDR SDRAM); Enhanced SDRAM (ESDRAM); SyncLmk DRAM (SLDRAM); Direct Rambus DRAM (DRDRAM); Ferroelectric RAM (FRAM); or any other type of memory.
- Further embodiments include a central processing unit 121 that can access the main memory 122 via: a system bus 150; a memory port 103; or any other connection, bus or port that allows the processor 121 to access memory 122
- One embodiment of the computing device 100 provides support for any one of the following installation devices 1 16: a CD-ROM drive, a CD-R/RW drive, a DVD-ROM drive, tape drives of various formats, USB device, a bootable medium, a bootable CD, a bootable CD for GNU/Linux distribution such as KNOPPIX®, a hard-drive or any other device suitable for installing applications or software.
- installation devices 1 16 a CD-ROM drive, a CD-R/RW drive, a DVD-ROM drive, tape drives of various formats, USB device, a bootable medium, a bootable CD, a bootable CD for GNU/Linux distribution such as KNOPPIX®, a hard-drive or any other device suitable for installing applications or software.
- Applications can in some embodiments include a client agent 120, or any portion of a client agent 120.
- the computing device 100 may further include a storage device 128 that can be either one or more hard disk drives, or one or more redundant arrays of independent disks; where the storage device is configured to store an operating system, software, programs applications, or at least a portion of the client agent 120.
- a further embodiment of the computing device 100 includes an installation device 1 16 that is used as the storage device 128.
- the computing device 100 may further include a network interface 1 18 to interface to a Local Area Network (LAN), Wide Area Network (WAN) or the Internet through a variety of connections including, but not limited to, standard telephone lines, LAN or WAN links (e.g., 802.1 1 , Tl , T3, 56kb, X.25, SNA, DECNET), broadband connections (e.g., ISDN, Frame Relay, ATM, Gigabit Ethernet, Ethernet-over-SONET), wireless connections, or some combination of any or all of the above.
- LAN Local Area Network
- WAN Wide Area Network
- Connections can also be established using a variety of communication protocols (e.g., TCP/IP, IPX, SPX, NetBIOS, Ethernet, ARCNET, SONET, SDH, Fiber Distributed Data Interface (FDDI), RS232, RS485, IEEE 802.1 1 , IEEE 802.1 1 a, IEEE 802.11b, IEEE 802. l lg, CDMA, GSM, WiMax and direct asynchronous connections).
- One version of the computing device 100 includes a network interface 1 18 able to communicate with additional computing devices 100' via any type and/or form of gateway or tunneling protocol such as Secure Socket Layer (SSL) or Transport Layer Security (TLS), or the Citrix
- Versions of the network interface 118 can comprise any one of: a built-in network adapter; a network interface card; a PCMCIA network card; a card bus network adapter; a wireless network adapter; a USB network adapter; a modem; or any other device suitable for interfacing the computing device 100 to a network capable of communicating and performing the methods and systems described herein.
- Embodiments of the computing device 100 include any one of the following I/O devices 130A-130N: a keyboard 126; a pointing device 127; mice; trackpads; an optical pen; trackballs; microphones; drawing tablets; video displays; speakers; inkjet printers; laser printers; and dye-sublimation printers; or any other input/output device able to perform the methods and systems described herein.
- An I/O controller 123 may in some embodiments connect to multiple I/O devices 103A- 130N to control the one or more I/O devices.
- I/O devices 130A-130N may be configured to provide storage or an installation medium 116, while others may provide a universal serial bus (USB) interface for receiving USB storage devices such as the USB Flash Drive line of devices manufactured by Twintech Industry, Inc.
- USB universal serial bus
- Still other embodiments include an I/O device 130 that may be a bridge between the system bus 150 and an external communication bus, such as: a USB bus; an Apple Desktop Bus; an RS-232 serial connection; a SCSI bus; a FireWire bus; a Fire Wire 800 bus; an Ethernet bus; an AppleTalk bus; a Gigabit Ethernet bus; an Asynchronous Transfer Mode bus; a HIPPI bus; a Super HIPPI bus; a SerialPlus bus; a SCI/LAMP bus; a FibreChannel bus; or a Serial Attached small computer system interface bus.
- the computing machine 100 can connect to multiple display devices 124A-124N, in other embodiments the computing device 100 can connect to a single display device 124, while in still other embodiments the computing device 100 connects to display devices 124A-124N that are the same type or form of display, or to display devices that are different types or forms.
- Embodiments of the display devices 124A-124N can be supported and enabled by the following: one or multiple I/O devices 130A-130N; the I/O controller 123; a combination of I/O device(s) 13 OA- 13 ON and the I/O controller 123; any combination of hardware and software able to support a display device 124A-124N; any type and/or form of video adapter, video card, driver, and/or library to interface, communicate, connect or otherwise use the display devices 124A-124N.
- the computing device 100 may in some embodiments be configured to use one or multiple display devices 124A-124N, these configurations include: having multiple connectors to interface to multiple display devices 124A-124N; having multiple video adapters, with each video adapter connected to one or more of the display devices 124A-124N; having an operating system configured to support multiple displays 124A-124N; using circuits and software included within the computing device 100 to connect to and use multiple display devices 124A-124N; and executing software on the main computing device 100 and multiple secondary computing devices to enable the main computing device 100 to use a secondary computing device's display as a display device 124A-124N for the main computing device 100. Still other embodiments of the computing device 100 may include multiple display devices 124A-124N provided by multiple secondary computing devices and connected to the main computing device 100 via a network.
- the computing machine 100 can execute any operating system, while in other embodiments the computing machine 100 can execute any of the following operating systems: versions of the MICROSOFT WINDOWS operating systems such as WINDOWS 3.x; WINDOWS 95;
- the computing machine 100 can execute multiple operating systems.
- the computing machine 100 can execute PARALLELS or another virtualization platform that can execute or manage a virtual machine executing a first operating system, while the computing machine 100 executes a second operating system different from the first operating system.
- the computing machine 100 can be embodied in any one of the following computing devices: a computing workstation; a desktop computer; a laptop or notebook computer; a server; a handheld computer; a mobile telephone; a portable telecommunication device; a media playing device; a gaming system; a mobile computing device; a netbook; a device of the IPOD family of devices manufactured by Apple Computer; any one of the PLAYSTATION family of devices manufactured by the Sony Corporation; any one of the Nintendo family of devices manufactured by Nintendo Co; any one of the XBOX family of devices manufactured by the Microsoft Corporation; or any other type and/or form of computing, telecommunications or media device that is capable of communication and that has sufficient processor power and memory capacity to perform the methods and systems described herein.
- the computing machine 100 can be a mobile device such as any one of the following mobile devices: a JAVA-enabled cellular telephone or personal digital assistant (PDA), such as the i55sr, i58sr, i85s, i88s, i90c, i95cl, or the iml 100, all of which are
- PDA personal digital assistant
- the computing device 100 can be any one of the following mobile computing devices: any one series of Blackberry, or other handheld device manufactured by Research In Motion Limited; the iPhone manufactured by Apple Computer; Palm Pre; a Pocket PC; a Pocket PC Phone; or any other handheld mobile device.
- each server 106 can be logically linked to create a server farm 38, where each server 106 can include a network-side interface 202 and a farm-side interface 204.
- the network- side interface 202 can be in communication with one or more clients 102 or a network 104.
- the network 104 can be a WAN, LAN, or any other network described herein.
- each server 106 has a farm-side interface 204 that can connect with one or more servers 106 in the farm 38.
- each server 106 can connect with another server 106 in the farm 38 via the other server's farm-side interface and such that both servers' farm-side interfaces 204 are interconnected.
- servers 106 within the server farm 38 can communicate with one another.
- the farm-side interface 204 can communicate with the network-side interface 202, a persistent store 230 and, in some embodiments, with a dynamic store 240.
- the farm 38 in some embodiments, can include a combination of servers 106, a persistent store 230, and a dynamic store 240.
- the server 106 in some embodiments, can communicate with the persistent store 230, while other servers 106' communicate with the server 106 to access information stored in the persistent store.
- the persistent store 230 may be physically implemented on a disk, disk farm, a redundant array of independent disks (RAID), writeable compact disc, or any other device that allows data to be read and written and that maintains written data if power is removed from the storage device.
- a single physical device may provide storage for a plurality of persistent stores, i.e., a single physical device may be used to provide the persistent store 230 for more than one farm 38.
- the persistent store 230 can maintain static data associated with each server 106 in the farm 38, as well as the global data used by the servers 106 within the farm 38.
- the persistent store 230 may maintain the server data in a Lightweight Directory Access Protocol (LDAP) data model.
- LDAP Lightweight Directory Access Protocol
- the persistent store 230 can store the server data in an ODBC- compliant database.
- static data can refer to data that does not change frequently, i.e., data that changes only on an hourly, daily, or weekly basis, or data that never changes.
- Each server can, in some embodiments, use a persistent storage subsystem to read data from and write data to the persistent store 230.
- the data stored by the persistent store 230 may be replicated for reliability purposes either physically or logically.
- physical redundancy may be provided using a set of redundant, mirrored disks, where each disk provides a copy of the data.
- the database itself may be replicated using standard database techniques to provide multiple copies of the database.
- both physical and logical replication may be used
- the dynamic store 240 can be centralized such that the runtime data is stored in the memory of one server 106 in the farm 38. That server can then operate as a master network node with which all other servers 106 in the farm 38 communicate when seeking access to runtime data.
- each server 106 in the farm 38 can keep a full copy of the dynamic store 240 such that each server 106 communicates with every other server 106 to keep its copy of the dynamic store 240 up to date.
- each server 106 maintains its own runtime data and communicates with other servers 106 when seeking to obtain runtime data from them.
- a server 106 attempting to find an application program requested by the client 102 may communicate directly with every other server 106 in the farm 38 to identify one or more servers hosting the requested application.
- a collector point in some embodiments, can be a server 106 that collects and stores runtime data collected from other servers 106 in the farm 38.
- each collector point can store a copy of the entire dynamic store 240, while in other embodiments, each collector point stores a portion of the dynamic store 240, i.e. the collector point maintains runtime data of a particular data type.
- the type of data stored by a server 106 may be predetermined according to one or more criteria.
- servers 106 may store different types of data based on their boot order.
- the type of data stored by a server 106 may be configured by an administrator using an administration tool (Not Shown.)
- the dynamic store 240 can be distributed amongst two or more servers 106 in the farm 38.
- Servers 106 not designated as collector points can, in some embodiments, identify servers 106 that have been designated as collector points. Upon identifying these collector points, the servers 106 can communicates with the identified collector points to deliver and request runtime data. Using collector points, in one embodiment, can mitigate traffic caused by a large server farm 38 by allowing servers 106 to communicate with one or more collector points rather than every other server 106. Thus, requesting and delivering access to runtime data can be a less bandwidth intensive process.
- Each server 106 can operate as a collector point for more than one type of data.
- a first server 106" can operate as a collector point for licensing information and for loading information.
- each collector point may amass a different type of run-time data.
- a second server 106"' can collect licensing information, while the first server 106" collects loading
- each collector point can store data that is shared between all servers 106 in a farm 38.
- each collector point of a particular type of data exchanges the data collected by that collector point with every other collector point for that type of data in the farm 38.
- each collector point 106" and 106 possesses the same data.
- each collector point 106 and 106" also keeps every other collector point abreast of any updates to the runtime data.
- a client 102 can view applications executing within the farm 38 and can access available information and application sessions executing within the farm 38, by browsing the contents and activity within a farm 38.
- each server 106 can include an ICA browsing subsystem 260 that can provide the client 102 with browsing capabilities. After the client 102 establishes a connection with the ICA browser subsystem 260 of any of the servers 106, that browser subsystem can support a variety of client requests.
- client requests can include: (1) enumerating names of servers in the farm, (2) enumerating names of applications published in the farm, (3) resolving a server name and/or application name to a server address that is useful the client 102.
- the ICA browser subsystem 260 can also support requests made by clients 102 running a program neighborhood application that provides the client 102, upon request, with a view of those applications within the farm 38 for which the user is authorized. In some embodiments, the ICA browser subsystem 260 can forward all of the above- mentioned client requests to the appropriate subsystem in the server 106.
- each server 106 in the farm 38 that has a program neighborhood subsystem 270 can provide the user of a client 102 with a view of applications within the farm 38.
- the program neighborhood subsystem 270 may limit the view to those applications for which the user of the client 102 has authorization to access.
- this program neighborhood service presents the applications to the user as a list or a group of icons.
- the functionality provided by the program neighborhood subsystem 270 can be available to two types of clients, (1) program neighborhood-enabled clients that can access the functionality directly from a client desktop, and (2) non-program neighborhood-enabled clients (e.g., legacy clients) that can access the functionality by running a program neighborhood-enabled desktop on the server.
- program neighborhood-enabled clients that can access the functionality directly from a client desktop
- non-program neighborhood-enabled clients e.g., legacy clients
- Communication between a program neighborhood-enabled client and the program neighborhood subsystem 270 may occur over a dedicated virtual channel that is established on top of an ICA virtual channel. In other embodiments, the communication occurs using an XML service. In one of these embodiments, the program neighborhood-enabled client communicates with an XML subsystem.
- the program neighborhood-enabled client does not have a connection with the server with a program neighborhood subsystem 270.
- the client 102 sends a request to the ICA browser subsystem 260 to establish an ICA connection to the server 106 in order to identify applications available to the client 102.
- the client 102 then runs a client-side dialog that acquires the credentials of a user.
- the credentials are received by the ICA browser subsystem 260 and sent to the program neighborhood subsystem 270.
- the program neighborhood subsystem 270 sends the credentials to a user management subsystem for authentication.
- the user management subsystem may return a set of distinguished names representing the list of accounts to which the user belongs.
- the program neighborhood subsystem 270 Upon authentication, the program neighborhood subsystem 270 establishes the program neighborhood virtual channel. This channel remains open until the application filtering is complete. The program neighborhood subsystem 270 can then request the program neighborhood information from the common application subsystem 524 associated with those accounts. The common application subsystem 524 can then obtain the program neighborhood information from the persistent store 230. On receiving the program neighborhood information, the program neighborhood subsystem 270 formats and returns the program
- the partial ICA connection is closed.
- the program neighborhood-enabled client establishes a partial ICA connection with a server
- the program neighborhood-enabled client establishes a partial ICA connection with a server
- the selection of the farm 38 sends a request from the client 102 to the ICA browser subsystem 260 to establish an ICA connection with one of the servers 106 in the selected farm 38.
- the ICA browser subsystem 260 can then send a request to the program neighborhood subsystem 270, which selects a server 106 in the farm 38.
- Address information associated with the server 106 can be identified and returned to the client 102 by way of the ICA browser subsystem 260.
- the client 102 can then subsequently connect to the server 106 corresponding to the received address information.
- the program neighborhood-enabled client 102 establishes an ICA connection upon which the program neighborhood-virtual channel can be established, where the connection can remain open for as long as the ICA connection persists.
- the program neighborhood subsystem 270 can push program neighborhood information updates to the client 102.
- the program neighborhood subsystem 270 subscribes to events from the common application subsystem 524 to allow the program neighborhood subsystem 270 to detect changes to published applications.
- a computing device 100 includes a hypervisor layer, a virtualization layer, and a hardware layer.
- the hypervisor layer includes a hypervisor 401 (also referred to as a virtualization manager) that allocates and manages access to a number of physical resources in the hardware layer (e.g., the processor(s) 421, and disk(s) 428) by at least one virtual machine executing in the virtualization layer.
- the virtualization layer includes at least one operating system 410 and a plurality of virtual resources allocated to the at least one operating system 410.
- Virtual resources may include, without limitation, a plurality of virtual processors 432a, 432b, 432c (generally 432), and virtual disks 442a, 442b, 442c (generally 442), as well as virtual resources such as virtual memory and virtual network interfaces.
- the plurality of virtual resources and the operating system 410 may be referred to as a virtual machine 406.
- a virtual machine 406 may include a control operating system 405 in communication with the hypervisor 401 and used to execute applications for managing and configuring other virtual machines on the computing device 100.
- a hypervisor 401 may provide virtual resources to an operating system in any manner which simulates the operating system having access to a physical device.
- a hypervisor 401 may provide virtual resources to any number of guest operating systems 410a, 410b (generally 410).
- a computing device 100 executes one or more types of hypervisors.
- hypervisors may be used to emulate virtual hardware, partition physical hardware, virtualize physical hardware, and execute virtual machines that provide access to computing environments. Hypervisors may include those manufactured by VMWare, Inc., of Palo Alto, California; the XEN hypervisor, an open source product whose development is overseen by the open source Xen.org community; HyperV, VirtualServer or virtual PC hypervisors provided by
- a computing device 100 executing a hypervisor that creates a virtual machine platform on which guest operating systems may execute is referred to as a host server.
- the computing device 100 is a XEN SERVER provided by Citrix Systems, Inc., of Fort Lauderdale, FL.
- a hypervisor 401 executes within an operating system executing on a computing device.
- a computing device executing an operating system and a hypervisor 401 may be said to have a host operating system (the operating system executing on the computing device), and a guest operating system (an operating system executing within a computing resource partition provided by the hypervisor 401).
- a hypervisor 401 interacts directly with hardware on a computing device, instead of executing on a host operating system.
- the hypervisor 401 may be said to be executing on "bare metal," referring to the hardware comprising the computing device.
- a hypervisor 401 may create a virtual machine 406a-c (generally 406) in which an operating system 410 executes.
- the hypervisor 401 loads a virtual machine image to create a virtual machine 406.
- the hypervisor 401 executes an operating system 410 within the virtual machine 406.
- the virtual machine 406 executes an operating system 410.
- the hypervisor 401 controls processor scheduling and memory partitioning for a virtual machine 406 executing on the computing device 100. In one of these embodiments, the hypervisor 401 controls the execution of at least one virtual machine 406. In another of these embodiments, the hypervisor 401 presents at least one virtual machine 406 with an abstraction of at least one hardware resource provided by the computing device 100. In other embodiments, the hypervisor 401 controls whether and how physical processor capabilities are presented to the virtual machine 406.
- a control operating system 405 may execute at least one application for managing and configuring the guest operating systems.
- the control operating system 405 may execute an administrative application, such as an application including a user interface providing administrators with access to functionality for managing the execution of a virtual machine, including functionality for executing a virtual machine, terminating an execution of a virtual machine, or identifying a type of physical resource for allocation to the virtual machine.
- the hypervisor 401 executes the control operating system 405 within a virtual machine 406 created by the hypervisor 401.
- the control operating system 405 executes in a virtual machine 406 that is authorized to directly access physical resources on the computing device 100.
- a control operating system 405a on a computing device 100a may exchange data with a control operating system 405b on a computing device 100b, via communications between a hypervisor 401 a and a hypervisor 401b.
- one or more computing devices 100 may exchange data with one or more of the other computing devices 100 regarding processors and other physical resources available in a pool of resources.
- this functionality allows a hypervisor to manage a pool of resources distributed across a plurality of physical computing devices.
- multiple hypervisors manage one or more of the guest operating systems executed on one of the computing devices 100.
- control operating system 405 executes in a virtual machine 406 that is authorized to interact with at least one guest operating system 410.
- a guest operating system 410 communicates with the control operating system 405 via the hypervisor 401 in order to request access to a disk or a network.
- the guest operating system 410 and the control operating system 405 may communicate via a communication channel established by the hypervisor 401 , such as, for example, via a plurality of shared memory pages made available by the hypervisor 401.
- the control operating system 405 includes a network back-end driver for communicating directly with networking hardware provided by the computing device 100.
- the network back-end driver processes at least one virtual machine request from at least one guest operating system 1 10.
- the control operating system 405 includes a block back-end driver for communicating with a storage element on the computing device 100.
- the block back-end driver reads and writes data from the storage element based upon at least one request received from a guest operating system 410.
- control operating system 405 includes a tools stack 404.
- a tools stack 404 provides functionality for interacting with the hypervisor 401, communicating with other control operating systems 405 (for example, on a second computing device 100b), or managing virtual machines 406b, 406c on the computing device 100.
- the tools stack 404 includes customized applications for providing improved
- At least one of the tools stack 404 and the control operating system 405 include a management API that provides an interface for remotely configuring and controlling virtual machines 406 running on a computing device 100.
- the control operating system 405 communicates with the hypervisor 401 through the tools stack 104.
- the hypervisor 401 executes a guest operating system 410 within a virtual machine 406 created by the hypervisor 401.
- the guest operating system 410 provides a user of the computing device 100 with access to resources within a computing environment.
- a resource includes a program, an application, a document, a file, a plurality of applications, a plurality of files, an executable program file, a desktop environment, a computing environment, or other resource made available to a user of the computing device 100.
- the resource may be delivered to the computing device 100 via a plurality of access methods including, but not limited to, conventional installation directly on the computing device 100, delivery to the computing device 100 via a method for application streaming, delivery to the computing device 100 of output data generated by an execution of the resource on a second computing device 100' and communicated to the computing device 100 via a presentation layer protocol, delivery to the computing device 100 of output data generated by an execution of the resource via a virtual machine executing on a second computing device 100', or execution from a removable storage device connected to the computing device 100, such as a USB device, or via a virtual machine executing on the computing device 100 and generating output data.
- the computing device 100 transmits output data generated by the execution of the resource to another computing device 100'.
- the guest operating system 410 in conjunction with the virtual machine on which it executes, forms a fully-virtualized virtual machine which is not aware that it is a virtual machine; such a machine may be referred to as a "Domain U HVM (Hardware Virtual Machine) virtual machine".
- a fully-virtualized machine includes software emulating a Basic Input/Output System (BIOS) in order to execute an operating system within the fully-virtualized machine.
- a fully-virtualized machine may include a driver that provides functionality by communicating with the hypervisor 401. In such an embodiment, the driver may be aware that it executes within a virtualized environment.
- BIOS Basic Input/Output System
- the guest operating system 410 in conjunction with the virtual machine on which it executes, forms a paravirtualized virtual machine, which is aware that it is a virtual machine; such a machine may be referred to as a "Domain U PV virtual machine".
- a paravirtualized machine includes additional drivers that a fully- virtualized machine does not include.
- paravirtualized machine includes the network back-end driver and the block back- end driver included in a control operating system 405, as described above.
- FIG. 2B a block diagram depicts one embodiment of a plurality of networked computing devices in a system in which at least one physical host executes a virtual machine.
- the system includes a management component 404 and a hypervisor 401.
- the system includes a plurality of computing devices 100, a plurality of virtual machines 406, a plurality of hypervisors 401 , a plurality of management components referred to as tools stacks 404, and a physical resource 421 , 428.
- the plurality of physical machines 100 may each be provided as computing devices 100 such as those described herein.
- a physical disk 428 is provided by a computing device 100 and stores at least a portion of a virtual disk 442.
- a virtual disk 442 is associated with a plurality of physical disks 428.
- one or more computing devices 100 may exchange data with one or more of the other computing devices 100 regarding processors and other physical resources available in a pool of resources, allowing a hypervisor to manage a pool of resources distributed across a plurality of physical computing devices.
- a computing device 100 on which a virtual machine 406 executes is referred to as a physical host 100 or as a host machine 100.
- the hypervisor executes on a processor on the computing device 100.
- the hypervisor allocates, to a virtual disk, an amount of access to the physical disk.
- the hypervisor 401 allocates an amount of space on the physical disk.
- the hypervisor 401 allocates a plurality of pages on the physical disk.
- the hypervisor provisions the virtual disk 442 as part of a process of initializing and executing a virtual machine 450.
- the management component 404a is referred to as a pool management component 404a.
- a management operating system 405a which may be referred to as a control operating system 405a, includes the management component.
- the management component is referred to as a tools stack.
- the management component is the tools stack 404 described above in connection with Figure 2A.
- the management component 404 provides a user interface for receiving, from a user such as an administrator, an identification of a virtual machine 406 to provision and/or execute.
- the management component 404 provides a user interface for receiving, from a user such as an administrator, the request for migration of a virtual machine 406b from one physical machine 100 to another.
- the management component 404a identifies a computing device 100b on which to execute a requested virtual machine 406d and instructs the hypervisor 401b on the identified computing device 100b to execute the identified virtual machine; such a management component may be referred to as a pool management component.
- FIG. 2C embodiments of a virtual application delivery controller or virtual appliance 450 are depicted.
- any of the functionality and/or embodiments of the appliance 200 e.g., an application delivery controller
- the appliance 200 may be deployed in any embodiment of the virtualized environment described above in connection with FIGs 4A and 4B.
- the functionality of the application delivery controller being deployed in the form of an appliance 200, such
- functionality may be deployed in a virtualized environment 400 on any computing device 100, such as a client 102, server 106 or appliance 200.
- FIG. 2C a diagram of an embodiment of a virtual appliance 450 operating on a hypervisor 401 of a server 106 is depicted.
- the virtual appliance 450 may provide functionality for availability, performance, offload and security.
- the virtual appliance may perform load balancing between layers 4 and 7 of the network and may also perform intelligent service health monitoring.
- the virtual appliance may perform caching and compression.
- the virtual appliance may perform connection multiplexing and pooling and/or SSL processing.
- the virtual appliance may perform any of the application firewall functionality and SSL VPN function of appliance 200.
- any of the modules of the appliance 200 as described in connection with FIGs. 2A may be packaged, combined, designed or constructed in a form of the virtualized appliance delivery controller 450 deployable as one or more software modules or components executable in a virtualized environment 300 or non- virtualized environment on any server, such as an off the shelf server.
- the virtual appliance may be provided in the form of an installation package to install on a computing device.
- any of the cache manager 232, policy engine 236, compression 238, encryption engine 234, packet engine 240, GUI 210, CLI 212, shell services 214 and health monitoring programs 216 may be designed and constructed as a software component or module to run on any operating system of a computing device and/or of a virtualized environment 300.
- the virtualized appliance 400 may use any of these resources as provided by the virtualized environment 400 or as otherwise available on the server 106.
- any one or more vServers 275A-275N may be in operation or executed in a virtualized environment 400 of any type of computing device 100, such as any server 106.
- Any of the modules or functionality of the appliance 200 described in connection with Figure 2B may be designed and constructed to operate in either a virtualized or non- virtualized environment of a server.
- Any of the vServer 275, SSL VPN 280, Intranet UP 282, Switching 284, DNS 286, acceleration 288, App FW 280 and monitoring agent may be packaged, combined, designed or constructed in a form of application delivery controller 450 deployable as one or more software modules or components executable on a device and/or virtualized environment 400.
- a server may execute multiple virtual machines 406a-406n in the virtualization environment with each virtual machine running the same or different embodiments of the virtual application delivery controller 450.
- the server may execute one or more virtual appliances 450 on one or more virtual machines on a core of a multi-core processing system.
- the server may execute one or more virtual appliances 450 on one or more virtual machines on each processor of a multiple processor device.
- FIG. 3 A Illustrated in Figure 3 A is one embodiment of a system that retrieves multimedia data generated by a multimedia device 302, intercepts that multimedia data, and transmits the multimedia data to a remote computing machine where the data is formatted and sent to a display application 314' where the data can be displayed, played or presented to a user.
- the system can include two computing machines, a first computing machine 301 and a second computing machine 300, that communicate over a virtual channel 306. Each of the first computing machine 301 and the second computing machine 300 can execute a version of an
- each of the first computing machine 301 and the second computing machine 300 may execute a multimedia device driver 316, 316' communicating with a multimedia device 302, 302', and each machine 301, 300 may execute a display application 314, 314'.
- Executing on both the first and second computing machine 301, 300 is a DIRECTSHOW Framework 320 which can comprise any of the following modules, filters and elements: source filters 330, 330' further comprising either a streaming module 310 or a proxy streaming module 308; an application sink filter 330, 330'; a multimedia sink filter 312, 312'; a transform filter 326, 326'; and a renderer filter 328, 328'.
- source filters 330, 330' further comprising either a streaming module 310 or a proxy streaming module 308; an application sink filter 330, 330'; a multimedia sink filter 312, 312'; a transform filter 326, 326'; and a renderer filter 328, 328'.
- either or both of the first computing machine 301 and the second computing machine 300 can execute a display preview application 323, 323'.
- Multimedia data generated by a multimedia capture device may include any audio and/or video data or format.
- Multimedia data may include any type and form of graphical presentation.
- Multimedia data may include portions of audio and/or video.
- Multimedia data may include digital video data made up of frames or images.
- Multimedia data may include a key frame, such as a self-contained frame or an encoded frame without any reference to other images. In the MPEG standard, these may include I-frames and D-frames as opposed to P-frames and B-frames that require information of the previous frames, or both the previous and the following frames for encoding and decoding, respectively.
- DIRECTSHOW may include Extensible Core Microsoft Media Technology and may include a streaming architecture designed for audio and video data which offers a high-level application model enabling rapid development of digital media applications.
- DMO DirectX Media Object
- WMF Windows Media Foundation
- the framework may include Extensible Core Microsoft Media Technology and may include a streaming architecture designed for audio and video data which offers a high-level application model enabling rapid development of digital media applications.
- Such framework may also include a low-level plug-in model that enables third parties to create custom audio or video processing components. It may include features, such as DirectDraw and DirectSound of Microsoft Corporation.
- the system can comprise a first computing machine 301 and a second computing machine 300.
- the first computing machine 301 in some embodiments, can be any computing machine, computing machine or computer.
- the second computing machine 300 in some embodiments, can be any computing machine, computing machine or computer. Both the first computing machine 301 and the second computing machine 300 can be any computer, computing machine, or device described herein.
- the first computing machine 301 is a server, while the second computing machine 300 is a client.
- the first computing machine 301 is a client, while the second computing machine 300 is a server.
- the first computing machine 301 and the second computing machine 300 are clients.
- the first computing machine 301 and the second computing machine 300 are clients that communicate via one or more proxy machines.
- the proxy machines in this embodiment, can be a device, a client or a server.
- Figure 3 A depicts a single first computing machine 301
- the first computing machine 301 is a server farm comprising a plurality of servers.
- the first computing machine 301 is symbolic for any server within the server farm.
- the first computing machine 301 can be symbolic of any server- farm server designated to communicate with the second computing machine 300 or with a user accessing the server farm via the second computing machine 300.
- each of the first computing machine 301 and the second computing machine 300 execute an application/desktop delivery system 322, 322'.
- This delivery system 322, 322' can be any program, module, client, process, or service that, when executing on a computing machine, obtains graphical and non- graphical data and transmits that data to a remote computing machine via a virtual channel 306 established between the computing machine on which the delivery system 322, 322' executes and the remote computing machine.
- the application/desktop delivery system 322, 322' obtains application output data from applications executing on a local processor, and transmits that application output data to a remote computing machine.
- the application/desktop delivery system 322, 322' obtains application output data from applications executing on a local processor, and transmits that application output data to a remote computing machine.
- application/desktop delivery system 322, 322' obtains desktop output data from desktops executing on a local processor and transmits that desktop output data to a remote computing machine. In still other embodiments, the application/desktop delivery system 322, 322' obtains application or desktop output data from a virtual machine executing on a local processor and transmits that output to a remote computing machine. [00118] In one embodiment, the application/desktop delivery system 322, 322' can comprise any number of codecs 324, 324' which can be used by the
- Encoding or decoding multimedia data can, in some embodiments, include compressing or decompressing multimedia data. In other embodiments, encoding or decoding multimedia data can include encrypting or decrypting multimedia data. Any of the following codecs can be used to encode and decode video data included in a multimedia data stream: Theora; MPEG-4 ASP; VC-1 ; VC-3; TruDef;
- Any of the following codecs can be used to encode and decode audio data included in a multimedia data stream: Apple Lossless Audio Codec; Direct Stream Transfer; Dolby TrueHD; RealAudio; WavPack; Windows Media Audio 9 Lossless; Dolby Digital; ADX; MPEG Audio; Windows Media Audio; or any other audio codec. Any of the following codecs can be used to encode and decode textual data included in a multimedia data stream: BiM; MPEG-4 Part 17; Ogg Writ; and any other text codec.
- the codecs can compress using a lossy compression algorithm, while in other embodiments the codecs can compress using a lossless compression algorithm.
- the codecs used to encode the multimedia data are bundled together with the encoded multimedia data and sent to the remote computing machine.
- the remote computing machine can use the received codecs to decode the encoded multimedia data.
- the codecs 324, 324' can use any encryption algorithm to encrypt the data prior to transmitting the data.
- codecs 324, 324' can be used to decrypt encrypted data.
- the application/desktop delivery system 322, 322' can transmit presentation level protocol data from one computing machine to another over a virtual channel 306.
- the virtual channel 306, in one embodiment, can be a secure communicative connection between two or more computing machines.
- This virtual channel 306 can be established over a network such as any of the networks described herein, and often transmits data via a presentation level protocol such as the CITRIX ICA protocol, or any other presentation level protocol described herein.
- the application/desktop delivery system 322, 322' further comprises a module that encapsulates formatted multimedia data to form
- the application/desktop delivery system 322, 322' establishes the virtual channel 306 when a local computing machine requests access to an application executing on a remote computing machine. Establishing the virtual channel 306 can include any of the following actions: authenticating a user issuing a request to access an application executing on a remote computing machine;
- the second computing machine 300 can be connected to a multimedia device 302, while the first computing machine 301 is not connected to a multimedia device 302.
- the first computing machine 301 can be connected to a multimedia device 302' while the second computing machine 300 is not connected to a multimedia device 302.
- each of the first computing machine 301 and the second computing machine 300 are connected to different multimedia devices 302. While Figure 3A illustrates a single multimedia device 302, 302', either computing machine can be connected to multiple multimedia devices.
- the multimedia device(s) 302, 302' can be any of the following multimedia devices: web camera; microphone; projector; smart phone; DVD player; CD player; television tuner; musical instrument digital interface; MP3 player; digital cable interface; camera; video camera; infrared camera; optical input device; keyboard; mouse; Blu-Ray Player; satellite interface device; and any other multimedia device.
- the multimedia device 302, 302' interfaces with a multimedia driver 316, 316' on the computing machine.
- the multimedia driver 316, 316' receives raw multimedia output generated by the multimedia device 302, 302' and converts the raw output to a format able to be processed by the
- the multimedia driver 316, 316' can interface with the DIRECTSHOW framework 320, 320', the display application 314, 314' or any other application that can input formatted multimedia output in its native form and process the output for eventual presentation to a user.
- the multimedia data can be any one of the following data: video; audio; text; graphical; or any other type of multimedia data.
- Figure 3 A illustrates a first computing machine 301 and a second computing machine 300 where each computing machine can execute a multimedia device driver 316, 316'.
- the computing machine 300, 301 connected to a multimedia device 302, 302' can include a multimedia device driver 316, 316', while the other computing machine that is not connected to the multimedia device 302, 302' does not include a multimedia device driver 316, 316'.
- a multimedia driver 316, 316' in some embodiments, is not needed by a computing machine 300, 301 unless that computing machine 300, 301
- the second computing machine 300 communicates directly with a multimedia device 302, 302'.
- the second computing machine 300 can include a multimedia device driver 316.
- the second computing machine 300 can transmit the multimedia data to a first computing machine 301 over a virtual channel 306.
- the first computing machine 301 does not communicate with a multimedia device 302, therefore the first computing machine 301 does not include a multimedia device driver 316.
- the first computing machine 301 in this embodiment, in this
- the display application 314, 314' can be any application able to display, play or otherwise present multimedia.
- the display application 314 can execute on the first computing machine 301 , and not on the second computing machine 300.
- the display application 314' can execute on the second computing machine 300 and not on the first computing machine 301.
- the system compensates for the lack of a display application on one of the computing machines, therefore in many embodiments, at least one computing machine within the system does not have an instance of the display application and therefore cannot execute the display application 314.
- the display application 314, 314' can by any of the following display applications: Microsoft Communicator; Skype; Windows Media Player; AOL Instant Messenger; Google Chat; RealPlayer; QuickTime; Adobe Media Player; DivX Video Player; Gmail Voice and Video Chat; iVideoChat; Camfrog; TokBox; NetMeeting; Yahoo Messenger; and any other multimedia player.
- the display application is a multimedia player for playing audio such as Windows Media Player, iTunes, or any other audio player.
- the display application is a video or web conference system able to replay audio, video and text.
- the display application 314, 314' is an application for displaying web camera data, or an application for facilitating web chats.
- the display application 314, 314' in still another embodiment, is an application for displaying video or a combination of video and audio.
- either or both of the first computing machine 301 and the second computing machine 300 can execute a display preview application 323, 323'.
- the display preview application 323, 323' can be a stand-alone application, while in other embodiments the display preview application 323, 323' can be included in the display application 314. In other embodiments, the display preview application 323, 323' can be included within the application/desktop delivery system 322.
- the display preview application 323, 323' receives multimedia data processed within the framework 320, 320' and locally displays the multimedia data within an application output window.
- the display preview application 323, 323' can communicate with the application/desktop delivery system 322, 322' on the remote computing machine to receive control information about the presentation window.
- Control information can include: parent application window properties; application window size and position; in-remote session clipping, or any other control information used by the presentation window to display the multimedia data.
- the parent application or remote presentation application is a display application 314, 314' executing on a computing machine located remote from the computing machine receiving the multimedia data.
- One example of local preview via the display preview application 323, 323 ' includes a second computing machine 300 receiving multimedia data from a multimedia device 302. The second computing machine 300 receives an instruction to display the multimedia data in a remote application executing on a first computing machine 301. Rather than transmit the multimedia data to the first computing machine 301 , the display preview application 323 queries the
- the application/desktop delivery system 32 ⁇ executing on the first computing machine 301 for control information about the remote display application 314'.
- the application/desktop delivery system 312' can then retrieve this information from an operating system executing on the first computing machine 301 and transmit the relevant control information to the display preview application 323 executing on the second computing machine 300.
- the display preview application 323 can then use the control information to generate an application output window for displaying the received multimedia data.
- Each of the first computing machine 301 and the second computing machine 300 execute an instance of the DIRECTSHOW framework 320, 320'.
- the framework can be a framework based on either of the following technologies DIRECTX MEDIA OBJECT (DMO), MEDIA
- the first computing machine can execute a first framework 320' based on a first type of multimedia technology, while the second computing machine can execute a second framework 320 based on a second type of multimedia technology.
- the first computing machine can execute a DIRECTSHOW framework 320' while the second computing machine can execute a framework 320 based on DMO multimedia technology.
- the first computing machine can execute a framework 320' based on MJ multimedia technology, while the second computing machine can execute a framework 320 based on DMO multimedia technology.
- the DIRECTSHOW framework 320, 320' is a software development kit or application program interface that can be used to format, modify and manipulate multimedia data and more specifically, multimedia data streams.
- the DIRECTSHOW framework 320, 320' can be used to perform operations on multimedia data inputted by a source filter 330, 330'. Filters, modules, clients or programs included within the framework can receive raw data, format the raw data, modify the raw data and perform other complex processing operations on inputted data.
- the processing operations are performed within the DIRECTSHOW framework 320. Outside applications can interface with data inputted and manipulated within the DIRECTSHOW framework via communication interfaces included within the DIRECTSHOW framework 320, 320'.
- These communication interfaces can use COM technology developed by Microsoft.
- the communication interfaces can be any communication interface able to relay information from modules, filters, clients or other programs executing within the DIRECTSHOW framework 320, 320' to programs interfacing with the framework 320, 320'.
- the DIRECTSHOW framework 320, 320' executes source filters 330, 330' which can retrieve information from multimedia driver(s) 316, 316' and forward that information on to other filters and modules executing within the framework 320, 320'.
- the source filters are modules, clients or programs executing on a computing machine and within the framework 320, 320'.
- the source filters are services and processes executed by a processor and executed by a processor within the kernel space of a computing machine.
- the source filters 330, 330' stream data from any of the following sources: a driver; a device interface; a device; a communication module; a virtual channel 306; a network; a memory repository; or any other information source.
- the source filters 330, 330' stream the multimedia data from a source at the kernel level rather than at the user level.
- the source filters 330, 330' can stream data at the user or program level. Data streamed by the source filters 330, 330' is often forwarded to additional modules, programs, clients or filters executing within the framework 320, 320'.
- the source filter 330, 330' can also stream the multimedia data to additional modules, clients, filters and programs executing within the framework 320, 320'.
- the source filters 330, 330' can comprise a single source filter, or can comprise multiple filters. These source filters 330, 330' can be associated with a specific multimedia device, or can be associated with a specific multimedia data type, i.e. video, audio, text.
- the source filters 330, 330' can comprise a streaming module 310 for streaming multimedia content from a source such as a multimedia device driver 316.
- the streaming module 310 in some embodiments can be the KSProxy filter included within the DIRECTSHOW framework 320, 320'. When the streaming module 310 is the KSProxy filter or a module, filter or program having substantially the same functionality as the
- the streaming module 310 will stream multimedia data from a source such as a driver, device interface or other interface or communication port able to relay multimedia data to a program executing within the framework 320, 320'.
- the streaming module 310 is included in the instance of the framework 320, 320' installed on the computing machine connected to or in direct communication with the multimedia device 302, just as Figure 3 A depicts the framework 320 on the second computing machine 300 as comprising a streaming module 310.
- the streaming module 310 can be any program able to retrieve raw, un-formatted multimedia data either from a multimedia device or from a driver or other interface communicating with the multimedia device.
- the source filters 330, 330' can comprise a proxy streaming module 308 that corresponds to a streaming module 310 on a remote computing machine.
- the proxy streaming module 308 executing within the instance of the DIRECTSHOW framework 320' executing on the first computing machine 301 corresponds to the streaming module 310 executing within the instance of the DIRECTSHOW framework 320 executing on the second computing machine 300.
- the proxy streaming module 308, however, streams multimedia data from one of either a network 104, a virtual channel 306 such as the virtual channel 306 between the first computing machine 301 and the second computing machine 300 and any other multimedia source.
- the proxy streaming module 308, in some embodiments streams the multimedia data from a virtual channel 306 by streaming the multimedia data from a temporary memory repository or cache on the computing machine.
- the application/desktop delivery system 322 or a program communicating with the application/desktop delivery system 322 stores the multimedia data into the temporary memory repository or cache after the application/desktop delivery system 322 receives the multimedia data from the remote computing machine, i.e. the second computing machine 300.
- the proxy streaming module 308 opens or otherwise establishes a channel within the virtual channel 306, and streams multimedia data directly from the remote computing machine, i.e. the second computing machine 300, over the established channel.
- Other filters included within the framework 320 are the transform filter 326, 326' and the renderer filter 328, 328'.
- the transform filter 326, 326' can input raw or formatted multimedia data and further manipulate, modify or transform the inputted data.
- the transform filter 326, 326' can input multiple video streams and splice those streams together according to a combination scheme or according to user instructions.
- the transform filter 326, 326' can input an audio stream and a video stream, synchronize the streams according to timestamp information included within each stream, and combine the streams into a single multimedia data stream.
- the transform filter 326, 326' can place text within video or perform other similar modifications to the inputted data stream.
- the renderer filter 328, 328' can render the multimedia streams.
- the renderer filter 328, 328' can input an audio stream, and format the audio stream as needed before forwarding the audio stream to a sound card within the local computing machine.
- the renderer filter 328, 328' in other embodiments can forward the audio stream to an audio player such as iTunes or Windows Media Player.
- the renderer filter 328, 328' can input a video stream, and format the video stream as needed before forwarding the video stream to a video card within the local computing machine.
- the renderer filter 328, 328' in other embodiments, can forward the video stream to a video player such as iVideo or Windows Media Player.
- a video player such as iVideo or Windows Media Player.
- the renderer filter 328, 328' can draw received video to a screen, draw received graphics to a screen, draw received text to a screen, or write received multimedia data to a memory element.
- the DIRECTSHOW framework 320, 320' can comprise an application sink filter 330, 330' which can be used to stream formatted multimedia data from the DIRECTSHOW framework 320, 320' to a display application 314, 314'.
- an application sink filter 330, 330' for each possible display application.
- the application sink filter 330, 330' forwards the multimedia data stream according to a file type or according to a configuration setting.
- a renderer filter 328 can be referred to as a sink filter.
- the application sink filter 330, 330' can be called the application renderer filter 330, 330'.
- the DIRECTSHOW framework 320, 320' can comprise a multimedia sink filter 312, 312' which can be used to receive the streamed multimedia data generated by the multimedia device 302 and transmit the streamed multimedia data over the virtual channel 306 to the remote computing machine, e.g. the first computing machine 301.
- the multimedia sink filter 312, 312' in some embodiments, can be a video/audio sink filter 312, 312' that receives, processes and transmits both video and audio data.
- the multimedia sink filter 312, 312' can be a camera/microphone sink filter 312, 312'.
- the video/audio sink filter 312, 312' can be a type of renderer filter 328.
- a renderer filter 328 can be referred to as a sink filter.
- the video/audio sink filter 312, 312' can be called the video/audio renderer filter 312, 312'.
- Figure 3B illustrates a single multimedia device 302, 302', in other embodiments the computing machines 300, 301 can communicate with multiple multimedia devices.
- the multimedia data stream received by the streaming module 310 and streamed from the virtual channel 306 by the proxy streaming module 308 can include one or more data streams originating from one or more multimedia devices.
- the multimedia data stream can include data generated by two different cameras.
- the streaming module 310 can stream data from both cameras and the remoting application or application/desktop delivery system 322, 322' can stream both camera data streams over the virtual channel 306.
- the proxy streaming module 308 can then stream from the virtual channel 306 a single multimedia data stream that includes both the first and second camera data stream.
- FIG. 3B Illustrated in Figure 3B is an embodiment of the system 303 illustrating another scheme for the DIRECTSHOW framework 320, 320'.
- the instance of the framework 320 executing on the second computing machine 300 comprises a KSProxy filter 340, an encoder 342 and an interceptor 344.
- the multimedia information/data obtained or received by the multimedia driver 316 executing on the second computing machine 300 is streamed to the KSProxy filter 340 at the kernel level.
- the interceptor 344 streams the encoded multimedia information over a virtual channel 306 installed between the first computing machine 301 and the second computing machine 300.
- the following components are included within the DIRECTSHOW framework 320' on the first computing machine 301 , a receiver 346, a decoder 347 and a proxy multimedia device 348.
- the proxy multimedia device 348 can communicate with a display application 314 executing on the first computing machine 301.
- the KSProxy filter 340 functions substantially the same as the streaming module 310.
- the KSProxy filter 340 in many embodiments, streams multimedia data directly from the multimedia device driver 316 and streams the received multimedia data to other filters or modules included in the DIRECTSHOW framework 320.
- the KSProxy filter 340 streams the multimedia data from the multimedia device driver 316 at the kernel level. In other embodiments, the KSProxy filter 340 streams the multimedia data from the multimedia device driver 316 at the user or application level.
- the encoder 342 can be any encoder.
- the encoder 342 can receive the multimedia data from the KSProxy filter 340 and encode the multimedia data using any codec.
- the encoder 342 can encode the multimedia data using any codec described herein.
- the type of codec used to encode the multimedia data can be based on the type of data included in the multimedia data stream. For example, a video encoding codec can be used to encode video data, while an audio encoding codec can be used to encode audio data.
- the encoder 342 can be communicatively connected to the KSProxy filter 340 or streaming module 310 such that the encoder 342 can stream multimedia data from either the filter 340 or the module 310.
- the encoder 342 and decoder 347 can negotiate security credentials and in some embodiments, can exchange encryption keys or other encryption agents.
- the interceptor 344 can be communicatively connected to the encoder 342 such that the interceptor 344 intercepts multimedia data once it is encoded by the encoder 342. There may be encoder/decoder negotiation involved. Such negotiation may be in accordance with ICA or RAVE virtual channel protocol. In some embodiments, the interceptor 344 can stream multimedia data directly from the KSProxy filter 340 or the streaming module 310 such that the multimedia data is not encoded by the encoder 342. In another embodiment, the interceptor 344 includes substantially the same functionality as the multimedia sink filter 312. The interceptor 344, in some embodiments, communicates with the receiver 346 using a presentation level protocol.
- the interceptor 344 Upon receiving the multimedia data either from the encoder 342, the KSProxy filter 340/streaming module 310 or some other filter or module; the interceptor 344 transmits or streams the multimedia data to the receiver 346.
- the interceptor 344 encapsulates the multimedia data packets in a presentation level protocol before transmitting the multimedia data to a remote computing machine.
- Figure 3B illustrates a framework 320 where the encoder 342 precedes the interceptor 344
- the interceptor 344 can intercept multimedia data and forward it to an encoder 342.
- the interceptor 344 precedes the encoder 342.
- the placement of the modules within the frameworks 320, 320' illustrated in Figure 3B does not indicate the order in which actions occur during the process of streaming multimedia data, processing the multimedia data, intercepting the multimedia data and transmitting the multimedia data, or the process of receiving and processing multimedia data transmitted over a virtual channel 306.
- the receiver 346 receives multimedia data from a remote computing machine and forwards the received data to other filters, modules, clients or programs executing within the DIRECTSHOW framework 320'.
- the receiver 346 can un-encapsulate multimedia data encapsulated in a presentation level protocol before streaming the multimedia data to other programs executing within the DIRECTSHOW framework 320'.
- the receiver 346 can stream the multimedia data to a decoder 347 or directly to a proxy multimedia device 348.
- the decoder 347 can decode received multimedia data using the codec used to encode the multimedia data.
- This codec can be any codec, and more specifically, can be any codec described herein. In one
- the decoder 347 can decompress, decrypt or otherwise decode the multimedia data before streaming the multimedia data to another program or component executing within the framework 320'.
- the proxy multimedia device 348 can stream multimedia data from any of the filters included in the framework 320', but more specifically can stream multimedia data from either the receiver 346 or the decoder 347. In one
- the proxy multimedia device 348 streams the received multimedia data directly to the display application 314 so that the display application 314 can display or present the received multimedia data.
- the framework 320' can comprise a proxy multimedia device 348 for each display application 314. In other embodiments, the framework 320' can comprise a proxy multimedia device 348 for each type of multimedia data.
- FIG. 4 Illustrated in Figure 4 is an embodiment of a flow diagram that depicts an embodiment of the path 350 multimedia data travels from the time it is captured by a multimedia device to the point in time that a user views or hears the multimedia data.
- the path or process 350 includes a streaming module 310 or KSProxy filter 340 streaming multimedia data from a multimedia driver 316, 316' that captures multimedia data generated by a multimedia device 302, 302' (Step 352).
- the multimedia data is then intercepted by a module within the framework 320, 320' (Step 354) and streamed to an encoder 342 or transformation filter 326 where the multimedia data is encoded using codecs 324 on the computing machine (Step 356).
- the encoded multimedia data is then transmitted 358 over the virtual channel 306 (Step 358) to a remote computing machine.
- Modules executing on the remote computing machine received the transmitted multimedia data and decode the data using codecs corresponding to those codecs used to encode the multimedia data (Step 360).
- a proxy module streams the multimedia data from the application that received the multimedia data (Step 362) and then transmits the multimedia data to a display application (Step 364).
- the display application receives the streamed multimedia data and presents the multimedia data to a user.
- the application/desktop delivery application can then capture the presented data and send it back to the second computing machine 300 (Step 366).
- a streaming module 310 or KSProxy filter 340 streams multimedia data from a multimedia driver 316, 316' that captures multimedia data generated by a multimedia device 302, 302'.
- the streaming module 310 is a KSProxy filter 340, while in other embodiments the streaming module is any application that can stream multimedia data directly from a multimedia driver 316, 316'.
- the streaming module 310 in some embodiments, executes at the kernel level. In other embodiments, the streaming module 301 can execute at the user level.
- a module executing within the DIRECTSHOW framework 320, 320' intercepts the multimedia data generated by the multimedia device 302, 302' (Step 354).
- this module can be a streaming module 310 executing on a computing machine.
- this module can be a proxy streaming module 308 executing on a computing machine.
- the module in some embodiments, can intercept the multimedia data by streaming the multimedia data directly from a device driver 316, 316'. In other embodiments, the module can intercept the multimedia data by streaming the data from an
- an encoder 342 or transformation filter 326 encodes the multimedia data (Step 356) prior to transmission. Encoding can include compressing, transforming or otherwise encrypting the multimedia data. In some embodiments, the encoder 342 or transformation filter 326 can encode the multimedia data using any encoding scheme. In other embodiments, the encoder 342 or transformation filter 326 can encode the multimedia data using any encryption algorithm, compressing algorithm or encoding algorithm described herein. While Figure 4 illustrates a process 350 that includes encoding the multimedia data stream, in some embodiments, the process 350 does not include an encoding step during which an encoder encodes the multimedia data.
- the computing machine in some embodiments, can then transmit the encoded multimedia data over the virtual channel 306 to a remote computing machine (Step 358).
- the application/desktop delivery system 322 transmits the encoded multimedia data over the virtual channel 306.
- the multimedia data is transmitted over a virtual channel 306 dedicated to a user of the computing machine.
- modules executing on the remote computing machine receive the transmitted multimedia data and decode the data using codecs corresponding to those codecs used to encode the multimedia data (Step 360).
- the codecs in some embodiments, can be transmitted along with the multimedia data.
- the remote computing machine can request the codecs from the local computing machine.
- a decoder or other decoding application can decode the encoded multimedia data. While Figure 4 illustrates a process 350 whereby the multimedia data is decoded, in some embodiments the multimedia data is not encoded and therefore need not be decoded.
- a streaming module 310 or proxy streaming module 308 streams the multimedia data from the receiving application (Step 362).
- the proxy streaming module 308 streams the multimedia data from the application/desktop delivery system 322'.
- the streaming module 310 streams the multimedia data from the application/desktop delivery system 322.
- the module 310, 308 can stream the multimedia data into the DIRECTSHOW framework 320, 320'.
- the multimedia data can be further processed by filters or applications executing within the framework 320, 320' subsequent to streaming the multimedia data into the framework 320, 320'.
- the proxy streaming module 308 can forward the multimedia display to a display application for display (Step 364).
- a display application in some embodiments, can be any display application able to display, play or present multimedia data. In other embodiments, a display application can be any display application described herein.
- Forwarding the multimedia data to the display application can include streaming, transmitting or otherwise sending the display data to the display application. In some embodiments, forwarding the multimedia data can include formatting the multimedia data using one or more filters included in the DIRECTSHOW framework 320, 320'. In other embodiments, forwarding the multimedia data can include formatting the multimedia data according to one or more filters of the display application.
- the display application displays, plays or presents the multimedia data upon receiving it from the proxy streaming module 308.
- the display application in other embodiments, can display and/or present the multimedia data on a display device communicating with the computing machine on which the display application executes. In other embodiments, the display application plays the multimedia data on speakers communicating with the computing machine on which the display application executes.
- a remoting application intercepts the application output generated by the display application, and sends the display data of the displayed or played multimedia data to a second computing machine or client (Step 366). Intercepting the application output can include intercepting graphics generated by the display application, audio data generated by the display application, and or video generated by the display application.
- the remoting application is a remote access client such as XEN APPLICATION, OR XEN DESKTOP manufactured by CITRX SYSTEMS, INC.
- the remoting application can be any client that uses a presentation level protocol to intercept application output and transmit that output to remote computers or devices.
- a multimedia device 302, 302' generates multimedia data (Step 502) and a multimedia device driver 316, 316' retrieves or receives the multimedia data from the multimedia device (Step 504).
- a streaming module 310 executing on a computing machine streams or passes the multimedia data from the multimedia device driver 316 (Step 506) to one or more filers and/or modules executing within a DIRECTSHOW framework 320, 320' (Step 508).
- An application/desktop delivery system 322, 322' or remoting application then intercepts the multimedia data (Step 510) and transmits the multimedia data over a virtual channel 306 (Step 512).
- a multimedia device 302, 302' generates multimedia data (Step 502).
- the multimedia device 302, 302' can be any multimedia device, while in some embodiments the multimedia device 302, 302' can be any multimedia device described herein.
- the multimedia device 302, 302' can be connected to or communicating with a second computing machine 300.
- the multimedia device 302, 302' can be connected to a first computing machine 301, or any other computing machine or computing machine.
- the multimedia device 302, 302' generates multimedia data by recording video, graphics or audio.
- the multimedia device 302, 302' can be a video or web camera that can capture an image of a scene over a period of time. Capturing the scene can include generating a video and/or an audio representation of events that occur within the scene during a period of time.
- the period of time in some embodiments, is a period of time the device 302, 302' is set to record the events that occur within the scene.
- the scene in some embodiments, can be an area able to be viewed or heard by a lens and/or a microphone of the device 302, 302'.
- the device 302, 302' can record the movements and speech of a person who talks in front of the device's lens and microphone.
- a multimedia device driver 316, 316' executing on the computing machine can retrieve the multimedia data from the multimedia device 302, 302'(Step 504).
- Retrieving the multimedia data in some embodiments, can include receiving data transmitted by the multimedia device 302, 302' to the device driver 316, 316'.
- the multimedia device 302, 302' can generate multimedia data and substantially at the same time transmit that multimedia data over a connection between the device 302, 302' and the computing machine, to the computing machine.
- the multimedia device driver 316, 316' can receive the raw output from the multimedia device 302, 302' and translate it into commands able to be to interpreted by the operating system, the DIRECTSHOW Framework 320, 320' and other applications executing on the computing machine.
- the multimedia device 302, 302' can transmit, stream or otherwise send generated multimedia data to the computing machine where the multimedia data is stored in memory. The multimedia device driver 316, 316' can then retrieve the multimedia data from the memory element.
- a streaming module 310 executing on the computing machine can, in some embodiments, stream the multimedia data from the multimedia device driver 316, 316' (Step 506). While Figure 5 A illustrates a method 500 where the streaming module 310 streams data from the multimedia device driver 316, 316', in some embodiments the streaming module 310 can stream the multimedia data directly from the multimedia device 302, 302'. In other embodiments, the streaming module 310 can stream the multimedia data from the driver 316, 316' at the kernel level rather than at the user level.
- the streaming module 310 executes within the DIRECTSHOW Framework 320, 320' on the computing machine. Thus, the streaming module 310 can stream multimedia data from the multimedia device driver 316, 316' into the framework 320, 320'.
- the streaming module 310 streams the multimedia data to other filters or modules executing within the DIRECTSHOW Framework 320, 320' (Step 508). Streaming the multimedia data to the filters and modules can include passing, transmitting or otherwise sending the multimedia data to the filters and modules. In some embodiments, the streaming module 310 can stream the multimedia data to a filter or module in response to a request from a filter or module for the multimedia data. In one embodiment, the streaming module 310 can stream the multimedia data from the multimedia device driver 316 to any transform filter 326, 326', Tenderer filter 328, 328', and/or video/application sink filter.
- the streaming module 310 in some embodiments, in a source filter 330 such that the streaming module 310 streams multimedia data from the multimedia device driver 316, 316' and provides the streamed multimedia data to any filter or module that wants or tries to access the multimedia data.
- the streaming module 310 streams the multimedia data to filters and modules executing in the framework 320, 320' and associated with the display application 314, 314'.
- a remoting application in some embodiments, can intercept the multimedia data at any time after it is streamed from the multimedia device 302, 302' (Step 510).
- the remoting application can be any remoting application described herein.
- the remoting application can be the application/desktop delivery system 332.
- the remoting application can be any application, thin client or program that executes on the computing machine and that uses a presentation level protocol to transmit application data to a remote computing machine. Intercepting the multimedia data, in some embodiments, can include hooking into the
- the remoting application can extract the multimedia data included in the call and transmit the multimedia data to a remote computing machine.
- the remoting application can be a module or program that executes within an application/desktop delivery system 322, 322'.
- the module or program can register as a display application such that each time multimedia data is processed by the DIRECTSHOW Framework 320, 320', the module or program receives the processed multimedia data instead of other display applications executing on the computing machine.
- the remoting application can receive or capture the multimedia data.
- the remoting application transmits the multimedia data over a virtual channel 306 to a remote computing machine (Step 512).
- the remoting application can encapsulate or modify the multimedia data using a remote presentation protocol tailored towards transmitting multimedia data.
- the delivery system 322, 322' can then transmit the encapsulated data to a remote computing machine.
- Figure 5A illustrates a method 500 where the multimedia data is transmitted to one remote computing machine
- the multimedia data can be transmitted to one or more remote computing machines communicating with the computing machine via a virtual channel 306.
- the method 500 can further include a step of locally display the captured or intercepted multimedia data.
- the display preview application 323, 323' executing on the second computing machine 300 can receive and process the multimedia data. Instead of transmitting the multimedia data over a virtual channel 306 (Step 512), the display preview application 323, 323' can query the This display preview application 323, 323 ' can query a remote computing machine for control information related to a display application, and using this information the display preview application 323, 323' can generate an application output window. The display preview application 323, 323' can then display the multimedia data in the application output window. Displaying the multimedia data can also include playing audio data included in the multimedia data.
- the remoting application or any application or program executing on a computing machine to transmit the intercepted multimedia data over a virtual channel can perform one or more quality control operations.
- a quality control operation can include determining an amount of available bandwidth on the virtual channel 306, and varying the transmission rate according to the determined bandwidth. For example, when there is a low amount of available bandwidth, the program may transmit multimedia data over the virtual channel 306 at a slower rate in order to achieve optimal throughput and maximum quality.
- a quality control operation can include sending a command to a streaming module 310, video filter 312 or any other program to sample the multimedia data from the multimedia device 302 at a slower rate. In one embodiment, this command can include data indicating an amount of available bandwidth. In other embodiments, this command can include a sample rate determined based on an amount of available bandwidth.
- a proxy streaming module 308 executing within the DIRECTSHOW Framework 320, 320' of a remote computing machine streams multimedia data from a virtual channel 306 (Step 532) and decodes the streamed multimedia data (Step 534).
- the proxy streaming module 308 then passes the multimedia data to other filters and modules within the DIRECTSHOW Framework 320, 320' (Step 536).
- a filter or module within the DIRECTSHOW Framework 320, 320' then forwards the multimedia data to a display application 314, 314' (Step 538) which displays the multimedia data (Step 540).
- a remoting application executing on the remote computing machine intercepts the output generated by the display application (Step 542).
- a proxy streaming module 308 streams multimedia data from the virtual channel 306 (Step 532).
- the proxy streaming module 308 executes on a remote computing machine, and streams the multimedia data from a virtual channel 306 between a computing machine and the remote computing machine.
- the multimedia data in some embodiments, is generated by a multimedia device communicating with the computing machine, transmitted over the virtual channel 306 to the remote computing machine, and streamed from the virtual channel 306 by the proxy streaming module 308.
- an application/desktop delivery system 322, 322' or remoting application receives the multimedia data transmitted over the virtual channel 306 and the proxy streaming module 308 streams the multimedia data from the application/desktop delivery system 322, 322'.
- the proxy streaming module 308 can be a part of the
- the multimedia data stream is decoded (Step 534).
- the proxy streaming module 308 in some embodiments, decodes the multimedia data stream, while in other embodiments the application/desktop delivery system 322, 322' or remoting application decodes the multimedia data stream.
- the multimedia data stream is decoded using codecs 324, 324'.
- the proxy streaming module 308 passes or streams the multimedia data to other filters or modules executing within the
- Passing the multimedia data to the filters and modules can include streaming, transmitting or otherwise sending the multimedia data to the filters and modules.
- the proxy streaming module 308 can stream the multimedia data to a filter or module in response to a request from a filter or module for the multimedia data.
- the proxy streaming module 308 can stream the multimedia data to any transform filter 326, 326', Tenderer filter 328, 328', and/or video/application sink filter.
- the proxy streaming module 308 in some embodiments, in a source filter 330 such that the proxy streaming module 308 streams multimedia data to any filter or module that wants or tries to access the multimedia data.
- the proxy streaming module 308 streams the multimedia data to filters and modules executing in the framework 320, 320' and associated with the display application 314, 314'.
- the multimedia data is forwarded to the display application (Step 538) and displayed by the display application (Step 540).
- Displaying the data can include playing, displaying or otherwise presenting the multimedia data.
- displaying the multimedia data can include displaying video chat or web camera output within a display application such as Google Talk ®, Microsoft Communicator ®, Instant Messenger ®, or any other display application.
- the remoting application or application/desktop delivery system 322, 322' can then intercept the display application output and transmit it back to the computing machine (Step 542).
- the remoting application can intercept output generated by Google Talk ®, Microsoft Communicator ®, Instant Messenger ®, or any other display application and transmit that output to the computing machine.
- the computing machine can display the output in an application window and/or play the output using one or more audio
- the method 530 may not include intercepting the output generated by the display application (Step 542). Instead, in this embodiment the display application merely displays the multimedia data (Step 540). Displaying the multimedia data can include playing audio, display video or any other display or presentation of the multimedia data.
- the second computing machine 300 (or the local computing machine communicating with the multimedia device) merely supplies the first computing machine 301 (or the remote computing machine executing the display application) with multimedia data. The first computing machine 301 displays the multimedia data and does not transmit application output to the second computing machine 300 for display on a display device connected to the second computing machine 300.
- the present system may, in some embodiments, be referred to as Helios, reverse RAVE or a bi-directional RAVE system.
- the system may be used for providing video conferencing solutions to users using remote XenApp & XenDesktop machines.
- the system may include various functions and features, such as codecs and protocols for communicating data and instructions between the client and the servers.
- the system may support Microsoft Office Communicator (OCS) video and audio communications between the client and the server.
- OCS Microsoft Office Communicator
- the present system supports conferencing applications like Skype and GoogleTalk.
- a client using a locally connected camera may use the instant messenger video conferencing that executes on a remote server.
- the system may be designed to be functional on multiple platforms and operating systems, including all the operating systems and platforms described herein.
- a client 102A also referred to as an ICA client, may be connected to a capture device, such as a microphone or a video camera. The capture device may capture and transmit to the client raw audio and/or video data.
- Client 102A may be interested in communicating with another client 102B via an audio and/or video conferencing application executing on the server 106.
- Client 102 may not have or use a locally executing multimedia videoconferencing application and may wish to use such an application executing on the server 106.
- the server 106 may host any number of audio and/or video conferencing applications, such as, for example, the Microsoft Office Communicator, Skype, GoogleTalk, Yahoo instant messenger, MSG instant messenger or any other type and form of application enabling audio and/or video conferencing.
- Clients 102A and 102B may establish a virtual channel with the server 106 in order to commence the audio/video conference.
- the capture device such as the webcam connected to client 102A, may generate an audio/video stream.
- the audio/video data may be generated in any number of formats.
- the ICA client 102 may pick a particular native format of the audio/video data and may compress the data in order to transmit it over the ICA to the server 106.
- the server 106 may decompress the data and deliver the data to the hosted application.
- the application may then process the data and take care of the jitter and conversion of the format as it would normally do with any data from a locally connected capture device.
- the multimedia application may load Citrix webcam filter which may interface with the multimedia application in a manner in which it appears to the application as a real capture device, or a camera.
- the filter may communicate with the application and appear as a standard capture device, thus enabling the application to receive the raw data from the remote capture device via the filter while treating the data as if received from a local capture device.
- the application may treat the filter which delivers the data from the remote client 102, as the standard, locally connected capture device or a camera. The application may therefore receive the raw data and process the data as if the data was generated from a local video camera, based on the settings by which the application believes that the filter provided is in fact the camera itself.
- FIG. 5D an embodiment of a system that uses one or more custom made filters to enable a multimedia application on a remote server to process audio/video data generated by a capture device connected to a client machine is illustrated.
- the system may leverage DirectShow technology as a part of the Microsoft Windows operating system in order to intercept video from the capture devices, transcode and compress the video is illustrated.
- manufacturers of capture devices may supply drivers that plugin to Microsoft's Kernel Streaming architecture.
- DirectShow may include a layer that sits on top of the Kernel Streaming layer that may exchange data between applications and the capture devices.
- Such a DirectShow layer may be implemented as a normal filter that interacts with the multimedia topology much the same way other filters do.
- a custom made filter may be designed and provided to replace the functionality of the DirectShow capture filter.
- DirectShow may provide an API for implementing this feature, so no hooks may be needed.
- the custom filter may be registered under capture devices category, thus making the filter available to the system device enumerator.
- hosted capture applications enumerate the capture devices, such as the cameras or microphones
- these hosted capture applications may instantiate the system device enumerator and then enumerate the capture devices category.
- the new capture filter may become visible to the hosted applications as just another webcam.
- the users may only need to select a selection option associated with the filter in the application options.
- the filter may communicate with the client over the ICA protocol.
- capture devices may provide several different formats that capture applications can choose from.
- DirectShow may provide an interface, such as IAMStreamConfig interface, which applications may use to select the output format of the capture device.
- the new capture filter may implement this interface and return raw formats that are compatible with the codec being used for the system.
- the ICA client may then create a filter graph and loads the capture device along with a video sink filter.
- the video sink filter may be designed to receive the video from the capture device and to communicate with the host-side capture filter over ICA.
- Such a filter may be visible to the applications as just another webcam, so users only need to select it in the application options. Once this webcam filter is loaded by the application, it may communicate with the client over the ICA protocol.
- FIG. 5D illustrates an embodiment of system in which a number of filters of the system are illustrated.
- the system supplies 3 filters that plug- in to the DirectShow filtergraph (pipeline). These filters may include the Citrix interceptor filter, the Citrix receiver filter and the aforementioned webcam capture filter. The rest of the filters illustrated in FIG. 5D may be standard Microsoft or third-party components.
- a webcam filter gets loaded by the application on the server, this filter may send a message to the client to create a filtergraph and load the real webcam along with the encoder and interceptor filter.
- the application begins requesting data from the webcam filter, the request may be forwarded to the client, which then may start pushing media samples upstream.
- the system may include functionality for audio and video synchronization to ensure that the audio and video streams delivered are in sync.
- the application such as the Microsoft Office Communicator, Skype or GoogleTalk may render the video incoming to the server using a graphics API, which is then remoted as a graphics virtual channel over ICA.
- the incoming as well as outgoing stream may be compressed.
- the multimedia incoming stream comprising audio and/or video, may be remoted over forward (host to client) RAVE, or the Multimedia Acceleration Virtual Channel.
- the client may then decompresses and render the multimedia stream.
- the multimedia stream gets rendered in the host session.
- the data is delivered as raw or PCM format to the Citrix Remote audio endpoint component in the remote session.
- the Citrix Remote audio endpoint might compress it before sending it over an audio Virtual Channel to the ICA client.
- the ICA client then may decompress the audio data and render it.
- the ICA engine may be referred to as a high level abstraction of the process operations on a client side with respect to the data capture and transmission to the server 106.
- a driver of a capture device such as a Webcam KS Driver operates in the kernel space of the client 102.
- Platform abstraction layer may be responsible for setting up the DirectShow pipeline and loading the actual webcam components, such as for example the data generated by the capture device.
- the webcam KS driver may communicate with KsProxy DirectShow Filter, which may then forward the data to the intermediate conversion filters which finally forward the data to Citrix Video Sink DirectShow filter, which may also be known as CtxDSSink.dll.
- the data may then be forwarded to Protocol Link, also referred to as ClientMMLink.dll.
- the protocol link component may abstract the virtual channel implementation details from the video sink filter.
- the protocol link may also take care of the common tasks, such as for example compression.
- One or more codecs may be used for compression or decompression. In some embodiments, system may use Theora codec for compression and/or decompression.
- the data may be forwarded to the virtual channel, which may also be referred to as vdmm.dll.
- Virtual channel may perform stream aggregation and session management. Virtual channel may also maintain communication with the server 106, such as the
- XenApp/XenDesktop server host using an extended version of RAVE ICA protocol.
- the data may then be transmitted over to the server 106.
- the system may compress the video on the client before sending it over the network. Compression may be needed in order to reduce the amount of bandwidth consumed by the video. For example, 30 frames per second of raw RGB24 data at CIF 352 x 288 resolution may require about 72990720 bps (almost 73 Mbps!). Clients may consume less CPU cycles compressing the video and then sending it over the network compared to just sending huge amounts of
- the codec used may be any codec for processing audio and/or video data.
- the codec comprises source code that is editable in order to modify it and incorporate it into the filters/sinks for each multimedia platform.
- the coded may also need it to be able to control the encoder settings in real-time.
- the codec does not have any hard dependencies on the platform which would preclude us from incorporating it into other platforms, such as for example Windows, Linux or Macintosh.
- the codec is fast enough to process at least 30 frames per second or more.
- the codec processes at any speed, such as 10 frames per second, 20 frames per second, 30 frames per second, 40 frames per second, 50 frames per second, 60 frames per second, 70, frames per second, 80 frames per second, 90 frames per second or 100 frames per second. In further embodiments, the codec processes data at more than 100 frames per second, such as 110, 120, 150, 180, 200 and up 500 frames per second.
- the video may be received from the virtual channel implemented in mmvd.dll.
- the virtual channel may be hosted in the Wfshell.exe process.
- the virtual channel may perform stream aggregation and session management and may communicate with the client using the RAVE ICA protocol.
- Data may be transmitted up to the protocol marshalling interface.
- the protocol marshalling interface may be a layer that communicates with the virtual channel using a named pipe.
- Codec may receive data and may compress or decompress the data using Theora functionality.
- the data may be forwarded to the protocol link, also referred to as HostMMLink.dll.
- the protocol link component may abstract the virtual channel implementation details from the video sink filter and take care of the compression/decompression, as necessary.
- Data may be received and processed by Citrix Webcam DirectShow filter, which may forward it to the intermediate conversion filters, which may then forward the data to application sink filter.
- Application which had already taken care of the DirectShow pipeline and setup the webcam filter, may receive the data, and display it.
- the protocol sequence may be designed to minimize the protocol related code inside of the filters.
- the filters may be modified to handle things that are specific to the multimedia platform, while being as independent from the protocol sequence as possible. All other common functionality may be abstracted out of the filters and put in a common component. Such feature may make it easier to port the filters to other multimedia frameworks, such as, for example Windows Media Foundation and GStreamer.
- the system may re-use and leverage the existing RAVE virtual channel instead of creating a new one.
- an existing ICA virtual channel system may provide any available virtual channels via which the present system may operate.
- a host such as a server 106
- the client may transmit create stream response back to the server 106.
- the client may create topology and load video sink filter.
- the host server may load the capture filter, such as the customized filter which the application treats as a capture device.
- Application on the host may start the capture.
- the instruction to play may be transmitted from the host server to the client.
- the client may produce the samples using the capture device and transmit the samples to the host server.
- the host server may deliver the samples to the application.
- the application mistaking the capture filter for the capture device will operate as if the data was being delivered from a locally connected capture device.
- Once the application sends the instruction to stop the capture the client tears down the topology.
- the host server may transmit the request to delete the stream request.
- the host may also unload the capture filter.
- the client machine can directly (locally) display the multimedia data captured by the local multimedia device in an application output window.
- This operation may be referred to as a local preview and may be an optimization compared to streaming the data from the server machine back to the client machine where it originated.
- the client machine may need to be made aware of where and how to display the data, such as for example the parent app window, preview window size, position, in-remote-session clipping.
- the client machine may receive control information instead of the multimedia data from the presentation application running on the server machine.
- the multimedia data captured by a local multimedia capture device on a first client may be processed, rendered and displayed locally on the first client directly, while only the application output multimedia data from a second client processed by the presentation application on the remote server is received from the remote server.
- the second client may also process and display the locally generated multimedia data on the second client, while receiving from the remote server the multimedia application output data of the first client from the remote server.
- both clients can help save the bandwidth by not requiring the remote server to transmit back the locally generated multimedia stream but instead using the locally generated multimedia to display it locally while relying on the remote server only to deliver the multimedia output data of the multimedia stream generated from the other remote client.
- Both host server and client machines may be configured to be senders as well as receivers of commands or instructions needed to transmit and/or process the multimedia data.
- virtual channel protocol communication between the client and the host server which may also be referred to as RAVE VC or RAVE virtual channel, may be negotiated in order to facilitate the transmission of multimedia data between the client and server.
- Client to host (or client to host server) media streams or transmissions may be supported based on a version of protocol or a type of settings.
- Client-to-host (C2H) media streams may allow for selection of a specific capture device by the host, negotiation of native media types (including resolution) supported by the device, and negotiation of optional RAVE VC-level compression media types, such as the Theora codec.
- the Multimedia Virtual Channel Protocol may comprise the support for C2H (capture) media streams, which may be known as reverse-RAVE, which may also be negotiated by means of the
- CTXMM_TYPE_CAPABILITY_C2H_MEDIA_STREAMS capability The exact media stream type may be determined by the type of media property elements supplied in the MMVD_CMD_CREATE_CONTEXT_REQUEST command.
- the client may notify the host of media capture devices over the Control Virtual Channel (VC) by means of Plug-and-Play (PnP) notifications.
- the Control VC may supply a media capture device description and a non-empty list of native media Types supported by the device.
- the capture resolution may be defined. For example, for clients that do not implement this capability over the Control VC, the host may assume the presence of one default Device with default (uncompressed/raw) native Type and default resolution.
- the host server, as well as the presentation application may select a specific media device, and also requests the usage of a specific native media type to be used with the selected device, plus the compression media types supported by the host that can be used over the RAVE VC, listed in order of decreasing preference.
- the client may decide whether to use compression over the RAVE VC itself by choosing one of the media types supplied by the host.
- Any compression type may be used for compressing the data.
- the compression type capability negotiation may be implemented on a per stream context basis, as opposed to globally for the ICA session in order to provide more flexibility.
- the RAVE VC might encode directly in a media type format that the host presentation application expects and avoid trans-coding over the VC.
- Plug and play notifications may be transmitted between the client and the hosting server in order to enable the server to treat a filter or a function associated with the client and executing on the server as a multimedia capture device.
- a plug and play notification is transmitted from the client to the server via a virtual channel.
- an agent such as a filter associated with multimedia communication between the server and the client, may receive the plug and play notification and pass it on to the relevant presentation application.
- the application may process the plug and play notification and identify the agent or the function executing on the server as a particular capture device, such as a web camera or a microphone locally connected to the server.
- the agent may only represent a function for the hosted application's communication with a capture device, such as a web camera connected to the client.
- the application may then communicate with the agent as if it is a multimedia capture device.
- the agent may forward the communcation, instructions and data, as necessary, between the client and the presentation application.
- the hosted application may process the data from the remote capture device, as necessary. Plug and play features are described in U.S. Patent No.: 7,702,750, entitled “System and Method for Event Detection and Re-direction over a Network using a Presentation Level Protocol", filed on September 29, 2004 and U.S. Patent Application No.: 12/763,492, entitled “System and Method for Event Detection and Re-direction over a Network Using a Presentation Level Protocol", filed on April 20, 2010, both of which are
- Endpoint A in the virtualized session on the server
- Endpoint A' may only serve as a proxy for hand- shake and control with Endpoint B, but the video is channeled from the local web cam (Endpoint A) directly to Endpoint B.
- Endpoint A may be a local Web Cam on the local client machine.
- Endpoint A' may be referred to as the applcation in the virtualized/remote session.
- Endpoint B may include the entity on the other side of the video/audio communication.
- the client side video sampling rate may vary based on bandwidth availability in order to achieve optimal throughput and maximum possible quality.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP10836637.8A EP2510454B1 (fr) | 2009-12-08 | 2010-12-08 | Systèmes et procédés destinés à une présentation distante côté client d'un flux multimédia |
CN201080063306.1A CN102741830B (zh) | 2009-12-08 | 2010-12-08 | 用于多媒体流的客户机侧远程呈现的***和方法 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US26768609P | 2009-12-08 | 2009-12-08 | |
US61/267,686 | 2009-12-08 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2011072049A2 true WO2011072049A2 (fr) | 2011-06-16 |
WO2011072049A3 WO2011072049A3 (fr) | 2011-10-13 |
Family
ID=44083116
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2010/059526 WO2011072049A2 (fr) | 2009-12-08 | 2010-12-08 | Systèmes et procédés destinés à une présentation distante côté client d'un flux multimédia |
Country Status (4)
Country | Link |
---|---|
US (2) | US9191425B2 (fr) |
EP (1) | EP2510454B1 (fr) |
CN (1) | CN102741830B (fr) |
WO (1) | WO2011072049A2 (fr) |
Families Citing this family (69)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1232610B1 (fr) | 1999-10-22 | 2009-01-07 | Nomadix, Inc. | Systemes et procedes de gestion dynamique de largeur de bande par abonne dans un reseau de communications |
JP5292297B2 (ja) | 2006-09-29 | 2013-09-18 | ノマディックス インコーポレイテッド | コンテンツを挿入するためのシステムおよび方法 |
ES2741532T3 (es) * | 2008-12-23 | 2020-02-11 | Ericsson Telefon Ab L M | Distribución de elementos de contenido a dispositivos de usuario en un entorno móvil |
TWI444044B (zh) * | 2009-03-31 | 2014-07-01 | Ibm | 遠距應用中傳輸資料之裝置、方法與電腦程式產品 |
US20110030037A1 (en) | 2009-07-07 | 2011-02-03 | Vadim Olshansky | Zone migration in network access |
US8140702B2 (en) * | 2009-12-28 | 2012-03-20 | General Instrument Corporation | System and method of maximizing utility in media delivery network |
US8799362B2 (en) * | 2010-03-09 | 2014-08-05 | Avistar Communications Corporation | Scalable high-performance interactive real-time media architectures for virtual desktop environments |
US9373152B2 (en) | 2010-06-17 | 2016-06-21 | Thinci, Inc. | Processing of graphics data of a server system for transmission including multiple rendering passes |
US8754900B2 (en) | 2010-06-17 | 2014-06-17 | Thinci, Inc. | Processing of graphics data of a server system for transmission |
US20120057633A1 (en) * | 2010-09-02 | 2012-03-08 | Fang Shi | Video Classification Systems and Methods |
US9237363B2 (en) * | 2011-02-12 | 2016-01-12 | Openwave Mobility, Inc. | Dynamic injection of metadata into flash video |
TW201249135A (en) * | 2011-05-30 | 2012-12-01 | Wistron Corp | Remote management method and remote management system |
US9712847B2 (en) * | 2011-09-20 | 2017-07-18 | Microsoft Technology Licensing, Llc | Low-complexity remote presentation session encoder using subsampling in color conversion space |
US20130083210A1 (en) * | 2011-09-30 | 2013-04-04 | Successfactors, Inc. | Screen and webcam video capture techniques |
US8872883B2 (en) * | 2011-11-02 | 2014-10-28 | Octavio Cesar Silva | ViSTORE |
TWI451268B (zh) | 2011-11-08 | 2014-09-01 | Inst Information Industry | 提供虛擬桌面之雲端系統、於虛擬桌面播放多媒體之方法以及其電腦可讀取記錄媒體 |
KR101467430B1 (ko) * | 2011-12-12 | 2014-12-01 | 주식회사 케이티 | 클라우드 컴퓨팅 기반 어플리케이션 제공 방법 및 시스템 |
US20130254417A1 (en) * | 2012-03-21 | 2013-09-26 | Jason Nicholls | System method device for streaming video |
US9092090B2 (en) * | 2012-05-17 | 2015-07-28 | Hong Kong Applied Science And Technology Research Institute Co., Ltd. | Structured light for touch or gesture detection |
US9325676B2 (en) * | 2012-05-24 | 2016-04-26 | Ip Ghoster, Inc. | Systems and methods for protecting communications between nodes |
US10778659B2 (en) | 2012-05-24 | 2020-09-15 | Smart Security Systems Llc | System and method for protecting communications |
US9218526B2 (en) | 2012-05-24 | 2015-12-22 | HJ Laboratories, LLC | Apparatus and method to detect a paper document using one or more sensors |
US20140122544A1 (en) * | 2012-06-28 | 2014-05-01 | Transoft Technology, Inc. | File wrapper supporting virtual paths and conditional logic |
US9626450B2 (en) | 2012-06-29 | 2017-04-18 | Dell Products L.P. | Flash redirection with browser calls caching |
US9489471B2 (en) | 2012-06-29 | 2016-11-08 | Dell Products L.P. | Flash redirection with caching |
US9354764B2 (en) | 2012-06-29 | 2016-05-31 | Dell Products L.P. | Playback of flash content at a client by redirecting execution of a script by a flash redirection plugin at a server to a flash redirection browser at the client |
CN103684970B (zh) * | 2012-09-05 | 2018-04-27 | 中兴通讯股份有限公司 | 媒体数据流的传输方法和瘦终端 |
US10171540B2 (en) * | 2012-09-07 | 2019-01-01 | High Sec Labs Ltd | Method and apparatus for streaming video security |
KR101720654B1 (ko) * | 2012-09-14 | 2017-03-28 | 한국전자통신연구원 | 가상화 시스템 및 가상화 서비스 제공방법 |
US9444862B2 (en) * | 2012-09-29 | 2016-09-13 | Intel Corporation | Dynamic media content output for mobile devices |
WO2014100384A1 (fr) * | 2012-12-19 | 2014-06-26 | Rabbit, Inc. | Système et procédé de diffusion en continu audio-vidéo |
WO2014100374A2 (fr) * | 2012-12-19 | 2014-06-26 | Rabbit, Inc. | Procédé et système de partage et de découverte de contenu |
US9369670B2 (en) | 2012-12-19 | 2016-06-14 | Rabbit, Inc. | Audio video streaming system and method |
US20140196038A1 (en) | 2013-01-08 | 2014-07-10 | Commvault Systems, Inc. | Virtual machine management in a data storage system |
US9363472B2 (en) | 2013-03-14 | 2016-06-07 | Samsung Electronics Co., Ltd. | Video injection for video communication |
US9124668B2 (en) | 2013-05-20 | 2015-09-01 | Citrix Systems, Inc. | Multimedia redirection in a virtualized environment using a proxy server |
US9398057B2 (en) | 2013-06-04 | 2016-07-19 | Dropbox, Inc. | System and method for group participation in a digital media presentation |
US9407728B2 (en) | 2013-11-08 | 2016-08-02 | Dropbox, Inc. | Content item presentation system |
US10382595B2 (en) | 2014-01-29 | 2019-08-13 | Smart Security Systems Llc | Systems and methods for protecting communications |
US20150312444A1 (en) * | 2014-04-29 | 2015-10-29 | Xiaomi Inc. | Method and device for synchronizing photograph |
CN105323534B (zh) * | 2014-07-14 | 2019-04-23 | 深圳市潮流网络技术有限公司 | 一种第三方应用的会议处理方法和通讯设备 |
US20160019317A1 (en) | 2014-07-16 | 2016-01-21 | Commvault Systems, Inc. | Volume or virtual machine level backup and generating placeholders for virtual machine files |
TWI554089B (zh) * | 2014-09-29 | 2016-10-11 | 緯創資通股份有限公司 | 影音分享方法與系統 |
US10355964B2 (en) | 2014-10-31 | 2019-07-16 | At&T Intellectual Property I, L.P. | Method and system to capture selected network data |
US9983936B2 (en) | 2014-11-20 | 2018-05-29 | Commvault Systems, Inc. | Virtual machine change block tracking |
CN104461698A (zh) * | 2014-12-29 | 2015-03-25 | 成都致云科技有限公司 | 虚拟磁盘动态挂载的方法、管理装置及分布式存储*** |
US10565067B2 (en) | 2016-03-09 | 2020-02-18 | Commvault Systems, Inc. | Virtual server cloud file system for virtual machine backup from cloud operations |
EP3422184A4 (fr) * | 2016-03-25 | 2019-03-20 | Huawei Technologies Co., Ltd. | Procédé et dispositif de gestion d'application dans un environnement de virtualisation de fonction de réseau |
US10912090B2 (en) * | 2016-04-10 | 2021-02-02 | Lg Electronics Inc. | Method and device for transmitting uplink reference signal in wireless communication system |
US10678758B2 (en) | 2016-11-21 | 2020-06-09 | Commvault Systems, Inc. | Cross-platform virtual machine data and memory backup and replication |
US10896100B2 (en) | 2017-03-24 | 2021-01-19 | Commvault Systems, Inc. | Buffered virtual machine replication |
US10387073B2 (en) | 2017-03-29 | 2019-08-20 | Commvault Systems, Inc. | External dynamic virtual machine synchronization |
US11394731B2 (en) * | 2017-05-16 | 2022-07-19 | Citrix Systems, Inc. | Computer system providing anomaly detection within a virtual computing sessions and related methods |
US10441885B2 (en) | 2017-06-12 | 2019-10-15 | Microsoft Technology Licensing, Llc | Audio balancing for multi-source audiovisual streaming |
US10545625B2 (en) * | 2017-09-11 | 2020-01-28 | Citrix Systems, Inc. | Redirection of web content |
US11194930B2 (en) | 2018-04-27 | 2021-12-07 | Datatrendz, Llc | Unobtrusive systems and methods for collecting, processing and securing information transmitted over a network |
AU2019262610B2 (en) | 2018-05-04 | 2022-03-03 | Citrix Systems, Inc. | Computer system providing hierarchical display remoting optimized with user and system hints and related methods |
US11824785B2 (en) | 2018-05-04 | 2023-11-21 | Citrix Systems, Inc. | Systems and methods for remote computing session display based upon user input event prioritization |
US10855755B2 (en) | 2018-05-04 | 2020-12-01 | Citrix Systems, Inc. | WebRTC API redirection with fallbacks |
CN118116572A (zh) * | 2018-05-18 | 2024-05-31 | 科林达斯公司 | 用于机器人介入过程的远程通信和控制*** |
US10819817B2 (en) * | 2019-02-04 | 2020-10-27 | Dell Products L.P. | HTML5 multimedia redirection |
US11868819B2 (en) * | 2019-11-01 | 2024-01-09 | Grass Valley Limited | System and method for constructing filter graph-based media processing pipelines in a browser |
WO2021120088A1 (fr) * | 2019-12-19 | 2021-06-24 | 威创集团股份有限公司 | Appareil de traitement de signaux pour affichage à juxtaposition et procédé de traitement de signaux associé |
US11687513B2 (en) | 2020-05-26 | 2023-06-27 | Molecula Corp. | Virtual data source manager of data virtualization-based architecture |
US11960616B2 (en) | 2020-05-26 | 2024-04-16 | Molecula Corp. | Virtual data sources of data virtualization-based architecture |
US11263026B2 (en) * | 2020-05-26 | 2022-03-01 | Molecula Corp. | Software plugins of data virtualization-based architecture |
WO2022212319A1 (fr) * | 2021-04-01 | 2022-10-06 | Activevideo Networks, Inc. | Systèmes et procédés de services d'application en réseau de remplacement pour des applications de vidéoconférence |
US11876844B2 (en) * | 2021-04-01 | 2024-01-16 | Active Video Networks, Inc. | Systems and methods of alternative networked application services for video-conferencing applications |
US20230044823A1 (en) * | 2021-08-09 | 2023-02-09 | Xcelastream, Inc. | Multi-presence application architecture |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090083432A1 (en) | 2007-08-31 | 2009-03-26 | Hiroshi Saito | Traffic control system and traffic control method |
US20090100460A1 (en) | 2007-10-11 | 2009-04-16 | At&T Intellectual Property I, L.P. | Methods, systems and computer program products for providing ad insertion via a multimedia applications gateway |
US20090254948A1 (en) | 2008-04-03 | 2009-10-08 | Justin Middleton | System And Method For Streaming Video Over A Wireless Network |
US7702750B2 (en) | 2004-09-29 | 2010-04-20 | Citrix Systems, Inc. | System and method for event detection and re-direction over a network using a presentation level protocol |
Family Cites Families (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5737011A (en) * | 1995-05-03 | 1998-04-07 | Bell Communications Research, Inc. | Infinitely expandable real-time video conferencing system |
GB2314729B (en) * | 1995-12-19 | 2001-01-17 | Motorola Inc | Method and apparatus for rate governing communications |
US6732183B1 (en) * | 1996-12-31 | 2004-05-04 | Broadware Technologies, Inc. | Video and audio streaming for multiple users |
US6687745B1 (en) * | 1999-09-14 | 2004-02-03 | Droplet, Inc | System and method for delivering a graphical user interface of remote applications over a thin bandwidth connection |
NZ518774A (en) * | 1999-10-22 | 2004-09-24 | Activesky Inc | An object oriented video system |
US6532467B1 (en) * | 2000-04-10 | 2003-03-11 | Sas Institute Inc. | Method for selecting node variables in a binary decision tree structure |
US20020161685A1 (en) * | 2001-04-25 | 2002-10-31 | Michael Dwinnell | Broadcasting information and providing data access over the internet to investors and managers on demand |
US7302465B2 (en) | 2001-10-22 | 2007-11-27 | Comverse, Inc. | Distributed multimedia transfer |
US7362349B2 (en) * | 2002-07-10 | 2008-04-22 | Seiko Epson Corporation | Multi-participant conference system with controllable content delivery using a client monitor back-channel |
US7574653B2 (en) * | 2002-10-11 | 2009-08-11 | Microsoft Corporation | Adaptive image formatting control |
CN1571508B (zh) * | 2003-07-19 | 2010-05-12 | 华为技术有限公司 | 一种实现多画面的方法 |
US7373395B2 (en) * | 2004-02-04 | 2008-05-13 | Perseus Wireless, Inc. | Method and system for providing information to remote clients |
JP4331049B2 (ja) * | 2004-04-30 | 2009-09-16 | 株式会社東芝 | データ伝送装置およびデータ伝送方法 |
US7176957B2 (en) * | 2004-05-25 | 2007-02-13 | Seiko Epson Corporation | Local video loopback method for a multi-participant conference system using a back-channel video interface |
CA2594118C (fr) * | 2004-12-08 | 2015-07-21 | Imagine Communications Ltd. | Multiplexage statistique distribue de multimedia |
US20060203758A1 (en) | 2005-03-11 | 2006-09-14 | Samsung Electronics Co., Ltd. | Mobile terminal for relaying multimedia data to an external display device |
KR20070019382A (ko) | 2005-08-12 | 2007-02-15 | 삼성전자주식회사 | 네트워크 디스플레이장치 및 그 제어방법 |
US20070070185A1 (en) * | 2005-09-27 | 2007-03-29 | Dy Lady C | System and method for remote display of security video images |
US7702279B2 (en) | 2005-12-20 | 2010-04-20 | Apple Inc. | Portable media player as a low power remote control and method thereof |
US8436889B2 (en) * | 2005-12-22 | 2013-05-07 | Vidyo, Inc. | System and method for videoconferencing using scalable video coding and compositing scalable video conferencing servers |
US8619865B2 (en) * | 2006-02-16 | 2013-12-31 | Vidyo, Inc. | System and method for thinning of scalable video coding bit-streams |
TWI335178B (en) * | 2006-12-20 | 2010-12-21 | Asustek Comp Inc | Apparatus, system and method for remotely opearting multimedia streaming |
TW200838228A (en) * | 2007-03-14 | 2008-09-16 | Imagetech Co Ltd | Virtual camera system and real-time communication method thereof |
US20090031381A1 (en) * | 2007-07-24 | 2009-01-29 | Honeywell International, Inc. | Proxy video server for video surveillance |
US8170123B1 (en) * | 2008-04-15 | 2012-05-01 | Desktone, Inc. | Media acceleration for virtual computing services |
US8793331B2 (en) * | 2008-05-16 | 2014-07-29 | Wyse Technology L.L.C. | Multimedia redirection |
EP2279602B1 (fr) * | 2008-05-19 | 2013-07-31 | Citrix Systems, Inc. | Systèmes et procédés pour la fourniture à distance d appels à un plugin multimédia |
US8345082B2 (en) * | 2008-10-08 | 2013-01-01 | Cisco Technology, Inc. | System and associated methodology for multi-layered site video conferencing |
US8775658B2 (en) * | 2009-03-27 | 2014-07-08 | Wyse Technology L.L.C. | Apparatus and method for transparent communication architecture in remote communication |
US8601097B2 (en) * | 2010-02-22 | 2013-12-03 | Ncomputing Inc. | Method and system for data communications in cloud computing architecture |
-
2010
- 2010-12-08 US US12/963,535 patent/US9191425B2/en active Active
- 2010-12-08 CN CN201080063306.1A patent/CN102741830B/zh active Active
- 2010-12-08 US US12/963,539 patent/US9203883B2/en active Active
- 2010-12-08 WO PCT/US2010/059526 patent/WO2011072049A2/fr active Application Filing
- 2010-12-08 EP EP10836637.8A patent/EP2510454B1/fr active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7702750B2 (en) | 2004-09-29 | 2010-04-20 | Citrix Systems, Inc. | System and method for event detection and re-direction over a network using a presentation level protocol |
US8190676B2 (en) | 2004-09-29 | 2012-05-29 | Citrix Systems, Inc. | System and method for event detection and re-direction over a network using a presentation level protocol |
US20090083432A1 (en) | 2007-08-31 | 2009-03-26 | Hiroshi Saito | Traffic control system and traffic control method |
US20090100460A1 (en) | 2007-10-11 | 2009-04-16 | At&T Intellectual Property I, L.P. | Methods, systems and computer program products for providing ad insertion via a multimedia applications gateway |
US20090254948A1 (en) | 2008-04-03 | 2009-10-08 | Justin Middleton | System And Method For Streaming Video Over A Wireless Network |
Non-Patent Citations (1)
Title |
---|
See also references of EP2510454A4 |
Also Published As
Publication number | Publication date |
---|---|
US9203883B2 (en) | 2015-12-01 |
US9191425B2 (en) | 2015-11-17 |
WO2011072049A3 (fr) | 2011-10-13 |
US20110138069A1 (en) | 2011-06-09 |
EP2510454A2 (fr) | 2012-10-17 |
US20110145431A1 (en) | 2011-06-16 |
CN102741830A (zh) | 2012-10-17 |
CN102741830B (zh) | 2016-07-13 |
EP2510454A4 (fr) | 2013-11-27 |
EP2510454B1 (fr) | 2018-04-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2510454B1 (fr) | Systèmes et procédés destinés à une présentation distante côté client d'un flux multimédia | |
US10721282B2 (en) | Media acceleration for virtual computing services | |
US11029990B2 (en) | Delivering a single end user experience to a client from multiple servers | |
US10827004B2 (en) | Native viewer use for service results from a remote desktop | |
US9069438B2 (en) | Allocating virtual machines according to user-specific virtual machine metrics | |
US8638336B2 (en) | Methods and systems for remoting three dimensional graphical data | |
KR101966921B1 (ko) | 멀티 세션 관리 방법 및 장치 | |
US8645559B2 (en) | Redirection of multiple remote devices | |
JP2023509630A (ja) | メディアアプリケーションのためのマルチメディアリダイレクト | |
JP2023509621A (ja) | ウェブブラウザ・マルチメディア・リダイレクション | |
US10834164B2 (en) | Virtualizing audio and video devices using synchronous A/V streaming | |
KR102247886B1 (ko) | 클라우드 스트리밍 서비스 시스템, 이미지 타입에 따른 클라우드 스트리밍 서비스 방법 및 이를 위한 장치 | |
WO2024061307A1 (fr) | Procédés d'interaction, de traitement et de commande pour un service d'application dans un bureau en nuage | |
KR102167505B1 (ko) | 고화질 영상 서비스를 위한 가상 데스크탑 시스템 및 그 데이터 처리 방법 | |
US20230006850A1 (en) | Automatic repair of web conference session recording | |
KR102172679B1 (ko) | 씬 클라이언트를 이용한 가상 데스크탑과 iptv 서비스 및 그 데이터 처리 방법 | |
von Oven et al. | Getting Started with VMware Horizon | |
Rodríguez‐Silva et al. | Improving the virtualization of rich applications by combining VNC and streaming protocols at the hypervisor layer |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201080063306.1 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10836637 Country of ref document: EP Kind code of ref document: A1 |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10836637 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2010836637 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 4559/CHENP/2012 Country of ref document: IN |
|
NENP | Non-entry into the national phase |
Ref country code: DE |