WO2022051691A1 - Touchless control of video conferencing system - Google Patents

Touchless control of video conferencing system Download PDF

Info

Publication number
WO2022051691A1
WO2022051691A1 PCT/US2021/049175 US2021049175W WO2022051691A1 WO 2022051691 A1 WO2022051691 A1 WO 2022051691A1 US 2021049175 W US2021049175 W US 2021049175W WO 2022051691 A1 WO2022051691 A1 WO 2022051691A1
Authority
WO
WIPO (PCT)
Prior art keywords
client
video conferencing
user
client device
bridging
Prior art date
Application number
PCT/US2021/049175
Other languages
French (fr)
Inventor
Brandon BARRON
Brent Gardner
Christopher Jaynes
Justin Mcbride
Ryan Lee
Original Assignee
Mersive Technologies, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mersive Technologies, Inc. filed Critical Mersive Technologies, Inc.
Publication of WO2022051691A1 publication Critical patent/WO2022051691A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/141Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/147Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/141Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/142Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/15Conference systems

Definitions

  • Video conference rooms typically include one or more shared displays mounted so they are visible to everyone in the room, and an in-room control device for allowing users to specify what is shown on the one or more shared displays.
  • Video conference rooms may also include a camera and audio system. While video conference rooms may be used for a collaborative work session by a group of users physically present in the room, there is increasing interest in video conferences that combine onsite and remote participants. The implementation of the video conferencing system has a large impact on the experience of both types of uses.
  • a video conference room is operated using a host device accessed by an in-room control device.
  • an in-room control device for a video conference room may be a tablet that is connected to the displays and a communications network, either wired or wirelessly.
  • the in-room control device is used to activate the shared displays and initiate a video conference using video conferencing software.
  • the host device and other components of the video conference room may be deployed in different ways. Each deployment and systems engineering approach will involve a different use-case.
  • This “hardware codec” system e.g., a WebEx Telepresense® room
  • This “hardware codec” system requires an in-room interface that users interact with to operate the room. Cameras/audio plug into the codec and if users want to share digital content from their device, they are required to plug into a content input channel on the hardware codec.
  • this type of room is only able to communicate remotely with other rooms, or endpoints, using the same hardware codec due to the use of proprietary communication protocols of the video conference application, i.e., WebEx.
  • FIG. 1 Another way to deploy a video conferencing room is by using a host device that runs video conferencing software, such as Zoom.
  • a camera and/or audio system is plugged into the host device. Users are expected to login to the host device to launch the software that they want to use for video conferencing, and then interact with the system directly on the host device or via a room tablet/controller.
  • An example of this type of video conferencing room is a Zoom® Room.
  • the host device is hosting the video conference and remote users with a link to the video conferencing room may j oin the meeting.
  • in-room panel For those systems with an in-room panel, it is typically used to enable and disable, as well as control zoom, pan, tilt parameters of the in-room camera.
  • audio controls such as volume and mute can be presented on the in-room control panel. Users may interact with the panel during a video conference by touching various buttons, sliders, or other user-interaction elements.
  • Some in-room control devices provide for this type of sharing by making ports available to users to connect their personal device to the in-room control device. These ports may use a physical connection such as a USB, HDMI cable, Lighting cable, or the like, or a wireless communication protocol, such as cellular (4G, 5G, LTE, Bluetooth, Bluetooth Low Energy, Wi-Fi, etc.)
  • An application or software in the in-room control device allows users to interact with their own content or other user’s content on the one or more shared displays in the room.
  • a video conferencing infrastructure includes a video conferencing system with a shared display and a host device coupled to the shared display and a network; a first client device further comprising a processor, a network interface for communicating with the network, a memory and machine-readable instructions, stored in the memory, that, when executed by the processor, control the processor to launch a client bridging and control software on a first client device, the client bridging and control software operable to bridge to the host device; wirelessly connect, by the client bridging and control software, the first client device to the video conferencing system wherein the shared display is controlled using the client bridging and control software on the first client device and content of a display of the first client device is shared to the shared display; launch a video conferencing application on the first client device to host a video conference that may be joined by one or more remote users; and share the shared display in the video conferencing application with the one or more remote users.
  • FIG. 1 is a diagram illustrating example components of a video conferencing infrastructure for use with touchless control of a video conferencing system, in embodiments.
  • FIG. 2 is a diagram illustrating example components of the video conferencing system of FIG. 1, in embodiments.
  • FIG. 3 is a diagram illustrating example components of a client device of FIG. 1, in embodiments.
  • FIG. 4 is a flowchart of a method of associating multiple devices for touchless control of a video conferencing system, in embodiments.
  • FIG. 5 is a flowchart of a method performed by addressing service 126 of FIG. 1, in embodiments.
  • FIG. 6 is a flowchart of a method of touchless control of a video conferencing system, in embodiments.
  • FIG. 7 is a flowchart of a method of touchless control of video conferencing system using a mobile device, in embodiments.
  • FIG. 8 shows one example contact tracing system deployed with one conference room, in embodiments
  • FIG. 9 is a functional block diagram illustrating one example client device of FIG. 8 in further example detail, in embodiments.
  • FIG. 10 is a functional block diagram illustrating one example host device of FIG. 8 in further example detail, in embodiments;
  • FIG. 11 is a functional block diagram illustrating one example server, in embodiments.
  • FIG. 12 is a functional block diagram illustrating one example database with example data structure and content, in embodiments.
  • FIG. 13 is a flowchart illustrating one example method for collecting data within a conference room to allow contact tracing, in embodiments
  • FIG. 14 is a flowchart illustrating one example method for contact tracing of conference rooms participants, in embodiments.
  • a video conferencing system may be installed permanently or temporarily in a conference room or other location such that two or more users may participate in a video conference using a shared display.
  • One or more of the users may be physically present in the conference room, and any number of users may be located remotely from the video conference room.
  • the shared display is coupled to a host device which controls the shared display.
  • the host device includes a variety of ports, both wired and wireless, for connecting to the shared display, a communications network, and one or more client devices.
  • the host device may also be connected to a user interface.
  • users who are physically located in the conference room may each connect a client device to the host device using a wired or wireless connection to display content on the shared display.
  • Client software on the client device and host software on the host device cooperate to allow users of connected client devices to share content to the shared display and manipulate their own and others content on the shared display.
  • Users may interact with the host device and therefore the shared display with a variety of different client devices.
  • these client devices may generally be considered in two categories: a user laptop device and a user mobile device.
  • the user laptop device should be understood to encompass a laptop computer that includes a physical keyboard and a display of approximately 13 inches or more.
  • the laptop computer may connect to one or more peripherals such displays, webcams, and external keyboards, either directly or through a docking station.
  • the user laptop device may connect to a network such as ethemet or Wi-Fi.
  • a user laptop device may include software for managing workflow that is maintained by the employer, such as calendar and email software in addition to other productivity software like Microsoft Office®. It may also include client software for interfacing with a video conferencing system.
  • a user laptop device may include one or more video conferencing applications such as WebEx® or Zoom® that allow client devices to participate in a video conference that may be joined by remote users.
  • the user mobile device should be understood to encompass a smart phone or other mobile device that includes a touch screen of less than approximately 7 inches and may connect to Wi-Fi and/or cellular networks.
  • a user mobile device may execute software applications for calendar and email as well as one or more video conferencing applications.
  • the user mobile device is very portable and may be carried by a user throughout the day whereas the user laptop device might be left in an office.
  • Other types of client devices such as a tablet computer, may be a hybrid of both categories.
  • a video conferencing system may be installed in a conference room or other defined location large enough for two or more people to gather for a meeting.
  • An employer or organization facility may have one or several video conference rooms.
  • the term “conference room” is used herein, but embodiments described herein may be used with any video conference system installed in a location where it may be shared among users who request access to it for a defined period of time, either by scheduling through a calendar or on demand.
  • Embodiments described herein provide for touchless control of a video conferencing system with a user’s personal devices, without requiring physical interaction with components of the video conferencing system.
  • FIGS. 1 - 3 An overview of representative components that may be used for touchless video conferencing is shown in FIGS. 1 - 3.
  • video conferencing infrastructure 100 includes video conferencing system 102, two or more client devices 122, 124 and an addressing service 126 connected over a communications network 120.
  • FIG. 2 illustrates details of video conferencing system 102
  • FIG. 3 illustrates components of a client device 122, although the discussion herein also applies to client device 124.
  • FIGS. 1, 2 and 3 are best viewed together in the following description.
  • FIG. 1 is a diagram illustrating example components of video conference infrastructure 100.
  • a video conferencing system 102 includes a host device 104 and a shared display 106.
  • Shared display 106 may be a single display or multiple displays.
  • shared display 106 is permanently mounted to a wall of a conference room and sized such that it is visible to everyone in the conference room.
  • Host device 104 includes at least one processor 114 and memory 116, which stores machine-readable instructions which, when executed by processor 114 cause host device to perform functions of video conferencing system 102.
  • the software components shown in FIG. 2 are for purposes of illustrating embodiments herein. More or fewer software components may be provided in memory 116 as needed.
  • Host software 130 includes machine-readable instructions for operating the components of video conferencing system 102, as well as interacting with client bridging and control software 150 in client device 122 of FIG. 3 to allow client device 122 to access shared display 106 and other components of video conferencing system 102.
  • Memory 116 may also store video conferencing software 132, for example WebEx or Zoom, for managing a video conference with remote participants through network interface 118 and communications network 120.
  • video conferencing software 132 for example WebEx or Zoom
  • addressing service 126 of FIG. 1 may be stored in memory 116 and executed by host device 104 as host addressing service 134 instead of a separate component on a network. Addressing service 126 will be discussed in more detail below.
  • Host device 104 also includes network interface 118 for interfacing with communications network 120.
  • Communications network 120 represents any wired or wireless network that facilitates communication between electronic devices, such as Wi-Fi, Bluetooth, WAN, LAN, radio frequency or cellular, for example.
  • a user interface 108 such as a keyboard or tablet device, may be provided with video conferencing system 102 to allow a user to interact with host device 104 to determine what is shown on shared display 106, however, user interface 108 is not required to operate host device 104.
  • Host software 130 may interact with client devices over network interface 118 so that a client device acting as a host provides video conferencing functions such as call set-up and control as well as control of the shared display and other components of the video conferencing system
  • video conferencing system 102 may also include a camera 110 and an audio system 112, such as one or more microphones and speakers.
  • Other peripheral devices may also be included in video conferencing system 102 depending on the needs of the users.
  • Video conferencing infrastructure 100 includes client devices 122 and 124. Although two client devices are shown, any number of client devices may be used with infrastructure 100. Each of the client devices 122 and 124 may be a user laptop device, user mobile device or any digital device capable of connecting to a communication network, as described above. Client devices 122 and 124 connect to communications network 120 with user network interface 156, which may be a wireless transceiver with an antenna or other device that may exchange information with a network. User network interface 156 may include the capability to connect using one or more different protocols, in embodiments. More than one interface and protocol may be used by each of client devices 122 and 124, which may also connect to video conferencing system 102 directly instead of through communications network 120 as shown by the dotted arrows in FIG. 1.
  • Client device 122 includes at least processor 140 and memory 142 for storing data and machine-readable instructions for execution by processor 140, as shown in FIG. 3.
  • Client device 122 also includes a display 144 and one or more ports 146, such as USB, charging ports or an external display port.
  • client device 122 may include one or more peripherals.
  • a user laptop device for example, may include keyboard, camera, speakers, etc, that may be connected to processor 140 and memory 142 through one or more of ports 146.
  • a user mobile device may include similar peripherals incorporated within the housing of the user mobile device or as external devices.
  • Memory 142 of client device 122 stores various software components as described herein.
  • Software components shown in FIG. 3 are for purposes of illustrating embodiments herein. More or fewer software components may be provided in memory 142 as needed.
  • Client device software 154 includes machine-readable instructions for operating the components of client device 122.
  • Memory 142 also stores client bridging and control software 150, which is used to login to and access video conferencing system 102.
  • Video conferencing software 152 may include, for example, WebEx or Zoom, for participating in a video conference with other participants.
  • communications network 120 such as addressing service 126 and calendar service 128, as well as other servers, storage devices, additional networks, etc.
  • client devices 122 and 124 connect to video conferencing system 102 to work collaboratively using shared display 106.
  • Other users may participate in a video conference using video conferencing system 102 by using camera 110 and audio system 112 without bridging a client device to video conferencing system 102.
  • Client software installed on client device 122 or 124 facilitates connection with host device 104, either directly or over communication network 120, and therefore collaboration using shared display 106.
  • Some of the interactions provided by video conferencing system 102 to users bridging with a client device 122 (or client device 124) for a collaboration using shared display 106 include:
  • each of client devices 122, 124 is capable of publishing content to shared display 106 and controlling the shared display simultaneously with other devices by executing client bridging and control software 150.
  • Client bridging and control software 150 is executed to form a bridge between client device 122, 124 and host device 104 to exchange media (video, audio, images, application window pixels) as well as control information and other metadata (user information, location of source, etc).
  • Bridging a client device to a video conferencing system may be understood as bridging all components of the video conferencing system 102 to client device 122, 124 including the shared display and audio/video devices for the purposes of exchanging data. Bridging may also include conversion between communication protocols.
  • Touchless control of a video conferencing system is provided when a user may connect with and control system components and a video conference call without physically contacting shared or public devices.
  • a user may accomplish touchless control using a laptop device, a mobile device and other types of devices.
  • FIG. 4 is a flowchart of a method 400 of associating multiple devices for touchless control of a video conferencing system, in embodiments.
  • the flowchart of FIG. 4 shows an embodiment of linking a user mobile device to a user laptop device, but more generally, any client device may be linked to any other client device using a similar method. Further, the order of the steps may be changed or steps described separately below may be combined.
  • a user opens client bridging and control software 150 on a user laptop device.
  • a user may open client bridging and control software 150 by any method of opening a software application such as clicking an icon or a menu item on a display of the user laptop device.
  • step 404 the user laptop device displays a window of client bridging and control software 150 including a menu with a variety of options for interacting with client bridging and control software 150.
  • one of those options may include “register mobile device” or “register another device.”
  • client bridging and control software 150 causes a QR (Quick Response) code to be displayed on the user laptop device.
  • QR Quick Response
  • a user scans the QR code using a camera of a user mobile device to link the user mobile device with the user laptop device.
  • a QR code is used to facilitate linking with another device however, other types of codes may be used, including a code that may be entered manually.
  • step 410 the user laptop device and user mobile device exchange private/public key certificates.
  • step 412 the user laptop device and user mobile device are registered as part of a device pool for the purposes of authentication and control. Registration of a device pool may be maintained in client bridging and control software 150, addressing service 126 and/or another server accessible over communications network 120.
  • FIG. 5 is a flowchart of a method 500 performed by addressing service 126 of FIG. 1 for providing touchless control of a video conferencing system.
  • each host device associated with video conferencing infrastructure 100 provided by an employer or organization is stores the network address of addressing service 126.
  • Each client device which has installed client bridging and control software 150 also stores the network address of addressing service 126.
  • a common network protocol such as DNS (Domain Name Server) is used to resolve a logical network address into a unique IP (Internet Protocol) address.
  • addressing service 126 is in a waiting state pending receipt of update messages from host and client devices.
  • addressing service 126 stores a list of host and client devices that are sending update messages, and is located on a server coupled to communications network 120, which may be a local network within a facility or the Internet in general.
  • the host and client devices send periodic update messages to addressing service 126.
  • host device 104 sends an update message including the IP address of the host device, ports to be used to communicate with the device, a human-readable name such as “Conference Room 1,” the number of users connected to the device and a current screen key.
  • a client device 122 or 124 sends a message including a device ID, it’s current IP addresses and other metadata as required by video conferencing infrastructure 100.
  • the screen key sent by host device 104 in step 506 is dynamic, i.e., it changes over time. This ensures that for each meeting using video conferencing system 102, the meeting host is bridging to the room in which they are physically located. This secure start to a meeting prevents users from joining a meeting from other locations simply by knowing a screen key that is assigned to a host device statically.
  • addressing service 126 receives the screen key it checks the stored list for duplicate screen keys that have been dynamically generated by other host devices 104. If a duplicate key is found at step 510, a rejection message is returned to the host device in step 512, which causes host device 104 to send another update message with a different screen key.
  • addressing service 126 may include a proposed screen key that is not a duplicate with the rejection message.
  • step 514 when addressing service 126 receives a message from client device 122 or 124, it checks to see if it is a request from client bridging and control software 150.
  • a user may access a menu using client bridging and control software 150 to initiate a video conference call, causing client bridging and control software 150 to request a list of available host devices from addressing service 126. If the update message is not a request, addressing service 126 returns to waiting state.
  • step 516 in response to a request, addressing service 126 sends the list of recently updated host devices to the requesting client device.
  • the list would include the human-readable name, the IP address of the host device 104, and other metadata.
  • FIG. 6 is a flowchart of a method 600 of touchless control of a video conferencing system.
  • Method 600 includes steps 604, 610, 612 and 618.
  • method 600 also includes at least one of steps 602, 606, 608, 614 and 616.
  • a user with a client device enters a conference room with a video conferencing system.
  • a client device 122, 124 may be a user laptop device, a user mobile device, or any other type of device that is running a client bridging and control software 150.
  • client bridging and control software is launched on a client device.
  • client bridging and control software 150 may be launched on client device 122 such as a user laptop device.
  • client bridging and control software 150 may be launched in several ways.
  • a user may select a menu item in client bridging and control software 150 that sends a request for a list of available host devices to addressing service 126.
  • the list received from addressing service 126 may include a human-readable name for each host device as well as an IP address for each host device and other metadata.
  • the list is displayed then the user is prompted to select the host device corresponding to the conference room.
  • client bridging and control software 150 may automatically send a request to addressing service 126 upon launch.
  • a user may provide the current screen key displayed on shared display 106 to client bridging and control software 150.
  • step 610 the video conferencing system is wirelessly connected to the client device and the display of the client device is shared to shared display.
  • client device 122 is wirelessly connected to video conferencing system 102 wherein the shared display 106 is controlled using client bridging and control software 150 on client device 122 and content of display of 144 client device 122 is shared to shared display 106.
  • a video conferencing application is launched on the client device.
  • a video conferencing application 152 such as WebEx or Zoom may be launched on a client device.
  • Remote users not physically located at video conferencing system 102 may join a video conference hosted on client device 122 using a link or other methods.
  • a video conferencing application 152 may be launched in several ways.
  • a calendar service 128 is accessed over communications network 120 to identify a video conference meeting scheduled by a user of the client device 122 on the user’s calendar. The video conference meeting may be started automatically using information in the meeting notice.
  • a calendar service 128 is accessed over network 120 to identify a video conference meeting scheduled for the video conferencing system 102 to which the client device 122 is connected in step 610. The video conference meeting may be started automatically using information in the meeting notice.
  • step 618 a shared display of the video conferencing system is shared in the video conferencing application.
  • shared display 106 is shared into video conferencing application 152 so that remote users who have joined the video conference can see the content sharing done by client devices who have connected to video conferencing system 102.
  • client devices may also connect to video conferencing system 102 using a version of client bridging and control software 150 installed on the client device.
  • FIG. 7 is a flowchart of a method 700 of touchless control of a video conferencing system.
  • Method 700 includes steps 704, 706, 708, and 710 .
  • method 700 also includes at least one of steps 702 and 712.
  • step 702 a user with a user mobile device enters a conference room with a video conferencing system.
  • a user mobile device is running a client bridging and control software 150 and has been associated with a user laptop device in a device pool for touchless control of video conferencing system 102 as described above in FIG. 4.
  • step 704 client bridging and control software is launched on a client device.
  • client bridging and control software 150 may be launched on client device 122 such as a user laptop device as described above in step 604 of FIG. 6.
  • step 706 an IP address of a host device and a device pool key are sent to a user laptop device.
  • an IP address of host device 104 of video conferencing system 102 is sent to a user laptop device that was previously authenticated with the user mobile device.
  • the authentication method of FIG. 4 results in a device pool key that can be exchanged by a user mobile device and a user laptop device when using the user mobile device to connect with video conferencing system 102.
  • step 708 the user mobile device is authenticated with the user laptop device using the device pool key.
  • step 710 the video conferencing system is wirelessly connected to the user mobile device and the user laptop device, and content from the display of the user mobile device is shared to shared display.
  • user mobile device 122 is wirelessly connected to video conferencing system 102 wherein the shared display 106 is controlled using client bridging and control software 150 on client device 122 and content of display of 144 client device 122 is shared to shared display 106.
  • a video conferencing application is launched on the user laptop device to host a video conference with remote users using a video conferencing system.
  • video conferencing application 152 is launched on a user laptop device in the device pool with the user mobile device and shared display 106 is shared in video conferencing application 152.
  • a video conference using video conferencing system 102 is hosted on a user laptop device whether or not the user laptop device is physically located in the conference room. Further, remote participants who have joined the video conference are able to see shared display 106 in the same way that users of client devices connected to video conferencing system 102 see shared display 106.
  • host device 104 may wirelessly detect the presence of a client device in a conference room using radio frequency (e.g., Bluetooth) signals generated by the client device, causing client bridging and control software 150 to be launched on the client device.
  • radio frequency e.g., Bluetooth
  • the presence of a user in a conference room may be detected using a camera 110 installed in the room.
  • This embodiment requires mapping facial or other features of a user to device IDs of the user’s devices. This mapping may be learned over time, for example, from repeated associations between a camera recognition image/room and a device ID.
  • This learning occurs each time a user connects, providing a tuple of information [camera recognition image, time of day, room] that is associated with a device ID.
  • This mapping maps the appearance of a user’s set of pixels in an observed image to a device ID when they connect to the display.
  • a neural network may be trained with this info over time then we store these mappings. Once we detect a user, the network is queried for that room, closest to the time of day, and then discover the device ID. An alert is sent to that device ID causing it to auto connect and launch client bridging and control software 150 as in steps 604 and 606 of FIG. 6.
  • the present embodiments solve these problems by using existing conference room equipment, which is primarily provided to improve meeting engagement and productivity, to track proximity of client devices within conference rooms, and to provide contact tracing of users in a private manner.
  • existing conference room equipment which is primarily provided to improve meeting engagement and productivity
  • to track proximity of client devices within conference rooms and to provide contact tracing of users in a private manner.
  • the anonymity of the users may be maintained while providing a way of tracking potential contact (exposure) and provide notification thereof.
  • FIG. 8 shows one example contact tracing system 800 deployed with a conference room 802 (or any other similar space, such as an auditorium, a classroom, etc.).
  • System 800 primarily operates as a media sharing service for meetings, conferences, and discussions, and includes a host device 804 (also may be known as a room host device, or room host) that is computer based and controlled by host software 850 and is communicatively coupled with a room display 806 (e.g., a computer monitor, a television screen, a projector, etc.) positioned within conference room 802.
  • Host device 804 is communicatively coupled to a network that may be one or both of wired and, preferably, wireless.
  • Host device 804 may also connect, via network 814 for example, with an administrator device 870 that allows an administrator to configure, manage, and interact with host device 804.
  • each user 808 e.g., an organizer, an attendee
  • client device 810 e.g., one or more of a smartphone, a tablet computer, and a laptop computer.
  • user 808(1) has two client devices 810(1) and 810(1)’
  • user 808(2) has one client device 810(2)
  • user 808(3) has one client device 810(3).
  • Host device 804 includes a database 852 and supports functionality that allows one or more of client devices 810 to share content to room display 806, such that is may be viewed by all users 808.
  • each user 808 arrives at conference room 802, their at least one client device 810 may connect to host device 804 via a wired and/or wireless network 814 so that content may be shared on room display 806, or to allow user 808 to host a video conference, or otherwise use system 800 within conference room 802.
  • Network 814 may have a range that is limited to include only conference room 802.
  • host device 804 captures and stores, in database 852, a connect entry that includes a device ID that uniquely identifies the connecting device and a timestamp of the connection time. Similarly, as each client device 810 disconnects from host device 804, host device 804 stores, within database 852, a disconnect entry that includes the unique ID of the disconnecting device and a timestamp of the disconnection time. Accordingly, database 852 includes pairs of connect and disconnect entries that define when the corresponding client device 810 was within conference room 802. Host device 804 may process database 852 to evaluates the paired entries to determine which client devices 810 were simultaneously present within conference room 802 at any time. Although database 852 is shown as part of host device 804, in embodiments, it may be located in a server or cloud storage accessible via network 814.
  • user 808 manually controls client device 810 to connect with host device 804 via network 814.
  • client device 810 automatically connects with host device 804 without user interaction. See for example, PCT International Application Number PCT/US19/14081, titled “Systems and Methods to Determine Room Occupancy,” filed January 17, 2019, which is incorporated herein as Appendices A.
  • Database 852 may also store, for each user 808 registered with host device 804, a device pool (see device pool 1214 of FIG. 12) that defines the device ID of each client device 810 that user 808 may connect with host device 804.
  • the device pool may also be used to register client devices 810 for the purposes of authentication and control, and may be maintained using one or more of application software of client device 810, an external addressing service, and/or another server accessible via network 814.
  • user 808(1) has two client devices 810(1) and 810(1)’, wherein the corresponding device pool stores the unique IDs for both client devices 810(1) and 810(1)’ on association with user 808(1).
  • host device 804 may determine that user 808(1) is present within conference room 802 when either, or both, of client devices 810(1) and 810(1)’ are connected.
  • host software 850 of host device 804 may determine that user 808(2) is present in conference room 802
  • client device 810(3) is connected
  • host device 804 may determine that user 808(3) is present in conference room 802.
  • presence of users 808 within conference room 802 may be determined by other methods without departing from the scope hereof.
  • an external server e.g., an analytic server
  • network 814 may automatically detect when client devices 810 connect to, and disconnect from, host device 804.
  • contact tracing systems may be deployed in any number of conference rooms.
  • Digital infrastructure may be provided to track a user through multiple conference rooms.
  • FIG. 9 is a functional block diagram illustrating one example client device 900 that may represent any of client devices 810 of FIG. 8.
  • Client device 900 includes at least one processor 902 communicatively coupled with memory 904 storing a device ID 306 and an application 908.
  • Client device 900 also includes a communication interface 910.
  • Device ID 906 uniquely identifies client device 900.
  • Application 908 is software with machine readable instructions that, when executed by processor 902, control processor 902 to communicate, via communication interface 910, with any one of host devices 804. Where client device 900 is a smartphone, application 908 may be an app that is downloaded to client device 900.
  • Application 908 implements functionality that allows client device 900 to communicate with, and/or control, host device 804 to display information on the corresponding room display 806.
  • Application 908 is further enhanced to control processor 902 to send, either autonomously or under user direction, device ID 906 to host device 804 when at the corresponding conference room 80. Accordingly, application 908 indicates presence of client device 900 to the corresponding host device when the user is at the conference room.
  • FIG. 10 is a functional block diagram illustrating example detail of one host device 1000 that may represent any of host device 804 of FIG. 8.
  • Host device 1000 includes at least one processor 1002 communicatively coupled to memory 1004 that includes host software 1008, a host ID 1010, and, in certain embodiments, a database 1052 that may represent database 852 of host device 804 of FIG. 8.
  • Host software 1008 may represent host software 850 of FIG. 8.
  • database 452 is external to host device 1000.
  • Host device 1000 also includes a communication interface 1006 that communicates with a network and/or any of client devices 810 of FIG. 8.
  • Host software 1008 has machine readable instructions that, when executed by processor 1002, control processor 1002 to provide control and access, client devices 810 via communication interface 1006, to a room display, such as room display 806 of FIG. 8.
  • host software 1008 collaborates with application 908 of client device 900 to provide features for media sharing and/or video conferencing using the room displays 806.
  • Host software 1008 is further enhanced to track connect and disconnect of client devices 900, using device ID 906, when each client device 900 communicates with host device 1000.
  • host software 1008 creates connect and disconnect entries in database 1052 as each client device 900 connects and disconnects from host device 1000. Where database 1052 is not included with host device 1000, host software 1008 may create connect and disconnect entries in an external database via communications interface 1006.
  • host software 1008 may also include a contact tracer 1012, having machine readable instructions that, when executed by processor 1002, control processor 1002 to process database 1052 to determine contact risk assessments for users 808 that attended meetings in conference roomer 802, such as in response to a request to trace one or more users and/or one or more client devices.
  • Contact risk assessment is described in detail below.
  • contact tracer 1012 generates a trace device list 1015, stored in memory 1004 for example, that defines a device ID 906 for each client device 900 used by the traced user.
  • Trace device list 1015 is processed by contact tracer 1012 to determine a tierl collaborator list 1016 of device IDs that were concurrently in the same conference room 802 (e.g., at the same location) as any client device of the traced user.
  • Contact tracer 1012 may also determine atier2 collaborator list 1018 of device IDs of client devices that were concurrently at the same location as any client device identified in tierl collaborator list 1016.
  • Host software 1008 may also include a trained classifier 1014 that implements artificial intelligence to match facial images to biometric data of users 808, as described in further detail below.
  • FIG. 11 is a functional block diagram illustrating one example server 1100
  • Server 1100 is a computer that includes a communications interface 116, and at least one processor 1102 communicatively coupled with memory 1104 storing a database 1152.
  • Memory 1104 also stores a contact tracer 1108 that is software with machine readable instructions that, when executed by processor 51102, cause processor 1102 to process database 1152 to generate a contact risk assessment for users that attended a meeting where one of the users was later determined as being sick and contagious (e.g., a virus carrier).
  • Contact risk assessment is described in further detail below.
  • contact tracer 1108 may generate a trace device list 1111, stored in memory 1104 for example, that defines device IDs 906 of each client device 900 used by the user being traced. Based on this list, contact tracer 1108 determines a tierl collaborator list 1112 of device IDs that were concurrently at the same location as any of the traced user’s client devices identified in trace device list 1111. Contact tracer 1108 may also determine a tier2 collaborator list 1114 of device IDs of client devices that were concurrently at the same location as any client device identified in tierl collaborator list 1112. Server 1100 may also include, within memory 1104, a trained classifier 1110 that implements artificial intelligence to match facial images to biometric data of users 808, as stored in user registration entries of database 1152 for example.
  • FIG. 12 is a functional block diagram illustrating a database 1200 with example data structure and content.
  • Database 1200 represents any of database 852, 1052, and 1152 of FIGs. 8, 10 and 11, respectively, or a combination thereof.
  • Database 1200 is shown with a user registration entry 1210, a connect entry 1220, and a disconnect entry 1230.
  • One user registration entry 610 may be generated for each user 808 that may use conference room 802.
  • an employer may create an entry for each employee and visitors that are expected to participate in meetings within their conference rooms.
  • User registration entry 1210 includes a unique user ID 1212 (e.g., a name, an employee number, etc.), and a device pool 1214 that stores unique device IDs 306 (see FIG.
  • each client device 9300 of the identified user that may connect to any host device 1000.
  • user 808(1) has two client devices 810(1) and 810(1)’, and therefore device pool 1214 would include the device ID 906 for both of client devices 810(1) and 810(1)’.
  • Connect entry 1220 and disconnect entry 1230 are stored in database 1200 by host device 1000.
  • host device 1000 stores one at least one connect entry 1220 for each client device 810 that connects to host device 1000.
  • host device 1000 stores one at least one disconnect entry 1230 for each client device 810 that disconnects from host device 1000.
  • Host device 1000 generates connect entry 1220 with its own host ID 1010 (to distinguish entries generated by host device 1000 from entries generated by other host devices), device ID 306 of the connecting client device 810, and a timestamp 1222 that defines the date and time that the connection occurred.
  • host device 1000 generates disconnect entry 1230 with its own host ID 1010 (to distinguish entries generated by host device 1000 from entries generated by other host devices), device ID 906 of the disconnecting client device 810, and a timestamp 1232 that defines the date and time that the disconnect occurred (or was determined to have occurred by host device 1000).
  • connect entries 1220 and disconnect entries 1230 of database 1200 indicate when the corresponding user was present in the conference room corresponding to the identified host device.
  • an administrator 1270 e.g., administrators 870 and of FIG. 1
  • receives a request to trace contact between one of users 808 e.g., when informed that the user was likely contagious with an illness while attending a meeting at conference rooms 802
  • administrator 1270 interrogates database 1200 to leam of other users 808 that were in proximity of the traced user.
  • the administrator interrogates database 1200 to leam which other users were concurrently in the same conference room (e.g., in physical proximity) as the infected user at any time during the previous week.
  • Pairs of connect and disconnect entries 1220 and 1230, having the same host ID 1010 and device ID 906, in database 1200 indicate a period, based upon a difference between timestamps 1222 and 1232, that the corresponding user 808 was in the conference room 802 served by host device 1000 with host ID 1010. The user 808 is assumed to have been co-located with their client device 810.
  • administrator 1270 may interact with one or more of contact tracer 1012 and/or contract tracer 1108 to run a query on database 1200 that determines, based on the corresponding user registration entry 1210, all device IDs 906 for client devices 900 corresponding to user ID 1212 of the user being traced.
  • This list may be stored as trace device list 1015 in host device 1000 and/or trace device list 1111 within server 1100. Administrator 1270 may then control contact tracer 1012 and/or contract tracer 1108 to retrieve connect entry 1220 and disconnect entry 1230 pairs corresponding to each device ID 906 of trace device list 1015/1111, and to use corresponding timestamps 1222 and 1232 to determine conferences rooms 802 and periods when and where those client devices were present.
  • Administrator 1270 may then retrieve connect entry 1220 and disconnect entry 1230 pairs corresponding to other client devices 900 for the same time window and thereby determine which other client devices 810 were concurrently in the same conference room as at least one of the client devices of trace device list 1015/1111.
  • the corresponding device IDs 906 of client devices 810 that were concurrently in the same conference room as at least one of the client devices of trace device list 1015/1111 are used to form tierl collaborator list 1016/1112.
  • the users 808 corresponding to the identified client devices 810 in tierl collaborator list 1016/1112 are referred to as Tierl Collaborators because they were in the same conference room at the same time as the user being traced (e.g., and thereby risk having been directly infected by the traced user).
  • Administrator 1270 may interact further with one or more of contact tracer 1012 and/or contract tracer 1108 to interrogate database 1200 also to determine other users that are once (or more) removed from being in the same conference room as the traced user. For example, administrator 1270 may use contact tracer 1012 and/or contract tracer 11508 to query database 1200 to return connect entry 1220 and disconnect entry 1230 pairs corresponding to device IDs 906 in tierl collaborator list 1016/1112, and then generate a tier2 collaborator list 1114 that include device IDs 906 of client devices 810 that were concurrently in the same conference room as any of the client device identified in tierl collaborator list 1112, and after the corresponding time of entry into tierl collaborator list 1016/1112. Administrator 1270 may search further levels of indirection as needed to determine other further tiers of collaboration.
  • application 908 running on client device 810 includes functionality, or user interface, that allows user 808 to report an illness (e.g., when showing symptoms and/or testing positive for a particular illness or infection) directly to contact tracer 1012 of host device 1000 and/or contact tracer 1108 of server 1100.
  • the user interface may include, for example, a list of illnesses being traced and/or a field for inputting additional information regarding symptoms, for example.
  • application 908 sends only its own device ID 906 (e.g., of client device 900 that it is running on) to contact tracer 1012/1108, which then automatically determines Tierl Collaborators, and optionally Tier2 Collaborators, based on that device ID.
  • contact tracer 1012/1108 may retrieve additional devices IDs 906 corresponding to the user being traced by retrieving them from the corresponding device pool 1214 containing the reported device ID 906.
  • Contact tracer 1012/1108 may then automatically send a notification of the direct risk of infection to corresponding client devices 900 of Tierl Collaborators, and a notification of indirect risk of infection to client devices 900 of Tier2 Collaborators.
  • Administrator 1270 may preconfigure contact tracer 1012/1108 with a default time window (e.g., one week, one month, one day, etc..) for searching in response to self-reported illness and may also configure one or more rules that define the number of tiers to search and report.
  • contact tracer 1012 and/or contact tracer 1108 may send a notification to a corresponding client device 900 of each Tierl Collaborator, indicating “You were in 3 meetings this week with someone who has reported testing positive.”
  • contact tracer 1012 and/or contact tracer 1108 may send a notification to a corresponding client device 900 of each Tier2 Collaborator indicating “You shared 5 meetings with someone who spent time with a user who has reported testing positive.”
  • contact tracer 1012 and/or contact tracer 1108 may report statistical information of traces performed to administrator 1270.
  • contact tracer 1012/1108 may display, on a device of administrator 1270, a notification indicating “A case was reported that impacts 10 employees directly and 30 indirectly.”
  • contact tracer 1012/1108 may be configured with additional rules that automatically send messages and/or notifications to client devices of identified in tierl collaborator list 1016/1112 and/or tier2 collaborator list 1018/1114, etc., when certain statistical thresholds are reached. For example, a rule may cause contact tracer 1012/1108 to send a notification and/or email message to each client device 300 indicating “Go home” when the number of users impacted by one or more event is above a predefined threshold.
  • system 800 may be further enhanced to visually identify users within conference rooms 802 and 202, respectively.
  • system 800 may further include at least one camera 812 that is communicatively coupled with host device 804 and positioned to capture facial images of users 808 within conference room 802.
  • camera 812 may be positioned near room display 806 such that users 808 face camera 812 when watching room display 806.
  • Host software 850 controls camera 812 to capture at least one image of users 808 at intervals and may use computer vision techniques to identify one or more of users 808 based upon previously captured biometric data 616, FIG. 12 (e.g., facial images) stored in a user registration entry 1210 of database 852/1200.
  • Host software 850 processes the captured images to detects regions of pixels that correspond to users 808 and may include a trained classifier that implements artificial intelligence to match each region of pixels to biometric data 1216 of users 808, as stored in user registration entries 1210 of database 852/1200. Host software 850 may then map devices IDs 906 defined by device pool 1214 of matched user registration entries 610 to device IDs 906 detected by host software 850. Where host software 850 is unable to match the captured image to any user registration entry 1210, host software 850 may automatically assign a temporary unique identifier to the captured facial image and store it within database 852.
  • FIG. 13 is a flowchart illustrating one example method 1300 for collecting data within a conference room to allow contact tracing.
  • Method 1300 is implemented at least in part in one or more of host device 804, 1000 and server and 500.
  • method 1300 receives a message including a device ID from a client device.
  • host device 804 receives device ID 306 from client device 810(2).
  • method 1300 maintains a connected device ID list of client devices that are currently connected to the host device.
  • an entry is added or updated for the device ID of the client device sending the message.
  • Block 1306 is a decision. If, in block 1306, method 1300 determines that the received device ID belongs to a new client device, method 1300 continues with block 1308; otherwise, method 1300 continues with block 1310.
  • method 1300 stores a connect entry with the received device ID of the client device and a timestamp in a database.
  • host software 850 of host device 804 stores connect entry 1220 with host ID 1010, device ID 306, and a timestamp 622 set to the current time, in database 852.
  • Block 1310 is a decision. If, in block 1310, method 1300 determines that the received message is a disconnect message, method 1300 continues with block 1316; otherwise, method 1300 continues with block 1312. In block 1312, method 1300 detects when a client device corresponding to a device ID stored in the device ID list is no longer connected. In one example of block 1312, when host software 850 has not received any message from client device 810(1) corresponding to one device ID in the connected device ID list for a predefined period (e.g., one minute, five minutes, etc.), host software 850 may determine that client device 810(1) has disconnected from host device 804.
  • a predefined period e.g., one minute, five minutes, etc.
  • Block 1314 is a decision. If, in block 1314, method 1300 determines that one or more devices are no longer connected, method 1300 continues with block 1316; otherwise, method 1300 terminates until a subsequent message is received. In block 1316, method 1300 stores a disconnect entry with the device ID of the client device and a timestamp in the database. In one example of block 1316, host software 850 of host device 804 stores disconnect entry 630 with host ID 1010, device ID 906, and a timestamp 1232 set to the current time, in database 852. Method 1300 then terminates until a next message is received or until another client device is determined to no longer be connected.
  • FIG. 13 is a flowchart illustrating one example method 1300 for contact tracing of conference rooms participants.
  • Method 1300 may be implemented in host devices 804 or 1000 and in server 500.
  • method 1300 determines a time window for contact tracing.
  • contact tracer 1108, running in server 1100 receives a request to trace user 808(2) for the previous two weeks. In embodiments, any preferred time window may be used.
  • method 1300 determines client devices of contact being traced.
  • contact tracer 1108 retrieves user registration entry 1210 having a matching user ID 1212 of the contact being traced, and retrieves one or more device IDs 906 from device pool 1214.
  • method 1300 determines atierl collaborator list of client devices that were in the same conference room concurrently with any client device of the traced user during the time window.
  • contact tracer 1108 queries database 1200 to retrieve pairs of connect and disconnect entries 1220/1230 based upon the device IDs 906 retrieved in block 1302 and having timestamps 1222 and 1232 occurring within the previous two weeks.
  • Contact tracer 1108 queries database 1200 to retrieve pairs of connect and disconnect entries 1220/1230 of other device IDs 906 that have both (a) host IDs 1010 matching the host IDs 1010 of retrieved pairs of connect and disconnect entries 1220/1230, and (b) have a defined period between timestamps 1222 and 1232 that at least partially overlaps one of the periods defined by the timestamps 1222 and 1232 of the retrieved pairs of connect and disconnect entries 620/630 of the user being traced.
  • Contact tracer 1108 adds corresponding device IDs 906 to tierl collaborator list 1112.
  • method 1300 sends a notification to all tierl collaborators to indicate a direct exposure.
  • contact tracer 1012 of host device 1000 sends a notification to each client device 810 identified (e.g., through device IDs 906) in tierl collaborator list 1112 indicating direct exposure.
  • method 1300 determines a tier2 collaborator list of client devices that were concurrently in the same conference room with any client device in the tierl collaborator list during a subsequent portion of the time window.
  • contact tracer 1012 of host device 1000 queries database 1200 to retrieve connect entry 620 and disconnect entry 1230 pairs corresponding to device IDs 906 stored in tierl collaborator list 1016 to define tier2 risk locations and periods, and then retrieves queries database 1052 to retrieve connect entry 1220 and disconnect entry 1230 pairs corresponding to other client devices to determine whether they were concurrently in the same conference room as any of the client devices identified in tierl collaborator list 1016.
  • method 1300 sends a notification to all tier2 collaborators to indicate an indirect exposure.
  • contact tracer 1012 of host device 1000 sends a notification to each client device 810 identified (e.g., through device IDs 906) in tier2 collaborator list 1114 indicating indirect exposure.
  • method 1200 generates statistical data on client devices in the tierl collaborator list and the tier2 collaborator list.
  • contact tracer 1012/1108 may display, on a device of administrator 1270, a notification indicating “A case was reported that impacts ten employees directly and thirty indirectly.”
  • a user reporting an illness may cause the system to alert administrator 1270 who may be monitoring the system from any location.
  • the system may generate a report including a number of reporting incidents and the number of users notified for each incident.
  • Administrator 1270 may be presented with a dashboard including this information and other data that may be used for decision making.
  • a dashboard may include, for example, a notification that “There have been 3 self-reported incidents of COVID this week impacting 40 people directly and 160 indirectly.” Other types of information may be reported on a dashboard, such as geographic location, severity of the illness reported, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Telephonic Communication Services (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

A system and method for touchless control of a video conferencing system in a conference room allows a user to initiate and host a video conference from a client device that is wirelessly connected to the video conferencing system without physically contacting the video conferencing system. A video conference may be hosted on a user laptop in the conference room. A user may also initiate a video conference using a mobile device in the conference room that is authenticated to a laptop device in a different location. The laptop device hosts the video conference by connected to the video conferencing system in the conference room over a network. The mobile device may be used to control the video conference hosted on the laptop device without physically contacting the video conferencing system.

Description

TOUCHLESS CONTROL OF VIDEO CONFERENCING SYSTEM
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to Provisional Patent Applications 63/074,916 filed September 4, 2020 titled “Touchless Control of Video Conferencing System” and 63/074,896 filed September 4, 2020 titled “Contact Tracing Systems and Methods,” the entire contents of which are incorporated by reference.
BACKGROUND
[0002] Many offices include one or more video conference rooms that may be used by all employees. A video conference room typically includes one or more shared displays mounted so they are visible to everyone in the room, and an in-room control device for allowing users to specify what is shown on the one or more shared displays. Video conference rooms may also include a camera and audio system. While video conference rooms may be used for a collaborative work session by a group of users physically present in the room, there is increasing interest in video conferences that combine onsite and remote participants. The implementation of the video conferencing system has a large impact on the experience of both types of uses.
[0003] A video conference room is operated using a host device accessed by an in-room control device. In its simplest form, an in-room control device for a video conference room may be a tablet that is connected to the displays and a communications network, either wired or wirelessly. The in-room control device is used to activate the shared displays and initiate a video conference using video conferencing software. The host device and other components of the video conference room may be deployed in different ways. Each deployment and systems engineering approach will involve a different use-case.
[0004] One type of deployment uses a dedicated, specialized hardware system (as opposed to a general-purpose computer) to run the video conference. This “hardware codec” system (e.g., a WebEx Telepresense® room) requires an in-room interface that users interact with to operate the room. Cameras/audio plug into the codec and if users want to share digital content from their device, they are required to plug into a content input channel on the hardware codec. In general, this type of room is only able to communicate remotely with other rooms, or endpoints, using the same hardware codec due to the use of proprietary communication protocols of the video conference application, i.e., WebEx.
[0005] Another way to deploy a video conferencing room is by using a host device that runs video conferencing software, such as Zoom. A camera and/or audio system is plugged into the host device. Users are expected to login to the host device to launch the software that they want to use for video conferencing, and then interact with the system directly on the host device or via a room tablet/controller. An example of this type of video conferencing room is a Zoom® Room. The host device is hosting the video conference and remote users with a link to the video conferencing room may j oin the meeting.
[0006] Yet another way to deploy a video conference room uses no in-room system other than the shared display, camera and/or audio. In this case users are expected to plug the various cables for these devices into their own laptop, which may be challenging because users must interact with the cables and may lack device drivers which now need to be installed. Although an interface to the video conferencing software is on the laptop, the users still need to “touch” all the cables. Sharing of content from the laptop to the shared display also requires an HDMI/video cable to be connected from the laptop to the display in the room, but this means the display cannot be shared between users.
[0007] In any office or public building, a significant number of employees or members of the public may touch common physical resources, such as pressing elevator buttons, opening doors or using copiers. With increased attention to public health and safety, it is desirable to minimize these situations where multiple people interact physically with the same object or device. The common physical resources in video conferencing rooms may pose a particular risk because video conferencing rooms are typically in enclosed areas and can be highly utilized by many people.
[0008] For those systems with an in-room panel, it is typically used to enable and disable, as well as control zoom, pan, tilt parameters of the in-room camera. In addition, audio controls such as volume and mute can be presented on the in-room control panel. Users may interact with the panel during a video conference by touching various buttons, sliders, or other user-interaction elements.
[0009] During a video conference, users often wish to share digital files from their own device with other participants. Some in-room control devices provide for this type of sharing by making ports available to users to connect their personal device to the in-room control device. These ports may use a physical connection such as a USB, HDMI cable, Lighting cable, or the like, or a wireless communication protocol, such as cellular (4G, 5G, LTE, Bluetooth, Bluetooth Low Energy, Wi-Fi, etc.) An application or software in the in-room control device allows users to interact with their own content or other user’s content on the one or more shared displays in the room. SUMMARY OF THE EMBODIMENTS
[0010] In a first aspect, a method for controlling a video conferencing system comprising a shared display controlled by a host device coupled to a network includes launching a client bridging and control software on a first client device, the client bridging and control software operable to bridge to the host device; wirelessly bridging, by the client bridging and control software, the first client device to the video conferencing system wherein the shared display is controlled using the client bridging and control software on the first client device and content of a display of the first client device is shared to the shared display; launching a video conferencing application on the first client device to host a video conference that may be joined by one or more remote users; and sharing the shared display in the video conferencing application.
[0011] In a second aspect, a method for controlling a video conferencing system using a user mobile device, the video conferencing system comprising a shared display controlled by a host device coupled to a network includes launching client bridging and control software on the user mobile device, the client bridging and control software operable to bridge to the host device over the network; sending an IP address of the host device and a device pool key to a user laptop device; authenticating the user mobile device with the user laptop device using the device pool key; wirelessly bridging the user mobile device to the video conferencing system wherein the shared display is controlled using the client bridging and control software on the user mobile device; and sharing a content of a display of the user mobile device to the shared display.
[0012] In a third aspect, a video conferencing infrastructure includes a video conferencing system with a shared display and a host device coupled to the shared display and a network; a first client device further comprising a processor, a network interface for communicating with the network, a memory and machine-readable instructions, stored in the memory, that, when executed by the processor, control the processor to launch a client bridging and control software on a first client device, the client bridging and control software operable to bridge to the host device; wirelessly connect, by the client bridging and control software, the first client device to the video conferencing system wherein the shared display is controlled using the client bridging and control software on the first client device and content of a display of the first client device is shared to the shared display; launch a video conferencing application on the first client device to host a video conference that may be joined by one or more remote users; and share the shared display in the video conferencing application with the one or more remote users.
BRIEF DESCRIPTION OF THE FIGURES
[0013] FIG. 1 is a diagram illustrating example components of a video conferencing infrastructure for use with touchless control of a video conferencing system, in embodiments.
[0014] FIG. 2 is a diagram illustrating example components of the video conferencing system of FIG. 1, in embodiments.
[0015] FIG. 3 is a diagram illustrating example components of a client device of FIG. 1, in embodiments.
[0016] FIG. 4 is a flowchart of a method of associating multiple devices for touchless control of a video conferencing system, in embodiments.
[0017] FIG. 5 is a flowchart of a method performed by addressing service 126 of FIG. 1, in embodiments.
[0018] FIG. 6 is a flowchart of a method of touchless control of a video conferencing system, in embodiments.
[0019] FIG. 7 is a flowchart of a method of touchless control of video conferencing system using a mobile device, in embodiments.
[0020] FIG. 8 shows one example contact tracing system deployed with one conference room, in embodiments;
[0021] FIG. 9 is a functional block diagram illustrating one example client device of FIG. 8 in further example detail, in embodiments;
[0022] FIG. 10 is a functional block diagram illustrating one example host device of FIG. 8 in further example detail, in embodiments;
[0023] FIG. 11 is a functional block diagram illustrating one example server, in embodiments;
[0024] FIG. 12 is a functional block diagram illustrating one example database with example data structure and content, in embodiments;
[0025] FIG. 13 is a flowchart illustrating one example method for collecting data within a conference room to allow contact tracing, in embodiments;
[0026] FIG. 14 is a flowchart illustrating one example method for contact tracing of conference rooms participants, in embodiments; and
DETAILED DESCRIPTION OF THE EMBODIMENTS
[0027] A video conferencing system may be installed permanently or temporarily in a conference room or other location such that two or more users may participate in a video conference using a shared display. One or more of the users may be physically present in the conference room, and any number of users may be located remotely from the video conference room. The shared display is coupled to a host device which controls the shared display. The host device includes a variety of ports, both wired and wireless, for connecting to the shared display, a communications network, and one or more client devices. In embodiments, the host device may also be connected to a user interface.
[0028] In embodiments, users who are physically located in the conference room may each connect a client device to the host device using a wired or wireless connection to display content on the shared display. Client software on the client device and host software on the host device cooperate to allow users of connected client devices to share content to the shared display and manipulate their own and others content on the shared display.
[0029] Users may interact with the host device and therefore the shared display with a variety of different client devices. In embodiments, these client devices may generally be considered in two categories: a user laptop device and a user mobile device. The user laptop device should be understood to encompass a laptop computer that includes a physical keyboard and a display of approximately 13 inches or more. The laptop computer may connect to one or more peripherals such displays, webcams, and external keyboards, either directly or through a docking station. The user laptop device may connect to a network such as ethemet or Wi-Fi. When provided by an employer or organization, a user laptop device may include software for managing workflow that is maintained by the employer, such as calendar and email software in addition to other productivity software like Microsoft Office®. It may also include client software for interfacing with a video conferencing system. Further, a user laptop device may include one or more video conferencing applications such as WebEx® or Zoom® that allow client devices to participate in a video conference that may be joined by remote users.
[0030] The user mobile device should be understood to encompass a smart phone or other mobile device that includes a touch screen of less than approximately 7 inches and may connect to Wi-Fi and/or cellular networks. A user mobile device may execute software applications for calendar and email as well as one or more video conferencing applications. The user mobile device is very portable and may be carried by a user throughout the day whereas the user laptop device might be left in an office. Other types of client devices, such as a tablet computer, may be a hybrid of both categories.
[0031] A video conferencing system may be installed in a conference room or other defined location large enough for two or more people to gather for a meeting. An employer or organization facility may have one or several video conference rooms. The term “conference room” is used herein, but embodiments described herein may be used with any video conference system installed in a location where it may be shared among users who request access to it for a defined period of time, either by scheduling through a calendar or on demand. Embodiments described herein provide for touchless control of a video conferencing system with a user’s personal devices, without requiring physical interaction with components of the video conferencing system.
[0032] An overview of representative components that may be used for touchless video conferencing is shown in FIGS. 1 - 3. As shown in FIG. 1, video conferencing infrastructure 100 includes video conferencing system 102, two or more client devices 122, 124 and an addressing service 126 connected over a communications network 120. FIG. 2 illustrates details of video conferencing system 102 and FIG. 3 illustrates components of a client device 122, although the discussion herein also applies to client device 124. FIGS. 1, 2 and 3 are best viewed together in the following description.
[0033] FIG. 1 is a diagram illustrating example components of video conference infrastructure 100. A video conferencing system 102 includes a host device 104 and a shared display 106. Shared display 106 may be a single display or multiple displays. In embodiments, shared display 106 is permanently mounted to a wall of a conference room and sized such that it is visible to everyone in the conference room.
[0034] Host device 104 includes at least one processor 114 and memory 116, which stores machine-readable instructions which, when executed by processor 114 cause host device to perform functions of video conferencing system 102. The software components shown in FIG. 2 are for purposes of illustrating embodiments herein. More or fewer software components may be provided in memory 116 as needed. Host software 130 includes machine-readable instructions for operating the components of video conferencing system 102, as well as interacting with client bridging and control software 150 in client device 122 of FIG. 3 to allow client device 122 to access shared display 106 and other components of video conferencing system 102.
[0035] Memory 116 may also store video conferencing software 132, for example WebEx or Zoom, for managing a video conference with remote participants through network interface 118 and communications network 120. In addition, addressing service 126 of FIG. 1 may be stored in memory 116 and executed by host device 104 as host addressing service 134 instead of a separate component on a network. Addressing service 126 will be discussed in more detail below.
[0036] Host device 104 also includes network interface 118 for interfacing with communications network 120. Communications network 120 represents any wired or wireless network that facilitates communication between electronic devices, such as Wi-Fi, Bluetooth, WAN, LAN, radio frequency or cellular, for example.
[0037] A user interface 108 such as a keyboard or tablet device, may be provided with video conferencing system 102 to allow a user to interact with host device 104 to determine what is shown on shared display 106, however, user interface 108 is not required to operate host device 104. Host software 130 may interact with client devices over network interface 118 so that a client device acting as a host provides video conferencing functions such as call set-up and control as well as control of the shared display and other components of the video conferencing system
[0038] In embodiments, video conferencing system 102 may also include a camera 110 and an audio system 112, such as one or more microphones and speakers. Other peripheral devices may also be included in video conferencing system 102 depending on the needs of the users.
[0039] Video conferencing infrastructure 100 includes client devices 122 and 124. Although two client devices are shown, any number of client devices may be used with infrastructure 100. Each of the client devices 122 and 124 may be a user laptop device, user mobile device or any digital device capable of connecting to a communication network, as described above. Client devices 122 and 124 connect to communications network 120 with user network interface 156, which may be a wireless transceiver with an antenna or other device that may exchange information with a network. User network interface 156 may include the capability to connect using one or more different protocols, in embodiments. More than one interface and protocol may be used by each of client devices 122 and 124, which may also connect to video conferencing system 102 directly instead of through communications network 120 as shown by the dotted arrows in FIG. 1.
[0040] Client device 122 includes at least processor 140 and memory 142 for storing data and machine-readable instructions for execution by processor 140, as shown in FIG. 3. Client device 122 also includes a display 144 and one or more ports 146, such as USB, charging ports or an external display port. In embodiments, client device 122 may include one or more peripherals. A user laptop device, for example, may include keyboard, camera, speakers, etc, that may be connected to processor 140 and memory 142 through one or more of ports 146. A user mobile device may include similar peripherals incorporated within the housing of the user mobile device or as external devices.
[0041] Memory 142 of client device 122 stores various software components as described herein. Software components shown in FIG. 3 are for purposes of illustrating embodiments herein. More or fewer software components may be provided in memory 142 as needed. Client device software 154 includes machine-readable instructions for operating the components of client device 122. Memory 142 also stores client bridging and control software 150, which is used to login to and access video conferencing system 102. Video conferencing software 152 may include, for example, WebEx or Zoom, for participating in a video conference with other participants.
[0042] Referring to FIG. 1, other components may be connected to communications network 120, such as addressing service 126 and calendar service 128, as well as other servers, storage devices, additional networks, etc.
[0043] In embodiments, client devices 122 and 124 connect to video conferencing system 102 to work collaboratively using shared display 106. Other users may participate in a video conference using video conferencing system 102 by using camera 110 and audio system 112 without bridging a client device to video conferencing system 102. Client software installed on client device 122 or 124 facilitates connection with host device 104, either directly or over communication network 120, and therefore collaboration using shared display 106. Some of the interactions provided by video conferencing system 102 to users bridging with a client device 122 (or client device 124) for a collaboration using shared display 106 include:
[0044] Connection to shared display 106 without requiring a specific cable with the appropriate connectors for both shared display 106 and client device 122.
[0045] Publishing the live output of an application running on display 144 of client device 122.
[0046] Selection of any image appearing on shared display 106 and manipulating it using scale and repositioning tools, for example. The selection may be accomplished through an interface on display 144 of client device 122 or by interacting with the shared display 106 directly.
[0047] Transmitting streams of video or other media from client device 122 to shared display 106.
[0048] Displaying output from two or more client devices simultaneously and facilitating control of shared display 106 by any connected client device.
[0049] In embodiments, each of client devices 122, 124 is capable of publishing content to shared display 106 and controlling the shared display simultaneously with other devices by executing client bridging and control software 150. Client bridging and control software 150 is executed to form a bridge between client device 122, 124 and host device 104 to exchange media (video, audio, images, application window pixels) as well as control information and other metadata (user information, location of source, etc). Bridging a client device to a video conferencing system may be understood as bridging all components of the video conferencing system 102 to client device 122, 124 including the shared display and audio/video devices for the purposes of exchanging data. Bridging may also include conversion between communication protocols.
[0050] Touchless control of a video conferencing system is provided when a user may connect with and control system components and a video conference call without physically contacting shared or public devices. In embodiments, a user may accomplish touchless control using a laptop device, a mobile device and other types of devices.
[0051] FIG. 4 is a flowchart of a method 400 of associating multiple devices for touchless control of a video conferencing system, in embodiments. For purposes of illustration, the flowchart of FIG. 4 shows an embodiment of linking a user mobile device to a user laptop device, but more generally, any client device may be linked to any other client device using a similar method. Further, the order of the steps may be changed or steps described separately below may be combined.
[0052] In step 402, a user opens client bridging and control software 150 on a user laptop device. In an example of step 402, a user may open client bridging and control software 150 by any method of opening a software application such as clicking an icon or a menu item on a display of the user laptop device.
[0053] In step 404, the user laptop device displays a window of client bridging and control software 150 including a menu with a variety of options for interacting with client bridging and control software 150. In an example of step 404, one of those options may include “register mobile device” or “register another device.”
[0054] In step 406, client bridging and control software 150 causes a QR (Quick Response) code to be displayed on the user laptop device. In step 408, a user scans the QR code using a camera of a user mobile device to link the user mobile device with the user laptop device. In an example of steps 406 and 408, a QR code is used to facilitate linking with another device however, other types of codes may be used, including a code that may be entered manually.
[0055] In step 410, the user laptop device and user mobile device exchange private/public key certificates. In step 412, the user laptop device and user mobile device are registered as part of a device pool for the purposes of authentication and control. Registration of a device pool may be maintained in client bridging and control software 150, addressing service 126 and/or another server accessible over communications network 120.
[0056] FIG. 5 is a flowchart of a method 500 performed by addressing service 126 of FIG. 1 for providing touchless control of a video conferencing system.
[0057] In step 502, each host device associated with video conferencing infrastructure 100 provided by an employer or organization is stores the network address of addressing service 126. Each client device which has installed client bridging and control software 150 also stores the network address of addressing service 126. In an example of step 502, a common network protocol such as DNS (Domain Name Server) is used to resolve a logical network address into a unique IP (Internet Protocol) address.
[0058] In step 504, addressing service 126 is in a waiting state pending receipt of update messages from host and client devices. In an example of step 504, addressing service 126 stores a list of host and client devices that are sending update messages, and is located on a server coupled to communications network 120, which may be a local network within a facility or the Internet in general.
[0059] In steps 506 and 508, the host and client devices send periodic update messages to addressing service 126. In an example of step 506, host device 104 sends an update message including the IP address of the host device, ports to be used to communicate with the device, a human-readable name such as “Conference Room 1,” the number of users connected to the device and a current screen key. In an example of step 508, a client device 122 or 124 sends a message including a device ID, it’s current IP addresses and other metadata as required by video conferencing infrastructure 100.
[0060] The screen key sent by host device 104 in step 506 is dynamic, i.e., it changes over time. This ensures that for each meeting using video conferencing system 102, the meeting host is bridging to the room in which they are physically located. This secure start to a meeting prevents users from joining a meeting from other locations simply by knowing a screen key that is assigned to a host device statically. When addressing service 126 receives the screen key it checks the stored list for duplicate screen keys that have been dynamically generated by other host devices 104. If a duplicate key is found at step 510, a rejection message is returned to the host device in step 512, which causes host device 104 to send another update message with a different screen key. In an embodiment, addressing service 126 may include a proposed screen key that is not a duplicate with the rejection message. [0061] In step 514, when addressing service 126 receives a message from client device 122 or 124, it checks to see if it is a request from client bridging and control software 150. In an example of step 514, a user may access a menu using client bridging and control software 150 to initiate a video conference call, causing client bridging and control software 150 to request a list of available host devices from addressing service 126. If the update message is not a request, addressing service 126 returns to waiting state. In step 516, in response to a request, addressing service 126 sends the list of recently updated host devices to the requesting client device. In an embodiment, the list would include the human-readable name, the IP address of the host device 104, and other metadata.
[0062] FIG. 6 is a flowchart of a method 600 of touchless control of a video conferencing system. Method 600 includes steps 604, 610, 612 and 618. In embodiments, method 600 also includes at least one of steps 602, 606, 608, 614 and 616.
[0063] In step 602, a user with a client device enters a conference room with a video conferencing system. In an example of step 602, a client device 122, 124 may be a user laptop device, a user mobile device, or any other type of device that is running a client bridging and control software 150.
[0064] In step 604, client bridging and control software is launched on a client device. In an example of step 604, client bridging and control software 150 may be launched on client device 122 such as a user laptop device. In embodiments, client bridging and control software 150 may be launched in several ways. In step 606, a user may select a menu item in client bridging and control software 150 that sends a request for a list of available host devices to addressing service 126. The list received from addressing service 126 may include a human-readable name for each host device as well as an IP address for each host device and other metadata. The list is displayed then the user is prompted to select the host device corresponding to the conference room. Alternatively, client bridging and control software 150 may automatically send a request to addressing service 126 upon launch. In step 608, a user may provide the current screen key displayed on shared display 106 to client bridging and control software 150.
[0065] In step 610, the video conferencing system is wirelessly connected to the client device and the display of the client device is shared to shared display. In an example of step 610, client device 122 is wirelessly connected to video conferencing system 102 wherein the shared display 106 is controlled using client bridging and control software 150 on client device 122 and content of display of 144 client device 122 is shared to shared display 106.
[0066] In step 612, a video conferencing application is launched on the client device. In an example of step 612, a video conferencing application 152 such as WebEx or Zoom may be launched on a client device. Remote users not physically located at video conferencing system 102 may join a video conference hosted on client device 122 using a link or other methods. In embodiments, a video conferencing application 152 may be launched in several ways. In step 614, a calendar service 128 is accessed over communications network 120 to identify a video conference meeting scheduled by a user of the client device 122 on the user’s calendar. The video conference meeting may be started automatically using information in the meeting notice. In step 616, a calendar service 128 is accessed over network 120 to identify a video conference meeting scheduled for the video conferencing system 102 to which the client device 122 is connected in step 610. The video conference meeting may be started automatically using information in the meeting notice.
[0067] In step 618, a shared display of the video conferencing system is shared in the video conferencing application. In an example of step 618, shared display 106 is shared into video conferencing application 152 so that remote users who have joined the video conference can see the content sharing done by client devices who have connected to video conferencing system 102.
[0068] In embodiments, other client devices may also connect to video conferencing system 102 using a version of client bridging and control software 150 installed on the client device.
[0069] FIG. 7 is a flowchart of a method 700 of touchless control of a video conferencing system. Method 700 includes steps 704, 706, 708, and 710 . In embodiments, method 700 also includes at least one of steps 702 and 712.
[0070] In step 702, a user with a user mobile device enters a conference room with a video conferencing system. In an example of step 602, a user mobile device is running a client bridging and control software 150 and has been associated with a user laptop device in a device pool for touchless control of video conferencing system 102 as described above in FIG. 4.
[0071] In step 704, client bridging and control software is launched on a client device. In an example of step 704, client bridging and control software 150 may be launched on client device 122 such as a user laptop device as described above in step 604 of FIG. 6.
[0072] In step 706, an IP address of a host device and a device pool key are sent to a user laptop device. In an example of step 706, an IP address of host device 104 of video conferencing system 102 is sent to a user laptop device that was previously authenticated with the user mobile device. The authentication method of FIG. 4 results in a device pool key that can be exchanged by a user mobile device and a user laptop device when using the user mobile device to connect with video conferencing system 102.
[0073] In step 708, the user mobile device is authenticated with the user laptop device using the device pool key.
[0074] In step 710, the video conferencing system is wirelessly connected to the user mobile device and the user laptop device, and content from the display of the user mobile device is shared to shared display. In an example of step 710, user mobile device 122 is wirelessly connected to video conferencing system 102 wherein the shared display 106 is controlled using client bridging and control software 150 on client device 122 and content of display of 144 client device 122 is shared to shared display 106.
[0075] In step 712, a video conferencing application is launched on the user laptop device to host a video conference with remote users using a video conferencing system. In an example of step 712, video conferencing application 152 is launched on a user laptop device in the device pool with the user mobile device and shared display 106 is shared in video conferencing application 152. A video conference using video conferencing system 102 is hosted on a user laptop device whether or not the user laptop device is physically located in the conference room. Further, remote participants who have joined the video conference are able to see shared display 106 in the same way that users of client devices connected to video conferencing system 102 see shared display 106.
[0076] Changes may be made in the above methods and systems without departing from the scope hereof. For example, host device 104 may wirelessly detect the presence of a client device in a conference room using radio frequency (e.g., Bluetooth) signals generated by the client device, causing client bridging and control software 150 to be launched on the client device. In embodiments, the presence of a user in a conference room may be detected using a camera 110 installed in the room. This embodiment requires mapping facial or other features of a user to device IDs of the user’s devices. This mapping may be learned over time, for example, from repeated associations between a camera recognition image/room and a device ID. This learning occurs each time a user connects, providing a tuple of information [camera recognition image, time of day, room] that is associated with a device ID. This mapping maps the appearance of a user’s set of pixels in an observed image to a device ID when they connect to the display. In embodiments, a neural network may be trained with this info over time then we store these mappings. Once we detect a user, the network is queried for that room, closest to the time of day, and then discover the device ID. An alert is sent to that device ID causing it to auto connect and launch client bridging and control software 150 as in steps 604 and 606 of FIG. 6.
[0077] When groups of people meet to collaborate, they become proximate to each other as they discuss topics on the agenda. Unfortunately, this proximity presents an opportunity for illness to spread between the people. Further, where the illness is caused by airborne transmission of a virus, and where symptoms of illness are suppressed until well after exposure, the virus may spread unknowingly to people through proximity. One aspect of the present embodiments includes the realization that contact tracing is important within the workplace environment, particularly where people group together to collaborate, but recognizes that there is a tradeoff between accurate tracking of individual contact and the need for privacy. People are forced to choose between either increased privacy by relaxing contact tracing approaches to the detriment of safety or giving up privacy to increase safety. The present embodiments solve these problems by using existing conference room equipment, which is primarily provided to improve meeting engagement and productivity, to track proximity of client devices within conference rooms, and to provide contact tracing of users in a private manner. Advantageously, by tracing proximity of client devices within conference rooms, the anonymity of the users may be maintained while providing a way of tracking potential contact (exposure) and provide notification thereof.
[0078] FIG. 8 shows one example contact tracing system 800 deployed with a conference room 802 (or any other similar space, such as an auditorium, a classroom, etc.). System 800 primarily operates as a media sharing service for meetings, conferences, and discussions, and includes a host device 804 (also may be known as a room host device, or room host) that is computer based and controlled by host software 850 and is communicatively coupled with a room display 806 (e.g., a computer monitor, a television screen, a projector, etc.) positioned within conference room 802. Host device 804 is communicatively coupled to a network that may be one or both of wired and, preferably, wireless. Host device 804 may also connect, via network 814 for example, with an administrator device 870 that allows an administrator to configure, manage, and interact with host device 804. Within conference room 802, each user 808 (e.g., an organizer, an attendee) has at least one client device 810 (e.g., one or more of a smartphone, a tablet computer, and a laptop computer). In the example of FIG. 8, user 808(1) has two client devices 810(1) and 810(1)’, user 808(2) has one client device 810(2), and user 808(3) has one client device 810(3). Host device 804 includes a database 852 and supports functionality that allows one or more of client devices 810 to share content to room display 806, such that is may be viewed by all users 808. As each user 808 arrives at conference room 802, their at least one client device 810 may connect to host device 804 via a wired and/or wireless network 814 so that content may be shared on room display 806, or to allow user 808 to host a video conference, or otherwise use system 800 within conference room 802. Network 814 may have a range that is limited to include only conference room 802.
[0079] As each client device 810 connects to host device 804, host device 804 captures and stores, in database 852, a connect entry that includes a device ID that uniquely identifies the connecting device and a timestamp of the connection time. Similarly, as each client device 810 disconnects from host device 804, host device 804 stores, within database 852, a disconnect entry that includes the unique ID of the disconnecting device and a timestamp of the disconnection time. Accordingly, database 852 includes pairs of connect and disconnect entries that define when the corresponding client device 810 was within conference room 802. Host device 804 may process database 852 to evaluates the paired entries to determine which client devices 810 were simultaneously present within conference room 802 at any time. Although database 852 is shown as part of host device 804, in embodiments, it may be located in a server or cloud storage accessible via network 814.
[0080] In certain embodiments, user 808 manually controls client device 810 to connect with host device 804 via network 814. In other embodiments, client device 810 automatically connects with host device 804 without user interaction. See for example, PCT International Application Number PCT/US19/14081, titled “Systems and Methods to Determine Room Occupancy,” filed January 17, 2019, which is incorporated herein as Appendices A.
[0081] Database 852 may also store, for each user 808 registered with host device 804, a device pool (see device pool 1214 of FIG. 12) that defines the device ID of each client device 810 that user 808 may connect with host device 804. The device pool may also be used to register client devices 810 for the purposes of authentication and control, and may be maintained using one or more of application software of client device 810, an external addressing service, and/or another server accessible via network 814. In the example of FIG. 8, user 808(1) has two client devices 810(1) and 810(1)’, wherein the corresponding device pool stores the unique IDs for both client devices 810(1) and 810(1)’ on association with user 808(1). Accordingly, based upon the device pool, host device 804 may determine that user 808(1) is present within conference room 802 when either, or both, of client devices 810(1) and 810(1)’ are connected. Similarly, when client device 810(2) is connected, host software 850 of host device 804 may determine that user 808(2) is present in conference room 802, and when client device 810(3) is connected, host device 804 may determine that user 808(3) is present in conference room 802. As described in further detail below, presence of users 808 within conference room 802 may be determined by other methods without departing from the scope hereof. For example, an external server (e.g., an analytic server) connected to network 814 may automatically detect when client devices 810 connect to, and disconnect from, host device 804.
[0082] Although embodiment described herein are in terms of one conference room, contact tracing systems may be deployed in any number of conference rooms. Digital infrastructure may be provided to track a user through multiple conference rooms.
Client Device
[0083] FIG. 9 is a functional block diagram illustrating one example client device 900 that may represent any of client devices 810 of FIG. 8. Client device 900 includes at least one processor 902 communicatively coupled with memory 904 storing a device ID 306 and an application 908. Client device 900 also includes a communication interface 910. Device ID 906 uniquely identifies client device 900. Application 908 is software with machine readable instructions that, when executed by processor 902, control processor 902 to communicate, via communication interface 910, with any one of host devices 804. Where client device 900 is a smartphone, application 908 may be an app that is downloaded to client device 900. Application 908 implements functionality that allows client device 900 to communicate with, and/or control, host device 804 to display information on the corresponding room display 806. Application 908 is further enhanced to control processor 902 to send, either autonomously or under user direction, device ID 906 to host device 804 when at the corresponding conference room 80. Accordingly, application 908 indicates presence of client device 900 to the corresponding host device when the user is at the conference room.
Host Device
[0084] FIG. 10 is a functional block diagram illustrating example detail of one host device 1000 that may represent any of host device 804 of FIG. 8. Host device 1000 includes at least one processor 1002 communicatively coupled to memory 1004 that includes host software 1008, a host ID 1010, and, in certain embodiments, a database 1052 that may represent database 852 of host device 804 of FIG. 8. Host software 1008 may represent host software 850 of FIG. 8. In certain other embodiments, database 452 is external to host device 1000. Host device 1000 also includes a communication interface 1006 that communicates with a network and/or any of client devices 810 of FIG. 8. [0085] Host software 1008 has machine readable instructions that, when executed by processor 1002, control processor 1002 to provide control and access, client devices 810 via communication interface 1006, to a room display, such as room display 806 of FIG. 8. For example, host software 1008 collaborates with application 908 of client device 900 to provide features for media sharing and/or video conferencing using the room displays 806. Host software 1008 is further enhanced to track connect and disconnect of client devices 900, using device ID 906, when each client device 900 communicates with host device 1000. For example, host software 1008 creates connect and disconnect entries in database 1052 as each client device 900 connects and disconnects from host device 1000. Where database 1052 is not included with host device 1000, host software 1008 may create connect and disconnect entries in an external database via communications interface 1006.
[0086] In embodiments where host device 1000 includes database 1052, host software 1008 may also include a contact tracer 1012, having machine readable instructions that, when executed by processor 1002, control processor 1002 to process database 1052 to determine contact risk assessments for users 808 that attended meetings in conference roomer 802, such as in response to a request to trace one or more users and/or one or more client devices. Contact risk assessment is described in detail below. In brief, contact tracer 1012 generates a trace device list 1015, stored in memory 1004 for example, that defines a device ID 906 for each client device 900 used by the traced user. Trace device list 1015 is processed by contact tracer 1012 to determine a tierl collaborator list 1016 of device IDs that were concurrently in the same conference room 802 (e.g., at the same location) as any client device of the traced user. Contact tracer 1012 may also determine atier2 collaborator list 1018 of device IDs of client devices that were concurrently at the same location as any client device identified in tierl collaborator list 1016. Host software 1008 may also include a trained classifier 1014 that implements artificial intelligence to match facial images to biometric data of users 808, as described in further detail below.
Server
[0087] FIG. 11 is a functional block diagram illustrating one example server 1100 Server 1100 is a computer that includes a communications interface 116, and at least one processor 1102 communicatively coupled with memory 1104 storing a database 1152. Memory 1104 also stores a contact tracer 1108 that is software with machine readable instructions that, when executed by processor 51102, cause processor 1102 to process database 1152 to generate a contact risk assessment for users that attended a meeting where one of the users was later determined as being sick and contagious (e.g., a virus carrier). Contact risk assessment is described in further detail below.
[0088] In summary, contact tracer 1108 may generate a trace device list 1111, stored in memory 1104 for example, that defines device IDs 906 of each client device 900 used by the user being traced. Based on this list, contact tracer 1108 determines a tierl collaborator list 1112 of device IDs that were concurrently at the same location as any of the traced user’s client devices identified in trace device list 1111. Contact tracer 1108 may also determine a tier2 collaborator list 1114 of device IDs of client devices that were concurrently at the same location as any client device identified in tierl collaborator list 1112. Server 1100 may also include, within memory 1104, a trained classifier 1110 that implements artificial intelligence to match facial images to biometric data of users 808, as stored in user registration entries of database 1152 for example.
Database
[0089] FIG. 12 is a functional block diagram illustrating a database 1200 with example data structure and content. Database 1200 represents any of database 852, 1052, and 1152 of FIGs. 8, 10 and 11, respectively, or a combination thereof. Database 1200 is shown with a user registration entry 1210, a connect entry 1220, and a disconnect entry 1230. One user registration entry 610 may be generated for each user 808 that may use conference room 802. For example, an employer may create an entry for each employee and visitors that are expected to participate in meetings within their conference rooms. User registration entry 1210 includes a unique user ID 1212 (e.g., a name, an employee number, etc.), and a device pool 1214 that stores unique device IDs 306 (see FIG. 9) of each client device 9300 of the identified user that may connect to any host device 1000. Using the example of FIG. 8, user 808(1) has two client devices 810(1) and 810(1)’, and therefore device pool 1214 would include the device ID 906 for both of client devices 810(1) and 810(1)’.
[0090] Connect entry 1220 and disconnect entry 1230 are stored in database 1200 by host device 1000. For example, host device 1000 stores one at least one connect entry 1220 for each client device 810 that connects to host device 1000. Similarly, host device 1000 stores one at least one disconnect entry 1230 for each client device 810 that disconnects from host device 1000. Host device 1000 generates connect entry 1220 with its own host ID 1010 (to distinguish entries generated by host device 1000 from entries generated by other host devices), device ID 306 of the connecting client device 810, and a timestamp 1222 that defines the date and time that the connection occurred. Similarly, host device 1000 generates disconnect entry 1230 with its own host ID 1010 (to distinguish entries generated by host device 1000 from entries generated by other host devices), device ID 906 of the disconnecting client device 810, and a timestamp 1232 that defines the date and time that the disconnect occurred (or was determined to have occurred by host device 1000). Advantageously, connect entries 1220 and disconnect entries 1230 of database 1200 indicate when the corresponding user was present in the conference room corresponding to the identified host device.
Contact Risk Assessment
[0091] When an administrator 1270 (e.g., administrators 870 and of FIG. 1) receives a request to trace contact between one of users 808 (e.g., when informed that the user was likely contagious with an illness while attending a meeting at conference rooms 802), administrator 1270 interrogates database 1200 to leam of other users 808 that were in proximity of the traced user. For example, where the user being traced may have been infectious for the previous week, the administrator interrogates database 1200 to leam which other users were concurrently in the same conference room (e.g., in physical proximity) as the infected user at any time during the previous week. Pairs of connect and disconnect entries 1220 and 1230, having the same host ID 1010 and device ID 906, in database 1200 indicate a period, based upon a difference between timestamps 1222 and 1232, that the corresponding user 808 was in the conference room 802 served by host device 1000 with host ID 1010. The user 808 is assumed to have been co-located with their client device 810.
[0092] In one example of operation, administrator 1270 may interact with one or more of contact tracer 1012 and/or contract tracer 1108 to run a query on database 1200 that determines, based on the corresponding user registration entry 1210, all device IDs 906 for client devices 900 corresponding to user ID 1212 of the user being traced. This list may be stored as trace device list 1015 in host device 1000 and/or trace device list 1111 within server 1100. Administrator 1270 may then control contact tracer 1012 and/or contract tracer 1108 to retrieve connect entry 1220 and disconnect entry 1230 pairs corresponding to each device ID 906 of trace device list 1015/1111, and to use corresponding timestamps 1222 and 1232 to determine conferences rooms 802 and periods when and where those client devices were present. Administrator 1270 may then retrieve connect entry 1220 and disconnect entry 1230 pairs corresponding to other client devices 900 for the same time window and thereby determine which other client devices 810 were concurrently in the same conference room as at least one of the client devices of trace device list 1015/1111. The corresponding device IDs 906 of client devices 810 that were concurrently in the same conference room as at least one of the client devices of trace device list 1015/1111 are used to form tierl collaborator list 1016/1112. The users 808 corresponding to the identified client devices 810 in tierl collaborator list 1016/1112 are referred to as Tierl Collaborators because they were in the same conference room at the same time as the user being traced (e.g., and thereby risk having been directly infected by the traced user).
[0093] Administrator 1270 may interact further with one or more of contact tracer 1012 and/or contract tracer 1108 to interrogate database 1200 also to determine other users that are once (or more) removed from being in the same conference room as the traced user. For example, administrator 1270 may use contact tracer 1012 and/or contract tracer 11508 to query database 1200 to return connect entry 1220 and disconnect entry 1230 pairs corresponding to device IDs 906 in tierl collaborator list 1016/1112, and then generate a tier2 collaborator list 1114 that include device IDs 906 of client devices 810 that were concurrently in the same conference room as any of the client device identified in tierl collaborator list 1112, and after the corresponding time of entry into tierl collaborator list 1016/1112. Administrator 1270 may search further levels of indirection as needed to determine other further tiers of collaboration.
Self-Reporting
[0094] In certain embodiments, application 908 running on client device 810 includes functionality, or user interface, that allows user 808 to report an illness (e.g., when showing symptoms and/or testing positive for a particular illness or infection) directly to contact tracer 1012 of host device 1000 and/or contact tracer 1108 of server 1100. The user interface may include, for example, a list of illnesses being traced and/or a field for inputting additional information regarding symptoms, for example. Particularly, application 908 sends only its own device ID 906 (e.g., of client device 900 that it is running on) to contact tracer 1012/1108, which then automatically determines Tierl Collaborators, and optionally Tier2 Collaborators, based on that device ID. Optionally, contact tracer 1012/1108 may retrieve additional devices IDs 906 corresponding to the user being traced by retrieving them from the corresponding device pool 1214 containing the reported device ID 906. Contact tracer 1012/1108 may then automatically send a notification of the direct risk of infection to corresponding client devices 900 of Tierl Collaborators, and a notification of indirect risk of infection to client devices 900 of Tier2 Collaborators. Administrator 1270 may preconfigure contact tracer 1012/1108 with a default time window (e.g., one week, one month, one day, etc..) for searching in response to self-reported illness and may also configure one or more rules that define the number of tiers to search and report.
[0095] Advantageously, the workplace is automatically notified of potential infection while protecting the privacy of the infected user. In one example, contact tracer 1012 and/or contact tracer 1108 may send a notification to a corresponding client device 900 of each Tierl Collaborator, indicating “You were in 3 meetings this week with someone who has reported testing positive.” In another example, contact tracer 1012 and/or contact tracer 1108 may send a notification to a corresponding client device 900 of each Tier2 Collaborator indicating “You shared 5 meetings with someone who spent time with a user who has reported testing positive.”
[0096] In certain embodiments, contact tracer 1012 and/or contact tracer 1108 may report statistical information of traces performed to administrator 1270. For example, contact tracer 1012/1108 may display, on a device of administrator 1270, a notification indicating “A case was reported that impacts 10 employees directly and 30 indirectly.”
[0097] In certain embodiments, contact tracer 1012/1108 may be configured with additional rules that automatically send messages and/or notifications to client devices of identified in tierl collaborator list 1016/1112 and/or tier2 collaborator list 1018/1114, etc., when certain statistical thresholds are reached. For example, a rule may cause contact tracer 1012/1108 to send a notification and/or email message to each client device 300 indicating “Go home” when the number of users impacted by one or more event is above a predefined threshold.
Visual Identification
[0098] In certain embodiments, system 800 may be further enhanced to visually identify users within conference rooms 802 and 202, respectively. As shown in FIG. 8, system 800 may further include at least one camera 812 that is communicatively coupled with host device 804 and positioned to capture facial images of users 808 within conference room 802. For example, camera 812 may be positioned near room display 806 such that users 808 face camera 812 when watching room display 806. Host software 850 controls camera 812 to capture at least one image of users 808 at intervals and may use computer vision techniques to identify one or more of users 808 based upon previously captured biometric data 616, FIG. 12 (e.g., facial images) stored in a user registration entry 1210 of database 852/1200. Host software 850 processes the captured images to detects regions of pixels that correspond to users 808 and may include a trained classifier that implements artificial intelligence to match each region of pixels to biometric data 1216 of users 808, as stored in user registration entries 1210 of database 852/1200. Host software 850 may then map devices IDs 906 defined by device pool 1214 of matched user registration entries 610 to device IDs 906 detected by host software 850. Where host software 850 is unable to match the captured image to any user registration entry 1210, host software 850 may automatically assign a temporary unique identifier to the captured facial image and store it within database 852.
[0099] FIG. 13 is a flowchart illustrating one example method 1300 for collecting data within a conference room to allow contact tracing. Method 1300 is implemented at least in part in one or more of host device 804, 1000 and server and 500.
[00100] In block 1302, method 1300 receives a message including a device ID from a client device. In one example of block 1302, host device 804 receives device ID 306 from client device 810(2). In block 1304, method 1300 maintains a connected device ID list of client devices that are currently connected to the host device. In one example of block 1304, an entry is added or updated for the device ID of the client device sending the message. Block 1306 is a decision. If, in block 1306, method 1300 determines that the received device ID belongs to a new client device, method 1300 continues with block 1308; otherwise, method 1300 continues with block 1310. In block 1308, method 1300 stores a connect entry with the received device ID of the client device and a timestamp in a database. In one example of block 1308, host software 850 of host device 804 stores connect entry 1220 with host ID 1010, device ID 306, and a timestamp 622 set to the current time, in database 852.
[00101] Block 1310 is a decision. If, in block 1310, method 1300 determines that the received message is a disconnect message, method 1300 continues with block 1316; otherwise, method 1300 continues with block 1312. In block 1312, method 1300 detects when a client device corresponding to a device ID stored in the device ID list is no longer connected. In one example of block 1312, when host software 850 has not received any message from client device 810(1) corresponding to one device ID in the connected device ID list for a predefined period (e.g., one minute, five minutes, etc.), host software 850 may determine that client device 810(1) has disconnected from host device 804.
[00102] Block 1314 is a decision. If, in block 1314, method 1300 determines that one or more devices are no longer connected, method 1300 continues with block 1316; otherwise, method 1300 terminates until a subsequent message is received. In block 1316, method 1300 stores a disconnect entry with the device ID of the client device and a timestamp in the database. In one example of block 1316, host software 850 of host device 804 stores disconnect entry 630 with host ID 1010, device ID 906, and a timestamp 1232 set to the current time, in database 852. Method 1300 then terminates until a next message is received or until another client device is determined to no longer be connected.
[00103] FIG. 13 is a flowchart illustrating one example method 1300 for contact tracing of conference rooms participants. Method 1300 may be implemented in host devices 804 or 1000 and in server 500.
[00104] In block 1301, method 1300 determines a time window for contact tracing. In one example of block 1301, contact tracer 1108, running in server 1100, receives a request to trace user 808(2) for the previous two weeks. In embodiments, any preferred time window may be used. In block 1302, method 1300 determines client devices of contact being traced. In one example of block 1302, contact tracer 1108 retrieves user registration entry 1210 having a matching user ID 1212 of the contact being traced, and retrieves one or more device IDs 906 from device pool 1214. In block 1304, method 1300 determines atierl collaborator list of client devices that were in the same conference room concurrently with any client device of the traced user during the time window. In one example of block 804, contact tracer 1108 queries database 1200 to retrieve pairs of connect and disconnect entries 1220/1230 based upon the device IDs 906 retrieved in block 1302 and having timestamps 1222 and 1232 occurring within the previous two weeks. Contact tracer 1108 then queries database 1200 to retrieve pairs of connect and disconnect entries 1220/1230 of other device IDs 906 that have both (a) host IDs 1010 matching the host IDs 1010 of retrieved pairs of connect and disconnect entries 1220/1230, and (b) have a defined period between timestamps 1222 and 1232 that at least partially overlaps one of the periods defined by the timestamps 1222 and 1232 of the retrieved pairs of connect and disconnect entries 620/630 of the user being traced. Contact tracer 1108 adds corresponding device IDs 906 to tierl collaborator list 1112.
[00105] In block 1306, method 1300 sends a notification to all tierl collaborators to indicate a direct exposure. In one example ofblock 806, contact tracer 1012 of host device 1000 sends a notification to each client device 810 identified (e.g., through device IDs 906) in tierl collaborator list 1112 indicating direct exposure.
[00106] In block 1308, method 1300 determines a tier2 collaborator list of client devices that were concurrently in the same conference room with any client device in the tierl collaborator list during a subsequent portion of the time window. In one example of block 1308, contact tracer 1012 of host device 1000 queries database 1200 to retrieve connect entry 620 and disconnect entry 1230 pairs corresponding to device IDs 906 stored in tierl collaborator list 1016 to define tier2 risk locations and periods, and then retrieves queries database 1052 to retrieve connect entry 1220 and disconnect entry 1230 pairs corresponding to other client devices to determine whether they were concurrently in the same conference room as any of the client devices identified in tierl collaborator list 1016.
[00107] In block 1310, method 1300 sends a notification to all tier2 collaborators to indicate an indirect exposure. In one example of block 1310, contact tracer 1012 of host device 1000 sends a notification to each client device 810 identified (e.g., through device IDs 906) in tier2 collaborator list 1114 indicating indirect exposure.
[00108] In block 1212, method 1200 generates statistical data on client devices in the tierl collaborator list and the tier2 collaborator list. In one example of block 1212, contact tracer 1012/1108 may display, on a device of administrator 1270, a notification indicating “A case was reported that impacts ten employees directly and thirty indirectly.”
[00109] In further embodiments, a user reporting an illness may cause the system to alert administrator 1270 who may be monitoring the system from any location. The system may generate a report including a number of reporting incidents and the number of users notified for each incident. Administrator 1270 may be presented with a dashboard including this information and other data that may be used for decision making. A dashboard may include, for example, a notification that “There have been 3 self-reported incidents of COVID this week impacting 40 people directly and 160 indirectly.” Other types of information may be reported on a dashboard, such as geographic location, severity of the illness reported, etc.
[00110] It should thus be noted that the matter contained in the above description or shown in the accompanying drawings should be interpreted as illustrative and not in a limiting sense. Herein, and unless otherwise indicated: (a) the adjective "exemplary" means serving as an example, instance, or illustration, and (b) the phrase “in embodiments” is equivalent to the phrase “in certain embodiments,” and does not refer to all embodiments. The following claims are intended to cover all generic and specific features described herein, as well as all statements of the scope of the present method and system, which, as a matter of language, might be said to fall therebetween.

Claims

CLAIMS What is claimed is:
1. A method for controlling a video conferencing system comprising a shared display controlled by a host device coupled to a network, the method comprising: launching a client bridging and control software on a first client device, the client bridging and control software operable to bridge to the host device; wirelessly bridging, by the client bridging and control software, the first client device to the video conferencing system wherein the shared display is controlled using the client bridging and control software on the first client device and content of a display of the first client device is shared to the shared display; launching a video conferencing application on the first client device to host a video conference that may be joined by one or more remote users; and sharing the shared display in the video conferencing application.
2. The method of claim 1, further comprising: launching the client bridging and control software on a second client device; wirelessly bridging the second client device to the video conferencing system; and sharing, by the client bridging and control software on the second client device, a content of a display of the second client device to the shared display; wherein the content of the second client device is visible to the one or more remote users who have joined the video conference.
3. The method of claim 2, wherein the first or second client device is a laptop.
4. The method of claim 2, wherein the first or second client device is a user mobile device.
5. The method of claim 1, wherein the video conferencing system further comprises a camera and an audio system and the method further comprising sharing the camera and audio system in the video conferencing application.
6. The method of claim 1, wherein launching the client bridging and control software further comprises entering a code shown on the shared display into the client bridging and control software on the first client device.
25 The method of claim 1, wherein launching the client bridging and control software further comprises: receiving a list of video conferencing systems from an addressing service over the network, the list comprising a human-readable name and an IP address for each host device; and prompting a user to select a video conferencing system from the list. The method of claim 1, wherein launching a video conferencing application on the first client device further comprises accessing a calendar service over the network to identify a meeting scheduled by a user of the first client device on a calendar of the user. The method of claim 1, wherein launching a video conferencing application on the first client device further comprises accessing a calendar service over the network to identify a meeting scheduled for the video conferencing system. A method for controlling a video conferencing system using a user mobile device, the video conferencing system comprising a shared display controlled by a host device coupled to a network, the method comprising: launching client bridging and control software on the user mobile device, the client bridging and control software operable to bridge to the host device over the network; sending an IP address of the host device and a device pool key to a user laptop device; authenticating the user mobile device with the user laptop device using the device pool key; wirelessly bridging the user mobile device to the video conferencing system wherein the shared display is controlled using the client bridging and control software on the user mobile device; and sharing a content of a display of the user mobile device to the shared display. The method of claim 10, further comprising: launching a video conferencing application on the user laptop device to host a video conference that may be joined by one or more remote users. The method of claim 10, further comprising prior to launching client bridging and control software: exchanging a public/private key certificate between the user mobile device and the user laptop device to form a device pool key for authentication of the user mobile device with the user laptop device. A video conferencing infrastructure comprising: a video conferencing system comprising: a shared display; a host device coupled to the shared display and a network; a first client device further comprising a processor, a network interface for communicating with the network, a memory and machine-readable instructions, stored in the memory, that, when executed by the processor, control the processor to: launch a client bridging and control software on a first client device, the client bridging and control software operable to bridge to the host device; wirelessly connect, by the client bridging and control software, the first client device to the video conferencing system wherein the shared display is controlled using the client bridging and control software on the first client device and content of a display of the first client device is shared to the shared display; launch a video conferencing application on the first client device to host a video conference that may be joined by one or more remote users; and share the shared display in the video conferencing application with the one or more remote users. The video conferencing infrastructure of claim 13, further comprising a second client device further comprising a processor, a network interface for communicating with the network, a memory and machine-readable instructions, stored in the memory, that, when executed by the processor, control the processor to: launch the client bridging and control software on a second client device; connect the second client device to the video conferencing system; and share, by the client bridging and control software on the second client device, a content of a display of the second client device to the shared display; wherein the content of the second client device is visible to the one or more remote users. The video conferencing infrastructure of claim 13, wherein the first client device is a user laptop device. The video conferencing infrastructure of claim 13, wherein the first client device is a user mobile device. The video conferencing infrastructure of claim 13, further comprising an addressing service maintaining a list of current IP addresses of host devices and client devices in the video conferencing infrastructure. The video conferencing infrastructure of claim 13, the memory in the first client device further comprising instructions that when executed by the processor, further control the processor to: receive a list of video conferencing systems from an addressing service over the network, the list comprising a human-readable name and an IP address for each video conferencing system; and prompt a user to select a video conferencing system from the list. The video conferencing infrastructure of claim 18, the memory in the first client device further comprising instructions that when executed by the processor, further control the processor to: send the IP address of the host device of the selected video conferencing system and the device pool key to a user laptop device; authenticate the first client device with the user laptop device. The video conferencing infrastructure of claim 13, wherein the video conferencing system further comprises a camera and an audio system.
28
PCT/US2021/049175 2020-09-04 2021-09-04 Touchless control of video conferencing system WO2022051691A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202063074896P 2020-09-04 2020-09-04
US202063074916P 2020-09-04 2020-09-04
US63/074,916 2020-09-04
US63/074,896 2020-09-04

Publications (1)

Publication Number Publication Date
WO2022051691A1 true WO2022051691A1 (en) 2022-03-10

Family

ID=80492084

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2021/049175 WO2022051691A1 (en) 2020-09-04 2021-09-04 Touchless control of video conferencing system

Country Status (1)

Country Link
WO (1) WO2022051691A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150341570A1 (en) * 2014-05-21 2015-11-26 Mersive Technologies, Inc. Intelligent shared display infrastructure and associated methods
US20170041356A1 (en) * 2015-08-06 2017-02-09 Ford Global Technologies, Llc Methods and Systems to Customize a Vehicle Computing System Based on an Electronic Calendar

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150341570A1 (en) * 2014-05-21 2015-11-26 Mersive Technologies, Inc. Intelligent shared display infrastructure and associated methods
US20170041356A1 (en) * 2015-08-06 2017-02-09 Ford Global Technologies, Llc Methods and Systems to Customize a Vehicle Computing System Based on an Electronic Calendar

Similar Documents

Publication Publication Date Title
US9137376B1 (en) Joining a teleconference
US11558437B2 (en) Communication system and method of using the same
CN109565454B (en) System and method for streaming data including multimedia content
US20160261648A1 (en) Communication system and method of using the same
US20150046581A1 (en) Communication system, management apparatus, communication method and computer-readable recording medium
WO2016074534A1 (en) Method, device, and system for signing in to video conference
CN102843355A (en) Conferencing and collaboration system and method thereof
US10735477B2 (en) System, apparatus and associated methodology for establishing multiple data communications between terminals
US10775761B2 (en) Dynamic personalized room control panel and personal preference room setup
US20170006146A1 (en) Information processing apparatus and recording medium
US20180109758A1 (en) Transmission system, transmission terminal, method and program
US11558210B2 (en) Systems and methods for initiating actions based on multi-user call detection
US20100289867A1 (en) Method and System for Launching a Scheduled Conference Based on the Presence of a Scheduled Participant
US20170163696A1 (en) Transmission control system, communications terminal, communications system, transmission control method, and storage medium
US9185144B2 (en) Apparatus, system, and method of managing data transmission, and recording medium storing transmission management program
US20180167580A1 (en) Communication terminal, communication system, communication method, and non-transitory computer-readable medium
US10645229B1 (en) Intercom system and intercom managing method
US20160262206A1 (en) Communication system, communication method, relay device, and computer program product
CN108665595A (en) A kind of Intelligent visible access control system
US8963990B2 (en) Transmission management system, transmission terminal, transmission system, method of managing data transmission, and recording medium storing data transmission management program
WO2023235225A1 (en) Workspace reservation user verification
US20220303314A1 (en) Image Sharing and Conference Participant Identification Method and Image Sharing and Conference Participant Identification System Capable of Performing Bi-directional Communications and Partitioning Images
WO2022051691A1 (en) Touchless control of video conferencing system
US11863905B1 (en) Application-based control of devices within an environment
KR101231827B1 (en) Home network system linkaging with smart work syatem and terminal linkage method of home network system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21865243

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21865243

Country of ref document: EP

Kind code of ref document: A1