WO2013062582A1

WO2013062582A1 - Grouping a participant and a resource

Info

Publication number: WO2013062582A1
Application number: PCT/US2011/058342
Authority: WO
Inventors: Mark E. Gorzynski; Michael D. Derocher
Original assignee: Hewlett-Packard Development Company, L.P.
Priority date: 2011-10-28
Filing date: 2011-10-28
Publication date: 2013-05-02

Abstract

Groups of participants of a meeting established over a network are provided, where the groups include a first group that has an active participant and a resource, where the resource is to be referenced by the active participant during the meeting. A display layout is provided, where the display layout includes display regions for displaying media streams corresponding to the groups. The display regions corresponding to the first group include display regions for the active participant and the resource that are placed in proximity to each other.

Description

GROUPING A PARTICIPANT AND A RESOURCE

Background

[0001 ] Remote conferencing systems allow for collaboration between people at different locations. These systems allow participants to interact with one another through the use of audio and/or video equipment that provides audio and/or video communications.

Brief Description Of The Drawings

[0002] Some implementations are described with respect to the following figures:

Figs. 1 -5 are schematic diagrams of display layouts according to various example implementations;

Fig. 6 is a flow diagram of an example process according to some

implementations; and

Fig. 7 is a block diagram of an example arrangement that includes equipment for establishing meetings over a communications network, according to some implementations.

Detailed Description

[0003] Remote conferencing allows people at remote locations ("conference sites") to conduct a meeting over a communications network (e.g. the Internet, a local area network, a wide area network, etc.). Note that "communications network" or "network" can refer to a single network or multiple networks. People at separate conference sites can communicate in the meeting, where each site may include audio/video equipment for capturing audio/video that is to be transmitted over the communications network to other conference sites involved in the meeting.

"Audio/video data" can refer to audio data (including voice from participants), or video data (including video of participants and/or resources), or both audio and video data. In the ensuing discussion, unless otherwise noted, the term "meeting" can refer to a session established over a communications network for exchanging audio/video data among multiple conference sites.

[0004] In addition to exchanging audio/video data, participants in a meeting can also share and collaboratively work on data, such as a document (word processing document, slide presentation document, spreadsheet document, and so forth), an audio and/or video presentation, and so forth. When a meeting is held in person (where the participants of the meeting are physically together in a common location), non-verbal communications are typically easily discernible to supplement verbal communications of the participants. Examples of non-verbal communications include hand gestures, facial expressions, pointing references, body language, and associations of people with data (seeing who is working on what).

[0005] However, with a meeting established over a communications network among participants that are physically at different conference sites, it can be challenging to perform and/or comprehend non-verbal communications among the participants of the meeting. For example, a presenter may use a sweeping hand gesture to indicate that the presenter is referring to "all the data," such as data associated with a document the presenter is discussing. However, if media streams corresponding to respective participants of the meeting are not properly laid out in respective display devices of endpoints at the corresponding conference sites, then the remote conferencing system may not accurately capture that the presenter is making a hand gesture towards the data that the presenter is referencing. In such a scenario, the non-verbal communication expressed by the sweeping hand gesture may be lost to the other participants of the meeting.

[0006] In accordance with some implementations, a grouping technique is provided to allow for logical and visual association of participant(s) with data that the participant(s) is (are) referencing. More generally, data that a participant is referencing is referred to as a "resource," which can include any type of visual material that can be displayed during the meeting, where the visual material can include a document (e.g. word processing document, presentation slides, spreadsheet document, etc.), an image, a movie, or any other visual material that is not a participant of the meeting. As discussed further below, associating a participant with a resource can refer to associating a media stream of the participant with a media stream of the resource in a display layout. A "media stream" can refer to content for depicting respective participant(s) or resource(s), where the media stream can contain audio/video data, an image, and so froth. Note that a media stream for a participant can include audio (e.g. voice), video, and in some cases, text (e.g. name label for the participant).

[0007] The group that associates participant(s) with resource(s) can be referred to as a collaborative group. A participant of the collaborative group that is

referencing a resource is considered an active participant, where an "active participant" is a person who is actively referencing the resource. Such an active participant can be associated with an active collaboration status (which can be indicated with a flag or other indicator being set to a true value, for example). A participant "referencing" a resource can include the participant making a presentation (e.g. discussing) with respect to the resource, gesturing towards the resource, being located at a location (e.g. a presentation podium or a location next to a whiteboard) corresponding to the resource.

[0008] The grouping technique also allows other groups of participants of the meeting to be defined, such as an audience group (including participants who are audience members or non-active participants). A "non-active participant" can refer to a participant that is not actively referencing a particular resource. "Non-active participant" does not necessarily mean that a participant is being passive; in fact, such a participant can be actively involved in doing something else during the meeting, such as engaging in discussions with another participant, performing another task during the meeting, or referencing another resource in the meeting.

[0009] Other types of groups of participants can also be defined. In a meeting, two or more groups can be defined. The defined groups can be described by metadata or other group information that contains definitions of memberships of the groups (e.g. information containing identifiers of the members of a group, information describing roles of the members of a group, an indicator for specifying a

collaboration status of each member, and so forth).

[0010] Groups can be associated with properties. An example property for a group can specify a relative size of the display region for a particular group as compared to a display region for another group. For example, the property can specify that a display region for a collaborative group is larger than a display region for an audience group. Properties can also be defined between groups. For example, one such property can specify that audio/video output from a first group is to be presented to a second group.

[001 1 ] Grouping of participants can be hierarchical in some implementations. Participants of a meeting can be part of a larger group ("general group")— within this general group, subgroups can be defined. The collaborative group can be considered a subgroup that is part of the general group. In such implementations, the active participant can be part of both the general group and the collaborative group. The general group can further include another subgroup, such as an audience group that includes non-active participants.

[0012] The assignment of participants to groups is not static, but rather the group membership of participants can be dynamically changed during a meeting. In fact, during a meeting, participants can move between groups. At a first time, a first participant can be considered an active participant, and is thus associated with the collaborative group. Other participants can be part of a second group that includes non-active participants. At a second, later time, a second participant can become an active participant, at which point the second participant is moved to the collaborative group, and the first participant is moved to the second group.

[0013] To enhance non-verbal communication associated with references to a particular resource that is part of the collaborative group, a display layout can be provided by the remote conferencing system to position a media stream

corresponding to the resource in proximity to a media stream corresponding to the active participant, to visually associate the resource with the active participant. The display layout provided by the remote conferencing system may be a rule-based display layout, where a specific display layout is defined according to a predefined set of rules that are intended to improve understanding of non-verbal

communications among participants of a meeting established over a

communications network. For example, one rule of the predefined set of rules is that the media stream for an active participant is to be positioned in the proximity of the media stream for the associated resource (that is in the collaborative group).

"Proximity" can be defined in different ways. In some examples, the media streams for the active participant and the resource are in "proximity" if the media streams are adjacent each other (or within some predefined distance of each other) along a particular direction (e.g. along a row or column) in a display area of a display device. As other examples, the media streams for the active participant and the resource are in "proximity" if the media stream of the participant is combined with the media stream of the resource (e.g. the media stream of the participant overlays the media stream of the resource, or vice versa). As yet further examples, the media streams for the active participant and the resource are in "proximity" if there is some indication of association between the media streams (e.g. , a graphic linking the media streams).

[0014] Fig. 1 illustrates an example display layout of media streams

corresponding to participants of a meeting and a resource in a display device 102, where the display device 1 02 can be part of an endpoint. An endpoint can refer to equipment that is usable by a participant to communicate audio/video data and other data in the meeting. Examples of an endpoint can include a computer (e.g. a desktop computer, a notebook computer, a tablet computer, etc.), a personal digital assistant (PDA), a smartphone, a conferencing system in a meeting room, and so forth.

[0015] The display device 102 includes an overall display area 104 (e.g. a display window) for displaying media streams associated with the meeting. The media streams include media streams for various participants, which are displayed in respective participant display regions 106 and 108 of the display area 104. In addition, the media stream for a resource that can be referenced during the meeting is displayed in a resource display region 1 10. [0016] In the Fig. 1 example, one of the participants is designated the active participant, while the other participants are non-active participants. The media stream for the active participant is displayed in the participant display region 106, while the media streams for the non-active participants are displayed in the participant display regions 108.

[0017] The display layout for the media streams of the participants and the resource of the meeting is defined based on the groups defined as well as a predefined set of rules. In the example of Fig. 1 , the media stream for the resource and the media stream for the active participant are grouped into a collaborative group, while the media streams for the non-active participants are grouped into at least one other group.

[0018] The display layout positions the display regions 106 and 1 10

corresponding to the collaborative group (which includes the active participant and the resource) in proximity with each other to visually associate the resource with the active participant. Additionally, the display layout also contains display regions 108 corresponding to the at least one other group that includes the non-active participants.

[0019] The participant display regions 108 that display the media streams for the non-active participants are also visually associated in a manner that allows for ease of recognition that the non-active participants are distinct from the members of the collaborative group. In the example of Fig. 1 , the participant display regions 108 are each generally smaller in size than the display regions 106 and 1 10.

[0020] The participant display regions 108 for displaying the media streams for the non-active participants are provided in a first portion of the display area 104, while the display regions 106 and 1 10 (corresponding to the media streams for the active participant and the resource) are provided in a second portion of the display area 104. In this manner, any non-verbal communication made by the active participant with respect to the resource may be captured and conveyed to the remaining participants of the meeting. [0021 ] The display regions for the non-active participants can also have some predetermined order, such as a seating order of the non-active participants at respective conference sites. Such predetermined order is considered in producing a display layout that includes display regions corresponding to media streams of the meeting. Note that an order can also be specified in a display layout between a display region for a resource and a display region for a participant.

[0022] While Fig. 1 shows an example that includes one active participant, a different example can involve multiple active participants that are associated with respective resources, as depicted in Fig. 2. Fig. 2 shows display areas 202 and 204, where the display area 202 includes participant display regions 206 for a group (or groups) of non-active participants. The display area 204 includes participant display regions 208 and 210 to display corresponding media streams for active participants A and B. The display area 204 also includes resource display regions 212 and 214 to display corresponding media streams for resources A and B.

[0023] The participant display region 208 and the resource display region 212 are placed in proximity to each other (along a particular row of the display area 204, for example) due to the grouping of active participant A and resource A. Similarly, the display regions 210 and 214 are placed in proximity to each other (along a corresponding row, for example) due to the grouping of active participant B with resource B. In an example according to Fig. 2, active participant A and resource A can be part of a first collaborative group, while active participant B and resource B can be part of a second collaborative group.

[0024] In some examples, the display areas 202 and 204 can be displayed on separate display devices, such as in an implementation where an endpoint has two display devices. Alternatively, the display areas 202 and 204 can be displayed in different portions of a single display device.

[0025] Fig. 3 illustrates another example collaboration, in which two active participants (A and B) are working on a common resource, displayed in resource display region 302. Media streams of the active participants are displayed in participant display regions 304 and 308. In the Fig. 3 example, the active participants displayed in the participant display regions 304 and 306 can be provided in the same collaborative group as the resource displayed in the resource display region 302. The display regions 302, 304, and 306 are part of a display area 308.

[0026] Another display area 310 includes display regions 312 for non-active participants. As with the Fig. 2 example, the display areas 308 and 310 can be displayed by multiple respective display devices or a single display device.

[0027] Fig. 4 illustrates a further example, in which participant display regions 402 and 404 for active participants A and B overlay a resource display region 406 that contains a media stream for a resource that the active participants are referencing. The display regions 402, 404, and 406 are part of a display area 408. Another display area 410 contains display regions 412 for displaying media streams for the non-active participants.

[0028] Fig. 5 illustrates yet a further example, which depicts a virtual room 502 that has a display area 504 representing a whiteboard or other presentation surface. The display area 504 has a display region 506 to display a media stream for a resource that is referenced by active participants displayed in display regions 508 and 510, which are also part of the display area 504. Fig. 5 further shows display regions 512 for displaying media streams for the non-active participants.

[0029] Fig. 6 is a flow diagram of a process for providing a display layout for a meeting, in accordance with some implementations. The process of Fig. 6 can be performed by an electronic device or a combination of electronic devices, including as examples endpoints of the meeting, a meeting controller (e.g., a multipoint control unit (MCU)), and/or any other intermediate device between endpoints.

[0030] The process identifies (at 602) an active participant from among multiple participants of a meeting. The identification of an active participant can employ any of various techniques. In some examples, the identification of the active participant can be based on processing the media streams of the multiple participants of the meeting and determining which of the multiple participants is associated with the most activity (e.g. loudest audio and/or most consistent stream of audio, most animated activity, etc.)- Alternatively, or additionally, the identification of the active participant may be based on a physical location of the participant, such as a participant who has moved to a position (a presentation podium, a position next to a whiteboard, etc.) in the proximity of the resource. Also alternatively, or additionally, the participants of the meeting (or a host participant) can identify which participant is the active participant. Although reference is made to identifying an active participant in the singular sense, note that in alternative examples, plural active participants can be identified.

[0031 ] The process then associates (at 604) the active participant with the resource that is to be referenced by the active participant, in a collaborative group. The collaborative group is distinct from other group(s) that is (are) defined for the meeting, where the other group(s) can include a group of non-active participants.

[0032] The process then provides (at 606) a display layout containing display regions for displaying media streams corresponding to the groups of the meeting, including the collaborative group, the group of non-active participants, and possibly other groups. The display regions corresponding to the collaborative group includes display regions for media streams of the resource and the active participant, and these display regions are placed in proximity with each other to visually associate the resource with the active participant. The display regions corresponding to the other group(s) include display regions for other participants of the meeting.

[0033] As noted above, group membership can dynamically change during a meeting. Different participants of the meeting can become active participants at different times. In response to such changes in group membership during the meeting, the process of Fig. 6 can be reiterated.

[0034] A change in membership of a group is an example of change in the meeting that is propagated to the various endpoints involved in the meeting. In some examples, local changes can also be made at endpoints. For example, a local user may wish to resize or reorganize their local display device. Layout rules for a collaborative group may be flexible enough to allow a number of local user adjustments. These local adjustments would not be seen by other participants. Some local changes may imply changes to encoding, such as when an image is resized which can lead to reduced or increased resolution. Such changes can be cascaded to a meeting controller and to remote conference sites, but are not classified as meeting changes.

[0035] Fig. 7 is a block diagram illustrating an example remote conferencing system that includes N (N > 1 ) conference sites (conference site A, conference site B, conference site N shown in Fig. 7). Each conference site has a respective endpoint 702, 704, and 706, as shown in Fig. 7. The endpoints at the respective conference sites can communicate with one another directly over a network 708, such as the Internet or other network. Alternatively, the endpoints can communicate with each other using one or multiple intermediate devices, including a meeting controller 710 (e.g. an MCU).

[0036] Each endpoint 702, 704, or 706 is connected to a respective display device(s) 712, 714, or 716, as well as to a respective camera(s) 718, 720, or 722. The display devices are used to display media streams of respective participants of a meeting established using the remote conferencing system of Fig. 7. The cameras are used to capture video streams of the participants. In alternative examples, a particular endpoint can be connected to multiple display devices and/or cameras (such as multiple cameras to capture participants at a site from different angles).

[0037] Although not shown, the endpoints 702, 704, and 706 can also include respective audio capture equipment to capture audio streams. Additionally, the endpoints can include respective equipment to allow for collaboration of resources, such as documents and so forth. The endpoints 702, 704, and 706 are able to exchange audio/video streams and resources with each other over the network 708.

[0038] As shown in Fig. 7, the endpoint 702 includes a grouping module 724 and a display layout module 726 that are executable on one or multiple processors 728. The processor(s) 728 is (are) connected to a storage medium 730. The grouping module 724 is able to determine the group membership of participants and resources, including the identification of the membership of the collaborative group and other group(s) of a meeting, as discussed above. Group information 732 containing a description of membership of the groups is stored in the storage medium 730.

[0039] The display layout module 726 is able to generate a display layout, given identified groups and display layout rules 734 (which are stored in the storage medium 730). The display layout rules 734 can include central layout rules that are applicable to a meeting in general (to be applied for display layouts for the various endpoints of the meeting) and local layout rules (which can depend upon the type of endpoint, e.g., an endpoint having a smaller display device versus an endpoint having a larger display device).

[0040] The other endpoints 704 and 706 can include similar components as the endpoint 702.

[0041 ] Instead of including the grouping module 724 and/or display layout module 726 in each endpoint, the grouping module and/or display layout module can instead be included in another device, such an the meeting controller 710. As shown in Fig. 7, the meeting controller 710 includes a grouping module 736 and a display layout module 738 executable on processor(s) 740. The grouping module 736 and display layout module 738 can perform similar tasks as those of the grouping module 724 and display layout module 726 in the endpoint 702.

[0042] The meeting controller 710 includes a storage medium 742 storing group information 744 (describing group memberships in respective groups) and display layout rules 746.

[0043] By using techniques or mechanisms according to some implementations, association of an active participant media stream and a media stream of a resource can be provided, to enhance recognition of non-verbal communications made with respect to the resource. Participants of the meeting can also easily see who is actively referencing the resource, and what resource the active participant is working on. A rule-based layout system is provided that can respond to changes in the meeting. [0044] Various modules discussed above, including modules 724, 7826, 736, and 738 can be in the form of machine-readable instructions that can be loaded for execution on a processor or processors (e.g. 728 or 740 in Fig. 7). A processor can include a microprocessor, microcontroller, processor module or subsystem, programmable integrated circuit, programmable gate array, or another control or computing device.

[0045] Data and instructions are stored in respective storage devices, which are implemented as one or more computer-readable or machine-readable storage media. The storage media include different forms of memory including

semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; optical media such as compact disks (CDs) or digital video disks (DVDs); or other types of storage devices. Note that the instructions discussed above can be provided on one computer-readable or machine-readable storage medium, or alternatively, can be provided on multiple computer-readable or machine-readable storage media distributed in a large system having possibly plural nodes. Such computer-readable or machine-readable storage medium or media is (are) considered to be part of an article (or article of manufacture). An article or article of manufacture can refer to any manufactured single component or multiple components. The storage medium or media can be located either in the machine running the machine-readable instructions, or located at a remote site from which machine-readable instructions can be downloaded over a network for execution.

[0046] In the foregoing description, numerous details are set forth to provide an understanding of the subject disclosed herein. However, implementations may be practiced without some or all of these details. Other implementations may include modifications and variations from the details discussed above. It is intended that the appended claims cover such modifications and variations.

Claims

What is claimed is:

1 . A method of a system having a processor, comprising:

identifying an active participant from among a plurality of participants of a meeting established over a network, where the active participant is to reference a resource;

associating, in a first of plural groups, the active participant and the resource, and associating, in a second of the plural groups, further participants of the meeting; and

providing a display layout containing display regions for displaying media streams corresponding to the first and second groups, wherein the display regions corresponding to the first group include display regions for the resource and the active participant that are placed in proximity with each other to visually associate the resource with the active participant, and the display regions corresponding to the second group include display regions for the further participants.

2. The method of claim 1 , further comprising:

during the meeting, changing membership of participants in the first and second groups, and modifying the display layout in response to the changed membership.

3. The method of claim 1 , wherein identifying the active participant comprises identifying one of the plurality of participants that is performing at least one of the following: making a presentation with respect to the resource, gesturing towards the resource, or being located at a location corresponding to the resource.

4. The method of claim 1 , wherein the active participant and the further participants are members of a general group, and wherein identifying the active participant causes the active participant to also be designated as a member of the first group.

5. The method of claim 1 , wherein the identifying, associating, and providing are performed at an endpoint participating in the meeting.

6. The method of claim 1 , wherein the identifying, associating, and providing are performed at an intermediate device between endpoints participating in the meeting.

7. The method of claim 1 , wherein providing the display layout is based on layout rules, and wherein associating participants in the first and second groups comprises producing group information describing membership in the first and second groups.

8. The method of claim 1 , further comprising identifying a second active participant from among the plurality of participants of the meeting, where the second active participant is to reference a second resource;

associating, in a third of the plural groups, the second active participant with the second resource, and

wherein providing the display layout comprises providing the display layout that has display regions for displaying media streams corresponding to the second resource and the second active participant, wherein the display regions for displaying the media streams corresponding to the second resource and the second participant are placed in proximity with each other to visually associate the second resource with the second active participant.

9. The method of claim 1 , further comprising:

identifying a second active participant from among the plurality of participants, wherein the second active participant is to also reference the resource,

wherein the associating further associates, in the first group, the second active participant with the first active participant and the resource, and

wherein providing the display layout further comprises providing the display layout having display regions corresponding to the first group that includes display regions for the media streams of the resource and the active participants.

10. A system comprising:

a storage medium to store layout rules pertaining to a display layout for a meeting established over a communications network; and

at least one processor to:

group participants of the meeting into a plurality of groups, wherein the plurality of groups include a collaborative group that includes an active participant from among the participants;

group a resource in the collaborative group, wherein the resource is to be referenced by the active participant during the meeting;

generate a display layout based on the plurality of groups and the layout rules, wherein the display layout includes plural collections of display regions for displaying media streams corresponding to the plurality of groups, wherein the display regions corresponding to the collaborative group include a display region for displaying a media stream of the active participant placed in proximity to a display region for displaying a media stream of the resource.

1 1 . The system of claim 10, wherein the resource includes visual material that is other than any of the participants of the meeting.

12. The system of claim 10, wherein the at least one processor is to further: during the meeting, change membership of the collaborative group, and modify the display layout in response to the changed membership.

13. The system of claim 12, wherein the plurality of groups include a general group including plural ones of the participants, and wherein the active participant is part of both the general group and the collaborative group.

14. The system of claim 10, wherein the collaborative group further includes a second active participant who is to reference the resource.

15. An article comprising at least one machine-readable storage medium storing instructions that upon execution cause a system to:

identify an active participant from among a plurality of participants of a meeting established over a network, where the active participant is to reference a resource; associate, in a first of plural groups, the active participant and the resource, and associating, in a second of the plural groups, further participants of the meeting; and

provide a display layout containing display regions for displaying media streams corresponding to the first and second groups, wherein the display regions corresponding to the first group include display regions for the resource and the active participant that are placed in proximity with each other to visually associate the resource with the active participant, and the display regions corresponding to the second group include display regions for the further participants.