EP2469853A1 - Method and device for processing video image data, system and terminal for video conference - Google Patents
Method and device for processing video image data, system and terminal for video conference Download PDFInfo
- Publication number
- EP2469853A1 EP2469853A1 EP10815004A EP10815004A EP2469853A1 EP 2469853 A1 EP2469853 A1 EP 2469853A1 EP 10815004 A EP10815004 A EP 10815004A EP 10815004 A EP10815004 A EP 10815004A EP 2469853 A1 EP2469853 A1 EP 2469853A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- video image
- image data
- data
- correlative
- multiple channels
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 93
- 238000012545 processing Methods 0.000 title claims abstract description 39
- 230000008569 process Effects 0.000 claims abstract description 60
- 238000004891 communication Methods 0.000 claims description 61
- 230000001360 synchronised effect Effects 0.000 claims description 44
- 238000000638 solvent extraction Methods 0.000 claims description 19
- 238000010586 diagram Methods 0.000 description 18
- 230000000007 visual effect Effects 0.000 description 8
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 6
- 238000009434 installation Methods 0.000 description 4
- 101000969688 Homo sapiens Macrophage-expressed gene 1 protein Proteins 0.000 description 3
- 102100021285 Macrophage-expressed gene 1 protein Human genes 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 230000006798 recombination Effects 0.000 description 3
- 238000005215 recombination Methods 0.000 description 3
- GNFTZDOKVXKIBK-UHFFFAOYSA-N 3-(2-methoxyethoxy)benzohydrazide Chemical compound COCCOC1=CC=CC(C(=O)NN)=C1 GNFTZDOKVXKIBK-UHFFFAOYSA-N 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- YTAHJIFKAKIKAV-XNMGPUDCSA-N [(1R)-3-morpholin-4-yl-1-phenylpropyl] N-[(3S)-2-oxo-5-phenyl-1,3-dihydro-1,4-benzodiazepin-3-yl]carbamate Chemical compound O=C1[C@H](N=C(C2=C(N1)C=CC=C2)C1=CC=CC=C1)NC(O[C@H](CCN1CCOCC1)C1=CC=CC=C1)=O YTAHJIFKAKIKAV-XNMGPUDCSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000005242 forging Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000006249 magnetic particle Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
Definitions
- the present invention relates to the field of communications, and in particular, to a method and an apparatus for processing video image data, a videoconferencing system and a videoconferencing terminal.
- a videoconferencing terminal at a transmitting end captures an image by using a single high definition camera, where resolutions of captured high definition videos are generally 720p30f, 720p60f, 1080130f, 1080i60f, 1080p30f, and 1080p60f, and then the captured videos are compressed and coded to generate a video code stream; then, through a data transmission network, the video code stream is transmitted to a videoconferencing terminal at a receiving end; and the videoconferencing terminal at the receiving end decodes the received video code stream to obtain a high definition video image from the transmitting end and display the image.
- the videoconferencing terminal can provide a higher video resolution than that provided by a standard definition videoconferencing terminal, and can bring better visual experience to users.
- a viewing angle of a provided video image is limited.
- a structure of the system includes multiple video terminals, each video terminal is equipped with a high definition camera, and multiple cameras are placed strictly at physical positions, so that when collected multiple channels of video images are displayed on multiple display devices in the same horizontal plane, the viewer can get a consecutive feeling.
- this solution has a strict demand on the decoration and layout of a conference room, especially on a position of a camera set and a distance between a user and the camera set; otherwise, an overlapping phenomenon may occur on an image that is displayed on a display device, and this strict demand results in complex installation of the system.
- embodiments of the present invention provide a method and an apparatus for processing video image data, a videoconferencing system and a videoconferencing terminal, to solve a problem that installation of a system is complex in the prior art.
- a method for processing video image data includes:
- An apparatus for processing video image data includes:
- An apparatus for processing video image data includes:
- An apparatus for processing video image data includes:
- a videoconferencing system includes:
- a videoconferencing system includes:
- a videoconferencing system includes:
- a videoconferencing terminal includes:
- a videoconferencing terminal includes:
- a videoconferencing terminal includes:
- the multiple channels of video image data collected by the multiple cameras are obtained, the multiple channels of video image data are processed into the single channel of panoramic video image data, and single channel of are recombined into several channels of video image data according to a display requirement for display.
- an operation of processing the multiple channels of video image data into the single channel of panoramic video image data may eliminate an overlapping situation existing between each channel of video image data. Therefore, the overlapping situation existing between each channel of video image data collected by the camera is allowed, so that requirements on a position where athe camera set is placed and a distance between a user and the camera set are lowered, and installation complexity of the system is simplified.
- An embodiment of the present invention discloses a method for processing video image data, where obtained multiple channels of correlative video image data are combined into a single channel of panoramic video image data, and according to a display requirement, the panoramic video image data is recombined into one or multiple channels (equal to the number of display devices) of video image data, and the video image data is displayed by a display device.
- the number of display devices may be multiple, and after combined panoramic video image data is recombined into multiple channels of video image data, the recombined video image data may be respectively sent to each display device.
- the display device performs display according to a position of each channel of video image data in the panoramic video image data, so as to provide wide viewing-angle visual experience for users. As shown in FIG. 4 , a specific process includes the following steps.
- Step S41 Obtain multiple channels of correlative video image data and correlative information between each channel of video image data.
- the multiple channels of correlative video image data are from multiple cameras that are disposed in the same scenario, and these cameras are placed at different positions in the scenario.
- the correlative information includes: physical position information that is used to indicate a video image data, and capture timestamp information of the video image data.
- Step S42 Combine the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information.
- the multiple channels of video image data are combined according to physical position information and capture timestamp information of each channel of video image data, to form the single channel of panoramic video image data.
- Step S43 Recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement.
- the panoramic video image data is recombined and filtered into the corresponding number of channels of video image data according to the number of display devices, a supported size of a frame, and a supported format of the video image data.
- Step S44 Send each channel of recombined video image data to each display device for display respectively.
- the panoramic video image data is recombined into four channels according to display positions, and the four channels are respectively transmitted to corresponding display devices for display, so that displayed images may be combined into a wide viewing-angle video image if each display device is arranged according to positions of the images in the panoramic video image data.
- the number of the display devices is three, as shown in FIG. 5b , which are respectively 51b, 52b and 53b, the panoramic video image data is recombined into three channels according to image display positions, and the three channels are respectively transmitted to corresponding display devices for display.
- a size of a display frame supported by a display device may be different, so that when the panoramic video image data is recombined, the panoramic video image data needs to be recombined, according to the size of the display frame supported by the display device, into video image data with a corresponding size, for example, a display device supports an HDMI video input interface and meanwhile supports a 1080p video format, and a resolution of a panoramic image is 4000*1080, and therefore, when panoramic video image data is recombined, the panoramic video image data is properly recombined and filtered into two channels of 1080p video image data with a resolution of 1920*1080 for display.
- a type of a video interface for obtaining video images may be any one or several kinds of the following: a YPbPr interface, a DVI interface, an HDMI interface and a VGA interface, that is, video input interfaces provided by each camera may be the same, and may also be different.
- a format of the recombined video image data is consistent with a video image format supported by a display device, and is determined according to the video image format supported by the display device.
- a video input interface type in the forgoing step S41 1 and a video output interface type in step S44 may be the same (for example, the video input interface is a YPbPr interface, and the video output interface is also the YPbPr interface) or different (for example, the video input interface is the YPbPr interface, and the video output interface may be an HDMI interface).
- a format of each channel of video image data also needs to be converted respectively according to a video interface type and a video image format that are supported by a corresponding display device, and then the video image data is sent to the corresponding display device.
- the multiple channels of video image data collected by the multiple cameras are obtained, the multiple channels of video image data are combined into the single channel of panoramic video image data, and are recombined into several channels of video image data according to a display requirement, and then the several channels of video image data are sent to display devices for display.
- the displayed video images can be combined into a wide viewing-angle video image by merely arranging the display devices according to positions of the video images in the panoramic video image data, so as to provide better visual experience for users.
- a process of combining the multiple channels of video image data into the single channel of panoramic video image data may eliminate an overlapping situation existing between each channel of video image data. Therefore, an overlapping phenomenon existing between images obtained by each camera may be allowed, which means that no particularly strict requirement is imposed on a position where a camera is placed and a distance between a user and a camera set, so that installation complexity of the camera is lowered.
- the display device may also be a display device that is adaptive to the panoramic video image data, and in this case, the number of display devices may be one.
- the multiple channels of video image data are respectively sent to the display devices according to positions of the multiple channels of video image data in the panoramic video image data, and the display device combines each channel of video image data into panoramic video image data for display.
- This embodiment of the present invention may be applied to a remote panoramic videoconferencing process, where each party taking part in a conference may send their own video image data to an opposite party (that is, a video image data sending process), and receive and display video image data that is sent by the opposite party (that is, a video image data receiving process).
- the video image data sending process includes the following steps.
- Step S61 Obtain video image data collected by multiple cameras placed at a local conference site and correlative information between each channel of video image data.
- Each camera is placed at a different position, but obtained video image data is correlative, and the correlative information includes a physical position and a capture timestamp of each channel of video image data.
- Step S62 Combine the multiple channels of video image data into a single channel of panoramic video image data according to a physical position and a capture timestamp of each channel of video image data.
- Step S63 Send the panoramic video image data through a communication network.
- step S61 processes of obtaining the video image data collected by the multiple cameras and obtaining the correlative information between each channel of video image data are implemented simultaneously, and it is doubtless that, to enable a user in front of multiple displayers to view frames captured by the cameras at the same same, it must be ensured that the multiple cameras collect scenario images synchronously. In addition, to ensure integrity of transmitted video images, it must be ensured that no disconnection occurs between scenario images shot by adjacent cameras, and an overlapping area is preferred, where the overlapping area may be removed in an image combining process.
- a network interface of the communication network may be: an ISDN interface, an E1 interface, or a V35 interface, where the ISDN interface, the E1 interface, or the V35 interface is based on circuit switching, an Ethernet interface based on packet switching, or a wireless port based on a wireless connection.
- the video image data receiving process includes the following steps.
- Step 571 Obtain panoramic video image data sent from a remote conference site through a communication network.
- Step S72 Recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement.
- Step S73 Send each channel of video image data to a corresponding display device for display.
- the video image data sending process may also be: after obtaining multiple channels of video image data and correlative information between each channel of video image data, directly sending the obtained video image data and correlative information through a communication network.
- the video image data receiving process is: after receiving the multiple channels of video image data and the correlative information, combining the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information, and recombining the panoramic video image data into multiple channels of video image data according to the number of display devices and sending the recombined video image data to corresponding display devices for display.
- the correlative information between the multiple channels of video image data may be embedded in the video image data (or compressed video image data) for transmission, for example, when the communication network is the Ethernet, the correlative information may be embedded in a video RTP packet for transmission, which facilitates synchronization between the correlative information and the video image data.
- the correlative information may also be transmitted separately, for example, transmitted through an independent data channel.
- the video image data sending process may further be: after obtaining multiple channels of video image data and correlative information between each channel of video image data, combining the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information, and after recombining the panoramic video image data into multiple channels of video image data according to the number of display devices (display devices at the remote coferrence site), sending the recombined video image data through a communication network.
- the video image data receiving process is: receiving the multiple channels of recombined video image data, and directly sending the received video image data to display devices at a local conference site for display.
- a transmitting end may send the panoramic video image data directly, and may also send the panoramic video image data after coding.
- the coding manner may be: H.261, H.263, H.264, MPEG1, MPEG2 or MPEG4.
- the panoramic video image data received by a receiving end may be uncoded raw data, and may also be coded data. It should be noted that, a size of a combined image is generally several times larger than a size of an original image, and in this case, even if a coder is used for coding, the amount of transmitted data is still larger, which imposes a strict requirement on capability of the coder.
- multiple coders are adopted for parallel processing, and furthermore, due to randomness of image data, synchronization of a sequence of coded data cannot be ensured, and to ensure that images displayed by multiple displayers at a display end are shot at the same time, the coded data needs to be synchronized.
- the forgoing process of recombining the panoramic video image data is actually an image partitioning process, which includes the following steps.
- the forgoing method for processing video image data further includes a synchronization process and a reconstruction process, which are respectively introduced as follows.
- the synchronization process is as follows:
- a device for implementing the forgoing method includes a receiving buffer, a reconstruction buffer, and a sending buffer.
- the receiving buffer receives partitioned sub-images, where synchronous information of sub-images that belong to the same panoramic image meets a preset rule, for example, the synchronous information is the same or consecutive, the reconstruction buffer stores a sub-image to be reconstructed, and the sending buffer stores a reconstructed image.
- the reconstruction information may be a partitioning manner, and the reconstruction process is: reconstructing the classified sub-images according to the partitioning manner to obtain multiple channels of video image data, where each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data.
- the synchronization process and the reconstruction process may specifically include:
- the buffer is not released at once, so that when the process proceeds to step f from step e or step h, the image stored in the sending buffer is a previous frame of image that is successfully reconstructed; and the image stored in the sending buffer is updated in step k, where the update may be implemented in a data overwriting manner, and may also be implemented in a manner of releasing the sending buffer and then storing data in the sending buffer, or in another data updating manner.
- a coding manner may be a compression standard code stream format that meets various current mainstream standards, such as h261, h263, h263++, mpeg1, mpeg2 or mpeg4.
- the decoding is performed first, and being corresponding to multiple coders in the recombining process, multiple decoders may also be set. Afterward, the decoded sub-images are classified according to the synchronous information to find multiple sub-images obtained by partitioning the same panoramic video image data, that is, image information obtained at the same time.
- An embodiment of the present invention further discloses an apparatus for processing video image data, which may implement the method disclosed in the foregoing embodiment.
- FIG. 8 A structural form of the apparatus for processing video image data is shown in FIG. 8 , which includes a data combining unit 81, a data recombining unit 82, data input interfaces 83, and a data output unit 84.
- the data input interfaces 83 are multiple, which are respectively connected to multiple cameras, and configured to obtain multiple channels of video image data and correlative information between each channel of video image data, where the correlative information includes: information that is used to indicate a physical position of the video image data, and capture timestamp information of the video image data.
- the data combining unit 81 is configured to combine the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information. Specifically, according to a physical position and capture time of each channel of video image data, the multiple channels of video image data are combined into the single channel of panoramic video image data.
- the data recombining unit 82 is configured to, according to the number and the size of display devices and a video image format supported by the display devices, recombine the single channel of panoramic video image data into multiple channels of video image data satisfying a display requirment of multiple display devices.
- the data output unit 84 is configured to send, through a communication network, the multiple channels of video image data processed and obtained by the data recombining unit 82 to a remote videoconferencing device (which may be a terminal, and may also be an MCU).
- a remote videoconferencing device which may be a terminal, and may also be an MCU.
- the remote videoconferencing device may arrange the multiple channels of video image data according to positions of video images in the panoramic video image, and then transmits the video image data to the multiple display devices.
- Video images displayed by all the display devices may be combined into a wide viewing-angle video image, so as to bring panoramic visual experience to users.
- the data input interface 83 may be a YPbPr interface, a DVI interface, an HDMI interface, or a VGA interface.
- another structure of the apparatus for processing video image data may further include a functional unit for compression and coding.
- the apparatus includes a data combining unit 91, a data recombining unit 92, data input interfaces 93, a data output unit 94, and a data coder 95.
- Functions of the data combining unit 91, the data recombining unit 92, the data input interface 93, and the data output unit 94 are basically the same as functions of the data combining unit 81, the data recombining unit 82, the data input interface 83, and the data output unit 84 respectively.
- the data coder 95 is configured to obtain multiple channels of video image data recombined and obtained by the data recombining unit 92, and after the obtained video image data is coded, provide the coded video image data for the data output unit 94.
- a coding manner may be: H.261, H.263, H.264, MPEG1, MPEG2, or MPEG4.
- each channel of video image data processed and obtained by the data recombining unit 92 includes: each sub-image obtained by recombining the panoramic video image and corresponding synchronous information and reconstruction information of each sub-image.
- a device that receivs the multiple channels of video image data may perform a synchronization process and a reconstruction process according to the synchronous information and the reconstruction information, where specific content of the synchronization process and the reconstruction process may be made referrence to the description of the foregoing method, and is not repeated here.
- the device that receivs the multiple channels of video image data is another structural form of the apparatus for processing video image data, which includes multiple data input interfaces and multiple data output interfaces, and further includes data decoders, a data synchronizing unit, and a data reconstructing unit.
- the data input interface is configured to obtain multiple channels of coded video image data.
- the data decoders are multiple, which are configured to simultaneously decode the multiple channels of coded video image data, where multiple channels of decoded video image data include multiple sub-images obtained by partitioning the panoramic video image and corresponding synchronous information and reconstruction information of the multiple sub-images.
- the data synchronizing unit is configured to classify the decoded sub-images according to corresponding synchronous information of the decoded sub-images, and a specific process may be made referrence to the description in the foregoing method embodiment.
- the data reconstructing unit is configured to reconstruct the classified sub-images according to the reconstruction information to obtain multiple channels of video image data, and provide the obtained video image data to the data output interface, where each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data.
- FIG. 10 Another structural form of the apparatus for processing video image data is shown in FIG. 10 , which includes a data combining unit 101, a data recombining unit 102, a data input interface 103, and multiple data output interfaces 104.
- Functions of the data combining unit 101 and the data recombining unit 102 are basically the same as the functions of the data combining unit 81 and the data recombining unit 82 respectively.
- a difference between this structure and the structure shown in FIG. 8 is that, multiple channels of video image data and correlative information between each channel of video image data are sent by another device through a communication network, where the multiple channels of video image data and the correlative information between each channel of video image data are obtained by the data input interface 103.
- the multiple data output interfaces 104 are respectively connected to multiple display devices, and configured to arrange, according to positions of video images in the panoramic video image, multiple channels of video image data processed and obtained by the data recombining unit 102, and send the multiple channels of video image data to display devices, where video images displayed by all the display devices may be combined into a wide viewing-angle video image.
- the data input interface 103 may be formed by a network interface and a data receiving unit, where the network interface is configured to establish a connection with the communication network, and the data receiving unit is configured to receive, through the network interface, video image data transmitted by another device through the communication network.
- the network interface may be an ISDN interface, an E1 interface, or a V35 interface, where the ISDN interface, the E1 interface, or the V35 interface is based on circuit switching, an Ethernet interface based on packet switching, or a wireless port based on a wireless connection.
- FIG. 11 another structure of the apparatus for processing video image data needs to include a functional unit for decoding, as shown in FIG. 11 , which include a data combining unit 111, a data recombining unit 112, a data input interface 113, and multiple data output interfaces 114, and further include a data decoder 115.
- Functions of the data combining unit 111, the data recombining unit 112, the data input interface 113, and the data output interface 114 are basically the same as the functions of the data combining unit 101, the data recombining unit 102, the data input interface 103, and the data output interface 104 respectively.
- the data decoder 115 is configured to decode multiple channels of video image data and correlative information between each channel of video image data, where the multiple channels of video image data and the correlative information between each channel of video image data are obtained by the data input interface 113, and provide the decoded multiple channels of video image data and correlative information between each channel of video image data for the data combining unit 111.
- an embodiment of the present invention further provides a videoconferencing system, and a specific structure of the system is shown in FIG. 12 , which includes a data combining unit 121, a data sending unit 122, a data receiving unit 123, a data recombining unit 124, multiple data input interfaces 125, and multiple data output interfaces 126.
- the data combining unit 121, the data sending unit 122, and the data input interfaces 125 are located at a videoconferencing site at one side.
- the multiple data input interfaces 125 obtain multiple channels of video image data and correlative information between each channel of video image data, the data combining unit 121 combines the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information, and then the data sending unit 122 sends the single channel of panoramic video image data to a remote conference site at the other side through a communication network.
- the data receiving unit 123, the data recombining unit 124, and the data output interfaces 126 are located at the remote conference site at the other side.
- the data receiving unit 123 receives the single channel of panoramic video image data that is carried on the communication network, and then provides the panoramic video image data to the data recombining unit 124, the data recombining unit 124 recombines, according to the number of display devices, a supported size of a frame, and a supported video image format, the single channel of panoramic video image data into multiple channels of video image data satisfying a display requirement of multiple display devices, and the data output interfaces 126 provide the video image data to corresponding display devices.
- the display devices are placed according to positions of the video images in the panoramic video image, and the video images displayed by all the display devices may be combined into a wide viewing-angle video image, so as to bring panoramic visual experience to users.
- the data input interface 125 and the data output interface 126 may be YPbPr interfaces, DVI interfaces, HDMI interfaces or VGA interfaces.
- the types of the data input interface 125 and the data output interface 126 may be different, and a format of video image data obtained by the data input interface 125 may be converted according to the type of the data output interface 126 during recombination of the data recombining unit 123.
- the data input interface 125 is a DVI interface
- the obtained video image data is in a DVI format
- the data output interface 126 is an HDMI interface, so that when the data recombining unit 123 recombines video image data, video image data in a DVI format needs to be converted into video image data in an HDMI format.
- the conference sites at both sides are required to play roles of a transmitter and a receiver at the same time, that is, to send video image data from a local conference site through the data input interfaces 125, the data combining unit 121, and the data sending unit 122, and receive and process video image data from a remote conference site through the data receiving unit 123, the data recombining unit 124, and the data output interfaces 126.
- FIG. 13 is another schematic structural diagram of a videoconferencing system according to an embodiment of the present invention.
- the system includes a data combining unit 131, a data sending unit 132, a data receiving unit 133, a data recombining unit 134, multiple data input interfaces 135, and multiple data output interfaces 136.
- the data sending unit 132 and the multiple data input interfaces 135 are located at a conference site at one side, and the data receiving unit 133, the data combining unit 131, the data recombining unit 134, and the multiple data output interfaces 136 are located at a conference site at the other side.
- the multiple data input interfaces 135 obtain multiple channels of video image data and correlative information between each channel of video image data, and the data sending unit 132 sends the multiple channels of video image data and the correlative information between each channel of video image data to the conference site at the other side through a communication network; and the data receiving unit 133 at the conference site at the other side receives the multiple channels of video image data and the correlative information between each channel of video image data, and then provides the received video image data and correlative information for the data combining unit 131, the data combining unit 131 combines the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information, and then provides the single channel of panoramic video image data for the data recombining unit 134, the data recombining unit 134 recombines, according to the number of display devices, a supported size of a frame, and a supported video image format, the single channel of panoramic video image data into multiple channels of video image data satisfying a display requirement of multiple display devices, and
- the data sending units (the data sending units 122 and 132) send the video image data through the communication network, in order to reduce the amount of data to be transmitted and ensure transmission safety, data sent by the data sending units may be coded.
- the data receiving units after receiving the data sent through the communication network, the data receiving units (the data receiving units 123 and 133) decode the data.
- a transmitter after receiving multiple channels of video image data and correlative information, combines the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information, then recombines the single channel of panoramic video image data into several channels of video image data, and sends the several channels of video image data; and a receiver receives the several channels of video image data, and then provides the the several channels of video image data for display devices at a local conference site for display.
- FIG. 14 A specific structural form is shown in FIG. 14 , which includes a data combining unit 141, a data sending unit 142, a data receiving unit 143, a data recombining unit 144, multiple data input interfaces 145, and multiple data output interfaces 146.
- Functions of the units are basically the same as the units in FIG. 12 and FIG. 13 , and a difference lies in that, the data input interfaces 145, the data combining unit 141, the data recombining unit 144, and the data sending unit 142 are located at a conference site at one side, and the data receiving unit 143 and the data output interfaces 146 are located at a conference site at the other side, which means that the data recombining unit 144 located at the conference site at one side needs to recombine video image data according to the number of display devices, a supported size of a frame, and a supported video image format at the conference site at the other side.
- another structure may further include data coders, data decoders, a data synchronizing unit, and a data reconstructing unit.
- the data coders are multiple, which are disposed at the conference site where the data recombining unit 144 is located, and configured to simultaneously process multiple channels of video image data recombined and obtained by the data recombining unit 144, where each channel of video image data recombined and obtained by the data recombining unit 144 includes each sub-image obtained by partitioning the panoramic video image, and corresponding synchronous information and reconstruction information of each sub-image.
- the number of the data decoders is the same as the number of the data coders.
- the data decoders are disposed at the conference site where the data receiving unit 143 is located, and configured to simultaneously decode multiple channels of coded video image data received by the data receiving unit 143.
- the data synchronizing unit is configured to classify, according to corresponding synchronous information of sub-images, sub-images decoded by the data decoders, and a specific process may be made referrence to the description in the foregoing method embodiment.
- the data reconstructing unit is configured to reconstruct the classified sub-images according to the reconstruction information to obtain multiple channels of video image data, and provide the obtained multiple channels of video image data for the data output interface, where each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data.
- this embodiment of the present invention is suitable to a situation that videoconferencing sites at both sides are the same (that is, the number of display devices, a supported size of a frame, and a supported video image format are the same).
- an embodiment of the present invention further discloses a videoconferencing terminal at the same time, and since roles played by both sides of a video conference are mutual (that is, both act as a transmitter and a receiver at the same time), a specific structure of the videoconferencing terminal is shown in FIG. 15 , which includes a data combining unit 151, a data transceiving unit 152, a network interface 153, a data recombining unit 154, multiple data input interfaces 155, and multiple data output interfaces 156.
- the network interface 153 is configured to establish a connection with an external communication network
- the data transceiving unit 152 is configured to obtain data sent from the communication network and send data to the communication network.
- Functions of other functional units such as the data combining unit 151, the data recombining unit 154, the data input interfaces 155, and the data output interfaces 156, may be made referrence to the content of the apparatus for processing video image data and the videoconferencing system in the foregoing description.
- the videoconferencing terminal needs to obtain multiple channels of video image data and correlative information between each channel of video image data at a local conference site, combine the obtained video image data and correlative information into a single channel of panoramic video image data, and send the single channel of panoramic video image data to a remote conference site through the communication network; and meanwhile, as a receiver, the videoconferencing terminal needs to receive a panoramic video image data sent from the remote conference site through the communication network, recombine the panoramic video image data into multiple channels of video image data, and then transmit the multiple channels of video image data to display devices at the local conference site.
- FIG. 16 shows another structure of the videoconferencing terminal, which includes a data combining unit 161, a data transceiving unit 162, a network interface 163, a data recombining unit 164, multiple data input interfaces 165, and multiple data output interfaces 166, and further includes a data coder 167 and a data decoder 168.
- Functions of the data combining unit 161, the data transceiving unit 162, the network interface 163, the data recombining unit 164, the data input interfaces 165, and the data output interfaces 166 are basically the same as the functions of the data combining unit 151, the data transceiving unit 152, the network interface 153, the data recombining unit 154, the data input interfaces 155, and the data output interfaces 156 respectively.
- the data coder 167 codes data before the data transceiving unit 162 sends the data, and the data decoder 168 decodes the data after the data transceiving unit 162 receives the data.
- FIG. 17 Another structure of the videoconferencing terminal is shown in FIG. 17 , which includes a data combining unit 171, a data transceiving unit 172, a network interface 173, a data recombining unit 174, multiple data input interfaces 175, and multiple data output interfaces 176.
- Functions of the units are basically the same as the functions of the units in FIG. 15 respectively.
- FIG. 18 shows another structure of the videoconferencing terminal, which includes a data combining unit 181, a data transceiving unit 182, a network interface 183, a data recombining unit 184, multiple data input interfaces 185, and multiple data output interfaces 186, and further includes a data coder 187 and a data decoder 188.
- Functions of the units are basically the same as the functions of the units in FIG. 16 respectively.
- the videoconferencing terminal obtains multiple channels of video image data and correlative information between each channel of video image data at a local conference site, combines the obtained video image data and correlative information into a single channel of panoramic video image data, recombines, according to the number of display devices, a supported size of a frame, and a supported video image format at a remote conference site, the single channel of panoramic video image data into multiple channels of video image data satisfying a display requirement of multiple display devices, and sends the multiple channels of video image data to the remote conference site through a communication network (or through the communication network after coding).
- the videoconferencing terminal receives multiple channels of video image data sent from the other side through the communication network, provides the received video image data for display devices at the local conference site for display (or provides the received video image data for the display devices at the local conference site for display after decoding).
- multiple coders may be adopted to simultaneously code multiple channels of recombined video image data
- multiple decoders are adopted to simultaneously decode multiple channels of coded video image data
- a synchronization process and a reconstruction process are performed, that is, classifying, according to corresponding synchronous information of sub-images, sub-images decoded by the data decoders, and reconstructing the classified sub-images according to the reconstruction information to obtain multiple channels of video image data, where each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data.
- each embodiment emphasizes a difference from the other embodiments, and the identical or similar parts between the embodiments may be made referrence to each other. Since the apparatuses disclosed in the embodiments are corresponding to the methods disclosed in the embodiments, the description of the apparatuses is simple and relevant parts may be made reference to the description of the methods.
- information, a message, and a signal may be represented by using any one of many different techniques and technologies.
- the message and information mentioned in the forgoing description may be represented as a voltage, a current, an electromagnetic wave, a magnetic field or a magnetic particle, an optical field, or any combination of the forgoing.
- the program may be stored in a computer readable storage medium, and the storage medium may include a ROM, a RAM, a magnetic disk, or an optical disk.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
Description
- This application claims priority to Chinese Patent Application No.
200910161963.9 - The present invention relates to the field of communications, and in particular, to a method and an apparatus for processing video image data, a videoconferencing system and a videoconferencing terminal.
- With development of coding and information compression technologies and rapid development of digital networks, a videoconferencing system emerges and accesses to the market. Since the first series of international standards about the videoconferencing system (H.320) were granted and implemented in the beginning of the 1990s, the videoconferencing system has been applied more and more widely. Meanwhile, demands for voice experience and video experience that are provided by the videoconferencing system are closely concerned. The voice experience is required to evolve towards high-fidelity voice recurrence, and the video experience is required to evolve towards a high resolution and broad viewing angle.
- In an existing videoconferencing television system, a videoconferencing terminal at a transmitting end captures an image by using a single high definition camera, where resolutions of captured high definition videos are generally 720p30f, 720p60f, 1080130f, 1080i60f, 1080p30f, and 1080p60f, and then the captured videos are compressed and coded to generate a video code stream; then, through a data transmission network, the video code stream is transmitted to a videoconferencing terminal at a receiving end; and the videoconferencing terminal at the receiving end decodes the received video code stream to obtain a high definition video image from the transmitting end and display the image.
- The videoconferencing terminal can provide a higher video resolution than that provided by a standard definition videoconferencing terminal, and can bring better visual experience to users. However, a viewing angle of a provided video image is limited.
- The Cisco TelePresence System solves the forgoing problem to some extent. As shown in
FIG. 1 , a structure of the system includes multiple video terminals, each video terminal is equipped with a high definition camera, and multiple cameras are placed strictly at physical positions, so that when collected multiple channels of video images are displayed on multiple display devices in the same horizontal plane, the viewer can get a consecutive feeling. - The prior art has the following disadvantages: this solution has a strict demand on the decoration and layout of a conference room, especially on a position of a camera set and a distance between a user and the camera set; otherwise, an overlapping phenomenon may occur on an image that is displayed on a display device, and this strict demand results in complex installation of the system.
- In view of the forgoing description, embodiments of the present invention provide a method and an apparatus for processing video image data, a videoconferencing system and a videoconferencing terminal, to solve a problem that installation of a system is complex in the prior art.
- The embodiments of the present invention are implemented as follows.
- A method for processing video image data includes:
- obtaining multiple channels of correlative video image data and correlative informationwhere a;
- combining the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information; and
- after recombining the panoramic video image data into multiple channels of video image data satisfying a display requirement, sending the recombined video image data to a display device for display.
- An apparatus for processing video image data includes:
- a data input interface, configured to obtain multiple channels of correlative video image data and correlative information wherea;
- a data combining unit, configured to combine the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;
- a data recombining unit, configured to recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement; and
- multiple data output interfaces, connected to an external display device, and configured to transmit the video image data processed and obtained by the data recombining unit to the display device.
- An apparatus for processing video image data includes:
- a data input interface, configured to obtain multiple channels of correlative video image data and correlative information that are collected by multiple cameraswhere a;
- a data combining unit, configured to combine the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;
- a data recombining unit, configured to recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement; and
- a data sending unit, configured to send, through a communication network, the multiple channels of video image data processed and obtained by the data recombining unit to a remote videoconferencing device, so that the videoconferencing device displays the video image data through a corresponding display device.
- An apparatus for processing video image data includes:
- a data input interface, configured to obtain multiple channels of coded video image data;
- multiple data decoders, configured to simultaneously decode the multiple channels of coded video image data, where multiple channels of decoded video image data include multiple sub-images obtained by partitioning the panoramic video image, and corresponding synchronous information and reconstruction information;
- a data synchronizing unit, configured to classify decoded sub-images according to the corresponding synchronous information;
- a data reconstructing unit, configured to reconstruct, according to the reconstruction information, the classified sub-images, to obtain multiple channels of video image data, where each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data; and
- multiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data processed and obtained by the data reconstructing unit to a corresponding display device.
- A videoconferencing system includes:
- a data input interface, configured to obtain multiple channels of correlative video image data and correlative informationwhere a;
- a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;
- a data sending unit, configured to send, through a communication network, the panoramic video image data ;
- a data receiving unit, configured to receive the panoramic video image data that is carried on the communication network;
- a data recombining unit, configured to recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement, where the decoded panoramic video image data is received by the data receiving unit; and
- multiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data processed and obtained by the data recombining unit to a corresponding display device.
- A videoconferencing system includes:
- a data input interface, configured to obtain multiple channels of correlative video image data and correlative informationwhere a;
- a data sending unit, configured to send the multiple channels of correlative video image data and correlative information through a communication network;
- a data receiving unit, configured to receive the multiple channels of correlative video image data and correlative information that are carried on the communication network;
- a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;
- a data recombining unit, configured to recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement; and
- multiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data processed and obtained by the data recombining unit to a corresponding display device.
- A videoconferencing system includes:
- a data input interface, configured to obtain multiple channels of correlative video image data and correlative informationwhere a;
- a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;
- a data recombining unit, configured to recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement;
- a data sending unit, configured to send, through a communication network, the multiple channels of video image data , wherethe coded multiple channels of video image data is processed and obtained by the data recombining unit;
- a data receiving unit, configured to receive the multiple channels of video image data that are carried on the communication network; and
- multiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data to a corresponding display device, where each channel of decoded video image data is received by the data receiving unit.
- A videoconferencing terminal includes:
- a data input interface, configured to obtain multiple channels of correlative video image data and correlative informationwhere a;
- a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;
- a data transceiving unit, configured to send the panoramic video image data to a remote videoconferencing device through a communication network, and receive a single channel of panoramic video image data sent by the videoconferencing device through the communication network;
- a data recombining unit, configured to recombine the panoramic video image data received by the data transceiving unit into multiple channels of video image data satisfying a display requirement; and
- multiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data processed and obtained by the data recombining unit to a corresponding display device.
- A videoconferencing terminal includes:
- a data input interface, configured to obtain multiple channels of correlative video image data and correlative informationwhere a;
- a data transceiving unit, configured to send the multiple channels of video image data and the correlative information to a remote videoconferencing device through a communication network, and receive multiple channels of video image data and correlative information that are sent by the videoconferencing device through the communication network;
- a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;
- a data recombining unit, configured to recombine the panoramic video image data processed and obtained by the data combining unit into multiple channels of video image data satisfying a display requirement; and
- multiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data processed and obtained by the data recombining unit to a corresponding display device.
- A videoconferencing terminal includes:
- a data input interface, configured to obtain multiple channels of correlative video image data and correlative informationwhere a;
- a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;
- a data recombining unit, configured to recombine the panoramic video image data processed and obtained by the data combining unit into multiple channels of video image data satisfying a display requirement;
- a data transceiving unit, configured to send, through a communication network, the multiple channels of video image data processed and obtained by the data recombining unit to a remote videoconferencing device, and receive multiple channels of recombined video image data sent by the videoconferencing device through the communication network; and
- multiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data received by the data transceiving unit to a corresponding display device.
- It can be seen from the forgoing technical solutions that, compared with the prior art, in the embodiments of the present invention, after the multiple channels of video image data collected by the multiple cameras are obtained, the multiple channels of video image data are processed into the single channel of panoramic video image data, and single channel of are recombined into several channels of video image data according to a display requirement for display. In this process, an operation of processing the multiple channels of video image data into the single channel of panoramic video image data may eliminate an overlapping situation existing between each channel of video image data. Therefore, the overlapping situation existing between each channel of video image data collected by the camera is allowed, so that requirements on a position where athe camera set is placed and a distance between a user and the camera set are lowered, and installation complexity of the system is simplified.
- The accompanying drawings described herein are used to provide for further understanding of the present invention, which are a part of this application, but are not intended to limit the present invention. In the accompanying drawings:
-
FIG. 1 is a schematic structural diagram of a videoconferencing system in the prior art; -
FIG. 2 is a schematic diagram of correlative video images according to an embodiment of the present invention; -
FIG. 3 is a schematic diagram of combining correlative video images according to an embodiment of the present invention; -
FIG. 4 is a flow chart of a method for processing video image data according to an embodiment of the present invention; -
FIG. 5a and FIG. 5b are schematic diagrams of recombining video image data in a method for processing video image data according to an embodiment of the present invention; -
FIG. 6 is a schematic diagram of a video image data sending process in a method for processing video image data according to an embodiment of the present invention; -
FIG. 7 is a schematic diagram of a video image data receiving process in a method for processing video image data according to an embodiment of the present invention; -
FIG. 8 is a schematic structural diagram of an apparatus for processing video image data according to an embodiment of the present invention; -
FIG. 9 is another schematic sturctural diagram of an apparatus for processing video image data according to an embodiment of the present invention; -
FIG. 10 is another schematic structural diagram of an apparatus for processing video image data according to an embodiment of the present invention; -
FIG. 11 is another schematic structural diagram of an apparatus for processing video image data according to an embodiment of the present invention; -
FIG. 12 is a schematic structural diagram of a videoconferencing system according to an embodiment of the present invention; -
FIG. 13 is another schematic structural diagram of a videoconferencing system according to an embodiment of the present invention; -
FIG. 14 is another schematic structural diagram of a videoconferencing system according to an embodiment of the present invention; -
FIG. 15 is a schematic structural diagram of a videoconferencing terminal according to an embodiment of the present invention; -
FIG. 16 is another schematic structural diagram of a videoconferencing terminal according to an embodiment of the present invention; -
FIG. 17 is another schematic structural diagram of a videoconferencing terminal according to an embodiment of the present invention; and -
FIG. 18 is another schematic structural diagram of a videoconferencing terminal according to an embodiment of the present invention. - In order to make the objectives, technical solutions, and advantages of the present invention more comprehensible, the present invention is described in further detail in the following with reference to the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and descriptions of the embodiments are only used to explain the present invention, but are not intended to limit the present invention.
- For the purpose of reference and clarity, technical terms, short forms or abbreviations used in this specification are concluded as follows:
- H.320: ITU-T Recommendation H.320, Narrow-band visual telephone systems and terminal equipment, a standard defined by International Telecommunication Union Telecommunication Standardization Section, which specifies a multimedia communication system based on a narrow-band switching system;
- H.323: ITU-T Recommendation H.323, Packet-based Multimedia Communications Systems, a standard defined by International Telecommunication Union Telecommunication Standardization Section, which specifies an architecture of a multimedia communication system based on a packet switching system;
- IP: Internet Protocol, that is, network protocol;
- ISDN: Integrated Services Digital Network, that is, integrated services digital network;
- ITU-T: International Telecommunication Union Telecommunication Standardization Sector, that is, International Telecommunication Union Telecommunication Standardization Sector;
- RTP: Real-time Transport Protocol, real-time transport protocol;
- MCU: Multipoint Control Unit, multipoint control unit;
- UDP: User Datagram Protocol, user datagram protocol;
- YPbPr: luminance (Y) and color difference (Pb/Pr);
- DVI: Digital Visual Interface, digital visual interface;
- HDMI: High Definition Multimedia Interface, high definition multimedia interface;
- VGA: Video Graphic Array, video graphic array;
- MPEG: Moving Pictures Experts Group, that is, Moving Pictures Experts Group, where MPEG1, MPEG2 and MPEG4 are all MPEG standards;
- video images correlative to each other (which are referred to as correlative video images hereinafter in order to facilitate description): video images obtained by multiple cameras in the same scenario, where generally, since the cameras are placed randomly, an overlapping area exists between these images, and as shown in
FIG. 2 , shaded parts are an overlapping area between animage 21 and animage 22, and theimage 21 and theimage 22 are correlative images; - image combination: combining multiple small-sized (small viewing-angle) images from the same scenario into a large-sized (wide viewing-angle) image; and processing the overlapping area between the correlative images during combination, for example, the
image 21 and theimage 22 shown inFIG. 2 are processed to obtain animage 23, as shown inFIG. 3 ; and - image recombination: partitioning and filtering a large-sized video image to form multiple small-sized video images.
- The technical solutions in the embodiments of the present invention are clearly and fully described in the following with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the embodiments to be described are only a part rather than all of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
- An embodiment of the present invention discloses a method for processing video image data, where obtained multiple channels of correlative video image data are combined into a single channel of panoramic video image data, and according to a display requirement, the panoramic video image data is recombined into one or multiple channels (equal to the number of display devices) of video image data, and the video image data is displayed by a display device.
- The number of display devices may be multiple, and after combined panoramic video image data is recombined into multiple channels of video image data, the recombined video image data may be respectively sent to each display device. The display device performs display according to a position of each channel of video image data in the panoramic video image data, so as to provide wide viewing-angle visual experience for users. As shown in
FIG. 4 , a specific process includes the following steps. - Step S41: Obtain multiple channels of correlative video image data and correlative information between each channel of video image data.
- The multiple channels of correlative video image data are from multiple cameras that are disposed in the same scenario, and these cameras are placed at different positions in the scenario.
- The correlative information includes: physical position information that is used to indicate a video image data, and capture timestamp information of the video image data.
- Step S42: Combine the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information.
- Specifically, the multiple channels of video image data are combined according to physical position information and capture timestamp information of each channel of video image data, to form the single channel of panoramic video image data.
- Step S43: Recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement.
- The panoramic video image data is recombined and filtered into the corresponding number of channels of video image data according to the number of display devices, a supported size of a frame, and a supported format of the video image data.
- Step S44: Send each channel of recombined video image data to each display device for display respectively.
- If the number of the display devices is four, as shown in
FIG. 5a , which are respectively 51a, 52a, 53a and 54a, the panoramic video image data is recombined into four channels according to display positions, and the four channels are respectively transmitted to corresponding display devices for display, so that displayed images may be combined into a wide viewing-angle video image if each display device is arranged according to positions of the images in the panoramic video image data. If the number of the display devices is three, as shown inFIG. 5b , which are respectively 51b, 52b and 53b, the panoramic video image data is recombined into three channels according to image display positions, and the three channels are respectively transmitted to corresponding display devices for display. In addition, a size of a display frame supported by a display device may be different, so that when the panoramic video image data is recombined, the panoramic video image data needs to be recombined, according to the size of the display frame supported by the display device, into video image data with a corresponding size, for example, a display device supports an HDMI video input interface and meanwhile supports a 1080p video format, and a resolution of a panoramic image is 4000*1080, and therefore, when panoramic video image data is recombined, the panoramic video image data is properly recombined and filtered into two channels of 1080p video image data with a resolution of 1920*1080 for display. - It should be noted that, in the forgoing step S41, a type of a video interface for obtaining video images may be any one or several kinds of the following: a YPbPr interface, a DVI interface, an HDMI interface and a VGA interface, that is, video input interfaces provided by each camera may be the same, and may also be different. In the forgoing step S43, a format of the recombined video image data is consistent with a video image format supported by a display device, and is determined according to the video image format supported by the display device.
- Furthermore, it should be noted that, a video input interface type in the forgoing
step S41 1 and a video output interface type in step S44 may be the same (for example, the video input interface is a YPbPr interface, and the video output interface is also the YPbPr interface) or different (for example, the video input interface is the YPbPr interface, and the video output interface may be an HDMI interface). When a video interface type and a video image data format that are supported by each display device are different, after the single channel of panoramic video image data is recombined into multiple channels of video image data, a format of each channel of video image data also needs to be converted respectively according to a video interface type and a video image format that are supported by a corresponding display device, and then the video image data is sent to the corresponding display device. - In this embodiment of the present invention, after the multiple channels of video image data collected by the multiple cameras are obtained, the multiple channels of video image data are combined into the single channel of panoramic video image data, and are recombined into several channels of video image data according to a display requirement, and then the several channels of video image data are sent to display devices for display. The displayed video images can be combined into a wide viewing-angle video image by merely arranging the display devices according to positions of the video images in the panoramic video image data, so as to provide better visual experience for users. Moreover, in this embodiment of the present invention, a process of combining the multiple channels of video image data into the single channel of panoramic video image data may eliminate an overlapping situation existing between each channel of video image data. Therefore, an overlapping phenomenon existing between images obtained by each camera may be allowed, which means that no particularly strict requirement is imposed on a position where a camera is placed and a distance between a user and a camera set, so that installation complexity of the camera is lowered.
- The display device may also be a display device that is adaptive to the panoramic video image data, and in this case, the number of display devices may be one. After the combined panoramic video image data is recombined into multiple channels of video image data, the multiple channels of video image data are respectively sent to the display devices according to positions of the multiple channels of video image data in the panoramic video image data, and the display device combines each channel of video image data into panoramic video image data for display. This embodiment of the present invention may be applied to a remote panoramic videoconferencing process, where each party taking part in a conference may send their own video image data to an opposite party (that is, a video image data sending process), and receive and display video image data that is sent by the opposite party (that is, a video image data receiving process).
- As shown in
FIG. 6 , the video image data sending process includes the following steps. - Step S61: Obtain video image data collected by multiple cameras placed at a local conference site and correlative information between each channel of video image data.
- Each camera is placed at a different position, but obtained video image data is correlative, and the correlative information includes a physical position and a capture timestamp of each channel of video image data.
- Step S62: Combine the multiple channels of video image data into a single channel of panoramic video image data according to a physical position and a capture timestamp of each channel of video image data.
- Step S63: Send the panoramic video image data through a communication network.
- Persons skilled in the art may understand that, in the forgoing step S61, processes of obtaining the video image data collected by the multiple cameras and obtaining the correlative information between each channel of video image data are implemented simultaneously, and it is doubtless that, to enable a user in front of multiple displayers to view frames captured by the cameras at the same same, it must be ensured that the multiple cameras collect scenario images synchronously. In addition, to ensure integrity of transmitted video images, it must be ensured that no disconnection occurs between scenario images shot by adjacent cameras, and an overlapping area is preferred, where the overlapping area may be removed in an image combining process.
- In this embodiment, a network interface of the communication network may be: an ISDN interface, an E1 interface, or a V35 interface, where the ISDN interface, the E1 interface, or the V35 interface is based on circuit switching, an Ethernet interface based on packet switching, or a wireless port based on a wireless connection.
- Being corresponding to the forging video image sending process, as shown in
FIG. 7 , the video image data receiving process includes the following steps. - Step 571: Obtain panoramic video image data sent from a remote conference site through a communication network.
- Step S72: Recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement.
- Step S73: Send each channel of video image data to a corresponding display device for display.
- In other embodiments, the video image data sending process may also be: after obtaining multiple channels of video image data and correlative information between each channel of video image data, directly sending the obtained video image data and correlative information through a communication network. Correspondingly, the video image data receiving process is: after receiving the multiple channels of video image data and the correlative information, combining the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information, and recombining the panoramic video image data into multiple channels of video image data according to the number of display devices and sending the recombined video image data to corresponding display devices for display. It should be noted that, in these embodiments, the correlative information between the multiple channels of video image data may be embedded in the video image data (or compressed video image data) for transmission, for example, when the communication network is the Ethernet, the correlative information may be embedded in a video RTP packet for transmission, which facilitates synchronization between the correlative information and the video image data. Definitely, the correlative information may also be transmitted separately, for example, transmitted through an independent data channel.
- In other embodiments, the video image data sending process may further be: after obtaining multiple channels of video image data and correlative information between each channel of video image data, combining the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information, and after recombining the panoramic video image data into multiple channels of video image data according to the number of display devices (display devices at the remote coferrence site), sending the recombined video image data through a communication network. Correspondingly, the video image data receiving process is: receiving the multiple channels of recombined video image data, and directly sending the received video image data to display devices at a local conference site for display.
- In addition, in the forgoing embodiment, a transmitting end may send the panoramic video image data directly, and may also send the panoramic video image data after coding. The coding manner may be: H.261, H.263, H.264, MPEG1, MPEG2 or MPEG4. Correspondingly, the panoramic video image data received by a receiving end may be uncoded raw data, and may also be coded data. It should be noted that, a size of a combined image is generally several times larger than a size of an original image, and in this case, even if a coder is used for coding, the amount of transmitted data is still larger, which imposes a strict requirement on capability of the coder. Based on the forgoing description, in other embodiments of the present invention, multiple coders are adopted for parallel processing, and furthermore, due to randomness of image data, synchronization of a sequence of coded data cannot be ensured, and to ensure that images displayed by multiple displayers at a display end are shot at the same time, the coded data needs to be synchronized.
- Specifically, the forgoing process of recombining the panoramic video image data is actually an image partitioning process, which includes the following steps.
- a. Partition the panoramic video image into multiple sub-images, and meanwhile obtain multiple pieces of synchronous information for generating the multiple sub-images, where each sub-image is corresponding to one piece of the synchronous information.
The synchronous information is specifically a timestamp of the received panoramic video image data and may also be a self-defined sequence number. A manner for defining the sequence number needs to ensure that sequence numbers of multiple sub-images obtained by partitioning the same panoramic video image data meet a preset rule, for example, the sequence numbers may be the same or consecutive. - b. Allocate reconstruction information for a partitioning manner of each sub-image, where the reconstruction information is used for recording the partitioning manner of each sub-image.
- c. Send each sub-image and corresponding synchronous information and reconstruction information of each sub-image to another device.
- Therefore, the forgoing method for processing video image data further includes a synchronization process and a reconstruction process, which are respectively introduced as follows.
- The synchronization process is as follows:
- Receive each sub-image and corresponding synchronous information and reconstruction information of each sub-image, where the each sub-image and corresponding synchronous information and reconstruction information of each sub-image are sent by another device, and then classify the sub-images according to the synchronous information to find multiple sub-images obtained by partitioning the same panoramic video image data, that is, image information obtained at the same time.
- A device for implementing the forgoing method includes a receiving buffer, a reconstruction buffer, and a sending buffer. The receiving buffer receives partitioned sub-images, where synchronous information of sub-images that belong to the same panoramic image meets a preset rule, for example, the synchronous information is the same or consecutive, the reconstruction buffer stores a sub-image to be reconstructed, and the sending buffer stores a reconstructed image.
- The reconstruction information may be a partitioning manner, and the reconstruction process is: reconstructing the classified sub-images according to the partitioning manner to obtain multiple channels of video image data, where each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data.
- The synchronization process and the reconstruction process may specifically include:
- Step a: Implement an initialization operation, that is, determine minimum synchronous information MinSyinfo.
The "minimum synchronous information" in this step may not be minimum, and may be randomly selected and assumed to be the "minimum synchronous information". - Step b: Take an unselected sub-image from the receiving buffer, and obtain synchronous information CurrSyinfo.
- Step c: Judge whether the MinSyinfo is greater than the CurrSyinfo, and if the MinSyinfo is greater than the CurrSyinfo, proceed to step d; otherwise, proceed to step e.
- Step d: Determine the CurrSyinfo as the MinSyinfo, and return to step b.
- Step e: Perform CDT (Check Delay Time, check delay time) processing, and if the delay time is greater than a specified delay, proceed to step f; otherwise, proceed to step g.
- Step f: Directly output an image stored in the sending buffer, and return to step a.
- Step g: Judge whether the MinSyinfo is smaller than the CurrSyinfo, and if the MinSyinfo is smaller than the CurrSyinfo, return to step b; otherwise, proceed to step h.
- Step h: Perform CDT processing, and if the delay time is greater than a specified delay, proceed to step f; otherwise, proceed to step i.
- Step i: Store the sub-image in the reconstruction buffer.
- Step j: Judge whether an unselected sub-image exists in the receiving buffer, and if an unselected sub-image exists in the receiving buffer, return to step b; and if no unselected sub-image exists in the receiving buffer, proceed to step k.
- Step k: Reconstruct the sub-image stored in the reconstruction buffer according to the reconstruction information, store a reconstructed image to the sending buffer, and proceed to step f.
- It may be understood that, after the sending buffer sends data in step f, the buffer is not released at once, so that when the process proceeds to step f from step e or step h, the image stored in the sending buffer is a previous frame of image that is successfully reconstructed; and the image stored in the sending buffer is updated in step k, where the update may be implemented in a data overwriting manner, and may also be implemented in a manner of releasing the sending buffer and then storing data in the sending buffer, or in another data updating manner.
- In other embodiments, in a recombining process, before the multiple sub-images and the corresponding synchronous information and reconstruction information are sent, the multiple sub-images and the corresponding synchronous information and reconstruction information are coded, where a coding manner may be a compression standard code stream format that meets various current mainstream standards, such as h261, h263, h263++, mpeg1, mpeg2 or mpeg4.
- Correspondingly, in the synchronization process, after the sub-images and the corresponding synchronous information and reconstruction information are received, decoding is performed first, and being corresponding to multiple coders in the recombining process, multiple decoders may also be set. Afterward, the decoded sub-images are classified according to the synchronous information to find multiple sub-images obtained by partitioning the same panoramic video image data, that is, image information obtained at the same time.
- An embodiment of the present invention further discloses an apparatus for processing video image data, which may implement the method disclosed in the foregoing embodiment.
- A structural form of the apparatus for processing video image data is shown in
FIG. 8 , which includes adata combining unit 81, adata recombining unit 82, data input interfaces 83, and adata output unit 84. - The data input interfaces 83 are multiple, which are respectively connected to multiple cameras, and configured to obtain multiple channels of video image data and correlative information between each channel of video image data, where the correlative information includes: information that is used to indicate a physical position of the video image data, and capture timestamp information of the video image data.
- The
data combining unit 81 is configured to combine the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information. Specifically, according to a physical position and capture time of each channel of video image data, the multiple channels of video image data are combined into the single channel of panoramic video image data. - The
data recombining unit 82 is configured to, according to the number and the size of display devices and a video image format supported by the display devices, recombine the single channel of panoramic video image data into multiple channels of video image data satisfying a display requirment of multiple display devices. - The
data output unit 84 is configured to send, through a communication network, the multiple channels of video image data processed and obtained by thedata recombining unit 82 to a remote videoconferencing device (which may be a terminal, and may also be an MCU). - Therefore, the remote videoconferencing device may arrange the multiple channels of video image data according to positions of video images in the panoramic video image, and then transmits the video image data to the multiple display devices. Video images displayed by all the display devices may be combined into a wide viewing-angle video image, so as to bring panoramic visual experience to users.
- The
data input interface 83 may be a YPbPr interface, a DVI interface, an HDMI interface, or a VGA interface. - It should be noted that, in order to reduce the amount of data to be transmitted and ensure transmission safety, another structure of the apparatus for processing video image data may further include a functional unit for compression and coding. As shown in
FIG. 9 , the apparatus includes adata combining unit 91, adata recombining unit 92, data input interfaces 93, adata output unit 94, and adata coder 95. - Functions of the
data combining unit 91, thedata recombining unit 92, thedata input interface 93, and thedata output unit 94 are basically the same as functions of thedata combining unit 81, thedata recombining unit 82, thedata input interface 83, and thedata output unit 84 respectively. - The data coder 95 is configured to obtain multiple channels of video image data recombined and obtained by the
data recombining unit 92, and after the obtained video image data is coded, provide the coded video image data for thedata output unit 94. A coding manner may be: H.261, H.263, H.264, MPEG1, MPEG2, or MPEG4. - In order to accelerate a data processing speed to ensure real-time data transmission, in other embodiments, multiple data coders may be adopted to simultaneously perform coding processing on the multiple channels of video image data recombined and obtained by the
data recombining unit 92. In this case, each channel of video image data processed and obtained by thedata recombining unit 92 includes: each sub-image obtained by recombining the panoramic video image and corresponding synchronous information and reconstruction information of each sub-image. Afte thedata output unit 94 outputs the multiple channels of video image data coded by the multiple coders, a device that receivs the multiple channels of video image data may perform a synchronization process and a reconstruction process according to the synchronous information and the reconstruction information, where specific content of the synchronization process and the reconstruction process may be made referrence to the description of the foregoing method, and is not repeated here. - The device that receivs the multiple channels of video image data is another structural form of the apparatus for processing video image data, which includes multiple data input interfaces and multiple data output interfaces, and further includes data decoders, a data synchronizing unit, and a data reconstructing unit.
- The data input interface is configured to obtain multiple channels of coded video image data.
- The data decoders are multiple, which are configured to simultaneously decode the multiple channels of coded video image data, where multiple channels of decoded video image data include multiple sub-images obtained by partitioning the panoramic video image and corresponding synchronous information and reconstruction information of the multiple sub-images.
- The data synchronizing unit is configured to classify the decoded sub-images according to corresponding synchronous information of the decoded sub-images, and a specific process may be made referrence to the description in the foregoing method embodiment.
- The data reconstructing unit is configured to reconstruct the classified sub-images according to the reconstruction information to obtain multiple channels of video image data, and provide the obtained video image data to the data output interface, where each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data.
- Another structural form of the apparatus for processing video image data is shown in
FIG. 10 , which includes adata combining unit 101, adata recombining unit 102, adata input interface 103, and multiple data output interfaces 104. - Functions of the
data combining unit 101 and thedata recombining unit 102 are basically the same as the functions of thedata combining unit 81 and thedata recombining unit 82 respectively. - A difference between this structure and the structure shown in
FIG. 8 is that, multiple channels of video image data and correlative information between each channel of video image data are sent by another device through a communication network, where the multiple channels of video image data and the correlative information between each channel of video image data are obtained by thedata input interface 103. The multipledata output interfaces 104 are respectively connected to multiple display devices, and configured to arrange, according to positions of video images in the panoramic video image, multiple channels of video image data processed and obtained by thedata recombining unit 102, and send the multiple channels of video image data to display devices, where video images displayed by all the display devices may be combined into a wide viewing-angle video image. - Specifically, the
data input interface 103 may be formed by a network interface and a data receiving unit, where the network interface is configured to establish a connection with the communication network, and the data receiving unit is configured to receive, through the network interface, video image data transmitted by another device through the communication network. - The network interface may be an ISDN interface, an E1 interface, or a V35 interface, where the ISDN interface, the E1 interface, or the V35 interface is based on circuit switching, an Ethernet interface based on packet switching, or a wireless port based on a wireless connection.
- In addition, if multiple channels of video image data and correlative information between each channel of video image data are coded, where the multiple channels of video image data and the correlative information between each channel of video image data are received by the
data input interface 103, another structure of the apparatus for processing video image data needs to include a functional unit for decoding, as shown inFIG. 11 , which include adata combining unit 111, adata recombining unit 112, adata input interface 113, and multipledata output interfaces 114, and further include adata decoder 115. - Functions of the
data combining unit 111, thedata recombining unit 112, thedata input interface 113, and thedata output interface 114 are basically the same as the functions of thedata combining unit 101, thedata recombining unit 102, thedata input interface 103, and thedata output interface 104 respectively. - The
data decoder 115 is configured to decode multiple channels of video image data and correlative information between each channel of video image data, where the multiple channels of video image data and the correlative information between each channel of video image data are obtained by thedata input interface 113, and provide the decoded multiple channels of video image data and correlative information between each channel of video image data for thedata combining unit 111. - In addition, an embodiment of the present invention further provides a videoconferencing system, and a specific structure of the system is shown in
FIG. 12 , which includes adata combining unit 121, adata sending unit 122, adata receiving unit 123, adata recombining unit 124, multiple data input interfaces 125, and multiple data output interfaces 126. - The
data combining unit 121, thedata sending unit 122, and the data input interfaces 125 are located at a videoconferencing site at one side. The multiple data input interfaces 125 obtain multiple channels of video image data and correlative information between each channel of video image data, thedata combining unit 121 combines the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information, and then thedata sending unit 122 sends the single channel of panoramic video image data to a remote conference site at the other side through a communication network. - The
data receiving unit 123, thedata recombining unit 124, and thedata output interfaces 126 are located at the remote conference site at the other side. Thedata receiving unit 123 receives the single channel of panoramic video image data that is carried on the communication network, and then provides the panoramic video image data to thedata recombining unit 124, thedata recombining unit 124 recombines, according to the number of display devices, a supported size of a frame, and a supported video image format, the single channel of panoramic video image data into multiple channels of video image data satisfying a display requirement of multiple display devices, and thedata output interfaces 126 provide the video image data to corresponding display devices. - The display devices are placed according to positions of the video images in the panoramic video image, and the video images displayed by all the display devices may be combined into a wide viewing-angle video image, so as to bring panoramic visual experience to users.
- It should be noted that, the
data input interface 125 and thedata output interface 126 may be YPbPr interfaces, DVI interfaces, HDMI interfaces or VGA interfaces. In addition, the types of thedata input interface 125 and thedata output interface 126 may be different, and a format of video image data obtained by thedata input interface 125 may be converted according to the type of thedata output interface 126 during recombination of thedata recombining unit 123. For example, thedata input interface 125 is a DVI interface, the obtained video image data is in a DVI format, and thedata output interface 126 is an HDMI interface, so that when thedata recombining unit 123 recombines video image data, video image data in a DVI format needs to be converted into video image data in an HDMI format. - The conference sites at both sides are required to play roles of a transmitter and a receiver at the same time, that is, to send video image data from a local conference site through the data input interfaces 125, the
data combining unit 121, and thedata sending unit 122, and receive and process video image data from a remote conference site through thedata receiving unit 123, thedata recombining unit 124, and the data output interfaces 126. - It should be noted that, in a system with another structural form, a transmitter only needs to obtain multiple channels of video image data and correlative information between each channel of video image data through the data input interfaces 125, and then send the obtained video image data and correlative information through a communication network to a remote conference site. A receiver obtains the multiple channels of video image data and the correlative information between each channel of video image data from the communication network, and then performs operations such as combination and recombination.
FIG. 13 is another schematic structural diagram of a videoconferencing system according to an embodiment of the present invention. The system includes adata combining unit 131, adata sending unit 132, adata receiving unit 133, adata recombining unit 134, multiple data input interfaces 135, and multiple data output interfaces 136. - The
data sending unit 132 and the multiple data input interfaces 135 are located at a conference site at one side, and thedata receiving unit 133, thedata combining unit 131, thedata recombining unit 134, and the multipledata output interfaces 136 are located at a conference site at the other side. - The multiple data input interfaces 135 obtain multiple channels of video image data and correlative information between each channel of video image data, and the
data sending unit 132 sends the multiple channels of video image data and the correlative information between each channel of video image data to the conference site at the other side through a communication network; and thedata receiving unit 133 at the conference site at the other side receives the multiple channels of video image data and the correlative information between each channel of video image data, and then provides the received video image data and correlative information for thedata combining unit 131, thedata combining unit 131 combines the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information, and then provides the single channel of panoramic video image data for thedata recombining unit 134, thedata recombining unit 134 recombines, according to the number of display devices, a supported size of a frame, and a supported video image format, the single channel of panoramic video image data into multiple channels of video image data satisfying a display requirement of multiple display devices, and thedata output interfaces 136 provides the multiple channels of video image data for corresponding display devices. - It should be noted that, since the data sending units (the
data sending units 122 and 132) send the video image data through the communication network, in order to reduce the amount of data to be transmitted and ensure transmission safety, data sent by the data sending units may be coded. Correspondingly, after receiving the data sent through the communication network, the data receiving units (thedata receiving units 123 and 133) decode the data. - It should be noted that, in a system with another structural form, after receiving multiple channels of video image data and correlative information, a transmitter combines the multiple channels of video image data into a single channel of panoramic video image data according to the correlative information, then recombines the single channel of panoramic video image data into several channels of video image data, and sends the several channels of video image data; and a receiver receives the several channels of video image data, and then provides the the several channels of video image data for display devices at a local conference site for display. A specific structural form is shown in
FIG. 14 , which includes a data combining unit 141, adata sending unit 142, adata receiving unit 143, a data recombining unit 144, multiple data input interfaces 145, and multiple data output interfaces 146. - Functions of the units are basically the same as the units in
FIG. 12 andFIG. 13 , and a difference lies in that, the data input interfaces 145, the data combining unit 141, the data recombining unit 144, and thedata sending unit 142 are located at a conference site at one side, and thedata receiving unit 143 and thedata output interfaces 146 are located at a conference site at the other side, which means that the data recombining unit 144 located at the conference site at one side needs to recombine video image data according to the number of display devices, a supported size of a frame, and a supported video image format at the conference site at the other side. - In addition, another structure may further include data coders, data decoders, a data synchronizing unit, and a data reconstructing unit.
- The data coders are multiple, which are disposed at the conference site where the data recombining unit 144 is located, and configured to simultaneously process multiple channels of video image data recombined and obtained by the data recombining unit 144, where each channel of video image data recombined and obtained by the data recombining unit 144 includes each sub-image obtained by partitioning the panoramic video image, and corresponding synchronous information and reconstruction information of each sub-image.
- The number of the data decoders is the same as the number of the data coders. The data decoders are disposed at the conference site where the
data receiving unit 143 is located, and configured to simultaneously decode multiple channels of coded video image data received by thedata receiving unit 143. - The data synchronizing unit is configured to classify, according to corresponding synchronous information of sub-images, sub-images decoded by the data decoders, and a specific process may be made referrence to the description in the foregoing method embodiment.
- The data reconstructing unit is configured to reconstruct the classified sub-images according to the reconstruction information to obtain multiple channels of video image data, and provide the obtained multiple channels of video image data for the data output interface, where each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data.
- It can be seen that, this embodiment of the present invention is suitable to a situation that videoconferencing sites at both sides are the same (that is, the number of display devices, a supported size of a frame, and a supported video image format are the same).
- Being corresponding to the forgoing method and apparatus for processing video image data and the videoconferencing system, an embodiment of the present invention further discloses a videoconferencing terminal at the same time, and since roles played by both sides of a video conference are mutual (that is, both act as a transmitter and a receiver at the same time), a specific structure of the videoconferencing terminal is shown in
FIG. 15 , which includes adata combining unit 151, adata transceiving unit 152, anetwork interface 153, adata recombining unit 154, multiple data input interfaces 155, and multiple data output interfaces 156. - The
network interface 153 is configured to establish a connection with an external communication network, and thedata transceiving unit 152 is configured to obtain data sent from the communication network and send data to the communication network. - Functions of other functional units, such as the
data combining unit 151, thedata recombining unit 154, the data input interfaces 155, and thedata output interfaces 156, may be made referrence to the content of the apparatus for processing video image data and the videoconferencing system in the foregoing description. - As a transmitter, the videoconferencing terminal needs to obtain multiple channels of video image data and correlative information between each channel of video image data at a local conference site, combine the obtained video image data and correlative information into a single channel of panoramic video image data, and send the single channel of panoramic video image data to a remote conference site through the communication network; and meanwhile, as a receiver, the videoconferencing terminal needs to receive a panoramic video image data sent from the remote conference site through the communication network, recombine the panoramic video image data into multiple channels of video image data, and then transmit the multiple channels of video image data to display devices at the local conference site.
-
FIG. 16 shows another structure of the videoconferencing terminal, which includes adata combining unit 161, adata transceiving unit 162, anetwork interface 163, adata recombining unit 164, multiple data input interfaces 165, and multipledata output interfaces 166, and further includes adata coder 167 and adata decoder 168. - Functions of the
data combining unit 161, thedata transceiving unit 162, thenetwork interface 163, thedata recombining unit 164, the data input interfaces 165, and thedata output interfaces 166 are basically the same as the functions of thedata combining unit 151, thedata transceiving unit 152, thenetwork interface 153, thedata recombining unit 154, the data input interfaces 155, and thedata output interfaces 156 respectively. - The data coder 167 codes data before the
data transceiving unit 162 sends the data, and thedata decoder 168 decodes the data after thedata transceiving unit 162 receives the data. - Another structure of the videoconferencing terminal is shown in
FIG. 17 , which includes adata combining unit 171, adata transceiving unit 172, anetwork interface 173, adata recombining unit 174, multiple data input interfaces 175, and multiple data output interfaces 176. - Functions of the units are basically the same as the functions of the units in
FIG. 15 respectively. - A difference lies in that, as a transmitter, the videoconferencing terminal obtains multiple channels of video image data and correlative information between each channel of video image data at a local conference site, and then directly sends the obtained video image data and correlative information to a remote conference site through a communication network. Meanwhile, as a receiver, the videoconferencing terminal receives multiple channels of video image data and correlative information between each channel of video image data at the remote conference site through the communication network, combines the received video image data and correlative information into a single channel of panoramic video image data, recombines the single channel of panoramic video image data into multiple channels of video image data, and transmits the multiple channels of video image data to display devices at the local conference site.
-
FIG. 18 shows another structure of the videoconferencing terminal, which includes adata combining unit 181, adata transceiving unit 182, anetwork interface 183, adata recombining unit 184, multiple data input interfaces 185, and multipledata output interfaces 186, and further includes a data coder 187 and a data decoder 188. - Functions of the units are basically the same as the functions of the units in
FIG. 16 respectively. - A difference lies in that, as a transmitter, the videoconferencing terminal obtains multiple channels of video image data and correlative information between each channel of video image data at a local conference site, and codes and sends the obtained video image data and correlative information directly to a remote conference site through a communication network. Meanwhile, as a receiver, the videoconferencing terminal receives multiple channels of video image data and correlative information between each channel of video image data that are sent from the remote conference site through the communication network, decodes and combines the received video image data and correlative information into a single channel of panoramic video image data, then recombines the single channel of panoramic video image data into multiple channels of video image data, and transmits the multiple channels of video image data to display devices at the local conference site.
- In other embodiments, as a transmitter, the videoconferencing terminal obtains multiple channels of video image data and correlative information between each channel of video image data at a local conference site, combines the obtained video image data and correlative information into a single channel of panoramic video image data, recombines, according to the number of display devices, a supported size of a frame, and a supported video image format at a remote conference site, the single channel of panoramic video image data into multiple channels of video image data satisfying a display requirement of multiple display devices, and sends the multiple channels of video image data to the remote conference site through a communication network (or through the communication network after coding). Meanwhile, as a receiver, the videoconferencing terminal receives multiple channels of video image data sent from the other side through the communication network, provides the received video image data for display devices at the local conference site for display (or provides the received video image data for the display devices at the local conference site for display after decoding). It should be noted that, in this case, during coding, multiple coders may be adopted to simultaneously code multiple channels of recombined video image data, and during decoding, multiple decoders are adopted to simultaneously decode multiple channels of coded video image data; and furthermore, a synchronization process and a reconstruction process are performed, that is, classifying, according to corresponding synchronous information of sub-images, sub-images decoded by the data decoders, and reconstructing the classified sub-images according to the reconstruction information to obtain multiple channels of video image data, where each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data.
- The embodiments in this specification are described in a progressive manner, each embodiment emphasizes a difference from the other embodiments, and the identical or similar parts between the embodiments may be made referrence to each other. Since the apparatuses disclosed in the embodiments are corresponding to the methods disclosed in the embodiments, the description of the apparatuses is simple and relevant parts may be made reference to the description of the methods.
- Persons skilled in the art may understand that information, a message, and a signal may be represented by using any one of many different techniques and technologies. For example, the message and information mentioned in the forgoing description may be represented as a voltage, a current, an electromagnetic wave, a magnetic field or a magnetic particle, an optical field, or any combination of the forgoing.
- Persons skilled in the art may further realize that, units and steps of algorithms according to the description of the embodiments disclosed by the present invention can be implemented by electronic hardware, computer software, or a combination of the two. In order to describe interchangeability of hardware and software clearly, compositions and steps of the embodiments are generally described according to functions in the forgoing description. Whether these functions are executed by hardware or software depends upon specific applications and design constraints of the technical solutions. Persons skilled in the art may use different methods for each specific application to implement the described functions, and such implementation should not be construed as a departure from the scope of the present invention.
- Persons of ordinary skill in the art may understand that all or a part of the steps in the method of the forgoing embodiments may be accomplished through a program instructing relevant hardware. The program may be stored in a computer readable storage medium, and the storage medium may include a ROM, a RAM, a magnetic disk, or an optical disk.
- The objectives, technical solutions, and beneficial effects of the present invention have been described in further detail through the forgoing specific embodiments. It should be understood that the forgoing descriptions are merely specific embodiments of the present invention, but are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, or improvement made without departing from the principle of the present invention should fall within the protection scope of the present invention.
Claims (28)
- A method for processing video image data, comprising:obtaining multiple channels of correlative video image data and correlative information wherein a;combining the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information; andafter recombining the panoramic video image data into multiple channels of video image data satisfying a display requirement, sending the recombined video image data to a display device for display.
- The method according to claim 1, wherein the correlative information comprises:physical position relation information and time stamp information of each channel of video images.
- The method according to claim 1, wherein the obtaining the multiple channels of correlative video image data and correlative information specifically is: obtaining multiple channels of correlative video image data and correlative information that are carried on a communication network.
- The method according to claim 1, wherein the obtaining the multiple channels of correlative video image data and correlative information specifically comprisesis: obtaining multiple channels of correlative video image data and correlative information that are collected by multiple cameras placed at a videoconferencing site at one side.
- The method according to claim 1, wherein a process of recombining the panoramic video image data comprises: a partitioning process, and the partitioning process comprises:partitioning the panoramic video image into multiple sub-images, and meanwhile generating synchronous information of each sub-image;allocating reconstruction information for each sub-image according to a partitioning manner; andsending the multiple sub-images and corresponding synchronous information and reconstruction information of the multiple sub-images;the method further comprises a synchronization process and a reconstruction process, whereinthe synchronization process comprises:receiving the sub-images, the synchronous information, and the reconstruction information; andclassifying the sub-images according to the synchronous information, wherein the sub-images that belong to the same panoramic video image belong to the same category; andthe reconstruction process comprises:reconstructing the classified sub-images according to the reconstruction information to obtain multiple channels of video image data, wherein each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data.
- The method according to claim 5, wherein the synchronous information is a sequence number or a timestamp.
- The method according to claim 5, wherein before the sending the multiple sub-images and the corresponding synchronous information and reconstruction information of the multiple sub-images, the multiple sub-images and the corresponding synchronous information and reconstruction information of the multiple sub-images are coded, wherein the coding specifically is: using multiple coders to simultaneously code the partitioned multiple channels of sub-images and the corresponding synchronous information and reconstruction information of the partitioned multiple channels of sub-images; and
the receiving the sub-images, the synchronous information, and the reconstruction information in the synchronization process specifically is: using multiple decoders to simultaneously decode the received and coded information. - The method according to claim 5, wherein multiple pieces of synchronous information for generating the multiple sub-images specifically is: timestamps at the time of generating the panoramic video image data, or self-defined sequence numbers that are generated, wherein sequence numbers of multiple sub-images obtained by partitioning the same panoramic video image data are identical.
- An apparatus for processing video image data, comprising:a data input interface, configured to obtain multiple channels of correlative video image data and correlative information wherein a;a data combining unit, configured to combine the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;a data recombining unit, configured to recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement; andmultiple data output interfaces, connected to an external display device, and configured to transmit the video image data processed and obtained by the data recombining unit to the display device.
- The apparatus according to claim 9, wherein the multiple channels of correlative video image data and correlative information are from an external communication network, and the video image data input interface is an ISDN interface, an E1 interface, a V35 interface, an Ethernet interface based on packet switching, or a wireless port based on a wireless connection.
- The device apparatus according to claim 9, further comprising:a data decoder, configured to, when the video image data received by the data input interface is the coded video image data, decode the video image data, and provide the decoded video image data to for the data combining unit.
- The device apparatus according to claim 9, wherein the correlative information comprises: physical position relation information and time stamp information of each channel of video images.
- An apparatus for processing video image data, comprising:a data input interface, configured to obtain multiple channels of correlative video image data and correlative information that are collected by multiple cameras wherein a;a data combining unit, configured to combine the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;a data recombining unit, configured to recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement; anda data sending unit, configured to send, through a communication network, the multiple channels of video image data processed and obtained by the data recombining unit to a remote videoconferencing device, so that the videoconferencing device displays the video image data through a corresponding display device.
- The apparatus according to claim 13, wherein the multiple channels of video image data generated by the data recombining unit comprises: each sub-image obtained by recombining the panoramic video image, and synchronous information and reconstruction information that are corresponding to each sub-image.
- The device apparatus according to claim 13, further comprising:a data coder, configured to code the multiple channels of video image data processed and obtained by the data recombining unit, and provide the coded video image data to for the data sending unit.
- The device apparatus according to claim 15, wherein the number of the data coders is multiple, and the multiple data coders simultaneously code the multiple channels of video image data.
- An apparatus for processing video image data, comprising:a data input interface, configured to obtain multiple channels of coded video image data;multiple data decoders, configured to simultaneously decode the multiple channels of coded video image data, wherein multiple channels of decoded video image data comprise multiple sub-images obtained by partitioning the panoramic video image, and corresponding synchronous information and reconstruction information of the multiple sub-images;a data synchronizing unit, configured to classify the decoded sub-images according to corresponding synchronous information of the sub-imgaes;a data reconstructing unit, configured to reconstruct the classified sub-images according to the reconstruction information to obtain multiple channels of video image data, wherein each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data; andmultiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data processed and obtained by the data reconstructing unit to a corresponding display device.
- A videoconferencing system, comprising:a data input interface, configured to obtain multiple channels of correlative video image data and correlative information wherein a;a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;a data sending unit, configured to send, through a communication network, the panoramic video image data;a data receiving unit, configured to receive the panoramic video image data that is carried on the communication network;a data recombining unit, configured to recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement, wherein the decoded panoramic video image data is received by the data receiving unit; andmultiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data processed and obtained by the data recombining unit to a corresponding display device.
- The system according to claim 18, further comprising:a data coder, configured to code the panoramic video image data processed and obtained by the data combining unit, and provide the coded video image data to for the data sending unit; anda data decoder, configured to decode the panoramic video image data received by the data receiving unit, and provide the decoded video image data forto the data recombining unit.
- A videoconferencing system, comprising:a data input interface, configured to obtain multiple channels of correlative video image data and correlative information wherein a;a data sending unit, configured to send the multiple channels of correlative video image data and correlative information through a communication network;a data receiving unit, configured to receive the multiple channels of correlative video image data and correlative information that are carried on the communication network;a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;a data recombining unit, configured to recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement; andmultiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data processed and obtained by the data recombining unit to a corresponding display device.
- The system according to claim 20, further comprising:a data coder, configured to code the multiple channels of correlative video image data and correlative information that are obtained by the data input interface, and provide the coded video image data and correlative information forto the data sending unit; anda data decoder, configured to decode the multiple channels of correlative video image data and correlative information that are received by the data receiving unit, and provide the decoded video image data and correlative information tofor the data combining unit.
- A videoconferencing system, comprising:a data input interface, configured to obtain multiple channels of correlative video image data and correlative information wherein a;a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;a data recombining unit, configured to recombine the panoramic video image data into multiple channels of video image data satisfying a display requirement;a data sending unit, configured to send, through a communication network, the multiple channels of video image data, wherein the coded multiple channels of video image data is processed and obtained by the data recombining unit;a data receiving unit, configured to receive the multiple channels of video image data that are carried on the communication network; andmultiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data to a corresponding display device, wherein each channel of decoded video image data is received by the data receiving unit.
- The system according to claim 22, further comprising:a data coder, configured to code the multiple channels of video image data recombined and obtained by the data recombining unit, and provide the coded video image data forto the data sending unit; anda data decoder, configured to decode the multiple channels of video image data received by the data receiving unit, and provide the decoded video image data to for the data output interfaces.
- The system according to claim 23, wherein each channel of video image data recombiend and obtained by the data recombining unit comprises: each sub-image obtained by recombining the panoramic video image, and synchronous information and reconstruction information that are corresponding to each sub-image;
the number of the data coders and the number of the data decoders are both multiple, multiple data coders simultaneously code the multiple channels of video image data, and multiple data decoders simultaneously decode the multiple channels of video image data;
the system further comprises:a data synchronizing unit, configured to classify the decoded sub-images according to corresponding synchronous information of the sub-images; anda data reconstructing unit, configured to reconstruct the classified sub-images according to the reconstruction information to obtain multiple channels of video image data, and provide the obtained video image data for the multiple data output interfaces, wherein each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data. - A videoconferencing terminal, comprising:a data input interface, configured to obtain multiple channels of correlative video image data and correlative information wherein a;a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;a data transceiving unit, configured to send the panoramic video image data to a remote videoconferencing device through a communication network, and receive a single channel of panoramic video image data sent by the videoconferencing device through the communication network;a data recombining unit, configured to recombine the panoramic video image data received by the data transceiving unit into multiple channels of video image data satisfying a display requirement; andmultiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data processed and obtained by the data recombining unit to a corresponding display device.
- A videoconferencing terminal, comprising:a data input interface, configured to obtain multiple channels of correlative video image data and correlative information wherein a;a data transceiving unit, configured to send the multiple channels of video image data and correlative information to a remote videoconferencing device through a communication network, and receive multiple channels of video image data and correlative information that are sent by the videoconferencing device through the communication network;a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;a data recombining unit, configured to recombine the panoramic video image data processed and obtained by the data combining unit into multiple channels of video image data statisfying a display requirement; andmultiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data processed and obtained by the data recombining unit to a corresponding display device.
- A videoconferencing terminal, comprising:a data input interface, configured to obtain multiple channels of correlative video image data and correlative information wherein a;a data combining unit, configured to process the multiple channels of correlative video image data into a single channel of panoramic video image data by using the correlative information;a data recombining unit, configured to recombine the panoramic video image data processed and obtained by the data combining unit into multiple channels of video image data satisfying a display requirement;a data transceiving unit, configured to send, through a communication network, the multiple channels of video image data processed and obtained by the data recombining unit to a remote videoconferencing device, and receive multiple channels of recombined video image data sent by the videoconferencing device through the communication network; andmultiple data output interfaces, connected to multiple external display devices, and configured to respectively transmit each channel of video image data received by the data transceiving unit to a corresponding display device.
- The terminal according to claim 27, further comprising:multiple data coders, configured to code the multiple channels of video image data recombined and obtained by the data recombining unit, and then provide the coded video image data to the data transceiving unit, wherein each channel of video image data comprises each sub-image obtained by partitioning the panoramic video image, and synchronous information and reconstruction information that are corresponding to each sub-image;a data decoder, configured to decode the multiple channels of video image data received by the data transceiving unit, and then provide the decoded video image data to a data synchronizing unit;the data synchronizing unit, configured to classify, according to corresponding synchronous information of the sub-images, the sub-images decoded by the data decoder; anda data reconstructing unit, configured to reconstruct the classified sub-images according to the reconstruction information to obtain multiple channels of video image data, and provide the obtained multiple channels of video image data for the multiple data output interfaces, wherein each channel of video image data is arranged according to a position of each channel of video image data in the panoramic video image data.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN200910161963.9A CN101668160B (en) | 2009-09-10 | 2009-09-10 | Video image data processing method, device, video conference system and terminal |
PCT/CN2010/076763 WO2011029402A1 (en) | 2009-09-10 | 2010-09-09 | Method and device for processing video image data, system and terminal for video conference |
Publications (3)
Publication Number | Publication Date |
---|---|
EP2469853A1 true EP2469853A1 (en) | 2012-06-27 |
EP2469853A4 EP2469853A4 (en) | 2013-05-15 |
EP2469853B1 EP2469853B1 (en) | 2016-08-10 |
Family
ID=41804568
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP10815004.6A Active EP2469853B1 (en) | 2009-09-10 | 2010-09-09 | Method and device for processing video image data, system and terminal for video conference |
Country Status (4)
Country | Link |
---|---|
US (1) | US20120169829A1 (en) |
EP (1) | EP2469853B1 (en) |
CN (1) | CN101668160B (en) |
WO (1) | WO2011029402A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105681682A (en) * | 2016-01-19 | 2016-06-15 | 广东威创视讯科技股份有限公司 | Video data transmission method and system |
Families Citing this family (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101668160B (en) * | 2009-09-10 | 2012-08-29 | 华为终端有限公司 | Video image data processing method, device, video conference system and terminal |
CN101877775A (en) * | 2010-04-06 | 2010-11-03 | 中兴通讯股份有限公司 | Telepresence system and camera group thereof |
CN101951507A (en) * | 2010-10-11 | 2011-01-19 | 大道计算机技术(上海)有限公司 | Large screen IP (Internet Protocol) video stream access equipment and implementation method thereof |
KR101840008B1 (en) * | 2011-06-24 | 2018-05-04 | 에스케이플래닛 주식회사 | High quality video streaming service system and method |
CN102868880B (en) * | 2011-07-08 | 2017-09-05 | 中兴通讯股份有限公司 | It is a kind of based on the media transmission method remotely presented and system |
CN103096015B (en) * | 2011-10-28 | 2015-03-11 | 华为技术有限公司 | Video processing method and video processing system |
CN103248944B (en) * | 2012-02-03 | 2017-08-25 | 海尔集团公司 | A kind of image transfer method and system |
CN103517025B (en) * | 2012-06-28 | 2016-08-31 | 华为技术有限公司 | The method of video communication, device and telepresence system |
CN102801969A (en) * | 2012-07-25 | 2012-11-28 | 华为技术有限公司 | Method, device and system of processing multimedia data |
CN102802039B (en) * | 2012-08-14 | 2015-04-15 | 武汉微创光电股份有限公司 | Multi-channel video hybrid decoding output method and device |
GB2507127B (en) * | 2012-10-22 | 2014-10-08 | Gurulogic Microsystems Oy | Encoder, decoder and method |
FR3004565B1 (en) * | 2013-04-12 | 2016-11-11 | Kolor | FUSION OF SEVERAL VIDEO STREAMS |
CN104301746A (en) * | 2013-07-18 | 2015-01-21 | 阿里巴巴集团控股有限公司 | Video file processing method, server and client |
US9742995B2 (en) | 2014-03-21 | 2017-08-22 | Microsoft Technology Licensing, Llc | Receiver-controlled panoramic view video share |
CN104333731A (en) * | 2014-11-19 | 2015-02-04 | 成都实景信息技术有限公司 | Enterprise video conference system |
CN104735464A (en) * | 2015-03-31 | 2015-06-24 | 华为技术有限公司 | Panorama video interactive transmission method, server and client end |
CN106331576A (en) * | 2015-06-25 | 2017-01-11 | 中兴通讯股份有限公司 | Multimedia service processing method, system and device |
CN106550239A (en) * | 2015-09-22 | 2017-03-29 | 北京同步科技有限公司 | 360 degree of panoramic video live broadcast systems and its implementation |
CN106686523A (en) * | 2015-11-06 | 2017-05-17 | 华为终端(东莞)有限公司 | Data processing method and device |
JP6747158B2 (en) * | 2016-08-09 | 2020-08-26 | ソニー株式会社 | Multi-camera system, camera, camera processing method, confirmation device, and confirmation device processing method |
CN107018370B (en) * | 2017-04-14 | 2020-06-30 | 威盛电子股份有限公司 | Display method and system for video wall |
CN107481324B (en) * | 2017-07-05 | 2021-02-09 | 微幻科技(北京)有限公司 | Virtual roaming method and device |
CN108881927B (en) * | 2017-11-30 | 2020-06-26 | 视联动力信息技术股份有限公司 | Video data synthesis method and device |
CN108924470A (en) * | 2018-09-12 | 2018-11-30 | 湖北易都信息技术有限公司 | A kind of video image data processing method for video conferencing system |
CN111654644A (en) * | 2020-05-15 | 2020-09-11 | 西安万像电子科技有限公司 | Image transmission method and system |
CN113114985B (en) * | 2021-03-31 | 2022-07-26 | 联想(北京)有限公司 | Information processing method and information processing device |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020135682A1 (en) * | 2001-02-13 | 2002-09-26 | Hiroto Oka | Image pickup system |
EP1677534A1 (en) * | 2004-12-30 | 2006-07-05 | Microsoft Corporation | Minimizing dead zones in panoramic images |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0906695A4 (en) * | 1996-06-21 | 1999-06-16 | Bell Communications Res | System and method for associating multimedia objects |
US20030025599A1 (en) * | 2001-05-11 | 2003-02-06 | Monroe David A. | Method and apparatus for collecting, sending, archiving and retrieving motion video and still images and notification of detected events |
CN1184815C (en) * | 2001-12-20 | 2005-01-12 | 中国科学院计算技术研究所 | Multi viewing angle video frequency programme network retransmitting method based on multi process |
US7428000B2 (en) * | 2003-06-26 | 2008-09-23 | Microsoft Corp. | System and method for distributed meetings |
EG23651A (en) * | 2004-03-06 | 2007-03-21 | Ct For Documentation Of Cultur | Culturama |
US7183549B2 (en) * | 2004-09-09 | 2007-02-27 | Flir Systems, Inc. | Multiple camera systems and methods |
CN100353760C (en) * | 2004-09-10 | 2007-12-05 | 张保安 | Combined wide-screen television system |
DE102005012132A1 (en) * | 2005-03-16 | 2006-09-28 | Valenzuela, Carlos Alberto, Dr.-Ing. | Arrangement for conducting a video conference |
CN100355272C (en) * | 2005-06-24 | 2007-12-12 | 清华大学 | Synthesis method of virtual viewpoint in interactive multi-viewpoint video system |
US20080101410A1 (en) * | 2006-10-25 | 2008-05-01 | Microsoft Corporation | Techniques for managing output bandwidth for a conferencing server |
CN101146231A (en) * | 2007-07-03 | 2008-03-19 | 浙江大学 | Method for generating panoramic video according to multi-visual angle video stream |
CN101521745B (en) * | 2009-04-14 | 2011-04-13 | 王广生 | Multi-lens optical center superposing type omnibearing shooting device and panoramic shooting and retransmitting method |
US8310520B2 (en) * | 2009-08-19 | 2012-11-13 | Avaya Inc. | Flexible decomposition and recomposition of multimedia conferencing streams using real-time control information |
CN101668160B (en) * | 2009-09-10 | 2012-08-29 | 华为终端有限公司 | Video image data processing method, device, video conference system and terminal |
-
2009
- 2009-09-10 CN CN200910161963.9A patent/CN101668160B/en not_active Expired - Fee Related
-
2010
- 2010-09-09 WO PCT/CN2010/076763 patent/WO2011029402A1/en active Application Filing
- 2010-09-09 EP EP10815004.6A patent/EP2469853B1/en active Active
-
2012
- 2012-03-09 US US13/416,919 patent/US20120169829A1/en not_active Abandoned
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020135682A1 (en) * | 2001-02-13 | 2002-09-26 | Hiroto Oka | Image pickup system |
EP1677534A1 (en) * | 2004-12-30 | 2006-07-05 | Microsoft Corporation | Minimizing dead zones in panoramic images |
Non-Patent Citations (1)
Title |
---|
See also references of WO2011029402A1 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105681682A (en) * | 2016-01-19 | 2016-06-15 | 广东威创视讯科技股份有限公司 | Video data transmission method and system |
CN105681682B (en) * | 2016-01-19 | 2019-06-14 | 广东威创视讯科技股份有限公司 | Method of transmitting video data and system |
Also Published As
Publication number | Publication date |
---|---|
CN101668160A (en) | 2010-03-10 |
WO2011029402A1 (en) | 2011-03-17 |
US20120169829A1 (en) | 2012-07-05 |
EP2469853B1 (en) | 2016-08-10 |
CN101668160B (en) | 2012-08-29 |
EP2469853A4 (en) | 2013-05-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2469853B1 (en) | Method and device for processing video image data, system and terminal for video conference | |
US8649426B2 (en) | Low latency high resolution video encoding | |
US10237548B2 (en) | Video transmission based on independently encoded background updates | |
US8976220B2 (en) | Devices and methods for hosting a video call between a plurality of endpoints | |
EP1721462B1 (en) | Arrangement and method for generating continuous presence images | |
US20070263077A1 (en) | System and Method for Dynamic Control of Image Capture in a Video Conference System | |
JP2004536529A (en) | Method and apparatus for continuously receiving frames from a plurality of video channels and alternately transmitting individual frames containing information about each of the video channels to each of a plurality of participants in a video conference | |
US7864251B2 (en) | System and method for decreasing end-to-end delay during video conferencing session | |
CN110943909A (en) | Audio and video fusion communication server | |
US10511766B2 (en) | Video transmission based on independently encoded background updates | |
CN114584737B (en) | Method and system for customizing multiple persons in same scene in real time in cloud conference | |
CN103856809A (en) | Method, system and terminal equipment for multipoint at the same screen | |
WO2015167820A1 (en) | Content message for video conferencing | |
CN103957391A (en) | Method and system for displaying videos of all parties at same time during multi-party call in video intercom | |
CN112272281A (en) | Regional distributed video conference system | |
US6560280B1 (en) | Video transmission system | |
US20200213657A1 (en) | Transmission apparatus, transmission method, encoding apparatus, encoding method, reception apparatus, and reception method | |
Squibb | Video transmission for telemedicine | |
CN101340546A (en) | High-resolution video conference system | |
CN101742219A (en) | Video conference image station equipment, implementing system thereof and implementing method thereof | |
CN102404548A (en) | Vehicle-mounted video session system | |
Johanson | Designing an environment for distributed real-time collaboration | |
Zia | Evaluation of Potential Effectiveness of Desktop Remote Video Conferencing for Interactive Seminars | |
Chavarrías et al. | 3D videoconferencing system using spatial scalability | |
Sinharoy et al. | End-to-end system for real-time bidirectional holographic communication |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20120319 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR |
|
DAX | Request for extension of the european patent (deleted) | ||
A4 | Supplementary search report drawn up and despatched |
Effective date: 20130417 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04N 7/14 20060101ALI20130411BHEP Ipc: H04N 7/15 20060101AFI20130411BHEP |
|
17Q | First examination report despatched |
Effective date: 20140530 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20160218 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: AT Ref legal event code: REF Ref document number: 819982 Country of ref document: AT Kind code of ref document: T Effective date: 20160815 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 7 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602010035451 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20160810 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 819982 Country of ref document: AT Kind code of ref document: T Effective date: 20160810 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161210 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161110 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161212 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160930 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161111 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602010035451 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20161110 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 |
|
26N | No opposition filed |
Effective date: 20170511 |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20161110 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160930 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160909 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160930 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 8 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160909 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20161110 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20100909 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 Ref country code: MT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160930 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 9 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160810 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R081 Ref document number: 602010035451 Country of ref document: DE Owner name: HUAWEI DEVICE CO., LTD., DONGGUAN, CN Free format text: FORMER OWNER: HUAWEI DEVICE CO., LTD., SHENZHEN, GUANGDONG, CN |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230808 Year of fee payment: 14 Ref country code: DE Payment date: 20230802 Year of fee payment: 14 |