GB2559396A - Method of video transmission and display - Google Patents

Abstract

A method, in a video output device, for acquiring video data for outputting to a display, comprises subscribing to a multicast stream of a plurality of multicast streams. Each multicast stream is streamed from a video source and comprises video frame data corresponding to a video frame portion (region, area). Subscribing to a multicast stream may comprise transmitting a subscription request to a network switch, coupled to the video output device and video source; the subscription request identifies the multicast stream and may indicate desired video quality, e.g. frame resolution, degree of video compression, frame bit depth or chroma subsampling scheme. Determining the multicast stream to request may be based on a position of the video frame on the display, e.g. by determining a mapping of a pixel of the display to a location in the video frame; a video frame position change may lead to a new multicast stream being subscribed to. The method has application to a video (display) wall.

Description

(71) Applicant(s):

TV One Limited

Continental Approach, Westwood Industrial Estate, MARGATE, Kent, CT9 4JG, United Kingdom (72) Inventor(s):

Richard Peter Disney Mallett (56) Documents Cited:

EP 1064817 B1 WO 2016/072927 A1 (58) Field of Search:

INT CL G06F, G09G, H04N Other: WPI, EPODOC

WO 2016/182541 A1 US 20160034240 A1 (74) Agent and/or Address for Service:

EIP

Fairfax House, 15 Fulwood Place, LONDON, WC1V 6HU, United Kingdom (54) Title of the Invention: Method of video transmission and display

Abstract Title: Subscribing to a multicast stream comprising data corresponding to a video frame portion (57) A method, in a video output device, for acquiring video data for outputting to a display, comprises subscribing to a multicast stream of a plurality of multicast streams. Each multicast stream is streamed from a video source and comprises video frame data corresponding to a video frame portion (region, area). Subscribing to a multicast stream may comprise transmitting a subscription request to a network switch, coupled to the video output device and video source; the subscription request identifies the multicast stream and may indicate desired video quality, e.g. frame resolution, degree of video compression, frame bit depth or chroma subsampling scheme. Determining the multicast stream to request may be based on a position of the video frame on the display, e.g. by determining a mapping of a pixel of the display to a location in the video frame; a video frame position change may lead to a new multicast stream being subscribed to. The method has application to a video (display) wall.

aO	a1	a2	a3
a4	a5	a6	a7
a8	a9	a10	a11
a12	a13	a14	a15

bO	b1	b2	b3
b4	b5	b6	b7
b8	b9	b10	b11
b12	b13	b14	b15

cO	c1	c2	c3
c4	c5	c6	c7
c8	c9	c10	c11
c12	c13	c14	c15

1220

1210

1215

1225 Network switch b8 aO bO a2 b2 a12 a13 b9 b12 b13 a1 a4 b4 b5 a3 a7 b3 b6 b7

1205b

cO	c8
c1	c9
c2	c10
c3	c11
c4	c12
c5	c13
c6	c14
c7	c15

1205a

aO	a1		a2	a3
a4	bO	b1		b2	b3	a7
b4	b5		b6	b7

a8 c8 c9 c10 c11

1205c

Figure 12

1205d

1/24

100

115a

	7
	/
/—	7

115c

105

105a 105b 115b

Figure 1

200

2/24

Figure 2

3/24

300

Streams

325 <Subscription request 330

Stream

335

Video source 305

Network switch 310

Video output device 315

Figure 3

4/24

Figure 4

5/24

500

515 Group address

520 Display device address

525 Identify as subscription request

530 Checksum

Figure 5

6/24

Video

600 -►

Figure 6

7/24

710

Ports

Figure 7

8/24

800

810

Processor «

805

Network interface

Figure 8

9/24

900

Figure 9

10/24

1000

1015

++++++++»++++4 ++++++++»++++4 +++++++<+++++4 +++++++++++++4 +++++++++++++4 +++++++++++++4 +++++++++++++4 +++++++++++++4 +++++++++++++3

— 1005b ++++++++++++++++1 ++++++++++++++++1 ++++++++++++++++1 ++++++++++++++++1 ++++++++++++++++1 ++++++++++++++++1 ++++++++++++++++1 ++++++++++++++++1

I++++++++++++++++I

1005d

Figure 10

11/24

1105

1123

1130

Figure 11

12/24

aO	a1	a2	a3
a4	a5	a6	a7
a8	a9	a10	a11
a12	a13	a14	a15

bO	b1	b2	b3
b4	b5	b6	b7
b8	b9	b10	b11
b12	b13	b14	b15

cO	c1	c2	c3
c4	c5	c6	c7
c8	c9	c10	c11
c12	c13	c14	c15

1210

1215

1220

1225 Network switch

a8	b8		aO	bO		a2	b2
a12	b9	a1	b1	a3	b3
a13	b12	a4	b4	a7	b6
	b13			b5			b7
	1205a \ J			r

aO	a1
a4	bO	b1
b4	b5

1205b

cO	c8
c1	c9
c2	c10
c3	c11
c4	c12
c5	c13
c6	c14
c7	c15

a2	a3
b2	b3	a7
b6	b7

a8	b8	b9
b12	b13
a12	a13

cO	c1	c2	c3
c4	c5	c6	c7
c8	c9	c10	c11
c12	c13	c14	c15

1205c

1205d

Figure 12

13/24

Video resolution	Pixel rate (Mpix/s)	4:4:4 12bit (Gbits/s)	4:4:4 8bit (Gbit/s)	4:2:2 10bit (Gbit/s)	4:2:0 8bit (Gbit/s)
720p60	57.6	2.07	1.38	1.15	0.69
1080p60	129.6	4.67	3.11	2.59	1.56
4k30	259.2	9.33	6.22	5.18	3.11
4k60	518.4	18.7	12.4	10.4	6.22
8k60	2073.6	74.6	49.8	41.5	24.9

Figure 13

1400

14/24

No window

Figure 14

15/24

1500 %»

0	1	2	3	4	5	6	7
8	9	10	11	12	13	14	15
16	17	18	19	20	21	22	23
24	25	26	27	28	29	30	31
32	33	34	35	36	37	38	39
40	41	42	43	44	45	46	47
48	49	50	51	52	53	54	55
56	57	58	59	60	61	62	63

1505 %»

0	57	50	43	36	29	22	15
8	1	58	51	44	37	30	23
16	9	2	59	52	45	38	31
24	17	10	3	60	53	46	39
32	25	18	11	4	61	54	47
40	33	26	19	12	5	62	55
48	41	34	27	20	13	6	63
56	49	42	35	28	21	14	7

Figure 15

16/24

65	40
30	45

i

A=45

Figure 16

17/24

1710

1720

1730

X

1	2
3	4

χ

1	2
3	4

χ

1	2
3	4

Y Cr Cb

X

Y1,Cr1,Cb1	Y2,Cr2,Cb2
Y3,Cr3,Cb3	Y4,Cr4,Cb4

YCrCb 4:4:4

Figure 17

18/24

1810

1	2
3	4

Y Cr Cb

X

Y1,Cr1,Cb1	Y2,Cr1,Cb1
Y3,Cr2,Cb2	Y4,Cr2,Cb2

YCrCb 4:2:2

Figure 18

19/24

1910

1	2
3	4

Y Cr Cb

X

Y1,Cr1,Cb1	Y2,Cr1,Cb1
Y3,Cr1,Cb1	Y4,Cr1,Cb1

YCrCb 4:2:0

Figure 19

20/24

2010

X

44	77	77	44	44	44	44	44
77	44	44	77	77	77	77	77
44	0	0	44	77	77	77	77
0	0	0	44	77	44	44	44
0	0	0	44	77	44	0	0
0	0	0	44	77	44	0	0
128	64	0	44	77	44	0	0
128	128	0	44	77	44		0

Figure 20

21/24

2110

X

1108

2120 \

616		1001
624	1·^

Figure 21

22/24

2210

484		484
132
		242
536	βββββ

2220 \

	242		242
44			275
	88		0
448			242	—“T-

Figure 22

23/24

2310

2320

X

	77	77	44	11M11 I·:·:·:·:·:·:·:·: v/!<v!v5(H·:·:·:·:·:·:·:·:	44	11¾¾¾	44
77	44	44	lllBBllI	77	liiBlii	77	liiBlii
44	0	0	44	77	77	77	77
0	IIM	G	44	77	44	44
	0		44		44		0
0		0	II	77	iiiijiii	0	111
128	64	|o	44	77	44	lll·^	0
12S	128	0	44	77	44	0	11M^

Figure 23

24/24

	No, sent	8-Bit	10-Bft	12-Bit	16-Bit
HO	32	8	10	12	16
VO	16	9	11	13	17
Hl	8	10	12	14	18
VI	4	11	13	15	19
H2	2	12	14	16	20
V2	1	13	15	17	21
H3V3	1	14	16	18	22
Total for streams	575	703	831	1087
Original video frame:	512	640	768	1024
Efficiency:	89%	91%	92%	94%

Figure 24

Application No. GB1701860.7

RTM

Date :24 July 2017

Intellectual

Property

Office

The following terms are registered trade marks and should be read as such wherever they occur in this document:

HDMI (Page 13 & 37)

Intellectual Property Office is an operating name of the Patent Office www.gov.uk/ipo

METHOD OF VIDEO TRANSMISSION AND DISPLAY

Technical Field

The present invention relates to methods, apparatus and systems for transmitting and receiving video data for a display.

Background

A video wall is a video system in which a number of video monitors or displays are arranged in an array formation to create a relatively larger overall display screen. A typical video wall system includes a video processing unit which comprises a number of video input modules for providing a variety of video inputs, a central processing module for processing video data from the input modules, and a number of video output modules for providing video outputs for connection via standard video cables to the video monitors. The number of video outputs can be increased by adding further output modules to the video processing unit. However, the number of video outputs, and hence the number of monitors supported by the system, is limited by bandwidth capability of the central processing module. Increasing resolution of the video inputs places further load on the central processing module which can result in further limitations on the number of supported video outputs.

Summary

According to a first aspect of the present invention, there is provided a method, in a video output device, for acquiring video data for output to a display, the method comprising:

subscribing to a multicast stream from a plurality of multicast streams, wherein each multicast stream of the plurality of multicast streams is streamed from a video source and comprises video frame data corresponding to a portion of a video frame; and receiving video frame data from the subscribed multicast stream for display on the display.

A method in accordance with the first aspect of the invention may enable a video output device to subscribe to multicast streams corresponding to less than a full frame of video data from a video source. For example, where the display is a display of a video wall, the method enables the video output device to subscribe to streams corresponding to just the video frame data that is needed for a particular display of the video wall. Therefore, in comparison to a video output device that receives a full frame of video data, a video output device operating in accordance with the first aspect of the invention can receive relatively less video data but still provide a full set of video frame data to an associated display. This can have the benefit of enabling a reduction in the bandwidth requirements for the reception of video frame data by a video output device.

In accordance with the invention, the method may further comprise: subscribing to one or more further multicast streams of the plurality of multicast streams, and receiving video frame data from the further subscribed multicast streams for display on the display.

A video output device operating in accordance with the method can subscribe to a set of multicast streams. The set of multicast streams may contain video data which when output to a display may fill the entire frame of the display.

Suitably, subscribing to a particular multicast stream comprises transmitting a subscription request to a network switch, the network switch being coupled to the video output device and to the video source, wherein the subscription request identifies the particular multicast stream.

In some embodiments, each of the plurality of multicast streams comprises video frame data at a given video quality. In this case, the subscription request may comprise an indication of a desired video quality, and the received video frame data may comprise video frame data at the desired video quality.

This allows a reduction in network bandwidth, as video is not transmitted to the video output device at a higher quality than required. The desired video quality may comprise one or more of a desired video frame resolution, a desired degree of video compression, a desired video frame bit depth and a desired chroma subsampling scheme.

In embodiments, the method comprises determining the or each multicast stream to request based on a position of the video frame on the display.

In some embodiments, determining the position of the video frame on the display comprises determining a mapping of at least one pixel of the display to a location in the video frame.

In embodiments, the method may comprise:

determining a change of the position of the video frame on the display; based on the determined change in position, determining a new multicast stream to subscribe to; and transmitting a further subscription request to the network switch, the further subscription request identifying the new multicast stream.

The video output device can thus update the streams to which it subscribes on the fly, based on changes in the location at which a given video frame is displayed.

The or each portion may have predefined spatial dimensions.

In some examples, the method comprises outputting display data, based on the received video frame data, to the display.

In an embodiment, the method comprises:

subscribing to a second multicast stream from a second plurality of multicast streams, wherein each multicast stream of the second plurality of multicast streams is streamed from a second video source and comprises video frame data corresponding to a portion of a second video frame; and receiving video frame data from the subscribed second multicast stream for display on the display.

A given video output device can thus subscribe to streams originating from multiple video sources. Because a video output device only subscribes to streams corresponding to video frame portions that are for display on its corresponding display, a video output device may receive video data from multiple video sources without a significant increase in required bandwidth. Additionally, this allows a single video to be divided across multiple sources, which allows higher resolution video to be supported since each source only needs to provide a lower resolution video.

The display may be a display of a video wall.

According to a further aspect of the present disclosure, there is provided a 5 non-transitory computer-readable storage medium comprising a set of computerreadable instructions stored thereon which, when executed by at least one processor, cause the at least one processor to perform a method as described above.

According to an embodiment, there is provided a video output device comprising:

a network interface;

a processor coupled to the network interface and configured to:

subscribe, via the network interface, to a multicast stream from a plurality of multicast streams, wherein each multicast stream of the plurality of multicast streams is streamed from a video source and comprises video frame data corresponding to a portion of a video frame; and receive, via the network interface, video frame data from the subscribed multicast stream.

According to a further embodiment, there is provided a video system comprising:

a plurality of displays;

a network switch;

at least one video source coupled to the network switch and configured to provide a plurality of multicast streams via the network switch, each multicast stream of the plurality of multicast streams comprising video frame data corresponding to a portion of a corresponding video frame;

a plurality of video output devices coupled to the network switch, each video output device being coupled to a corresponding display and being configured to:

transmit a subscription request to the network switch, the subscription request being for at least one of the plurality of multicast streams;

the network switch being configured to, in response to receiving a request from one of the plurality of video output devices, transmit each requested multicast stream to the corresponding video output device.

In some examples, each video source of the least one video sources provides 5 its corresponding plurality of multicast streams such that, for a single video frame, a video source transmits video frame data of each multicast stream in series to the network switch; and the series is such that video frame data corresponding to adjacent portions of a single video frame is transmitted nonconsecutively.

A consequence of the non-consecutive transmission of adjacent portions is a reduction in the incidence of spikes in required bandwidth, for example where a given video output device subscribes to streams corresponding to equivalent portions of multiple videos.

The series may be such that multicast streams corresponding to adjacent portions of a single video frame are transmitted in a pseudorandom order.

The plurality of displays may be displays of a video wall.

According to a further embodiment, there is provided a method of video frame transmission in a video system, the method comprising:

transmitting, from a video source and to a network switch, a plurality of multicast streams, each multicast stream of the plurality of multicast streams comprising video frame data corresponding to a portion of a corresponding video frame;

transmitting, from a video output device and to the network switch, a subscription request for at least one of the plurality of multicast streams; and in response to receiving a said subscription request, transmitting each requested multicast stream from the network switch to the video output device. According to an aspect of the present disclosure, there is provided a method, in a video source, for providing streamed video data. The method comprises:

retrieving video frame data from a video input; and providing to a network a plurality of multicast streams, wherein each multicast stream of the plurality of multicast streams comprises video frame data corresponding to a portion of a video frame.

According to a further aspect, there is provided a video source comprising: a video input; a network interface; and a processor, the processor being configured to:

retrieve video frame data from the video input; and provide, via the network interface, a plurality of multicast streams, wherein each multicast stream of the plurality of multicast streams comprises video frame data corresponding to a portion of a video frame.

Further features and advantages of the invention will become apparent from the following description of preferred embodiments of the invention, given by way of example only, which is made with reference to the accompanying drawings.

Brief Description of the Drawings

Figure 1 shows an example video wall system.

Figure 2 shows a schematic representation of a method according to an embodiment.

Figure 3 shows a schematic representation of a method according to an embodiment.

Figure 4 shows an example of data flow in a video system.

Figure 5 shows a schematic representation of a subscription request data packet according to an embodiment.

Figure 6 shows a schematic representation of a video source according to an embodiment.

Figure 7 shows a schematic representation of a network switch according to an embodiment.

Figure 8 shows a schematic representation of a video output device according to some embodiments.

Figure 9 shows a schematic representation of a video output device according to an embodiment.

Figure 10 shows a schematic representation of a video wall.

Figure 11 shows a schematic representation of a layout of a display, a canvas and two video windows.

Figure 12 shows an example of transmission, receipt and display of streamed video data on displays of a video wall.

Figure 13 shows a table of example bit rates required to transmit video data at various resolutions and chroma subsampling schemes.

Figure 14 shows an example procedure for a video output device to subscribe to streams.

Figure 15 shows an ordering of blocks of video frame data for transmission, according to an embodiment.

Figures 16 to 19 show example method of downscaling video data.

Figures 20 to 23 show a method of transmitting downscaled video data, according to an embodiment.

Figure 24 shows a table of example efficiencies for transmitting downscaled video data.

Detailed Description

In the following description, reference is made to videos and video data. In general, videos that are processed by computers and video processing devices can be thought of as a sequence of individual still images which are often referred to as video frames. Each image or video frame consists of a number of pixels which are typically arranged in a grid of horizontal rows and vertical columns. The number of horizontal rows (or lines) and vertical columns (or lines) determines a resolution characteristic of the image or frame as well as the corresponding video.

Referring to Figure 1, there is shown an example video system 100. Links between entities of the system 100 may for example comprise 1G, 10G or 25G copper or fibre links.

The system 100 comprises a video wall 105 comprising an array of four display devices 105a-d arranged in a two-by-two layout. Although the video system 100 is shown as a video wall system, it will be appreciated that embodiments of the present disclosure are also applicable to other video systems including, for example, systems where a plurality of displays do not necessarily form a video wall. Examples video systems include systems used in courtrooms, government assemblies, boardrooms, command & control centers, simulators, operating theatres, television studios, live stage events, casinos, and sports stadiums.

Each display device may, for example, be a computer monitor, a video projector or a television screen. The displays may be general purpose display devices or, alternatively, they may be display devices designed specifically for use in a video wall 105.

The system 100 comprises a network switch 107, such as a non-blocking network switch. A network switch includes a number of bidirection input/output ports, and a non-blocking network switch is one where each port can typically pass data to every other port, and typically the internal bandwidth of the switch can handle all the port bandwidths, at the same time, at full capacity. The switch 107 is configured to handle multicast subscription requests. The switch 107 may be utilised to provide a master Dynamic Host Configuration Protocol (DHCP) server and/or time server for the system 100.

The video system 100 comprises one or more video sources 110. The video source 110 is coupled to the network switch 107 and together they are configured to provide a plurality of multicast streams via the network switch 107. Each multicast stream comprises video frame data corresponding to a portion of a video frame. For example, a video frame with a particular resolution may be divided into fixed-size lower-resolution portions as described in more detail below. Preferably, the video frame is divided in both the horizontal and vertical direction so that each portion of the video frame data corresponds to a block of pixels. Each block may be a rectangular block having the same aspect ratio as the original video frame. Or the blocks could be square blocks of pixels. Each portion of the video frame would then be transmitted in a respective multicast stream.

The portion of the video frame may be a region within the video frame, or an area within the video frame. If the video frame is available in different resolutions, the portion of a video frame may be a portion of a video frame at a particular resolution. If the video frame at a particular resolution has particular number of pixels then the portion of the video frame may be a portion, region or area of the video frame covering a subset of the full set of pixels. If the video frame at a particular resolution is represented by a particular set of video frame data then the portion of the video frame may be a portion, region or area of the video frame incorporating a subset of video frame data.

In some embodiments, a single video source 110 provides a single video for display on the video wall 105. In a further embodiment, the system 100 comprises a plurality of video sources 110, each of which are configured to provide respective video frame data for display on the video wall 105. Some or all of the plurality of sources may comprise separate hardware. Alternatively or additionally, some or all of the plurality of sources may comprise logically separate sources within the same source hardware. In this manner, a single physical video source 110 can provide multiple videos for simultaneous display on the video wall 105. In some embodiments wherein sources comprise separate hardware, multiple video sources 110 provide the same video frame data. Such an arrangement provides redundancy in the system so that a loss of video from one video source 110 can be compensated for by the provision of the same video from another video source.

In some embodiments, for example where provision of video frames would require particularly high bandwidth or processing power, a single source video can be divided, for example spatially, into separate videos, each corresponding to a region of the single source video. For example, video frames may be divided into a number of evenly spaced vertical strips. The separate videos can then be provided by physically separate video sources. This approach is useful for relatively high resolution source videos.

In the example shown in Figure 1, the video source 110 provides a single video for display across the entire video wall 105 such that the top-left quarter of a given video frame is displayed on display device 105a, the top-right quarter is displayed on video device 105b, the bottom-left quarter is displayed on display device 105c and the bottom-right quarter is displayed on video device 105d.

The video source 110 may include a storage medium such as a hard disk or flash memory for storing a video file, which can be retrieved when a video needs to be output by the video source. The video could also originate from a live source such as a video from a video camera. Such a storage medium or live source of video can be local to the video source 110 hardware. For example, a video may be provided from a flash memory in the same cabinet as the video source 110. Alternatively, the storage medium or live source of video can be remote from the video source 110 hardware. For example, a live source may comprise a video camera coupled via a wireless link to the video source 110.

The video system 100 further comprises a plurality of video output devices 115a-d. Each video output device 115a-d is coupled to the network switch 107 and is also coupled to a corresponding display device 105a-d of the video wall 105. In some embodiments, a given video output device 115a-d is incorporated within the functionality of its corresponding display device 105a-d. For example, a display device 105a-d may comprise a dedicated video output device 115a-d hardware. Alternatively, a general purpose processor of a display device 105a-d may implement the functionality of a video output device 115a-d. Each video output device 115a-d is configured to transmit a subscription request to the network switch 107. The subscription request is for at least one of the plurality of multicast streams provided by the video source 110. The requested stream comprises video frame data that is for display on the display device 105a-d to which that video output device 115a-d is coupled. As an example, video output device 115a, associated with the top-left display 105a, would transmit a subscription request for at least one multicast stream corresponding to the top-left quarter of the video frame.

In an embodiment, a video output device 115a-d is configured to subscribe to at least one further stream of the plurality. Each of the at least one further streams comprises video frame data that is for display on the corresponding display 105a-d such that the combined requested streams together provide the entirety of video frame data that is for display on the display device 105a-d. In an example, a video output device 115a-d may submit separate subscription requests for each such stream. Alternatively, a video output device 115a-d may submit a single subscription request for multiple streams.

In some embodiments, the multicast streams are managed by a stream controller 120. Each video source 110 transmits to the stream controller 120 a description of the streams that it provides. The description may for example include a unique identifier of each stream and details of the specific portion of the video frame to which each stream relates. The stream controller 120 then transmits such a description to each video output device 115a-d. Each video output device 115a-d can thus determine which streams it should subscribe to, as described in more detail below.

In other embodiments, the system 100 does not comprise a stream controller 120. In such embodiments, each video source 110 broadcasts a description of the streams that it provides to each video output device 115a-d. As above, the description may include a unique identifier of each stream and details of the specific portion of the video frame to which each stream relates. Each video output device 115a-d can thus receive the broadcast stream descriptions and hence determine which streams it should subscribe to.

The network switch 107 is configured to receive subscription requests. In response to receiving a subscription request from a video output device 115a-d, the network switch transmits each requested stream to that video output device 115a-d. A given video output device 115a-d thus only receives video frame data based on the portion or portions to which it subscribes. Hence, the system 100 avoids the need to transmit the entirety of the video frame data to each video output device. As such, the overall bandwidth requirements of the system 100 can be reduced.

By using a network switch to distribute video streams to different video output devices, as well as manage streaming requests, the number of components of the system 100 can be kept low. For example, the system 100 may be configured with the network switch, the video input(s) 110, and the video output devices but without using any separate video servers to receive video frame data from the video source 110 and provide the data to the video output devices 105a-d.

It has been observed that, in systems in which the entire video frame is sent to each video output device 105a-d, the required network bandwidth increases with the number of displays. Similarly, the required processing power of a video source 110 increases with the number of displays. This can increase to the point that no further displays can be added to the video wall. The present system 100 enables pixel data of a video frame, provided by a video source 110, to be transmitted only to a subset of the video output device 115a-d that require that pixel data for their respective displays 105a-d. The system 100 can also enable a multicast stream relating to a portion of the video frame which is to be partially displayed on one display and partially displayed on one or more further displays to be subscribed to by the video output devices of those displays. Furthermore, in the case of edge blending, where two or more projectors acting as display devices provide partially overlapping video output, the system 100 enables the output of overlapping video by providing pixel data for the overlapping portion to the two or more projectors.

As such, the system 100 enables the bandwidth of video frame data transmitted from the video source 110 to each video output devices 115a-d, via the network switch 107, to be significantly lower than the bandwidth required for transmitting a complete version of the video frame data from the video source 110 to each display device 115a-d, regardless of how many displays 105a-d the video frame is spread across. Additional displays, with corresponding video output devices, can thus be added to the system without requiring additional or upgraded video source hardware. This remains true if such an additional display is physically remote from the rest of the network hardware, but it may be necessary to connect the additional display via upgraded connection hardware capable of transmission over an extended distance.

Suitably, the number of videos and/or the resolution of video available in the system 100 can be increased by upgrading individual video source 110 devices so that they are capable of handling increased bandwidth. Additionally or alternatively, the number of videos and/or the resolution of video available in the system 100 can be increased by adding additional video source 110 devices.

Typically, a video output device 115a-d will be capable of outputting video frame data at the maximum resolution of its corresponding display 105a-d. Therefore, even if the number of videos and/or the resolution of video available in the system 100 is increased, this would typically not require an upgrade the video output devices 115a-d.

After receiving the video frame data from the network switch 107, each video output device 115a-d then outputs display data, based on the received video frame data, to its corresponding display 105a-d. For example, the display data may comprise data in accordance with the HDMI standard, the DVI standard, or the VGA standard. In some examples, a video output device 115a-d may produce the display data by identifying a part of the received video frame data as not being for display, and producing the display data so as to exclude the identified part of the received video frame data. For example, where each multicast stream corresponds to a fixed-size block of a video frame, a given video output device 115a-d subscribes to streams such that the blocks corresponding to each requested stream together include the portion of the video frame that is for display on its corresponding display 105a-d. The received blocks may also cover further portions of the video frame, adjoining the portion that is for display. The video output device 115a-d may thus identify such further portions as not for display, and produce the display data omitting such further portions.

In some embodiments, when a video output device 115a-d does not receive a given block, for example due to network errors, the video output device 115a-d can interpolate the missing data from surrounding pixel blocks. Alternatively, the video output device 115a-d may replace the missing block with a corresponding block from the previous frame, for example held in a buffer. However, provided that a large number of blocks are not missing, such errors are typically not very noticeable on a display.

In some embodiments, multiple switches 107 can be used. In an example video wall 105 in which the total required bandwidth, for all displayed videos, does not exceed 10 Gb, the one or more video sources 110 may be linked to a single 10 Gb switch. This is then linked via a single 10 Gb link to a similar switch, to which all video output devices 115a-d are connected. For large video walls 105, multiple switches could be configured, with each switch being connected to a subset of the video output devices 115a-d. In some embodiments in which multiple switches are used, each switch is located physically near the video output devices 115a-d and/or video sources 110 to which it is connected. The switches may then be connected to each other via a fast interconnection, such as a fibre interconnection. In such examples, it is preferable for the interconnection to be capable of supporting nonblocking usage over all connected ports. The interconnection should preferably be capable of handling the greater of the total video output device 115a-d bandwidth and the total video source 110 bandwidth.

As a further example of network hardware, two or more connections could be used to send video frame data from the network switch 107 to a given video output device 115a-d. Such a configuration is particularly useful in embodiments such as control rooms, wherein multiple lower resolution sources are displayed on a higher resolution display, as this configuration allows a higher resolution display to be used since more video frame data can be transmitted from the network switch 107 to a given video output device 115a-d. Alternatively, a single high-bandwidth connection could be used instead of two or more lower-bandwidth connections.

Alternatively or additionally, two or more connections could be used to transmit multicast streams from a video source 110 to the network switch 107. This configuration is particularly advantageous in embodiments where the resolution of the video from the video source 110 is higher than the display resolution, as it allows a higher resolution video frame to be displayed on the video wall 205. Alternatively, a single high-bandwidth connection could be used instead of two or more lowerbandwidth connections.

A further advantage of the use of two connections to video output devices 115a-d and/or video sources 110 is that such a system could, in response for example to a network issue, use only one of the two connections. Such a system thus provides redundancy in case of network faults. As an example, when two links from a video source 110 are in use, video frame data could be streamed at a lower compression ratio and when a single link is in use, video frame data could be streamed at a higher compression ratio. Switching between these modes of operation may be triggered by automated sensing of a network issue, for example by further network hardware such as a network controller.

Figure 2 shows a schematic representation of a method 200, in a video output device, for acquiring video data for outputting to a display according to an aspect of the present disclosure.

The method 200 comprises a subscribing step 205 of subscribing to a multicast stream of a plurality of multicast streams. Each multicast stream is streamed from a video source and comprises video frame data corresponding to a portion of a video frame. The multicast stream to which the video output device subscribes comprises video frame data that is for display on a display associated with the video output device.

The method then comprises a receiving step 210 of receiving the video frame data that is for display on the display.

Figure 3 shows a schematic representation of a similar method 300 of video frame transmission in a video system. The system, as described above in connection with Figure 1, comprises at least one video source 305, a network switch 310 and at least one video output device 315. The method comprises data flow between a given video source 305 and a given video output device 315 via the network switch 310.

The video source 305 transmits to the network switch 310 a plurality of multicast streams 320. As set out above, each multicast stream comprises video frame data corresponding to a portion of a corresponding video frame.

The video output device 315 transmits to the network switch 320 a subscription request 330 for at least one of the plurality of streams. Each requested stream comprises video frame data that is for display on a display device associated with the video output device.

In response to receiving a said subscription request, the network switch 310 transmits each requested stream 335 to the video output device 315.

Figure 4 shows an example of data flow in a video system. The system comprises video sources 405, 410 and video output devices 415, 420, connected to each other via a network switch 425. Video source 405 provides multicast streams 405a-c to the network switch 425, each of which relate to portions of a first video. Video source 410 provides multicast streams 410a-c to the network switch 425, each of which relate to portions of a second video. Multicast streams 405a and 405b correspond to portions of the first video that are for display on the display associated with video output device 415. Multicast streams 405b and 405c correspond to portions of the first video that are for display on the display associated with video output device 420. Multicast streams 410a and 410c correspond to portions of the second video that are for display on the display associated with video output device 420. Multicast stream 410b corresponds to a portion of the second video that is not for display on either output device 415, 420.

Video output device 415 transmits to the network switch 425 one or more subscription requests 430 identifying streams 405a and 405b. A single subscription request 430 may be transmitted, identifying both streams 405a and 405b. Alternatively, video output device 415 may transmit separate subscription requests 430 for each stream 405a, 405b.

Similarly, video output device 420 transmits to the network switch 425 one or more subscription requests 435 identifying streams 405b, 405c, 410a, 410c. A single subscription request 435 may be transmitted, identifying all four streams 405b, 405c, 410a, 410c. Alternatively, video output device 420 may transmit one request 435 for streams 405b, 405c from video source 405, and a second request for streams 410a, 410c from video source 410. As a further example, separate subscription requests 435 may be transmitted for each stream 405b, 405c, 410a, 410c.

Dashed lines within the switch 425 indicate data flow from video sources 405, 410 to video output devices 415, 420 via the switch 425. In response to receiving the above-described subscription requests, the switch 425 transmits each requested stream to the corresponding video output devices 415, 420. Specifically, the switch 425 transmits streams 405a, 405b to video output device 415, and transmits streams 405b, 405c, 410a, 410c to video output device 420.

Figure 5 shows a schematic representation of a subscription request data packet 500 that is transmitted from a video output device to a network switch according to an embodiment. In analogous embodiments, the packet may be structured according to a standard multicast protocol, for example Internet Group Management Protocol (IGMP).

The packet 500 comprises a network address 515, for example a multicast group IP address, of the source of the video data to which the subscription request relates. In some embodiments, a stream controller receives details of each available stream, including the corresponding multicast group addresses, from each video source. The stream controller transmits these details to each video output device. In other embodiments, video sources broadcast details of available streams to each video output device. Each video output device can thus maintain a record of the available streams, including the multicast group address of each stream.

The packet 500 comprises a network address 520 of the display device from which the subscription request originates.

In some embodiments, the packet 500 further comprises a field 525 identifying the packet as a subscription request. In some examples, the network address 515 is sufficient to identify the packet 500 as a subscription request. In such examples, the packet 500 does not include the identification field 525.

The packet 500 then comprises a checksum 530 to enable the network switch to detect errors in the packet 500. For example, the checksum may comprise the 16-bit one's complement of the one's complement sum of the entire packet 500.

A network switch is configured to identify the packet 500 as a subscription request, for example by identifying the network address 515 as a multicast group address or by way of the identification field 525 identifying the packet as a subscription request. The network switch is configured to maintain a record of subscriptions. The network switch receives each multicast stream from each video source and, based on the record of subscriptions, forwards each stream to the video output devices that have subscribed to that stream.

In some examples, the packet 500 comprises an identification of multiple streams to which the subscription request relates. For example, the packet 500 may identify multiple multicast group addresses. In one such example, the packet 500 comprises a field identifying the bit length of the packet and/or the number of streams to which the request relates.

Figure 6 shows a schematic representation of a video source 600 according to one embodiment. The source 600 comprises various modules. Any of all of these modules could, either alone or in any combination, be implemented by dedicated circuitry, for example one or more field-programmable gate arrays, or be implemented by routines in one or more general processors.

The source 600 comprises an input 605 to receive video data, for example from a storage medium such as a hard disk or solid-state memory, or from a live video feed.

In some examples, video frame data is stored as block-by-block video data. The video frame data is then received at the input as block-by-block data, with each block of a given frame to be sent in a separate multicast stream. Alternatively, video frame data may be stored as line-by-line video frame data and converted by a module of the video source 600 to block-by-block video frame data, with each block corresponding to a separate multicast stream. As an example of such a conversion, 135 lines of video frame data may be cached and, from this, blocks of size 240x135 pixels may be determined. A 1920x1080 pixel frame may thus be converted into an 8x8 grid of blocks. This allows the block size to be modified on the fly, for example in response to a command from a stream controller.

In some examples, block-by-block video frame data is stored such that a given block is stored in successive memory locations of a bank of a memory, with adjacent blocks being stored in different banks of the memory. A given block can thus be rapidly read out from a bank of the memory. In embodiments wherein adjacent blocks are streamed in succession, this arrangement facilitates rapid access to adjacent blocks as one bank of the memory can be read while another is being opened or closed. An example of such block-based storage is described in more detail in European patent EP2446413B1, the contents of which is incorporated herein by reference.

The source comprises a sampling rate and colour conversion module 610, configured to convert the received video to a data rate and/or colour space that is suitable for providing via the network switch to video output devices.

The source 600 comprises a scaling module 615 for downscaling the video data. The scaling module 615 may for example perform block-based downscaling, downscaling each block of video frame data to provide multiple streams for each block, each having a different overall downscaling factor and/or chroma subsampling scheme. The downscaled block data is stored in a block memory 620.

The source 600 comprises an encryption module 625, configured to encrypt the downscaled video frame data into a format suitable for transmission via the network switch to the video output devices.

The source 600 comprises a control processor 630 configured to control an ethernet controller 635 to transmit the encrypted video frame data to the network switch.

Figure 7 shows a schematic representation of a network switch 700 according to an embodiment. The switch 700 comprises various modules. Any of all of these modules could, either alone or in any combination, be implemented by dedicated circuitry, for example one or more field-programmable gate arrays, or be implemented by routines in one or more general processors.

The switch 700 comprises ports 705 for receiving multicast streams from one or more video sources. For example, if the switch 700 is an ethernet switch, the ports 705 are ethernet ports.

The switch 700 further comprises ports 710 for receiving subscription requests from a plurality of video display devices. It should be noted that although the ports 705 and the ports 710 are for ease of explanation shown as distinct in Figure 7, ports 705 are not specifically optimised for connection to video sources and ports 710 are not specifically optimised for connection to display devices. A video source could be connected to a port 710 and/or a video display device could be connected to a port 705.

The switch comprises a processor 715 and switching hardware 720. The processor 715 is configured to process subscription requests received via ports 710. In response to receiving such a request, the processor 715 controls the switching hardware 720 to transmit each requested stream to the video display device from which the request was received.

Figure 8 shows a schematic representation of a video output device 800 according to some embodiments.

The video output device 800 comprises a network interface 805 and a processor 810 coupled to the network interface. The processor 810 is configured to subscribe, via the network interface 805, to a multicast stream of a plurality of multicast streams. Each multicast stream is streamed from a video source and comprises video frame data corresponding to a portion of a video frame. The multicast stream to which the processor 810 subscribes comprises video frame data that is for display on a display associated with the video output device 800.

The processor 810 is further configured to receive, via the network interface 805, the video frame data that is for display.

Figure 9 shows a schematic representation of a video output device 900 according to an embodiment. The video output device 900 comprises various modules. Any of all of these modules could, either alone or in any combination, be implemented by dedicated circuitry, for example one or more field-programmable gate arrays, or be implemented by routines in one or more general processors.

The video output device 900 comprises an ethemet controller 905 controlled by a processor 910. The processor 910 is configured to transmit a subscription request for a multicast stream, via the ethernet controller 905 to a network switch, for example as described above.

The ethemet controller 905 is further configured to receive the video frame data of the requested stream from the network switch.

The video output device 900 comprises a decryption module 915, which is configured to decrypt the received video frame data into a format suitable for further processing and display.

The video output device 900 comprises a pixel cache 920 in which decrypted video frame data is stored and from which video frame data is output to a display of the video wall.

The video output device 900 comprises a transform module 925, configured to apply transforms to the stored video frame data to produce video frame data suitable for display. For example, the transforms may comprise warping, rotating, up-scaling, down-scaling, de-interlacing and/or edge blending, depending on the desired display configuration of the video frame. Additionally, the transform module 925 may determine the particular multicast streams for subscription, as described in more detail below, and transmit an identification of those streams to the processor 910.

The video output device 900 comprises a sync pulse generator 930 configured to provide a synchronisation signal for synchronising the outputting of video frame data with the refresh rate of the display.

Figure 10 shows a schematic representation of a video wall 1000. The video wall comprises an array of four displays 1005a-d in a two-by-two layout. The video wall 1000 is configured to display three windows 1010, 1015, 1020. Each window 1010, 1015, 1020 represents a video served by a video server, and by analogy the window also represents a video frame of the video. Each window 1010, 1015, 1020 has a corresponding layer number, with window 1015 having a higher number than window 1010 and window 1020 having a higher number than window 1015. In a region where two windows 1010, 1015, 1020 overlap, the window with the highest layer number is displayed.

Window 1010 (shown in horizontally hatched shading) is displayed at a size equal to the combined area of all four displays 1005a-d.

Window 1015 (shown in diamond shading) is displayed at a size equal to a single display, and located in the geometric centre of the video wall. As such, the topleft quarter of window 1015 is displayed on the top-left display 1005a, the top-right quarter of window 1015 is displayed on the top-right display 1005b and the bottomleft quarter of window 1015 is displayed on bottom-left display 1005c.

Window 1020 (shown in dotted shading) is displayed at a size equal to a single display, filling the bottom-right display 1005d. As such, no portion of of windows 1010 and 1015 is displayed on the bottom-right display 1020.

As set out above, each multicast stream corresponds to a portion of a video. Each such portion comprises a block of pixels in a video frame. A video output device determines the relevant streams to which it should subscribe based on determining a position of the video frame relative to the associated display. In some embodiments, determining the location on the display comprises determining a mapping of at least one pixel of the display to a location in the video frame. For example, pixels in the bottom-right quarter of the top-left display 1005a map to positions on the top-left quarter of the video frame in window 1015. Based on the mapping, it is thus determined that the video output device should subscribe to streams corresponding to the top-left comer of the video of window 1015.

Furthermore, pixels in the remaining three quarters of the top-left display 1005a map to positions in the top-left three quarters of the video frame in window 1010. Based on the mapping, it is thus determined that the video output device should subscribe to streams corresponding to the top-left comer of the video of window 1010. The “positions” referred to above may be defined by horizontal and vertical coordinates on the display, the canvas and/or the video frame.

In some examples, the mapping is such that a block of pixels received from a given stream is displayed on a block of pixels of the display, the block of pixels of the display having the same dimensions as the received block of pixels. In other examples, the mapping is such that a received block of pixels from a given stream is displayed on a block of pixels of the display, with pixel values of the block of pixels of the display being determined by interpolating between pixel values of the received block of pixels. For example, the video frame data may be displayed with a rotation angle with respect to the display such that a single pixel of the display does not correspond to a single pixel of received video frame data. In such a case, the pixel values of the display are determined by interpolating between pixel values of the received video frame data.

Figure 11 shows a schematic representation of a mapping as described above, according to an example. A canvas 1105 is defined. The canvas 1105 is a virtual area in which windows 1110, 1115 are positioned. Each window 1110, 1115 displays video frame data. An area 1120 of the canvas 1105 corresponds to a display, such that portions of windows 1110, 1115 that fall within the display area 1120 are displayed at pixels in corresponding portions of the display.

The video output device maps each pixel of the display 1120 to a corresponding position on the canvas 1105. From this, the video output device maps each such position on the canvas to a position in each relevant window. For example, a pixel 1123 of the display maps onto a corresponding position in window 1115, and a pixel 1125 of the display maps onto corresponding positions in both window 1110 and window 1115. A pixel 1130 does not map onto a corresponding position in any window.

As outlined above, each multicast stream corresponds to a portion of a video, for example a fixed-size 240x135 pixel block. Examples of such blocks are represented in Figure 11 as dotted squares within each window 1110, 1115. The video output device determines blocks that include the aforementioned positions in each window. The video output device then subscribes to streams corresponding to each determined block. For example, pixel 1123 maps to a position within block 1135 of window 1115; the video output device would thus subscribe to the stream corresponding to this block. Pixel 1130 does not correspond to a position within any window; thus no stream would be subscribed to based on this pixel.

Pixel 1125 maps to a position within window 1115 and also to a position within window 1110. The video output device determines which window 1110, 1115 should be displayed at this position. As described above, each window 1110, 1115 has an associated layer number and the window 1110, 1115 with the highest layer number is displayed in any such overlapping regions. In the present example, window 1110 has a higher layer number than window 1115. Pixel 1125 maps to block 1140 within window 1110, and thus the video output device would subscribe to a stream corresponding to this block. Window 1110 and thus block 1140 is rotated at an angle with respect to the display 1120. As such, once video frame data relating to this block has been received, the video output device may interpolate between pixel values of the block to determine corresponding display pixel values, based on the rotation angle.

Figure 12 shows an example of transmission, receipt and display of streamed video data on displays 1205a-d of a video wall.

Three videos 1210, 1215, 1220 are to be displayed on displays 1205a-d. Each display 1205a-d has a resolution of 1920x1080 pixels. The videos 1210, 1215, 1220 are to be displayed in the example arrangement shown in Figure 10, such that video 1210 is displayed at a size equal to the combined area of all four displays 1205a-d, video 1215 is displayed at a size equal to a single display and located in the geometric centre of the video wall, and video 1220 is displayed at a size equal to a single display, filling the bottom-right display 1205d. Video 1220 has a higher layer number than video 1215, and video 1215 has a higher layer number than 1210.

Each video 1210, 1215, 1220 is divided into a four-by-four grid of pixel blocks. Each video 1210 has a resolution of 3840x2160 pixels. Each block therefore has a size of 960x540 pixels. Large blocks are presented in Figure 12 for ease of explanation, but blocks of other dimensions may alternatively be used. For example, a 1920x1080 pixel video frame may be divided into an 8x8 grid of blocks, each with a size of 240x135 pixels.

A larger block size requires a smaller number of multicast streams for each video, but requires a higher bandwidth for each stream. A given video output device would need to subscribe to such a high-bandwidth stream even if only a small number of pixels were required from that stream. This would increase the required network bandwidth. Conversely, a smaller block size decreases the required bandwidth for each block, but increases the total number of streams that must be handled by the network switch. The size of the blocks may be optimised based on a balance between these factors.

The size of a given block may be predetermined. In other examples, the size of a given block is varied, for example by a stream controller connected to the network. This allows the block size to be varied based on network conditions, for example available bandwidth and/or network switch processor load.

In Figure 12, the blocks of video 1210 are identified sequentially as blocks aOal5, the blocks of video 1215 are identified sequentially as blocks bO-bl5, and the blocks of video 1220 are identified sequentially as blocks c0-cl5. A multicast stream corresponding to each such block is transmitted to a network switch 1225.

A video output device associated with each display 1205a-d determines which blocks of each video are to be displayed on its corresponding display 1205a-d. The video output device then subscribes to the streams corresponding to those blocks.

As such, the video output device corresponding to display 1205a subscribes to streams corresponding to blocks aO, al and a4 of video 1210, and blocks bO, bl, b4 and b5 of video 1215. It does not subscribe to the stream corresponding to block a5 of video 1210, because video 1215 overlaps video 1210 in that region and has a higher layer number. As such, block a5 is not displayed.

Similarly, the video output device corresponding to display 1205b subscribes to streams corresponding to blocks a2, a3 and a7 of video 1210, and blocks b2, b3, b6 and b7 of video 1215. It does not subscribe to the stream corresponding to block a6 of video 1210, because video 1215 overlaps video 1210 in that region and has a higher layer number. As such, block a6 is not displayed.

The video output device corresponding to display 1205c subscribes to streams corresponding to blocks a8, al2 and al3 of video 1210, and blocks b8, b9, bl2 and b 13 of video 1215. It does not subscribe to the stream corresponding to block a9 of video 1210, because video 1215 overlaps video 1210 in that region and has a higher layer number. As such, block a9 is not displayed.

Finally, the video output device corresponding to display 1205d subscribes to streams corresponding to all blocks c0-cl5 of video 1220. It does not subscribe to any of the streams of videos 1210 and 1215, because video 1220 is arranged to fill display 1205d and has the highest layer number. As such, no block of of videos 1210 and 1215 is displayed on display 1205d.

As noted above, the displays 1205a-d each have a resolution of 1920x1080 and the videos 1210, 1215, 1220 each have a resolution of 3840x2160. As such, video 1210 is displayed at full resolution and videos 1215 and 1220 are displayed at half resolution. In some embodiments, videos 1215 and 1220 are received at full resolution and downscaled to half resolution by the video output device. In other embodiments, each video source stores multiple downscaled versions of each block, with each version being streamed as a separate multicast stream. For example, streams may be provided with downscaling factors of 100%, 75%, 50% and 25%, or 100%, 50% and 25%, or 100%, 50%, 25% and 12.5%. This increases the required bandwidth from the video source to the network switch 1225. This also increases the number of multicast groups, which correspondingly increases the processing load on the network switch 1225. However, this reduces the required bandwidth from the network switch 1225 to a video display device that subscribes to a downscaled stream. Furthermore, this eliminates the need for the video output device to downscale received video from the full resolution, which reduces the degree of processing that must be performed by the video output device.

The provision of multiple scaled streams from the video sources may be provided using different approaches. Two such approaches are described in the section of the description entitled 'Multiple Scaled Streams' with reference to Figures 16 to 24.

In addition to resolution, other examples of video quality include bit depth and the degree of compression applied to each block. Separate multicast streams may be provided from a given video source based on any of these, or other examples of video quality. For example, different chroma subsampling schemes such as 4:2:2 and 4:2:0 may be provided in different streams. In such embodiments, a subscription request comprises an indication of a desired video quality. For example, each stream of a particular quality may have a separate multicast group address. The subscription request would then identify the specific multicast group address corresponding to the desired quality. The received video frame data then comprises video frame data at that desired video quality. In an embodiment, the available video qualities are determined on the fly by a stream controller, and communicated from the stream controller to each video output device and video source. For example, the maximum available video quality may be altered to take into account changing network conditions.

Although the blocks may be compressed using for example JPEG compression, such a compression algorithm would potentially cause spikes in the data rate over the network for example where a series of detailed blocks is transmitted in succession, as efficient JPEG compression relies on some areas of a compressed image having little detail and thus being highly compressed. A discrete cosine transform compression algorithm with a fixed compressed size per compressed block of data is particularly advantageous in the present system, because it ensures minimal variability in the data rate over the network. Such an algorithm may for example be based on a modification of the discrete cosine transform algorithm that is used in JPEG compression, to remove higher frequencies more harshly and to give more colour depth to less detailed areas. A constant compressed size per block ensures that the network data rate is more easily monitored to ensure that network capacity is not exceeded.

Figure 13 shows a table of example pixel rates of video data of various bit depths at various resolutions with corresponding data rates required to transmit uncompressed video data at each resolution with various subsampling schemes. For example, assuming for simplicity that a given source provides a single multicast stream for each block of a given video frame, a data rate of 0.69 Gbit/s would be required for a video source to provide uncompressed 720p60 video frame data to network switch. Similarly, a data rate of 0.69 Gbit/s would be required for the network switch to provide such 720p60 video frame data to a given video output device. Such data transmissions could thus be handled by a 1 Gb non-blocking network switch. Because the video source provides a multicast stream corresponding to each block of a video frame, the bandwidth of video frame data streamed from a video source will not change, regardless of how many displays the video frame is spread across. Similarly, depending on the number of blocks into which each video stream is broken, the bandwidth of video frame data streamed from the network switch to an video output device may not be significantly higher than the bandwidth required to stream a single video frame to that video output device. The aforementioned 1 Gb non-blocking network switch would thus be usable regardless of how many video sources or video output devices operate on the network, and regardless of how many different videos are simultaneously displayed on the video wall.

Conversely, if a video source could provide video at 4k60 resolution and each video output device could output 1080p60 video to its corresponding display, the system may be configured such that the video source compresses video using a 4:1 compression scheme. A network switch with a 10 Gb connection for the or each video source, and a 1 Gb connection for each video output device, could then be used. From the table in Figure 7 it may be seen that uncompressed 4k60 video, using a 4:4:4 sampling scheme with a bit depth of 8 bits, would require a data rate of 12.4 Gbit/s. If compressed at a 4:1 ratio this would require a quarter of that, i.e. 3.1 Gbit/s into the network switch. Given four video output devices, each video output device would then extract 1080p60 video data at a quarter of 3.1 Gbit/s i.e at around 0.78 Gbit/s.

In an embodiment, a video output device determines a change of the location, on its corresponding display, at which a video frame is displayed. Such a change in location may mean that a greater or lesser extent of that video frame is displayed. For example, a video frame may be moved partially on to the display or partially off the display. As another example, a video frame may be moved such that a previouslyoverlapped portion of another video frame is exposed. Based on the determined change in location, the video output device determines a particular further multicast stream to request. The video output device then transmits a further subscription request to the network switch, the further subscription request identifying the particular further multicast stream. In embodiments, this process is performed on a frame-by-frame basis such that streams are subscribed to in real time based on such changes in locations. The window locations may change faster than the time required for a video output device to subscribe to a new stream. In some embodiments the video output device does not render such a change in window location until video frame data has been received from the new stream. For example, there may be a delay of one frame in rendering the location update. In order for such changes in window location to be promptly rendered, it is preferable for the network switch to be capable of responding promptly to a subscription request, for example within a time corresponding to a single frame.

Figure 14 shows an example procedure 1400 for a video output device to subscribe to streams, based on the mapping approach discussed above in relation to Figure 11. The procedure takes into account changes in locations at which a given video is to be displayed.

At step 1405, the display coordinates are set to (0, 0) which may for example represent the top-left pixel of the display.

At step 1410, the display coordinates are mapped to the window coordinates as outlined above in relation to Figure 11. In other words, this step determines which particular window is to be displayed at that pixel of the display (if any) and, for that particular window, what coordinates of the window are relevant for that pixel of the display.

If no window is to be displayed at the (0, 0) pixel, the flow proceeds to determining whether the end of the frame has been reached at step 1430.

If a window is to be displayed, the procedure determines at step 1415 whether the determined window coordinate corresponds to a stream to which the video output device is subscribed.

If the video output device is already subscribed to that stream, video frame data corresponding to the window coordinates is retrieved, for example from a cache of the video output device which has earlier received video data from the corresponding stream, at step 1420. The corresponding display pixel value is then determined and stored in a display buffer in preparation for outputting to the display. Determining the display pixel value may comprise interpolating between window pixel values as described above in relation to Figure 11.

If the video output device is not already subscribed to that stream, the video output device subscribes to that stream at step 1425. For example, the video output device may maintain a record of available streams and the video frame blocks to which each stream relates. In some embodiments, updates to such a record are communicated to the video output device by a stream controller. For example, a stream controller may transmit such an update in response to receiving an indication of a change in available streams from a particular video source. Video frame data from that stream is thus received and cached. The corresponding display pixel value is then determined and stored in the display buffer.

The procedure then determines at step 1430 if all pixels of the display have been accounted for i.e. whether the end of the display frame has been reached.

If the end of the frame has not been reached, the procedure moves to step 1435 in which display coordinates are updated to the next coordinates of the display, and the flow returns to mapping the new display coordinates to window coordinates at step 1410. In this manner, the procedure can be repeated pixel-by-pixel across the display, determining a mapping of each display pixel to corresponding window coordinates, and subscribing to streams as required. The repeat loop for cycling through the pixels of the display may, for example, correspond to a scan of pixels across a first line of the display, followed by a scan of pixels across a second line of the display and so on. Hence, the procedure can perform a raster-like scan across the pixels of the display.

If the end of the display frame has been reached, the procedure moves to step 1440. At step 1440 it is determined whether the video output device is subscribed to any subscriptions that correspond to video frame regions that will not be displayed. For example, window locations may have changed such that a previously-displayed region is now overlapped by another window. As another example, window locations may have changed such that a previously-displayed region is now displayed on a different display of the video wall. In one embodiment, at step 1420 a flag is set corresponding to each subscription from which data is retrieved. Similarly, at step 1425 a flag is set corresponding to each new subscription. The flags thus identify necessary subscriptions. As such, any remaining unflagged subscriptions correspond to video frame regions that will not be displayed.

If such unnecessary subscriptions are determined, the video output device unsubscribes from these subscriptions at step 1445. In some embodiments, unsubscribing comprises transmitting an unsubscribe request to the network switch. In response to receiving such a request, the network switch stops transmitting that stream to the video output device.

The procedure then moves to step 1450. At step 1450, the buffered display pixel values for the complete frame are output to the display. In doing so, the display buffer is emptied in preparation for the next frame.

The positions of the window(s) on the canvas are then updated at step 1455, preferably within a vertical blank of the canvas frame. The positions of the windows may, for example, be updated based on a time-based movement of the windows across the display and/or a sudden appearance, disappearance or reconfiguration of a window or windows on the video wall.

Finally, the flow proceeds at step 1460 to the next video frame, whereby the display coordinates are reset to (0, 0) at 1405, and the aforementioned steps of the procedure repeated for the next frame and so on.

In an alternative procedure to Figure 14, the display of buffered display pixel values may occur before the end of the frame, for example, on a line-by-line or pixelby-pixel basis.

In a further alternative, the procedure of Figure 14 may be performed without retrieving or displaying data but just determining which streams to subscribe to or unsubscribe from.

Figure 15 shows an ordering of blocks of video frame data for transmission, according to an embodiment. A video frame 1500 is divided into an 8-by-8 grid of blocks. The blocks are numbered sequentially from 0 to 63. Video frame data of each block is transmitted in a separate multicast stream. Data of each multicast stream is transmitted from the video source in series.

In some embodiments, transmissions from each video source to the network switch are synchronised, for example with a display refresh rate. For example, each video source could transmit blocks in series from block 0 to block 63. Streams corresponding to equivalent areas of different videos would thus arrive at the network switch at the same time. For example, video frame data corresponding to the top-left corner of each video, i.e. block 0, would be transmitted from corresponding video sources simultaneously.

With some window arrangements, a given video output device may subscribe to streams corresponding to equivalent areas of multiple videos. For example, an output box may display the top portion of each of many videos. As such, in embodiments wherein transmissions from each video source are synchronised, a large number of streams would simultaneously arrive at the network switch and require transmitting to the output video device. This would require a high peak network bandwidth. In one example, such streams are buffered by the network switch and transmitted spread out in time. This would maintain the same average bandwidth but reduce the required peak bandwidth. However, the network switch may have insufficient memory for buffering such a quantity of video frame data.

In some embodiments, the blocks 0 to 63 of video frame 1500 are reordered for transmission such that video frame data corresponding to adjacent portions of a single video frame 1500 is transmitted nonconsecutively. For example, video frame data corresponding to adjacent portions of a single video frame 1500 may be transmitted in a pseudorandom order. The order may be optimised to avoid overloading the network switch as outlined above.

Figure 15 shows the blocks of video frame 1500 re-ordered in such a manner 1505. Each column of the re-ordering 1505 comprises a cyclic permutation of the corresponding column of video frame 1500. As an illustrative example, the reordering 1505 is such that blocks 0-7 corresponding to the top edge of video frame 1500 are not adjacent to each other. Similarly, blocks corresponding to the left, right and bottom edges of video frame 1500 are not adjacent to each other.

Adjacent blocks of the video frame 1500 are thus transmitted nonconsecutively. Each video source 1500 may re-order video frames based on a different re-ordering, such that for example top-left block 0 from one video source is not transmitted at the same time as top-left block 0 from another video source. This reduces the peak bandwidth required when a given output box subscribes to equivalent blocks, for example the top-left corner, from multiple video sources.

A consequence of such re-ordering is that delays are caused in providing streams from a video source. For example, a video source may retrieve a given frame from a video store, in preparation for re-ordering and transmitting, in a line-by-line or block-by-block fashion. The source would thus retrieve blocks 0-7 first, followed by blocks 8-15, and so on. However, in the re-ordering 1505, block 0 is transmitted first and followed by block 57. The video source would thus need to wait until it had retrieved block 57, which is located on the last line of the re-ordering 1505. This adds up to one frame to the system latency.

MULTIPLE SCALED STREAMS

As explained above, videos or video steams can be thought of as a sequence of individual still images which are often referred to as video frames, and each image or video frame consists of a number of pixels arranged in a grid of horizontal rows and vertical columns.

Figure 16 shows a simplified video frame 1605 consisting of 4 horizontal rows and 4 vertical columns. The video frame therefore has a resolution of 4 x 4 with 16 pixels in total. For a colour video frame, each pixel may have 3 colour component values that each range from 0 to 255 i.e. an 8-bit value. The range of values is often referred to as the colour depth, with 8-bit, 10-bit, and 12-bit colour depths being typical colour-depths used in current video. The component values may represent the colours red (R), green (G), and blue (B). The 4x4 grid of pixels can therefore be divided into a 4 x 4 grid of red pixels, a 4 x 4 grid of green pixels, and a 4 x 4 grid of blue pixels. Alternatively, the component values may represent luminance (Y), bluedifference chroma (Cb), and red-difference chroma (Cr). In this case, the 4 x 4 grid of pixels can be divided into a 4 x 4 grid of Y pixels, a 4 x 4 grid of Cb pixels, and a 4 x 4 grid of Cr pixels.

Because each pixel is represented by component values, the more pixels in an image, the larger the number of values, and the larger the number of bits needed to represent the video frame. Accordingly, the number of bits needed to represent a video frame can be reduced by lowering the resolution of the video frame. The second video frame 1610 in Figure 16 shows a 2 x 2 video frame which is half the resolution of the 4 x 4 video frame 1605, and requires only a quarter of the number of bits to represent it.

The process of lowering the resolution of a video frame and the associated video stream is known as downscaling. For example, the video frame 1605 in Figure 16 is downscaled to a half-resolution video frame 1610. The process of downscaling involves combining component values of pixels in the original video frame to determine the component values of each pixel in the downscaled video frame. In the example shown in Figure 16, the luminance component value for the 4 pixels in the top left portion of video frame 1605 are shown. The corresponding pixel in the downscaled image is shown in the top left corner of video frame 1610, and the luminance component A is calculated by summing the luminance components 65, 40, 30, 45 to give a summed value 180 and then dividing by the number of pixels 4 to give the pixel value 45.

The half resolution video frame 1510 can be further downscaled by a half, resulting in a quarter resolution image which will be just a single pixel value.

Following this principle, a 4K high-definition video stream with a resolution of 3840 (horizontal) x 2160 (vertical) can be downscaled to a half resolution version with a resolution 1920 x 1080 (also known as 1080p HD video) which in turn can be downscaled to a quarter resolution version with a resolution 960 x 540, and an eighth resolution version 480 x 270. The lower resolution versions of the 4K video stream can be sent and viewed on equipment that may not be capable of sending or viewing 4K video. Furthermore, the lower resolution video streams can be used to conserve bandwidth in a video system since the bit-rate for these lower resolution versions is significantly less than the original 4K video stream.

An alternative way of reducing the bit-rate of a video stream is by selectively reducing the resolution of particular pixel components of the video frame. In particular, this process is known as chroma subsampling when the video components consist of luminance (Y), blue-difference chroma (Cb), and red-difference chroma (Cr).

Figure 17 shows a simple 2x2 video frame comprising a 2 x 2 Y pixel grid 1710, a 2 x 2 Cr pixel grid 1720, and a 2 x 2 Cb pixel grid 1730. The pixel values are labeled 1, 2, 3, and 4 for each grid. The overall 2x2 pixel grid 1750 comprises a combination of the values from each of the component grids so that the sets of pixel values consist of (Yl, Crl, Cbl), (Y2, Cr2, Cb2), (Y3, Cr3, Cb3), and (Y4, Cr4, Cb4). When all the components share the same resolution like this, the video is known as a YCrCb 4:4:4 video stream.

Figure 18 shows a second simple 2x2 video frame comprising a 2 x 2 Y pixel grid 1810, a 1 x 2 Cr pixel grid 1820, and a 1 x 2 Cb pixel grid 1830. The pixel values are labeled 1, 2, 3, and 4 for the Y grid and just 1 and 2 for the Cr and Cb grids. The overall 2x2 pixel grid 1850 comprises a combination of the values from each of the component grids so that the sets of pixel values consist of (Yl, Crl, Cbl), (Y2, Crl, Cbl), (Y3, Cr2, Cb2), and (Y4, Cr2, Cb2). When the Cr and Cb components have a half resolution in just the horizontal direction, as in this example, the video is known as a YCrCb 4:2:2 video stream.

Figure 19 shows a third simple 2x2 video frame comprising a 2 x 2 Y pixel grid 1910, a single Cr pixel 1920, and a single Cb pixel 1930. The pixel values are labeled 1, 2, 3, and 4 for the Y grid and just 1 for the Cr and Cb pixels. The overall 2 x 2 pixel grid 1950 comprises a combination of the values from each of the component grids so that the sets of pixel values consist of (Yl, Crl, Cbl), (Y2, Crl, Cbl), (Y3, Crl, Cbl), and (Y4, Crl, Cbl). When the Cr and Cb components have a half resolution in both the horizontal and vertical direction, as in this example, the video is known as a YCrCb 4:2:0 video stream.

When sending video between components of a video system, it is often useful to provide the video at various downscaled resolutions and/or with different chroma subsampling versions. In this way, the sending component can provide video at different quality and bit-rates, and different destination components can receive video at their desired quality without having to downscale themselves.

According to a first approach, an original video stream can be downscaled by a component of a video system into half, quarter, and eighth resolution video streams. Each of the 4 video streams can then be multicast separately by the sending component.

According to an alternative novel approach, the video frame can be sent in multiple parts, including one part which is a true downscaled version of the video frame, and other parts which when processed together with the downscaled part can provide other, gradually higher resolutions of the video frame. By processing all the parts together, it is possible to reproduce the original video frame.

In other words, the novel approach involves sending a video image in multiple parts, whereby the most basic part is of very low resolution, then other parts with gradually higher resolution (and with gradually increasing bandwidth) add more detail until the last part 'fills-in' the last missing parts of the video image. The parts are then sent as separate video streams. The gradually higher resolution parts or streams may not repeat information that has already been sent from the lower-resolution streams. Instead, they may supplement the information from the lower resolution stream or streams in order to provide information needed to build a higher resolution version.

If the streams according to this novel approach are multi-cast on a network, then receiving devices can subscribe to only the streams that they want in order to receive a video of the desired quality. The approach therefore enables receiving devices to selectively choose the relevant streams that correspond to the desired resolution of video for display including, in some cases, the desired chroma subsampling format. By doing so, the system makes efficient use of bandwidth to the receiving device since the combined bit-rate of the lower resolution streams may not exceed the bit-rate of the original video frame.

The bandwidth required for sending the partial video streams according to the novel approach over a single connection is equal to the sum of the bit-rates for each of the partial video streams. Likewise, the bandwidth required for sending the various standalone downscaled video streams according to the first approach over a single connection is equal to the sum of the bit-rates for each of the downscaled video streams.

Ideally, the bandwidth requirement for the partial video streams according to the novel approach will be lower than the bandwidth requirement for sending the various standalone downscaled video streams according to the first approach. Also, the bandwidth requirement for the partial video streams according to the novel approach will ideally not significantly exceed the bandwidth requirement for streaming the full resolution video stream as will be explained in more detail below. However, if no downscaling is provided from a sending component of the system, i.e. only the full resolution video stream is multicast, then a relatively high bandwidth usage on the network connection will result, and the receiving devices may then be required to downscale the received full resolution video stream.

If the video image is broken-up into different colour components, specifically YCrCb, then the Y, Cr and Cb components could be sent separately, with each component being sent as multiple resolution partial video streams. Thus a 4:2:2 YCrCb image could be received from a full 4:4:4 YCrCb image by only subscribing to half horizontal resolution streams of the Cr and Cb components - saving about 20% of the bandwidth normally required for a 4:4:4 image. Similarly, a 4:2:0 YCrCb image could be received from a full 4:4:4 YCrCb image by only subscribing to half resolution (i.e. half horizontal and half vertical) streams of the Cr and Cb components - saving about 40% of the bandwidth normally required for a 4:4:4 image.

For example, a 4k resolution, 60 frame per second, 4:4:4 YCrCb 24-bit video, when placed onto a network, will require around 12.4Gbits/s of bandwidth. However, a receiving device with an SDI output might only support a 4:2:2YCrCb format. Hence, the receiving device might subscribe to the full Y resolution streams but just the lower resolution Cr and Cb streams and thereby reduce the overall bit-rate to around 9.8Gbits/s (and may permit the video to be sent on a single 10Gb ethernet connection instead of two 10Gb ethemet connections).

In another example, the receiving device may include an HDMI output that only supports 4k resolution, 60 frame per second, 4:2:0 YCrCb 24-bit video. In this case, instead of receiving the whole 4:4:4 resolution video and processing it down to the HDMI output, the receiving device could subscribe to the full resolution Y streams and just the half resolution Cr and Cb streams, thereby reducing the bandwidth requirement from 12.4Gbits/s to around 7.5Gbits/s.

Where a single source is multicasting the aforementioned streams of data, any outputs can effectively avoid the need to down-scale the image by only subscribing to the necessary streams - thus avoiding some image processing and excess bandwidth in the process. This is especially true of a multi-viewer, where a single video image on a display is replaced by multiple smaller resolution video images arranged side-byside on the display. For example, a 2x2 arrangement of 4k60 images, each having been down-scaled to half resolution in the horizontal and vertical directions in order to fit into a single 4k60 output, would normally require the bandwidth for all four original 4k60 video frames to be received. By just subscribing to the lower half resolution streams of all four videos, the resulting bandwidth and processing is greatly reduced.

Another benefit is where a temporary network failure results in the loss of data packets. For a normal image transmission system, this might result in a section of the image being missing or badly corrupted. When sent as partial streams, the bad packets can be ignored and the image partly restored from the good packets - giving only a reduction in image quality on the affected area.

A further benefit is when a cross-fade is required on an output - where two live video sources are faded between, so that at the mid-point both are visible as a 'dissolve' effect. Normally this would require twice the bandwidth of a single video feed to be sent and for the output to product the cross-fade. But by altering the stream subscription during the fade (reducing the outgoing image quality/resolution whilst the incoming image quality/resolution is increased) the bandwidth requirement can be kept to around the same level as that of a single video feed. Generally a fade or dissolve is done quite quickly to prevent this being noticed - and since the outgoing image's quality reduction occurs as it becomes less visible this should prevent the quality reduction from being a perceptable problem for a viewer.

The novel approach for encoding and sending partial video streams will now be described with reference to the example shown in Figures 20 to 23.

Referring first to Figure 20, there is shown an 8 x 8 video frame 2010 with 8 rows and 8 columns containing in total 64 pixel values. This grid of pixels may represent a particular colour component of the video such as a R, G, B, Y, Cr, and/or Cb component.

Each pixel value may range from 0 to 255 which corresponds to an 8-bit pixel depth. Therefore, 512 bits (8 x 64) would be required to represent this colour component of the video frame. For a video frame containing three such colour components, the total number of bits for the video frame would be 1536 bits (3 x 512).

The video frame 2010 can be encoded in parts represented by the downscaled video frame 2020, and the 6 partial video frames 2110, 2120, 2210, 2220, 2310, and 2320.

The downscaled video frame 2020 (known as H3V3) is an accumulated total of all the pixel values from the original video frame 2010, and so effectively gives a ‘pixel’ value as if the pixels were down-scaled to l/8th horizontally and vertically (although with greater colour depth). Since the original pixels are 8-bit values, then this accumulated value would need to be 14-bits in size, since it might need to store a minimum value of 0, and a maximum value of 64*255=16,320 (note: 2^Λ14=16,384).

The first partial video frame 2110 (known as V2) is shown in Figure 21, and comprises a single pixel value of 1,617 calculated by the accumulated total of all the pixels in the upper 4 rows of the 8x8 block of pixels 2010, and needs to be a 13-bit value.

In all the partial video frames, the pixel value or values stored and sent are just the values shown in grey highlighting in the Figures. The other values are values that would make up the video frame at a particular resolution (or subsampling format), and can be derived from a combination of the downscaled video frame 2020 and other partial video frames.

Taking the video frame 2110, the value of the lower pixel 1108 can be calculated by simply subtracting the upper pixel value from the pixel value 2725 from the downscaled video frame 2020. i.e. 2725-1617=1108.

The next partial video frame 2120 (known as Ή2) has two values stored and sent 616 and 484 (highlighted in grey), each being 12-bits in size, and each representing the total of values for a 4x4 block of pixels of the original video frame. The 2 other 4x4 blocks of pixels can have their total worked out using simple subtraction e.g. 616 was sent for the upper-left 4x4 block, and hence the upper-right block must therefore be 1,617-616=1,001. H2 effectively represents an image than has been down-scaled to l/4th in the horizontal and vertical directions.

The remaining partial video frames 2210 (VI), 2220 (Hl), 2310 (V0) and finally 2320 (HO) are all sent in similar ways, with half the values being sent and the other half derivable from lower-resolution streams. The bit-depth required also reduces to 11-bits for VI, 10 bits for Hl, 9 bits for V0 and finally 8-bits for the HO stream.

The pixel values sent in H2, VI, Hl, V0, and HO are all arranged in a checkerboard pattern. Alternative patterns may be used without deviating from the general concept. The use of a checkerboard pattern is preferred as it helps improve interpolation of data should lower-resolution streams become corrupted or are missing.

Since the lower-resolution accumulated values in the streams 2020, 2110, 2120, 2210, 2220, 2310, and 2320 can be quite large compared to the values of the original pixels in the video frame 2010, they require sending using more bits than the original 8 bit single-pixel values.

The additional number of bits needed to enable the transmission of lower resolution information can be referred to as the overhead.

In the example of Figures 20 to 23, the original video frame is represented by 512 bits (64 x 8).

The number of bits required for each of the 7 streams 2020, 2110, 2120, 2210, 2220, 2310, and 2320 are as follows:

H3V3 520 requires 1x14 = 14 bits.

V2 610 requires 1x13 = 13 bits.

H2 620 requires 2x12 = 24 bits.

VI 710 requires 4x11 = 44 bits.

Hl 720 requires 8x10=80 bits.

V0 810 requires 16x9 = 144 bits.

HO 820 requires 32x8 = 256 bits.

The total bits required for all 7 streams is therefore 575 bits (14+13+24+44+80+144+256). This represents an overheard of 63 bits compared to the original video frame, which is around 12% in addition to the original 512 bits and represents a bandwidth efficiency of 89%.

For higher original colour bit-depths, the resulting bandwidth efficiency improves, as detailed in the table in Figure 24 which shows the bandwidth efficiencies for 8-bit, 10-bit, 12-bit, and 16-bit original video frame colour depths.

These values demonstrate that the ability to encode and send multiple resolution streams does not increase bandwidth significantly over sending just the original video frame.

If the first approach is used for sending the original video frame together with half, quarter, and eighth standalone downscaled streams, the total number of bits required would be much greater. For example, the 8 x 8 video frame 2010 in Figure would be represented by 512 bits, while the half resolution version would be 128 bits (16 x 8), the quarter resolution 32 bits (4 x 8), and the eighth resolution 8 bits (1 x 8). This gives a total number of 680 bits (512+128+32+8) which corresponds to an overhead of 168 bits. This overhead is significantly greater than the overhead provided by the novel approach of 63 bits. At the same time, the downscaled streams would not provide the half horizontal resolution video frames that are beneficial for chroma subsampling format transmissions.

The following are examples of how a video receiving device can subscribe to the video streams encoded using the novel approach.

For a subscribing receiver to receive the chroma sub sampling format 4:2:0 equivalent of an 8-bit video image at full resolution, the Y component would need to receive all of the 7 Y component H3V3, V2, H2, VI, Hl, VO, HO streams. However, the Cr and Cb components do not require the Cr/Cb component HO or VO streams, just the Cr/Cb component H3V3, V2, H2, VI, Hl streams. Hence of the original 3x512 = 1536 bits of information, only 575 + 175 + 175 = 925 bits need to be received. This is 60.2% of the original bandwidth.

To receive a chroma subsampling format 4:2:2 equivalent of an 8-bit video image at full resolution, the Y component would again need to receive all of the 7 Y component H3V3, V2, H2, VI, Hl, VO, HO streams. However, the Cr and Cb components do not require the Cr/Cb component HO stream, just the Cr/Cb component H3V3, V2, H2, VI, Hl, V0 streams. Hence of the original 3x512 = 1536 is reduced to 575 + 319 + 319 = 1213 bits. This is 79.0% of the original bandwidth.

For a video system to display, on a single monitor, a small preview window of each video source would normally require a special (separate) video stream to be sent from each source (since receiving a full-resolution for all sources, and downscaling them, would be very difficult). But with the suggested mechanism that video ‘preview’ window is already available as the H3V3 stream, which is a single 14-bit value representing a l/8th scaled block of 8x8 pixels, being 2.7% of the original image’s bandwidth. Hence, it would be possible to display around 40 times the number of images to be sent instead of sending them in their original resolution.

The lower-resolution streams to do not need to stop at H3V3 (the lowest resolution above). Further lower resolutions could be offered when starting from a larger original video frame block such as 16 x 16.

An alternative to the above method may involve sending all the pixel values for the H2 video frame 2120 and omitting the video frames H3V3 and V2. However, this would only offer very slightly better efficiency (reducing the 575 bits down to 572, for 512 bits of data).

The method described here is most suited to loss-less transmission of video data, although it may be possible to add further compression techniques to the individual streams.

Methods of the present disclosure may be implemented by way of a nontransitory computer-readable storage medium comprising a set of computer-readable instructions stored thereon which, when executed by at least one processor, cause the at least one processor to perform a method according to the present disclosure. The computer readable instructions may be retrieved from a machine-readable media, e.g. any media that can contain, store, or maintain programs and data for use by or in connection with an instruction execution system. In this case, machine-readable media can comprise any one of many physical media such as, for example, electronic, magnetic, optical, electromagnetic, or semiconductor media. More specific examples of suitable machine-readable media include, but are not limited to, a hard drive, a random access memory (RAM), a read only memory (ROM), an erasable programmable read-only memory, or a portable disc.

The above embodiments are to be understood as illustrative examples of the invention. Further embodiments of the invention are envisaged. For example, the display devices may be video projectors used for projection mapping. In such embodiments, it may be desirable to mask certain areas of a given video frame such that no video data is displayed in those areas. Such a mapping may be defined by any of the video source, video output device and/or other networked components such as a projection mapping controller. In examples wherein the mapping is defined by the video output device, a given video output device may subscribe to video frame data based on the mapping such that unneeded portions are not received.

It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention, which is defined in the accompanying claims.

Claims (21)

1. A method, in a video output device, for acquiring video data for output to a display, the method comprising:

subscribing to a multicast stream from a plurality of multicast streams, wherein each multicast stream of the plurality of multicast streams is streamed from a video source and comprises video frame data corresponding to a portion of a video frame; and receiving video frame data from the subscribed multicast stream for display on the display.

2. A method according to claim 1, the method further comprising: subscribing to one or more further multicast streams of the plurality of multicast streams, and receiving video frame data from the further subscribed multicast streams for display on the display.

3. A method according to claim 1 or claim 2, wherein subscribing to a multicast stream comprises transmitting a subscription request to a network switch, the network switch being coupled to the video output device and to the video source, wherein the subscription request identifies the multicast stream.

4. A method according to claim 3, wherein:

each of the plurality of multicast streams comprises video frame data at a given video quality;

the subscription request comprises an indication of a desired video quality; and the received video frame data comprises video frame data at the desired video quality.

5. A method according to claim 4, wherein the desired video quality comprises one or more of a desired video frame resolution, a desired degree of video compression, a desired video frame bit depth and a desired chroma subsampling scheme.

6. A method according to any of claims 3 to 5, the method further comprising: determining the or each multicast stream to request based on a position of the video frame on the display.

7. A method according to claim 6, wherein:

determining the position of the video frame on the display comprises determining a mapping of at least one pixel of the display to a location in the video frame.

8. A method according to any of claims 3 to 7, the method comprising: determining a change of the position of the video frame on the display;

based on the determined change in position, determining a new multicast stream to subscribe to; and transmitting a further subscription request to the network switch, the further subscription request identifying the new multicast stream.

9. A method according to any preceding claim, wherein the or each portion of the video frame has predefined spatial dimensions.

10. A method according to any preceding claim, the method comprising outputting display data, based on the received video frame data, to the display.

11. A method according to any preceding claim, wherein the method comprises: subscribing to a second multicast stream from a second plurality of multicast streams, wherein each multicast stream of the second plurality of multicast streams is streamed from a second video source and comprises video frame data corresponding to a portion of a second video frame; and receiving video frame data from the subscribed second multicast stream for display on the display.

12. A method according to any preceding claim, wherein the display is a display of a video wall.

13. A non-transitory computer-readable storage medium comprising a set of computer-readable instructions stored thereon which, when executed by at least one processor, cause the at least one processor to perform a method according to any preceding claim.

14. A video output device comprising: a network interface;

a processor coupled to the network interface and configured to:

subscribe, via the network interface, to a multicast stream from a plurality of multicast streams, wherein each multicast stream of the plurality of multicast streams is streamed from a video source and comprises video frame data corresponding to a portion of a video frame; and receive, via the network interface, video frame data from the subscribed multicast stream.

15. A video system comprising: a plurality of displays;

a network switch;

at least one video source coupled to the network switch and configured to provide a plurality of multicast streams via the network switch, each multicast stream of the plurality of multicast streams comprising video frame data corresponding to a portion of a corresponding video frame;

a plurality of video output devices coupled to the network switch, each video output device being coupled to a corresponding display and being configured to:

transmit a subscription request to the network switch, the subscription request being for at least one of the plurality of multicast streams; the network switch being configured to, in response to receiving a request from one of the plurality of video output devices, transmit each requested multicast stream to the corresponding video output device.

16. A video system according to claim 15, wherein:

each video source of the least one video sources provides its corresponding plurality of multicast streams such that, for a single video frame, a video source transmits video frame data of each multicast stream in series to the network switch; and the series is such that video frame data corresponding to adjacent portions of a single video frame is transmitted nonconsecutively.

17. A video system according to claim 16, wherein the series is such that multicast streams corresponding to adjacent portions of a single video frame are transmitted in a pseudorandom order.

18. A video system according to any of claims 15 to 17, wherein the plurality of displays are displays of a video wall.

19. A method of video frame transmission in a video system, the method comprising:

transmitting, from a video source and to a network switch, a plurality of multicast streams, each multicast stream of the plurality of multicast streams comprising video frame data corresponding to a portion of a corresponding video frame;

transmitting, from a video output device and to the network switch, a subscription request for at least one of the plurality of multicast streams; and in response to receiving a said subscription request, transmitting each requested multicast stream from the network switch to the video output device.

20. A method, in a video source, for providing streamed video data, the method comprising:

retrieving video frame data from a video input; and 5 providing to a network a plurality of multicast streams, wherein each multicast stream of the plurality of multicast streams comprises video frame data corresponding to a portion of a video frame.

21. A video source comprising:

10 a video input;

a network interface; and a processor, the processor being configured to:

retrieve video frame data from the video input; and provide, via the network interface, a plurality of multicast streams,

15 wherein each multicast stream of the plurality of multicast streams comprises video frame data corresponding to a portion of a video frame.

Go?

Intellectual

Property

Office

Application No: GB1701860.7 Examiner: Matthew Males