WO2003084235A1

WO2003084235A1 - Video pre-processing

Info

Publication number: WO2003084235A1
Application number: PCT/GB2003/001323
Authority: WO
Inventors: Othon Kamariotis
Original assignee: British Telecommunications Public Limited Company
Priority date: 2002-03-28
Filing date: 2003-03-27
Publication date: 2003-10-09

Abstract

A sequence of complete video frames (V1) to (Vn) are compared with a corresponding sequence of mask frames (M1) to (Mn) according to an automatic background extraction process (55) to from a single background frame (60). Each mask frame (M1) to (Mn) includes an opaque part (51) and a transparent part (52). Each frame comprises MxN pixels arranged in (M) columns 1,2,3,…,M and (N) rows 1,2,3,… ,N. Where a pixel takes the value associated with transparency, it indicates that the corresponding pixel in the corresponding complete video frame is a visible background pixel. Similarly, where a pixel in a mask frame takes the corresponding to an opaque pixel, it indicates that the corresponding pixel in the corresponding complete video frame is a pixel associated with the foreground 'object'. All of the mask frames (M1) to (Mn) are considered in turn to identify all of the background pixels visible in at least one complete video frame of the sequence. As soon as a background pixel is visible for the first time, the value of the background pixel (ie brightness, colour, etc) at that time is stored in a reference background frame until all possible pixels within the reference background frame have been set to the corresponding values of background pixels in the complete video frames at which point the reference background frame is outputted as the single background frame (60).

Description

VIDEO PRE-PROCESSING

The present invention relates to a method and apparatus for pre-processing video frames, and in particular to a method and apparatus for processing a sequence of video frames to form a single background frame which can be used with each of a plurality foreground object frames (extracted from the original sequence of complete video frames) to recreate each of the original complete video frames respectively.

A video sequence comprises a sequential series of complete video frames intended to be displayed successively at a predetermined rate. Each frame comprises a matrix of pixels, each of which has, in the case of black and white video sequences, an associated value indicating the brightness of the pixel, or, in the case of colour video sequences, a triplet of values which together indicate both the brightness and the colour of each pixel.

MPEG-4 is a standard for transmitting such a video sequence from a transmitting unit to a receiving unit over a packet-switched data network, especially when employing the Internet Protocol. The MPEG-4 standard is distinguished from earlier versions of the standard in that it permits a video "object" to be transmitted separately from a background against which the object moves (in the foreground). Where the background is static (ie is substantially the same in all of the frames in a particular sequence), the amount of bandwidth required to transmit the sequence can be reduced by transmitting the background information only once for the complete sequence of frames in which the background is static and transmitting only the object information for each frame on a frame-by-frame basis.

Additionally, where there is a background which is larger than can be seen in a single frame, but which the camera pans around, the amount of bandwidth can still be reduced by transmitting the complete background information only once for the complete sequence of frames wherein different frames show different parts of the complete background. In some circumstances, the background information need not be transmitted at all and the receiving unit generates (or selects) an arbitrary (or pre-stored) background for combination with the foreground objects prior to displaying a set of complete video frames. Additionally, in some circumstances, the background information is separately available to the transmitting unit and the transmitting unit does not need to perform any pre-processing on the original video sequence to be transmitted in order to extract the background information.

However, in many circumstances no information is available to the transmitting unit apart from the actual original frames of the video sequence itself. Methods have been developed for processing sequences of video frames in such circumstances. In particular, a paper called "Approaches to static background identification and removal" by Shelley and Seed (Sheffield Univ., UK), IEE Colloquium on 'Image Processing for Transport Applications' (Digest No. 1 993/236), 9 Dec. 1 993, IEE pp 6/1 -4; 4631073 - Conference Paper, describes a number of such methods in which the rate of change of the values of a pixel between successive frames is used to try to determine which pixels (and which values associated with the pixels) represent background pixels and which represent foreground (object) pixels. As soon as a pixel is identified as a background pixel, the corresponding values associated with that pixel (for the particular frame used in identifying the pixel as a background pixel) are set as the values for the correspondingly positioned pixel in the single background frame being created. Pixel based routine are distinguished by the independence of all the pixels. Non-pixel based techniques employ a global updating control derived from some measurable image attribute.

However, the present inventor has realized that there are some important cases where some information about the object moving in the foreground is already available to the transmitting unit, (for example in the form of a binary mask specifying, in respect of each frame in the sequence, which pixels are background pixels and which are object pixels in that particular frame), and that methods such as those described above are inefficient ways to produce a single background frame in such cases. According to a first aspect of the present invention, there is provided a method of detecting camera panning occurring from one video frame to another, the method comprising the steps of comparing an input pair of pre-processing complete video frames with an input corresponding pair of comparison frames each of which indicates which of the pixels, in the corresponding input pre-processing complete video frame, represent object pixels, to thereby determine which of the pixels in each of the complete video frames represent background pixels and performing motion estimation in respect of the background pixels only, ignoring the object pixels.

According to a second aspect of the present invention, there is provided a method of generating a single background frame for combining with a series of object frames to create a series of post-processing complete video frames each of which consists of the respective object frame superimposed onto the single background frame or a part of the single background frame, the method comprising comparing an input series of pre-processing complete video frames with an input corresponding series of comparison frames each of which indicates which of the pixels, in the corresponding input pre-processing complete video frame, represent object pixels, to thereby derive said single background video frame.

Each of the comparison frames may comprise a binary mask having a number of pixels each of which corresponds to a respective one of the pixels in each input preprocessing complete video frame and each of which is allocated a first or a second value (eg a zero or a one) in dependence upon whether or not the corresponding pixel in the corresponding input pre-processing complete video frame is a foreground object pixel or a background pixel. Instead of allocating a first or second value to each pixel, the pixels may be grouped into blocks of pixels, and each block may be assigned a first or second value in dependence upon whether or not the corresponding block of pixels includes one or more object pixels or not. Each block may comprise two hundred and fifty six pixels arranged in a sixteen by sixteen square arrangement, which blocks are hereinafter referred to as macroblocks.

The comparison step may comprise the sub-steps of: identifying a first pair of one of the input series of pre-processing complete video frames and the corresponding comparison frame; generating a reference background video frame by providing a plurality of pixels each of which corresponds to a respective one of the pixels in each of the input pre-processing complete video frames, setting each pixel corresponding to a pixel not indicated by the comparison frame of the first pair as being an object pixel to the value (for black-and-white) or triplet of values (for colour) associated with the corresponding pixel in the input pre-processing complete video frame of the first pair, and marking each such pixel as set, whilst leaving each pixel corresponding to a pixel which is indicated by the comparison frame of the first pair as being an object pixel to the default value or triplet of values and marking such pixels as unset; identifying a next pair of a complete pre-processing video frame and its corresponding comparison frame to form a current pair; identifying unset pixels in the reference background video frame which correspond to pixels indicated by the comparison frame of the current pair as not corresponding to object pixels, and setting any thus identified pixels in the reference background frame to the value or values associated with the corresponding pixels in the complete pre-processing video frame of the current pair; and repeating the last two preceding steps until all of the input complete pre- processing video frames and comparison frames have been processed.

According to a third aspect of the present invention, there is provided apparatus for generating a single background frame for combining with a series of object frames to create a series of post-processing complete video frames each of which consists of the respective object frame superimposed onto the single background frame, the apparatus comprising means for comparing an input series of pre-processing complete video frames with an input corresponding series of comparison frames each of which indicates which of the pixels, in the corresponding input pre-processing complete video frame, represent object pixels, to thereby derive said single background video frame. In order that the present invention may be better understood, an embodiment thereof will now be described in greater detail with reference to the accompanying drawings in which:

Figure 1 is a block diagram of a system incorporating an embodiment of the present invention;

Figure 2 is a flow chart illustrating the steps performed in accordance with a method according to the present invention;

Figure 3 is a flow chart illustrating the steps performed in accordance with a subroutine forming one of the steps illustrated in Figure 2; Figure 4 is a schematic diagram illustrating the effects of the steps illustrated in Figure 3; and

Figure 5 is a schematic diagram showing two of the video frames and two of the mask frames illustrated in Figure 4 in more detail.

Figure 1 is a schematic illustration of a system comprising a mobile device 10 connected via an air interface 1 5 to a base station 20 and thence, via a computer network 30,40 to the Internet 50. In the present embodiment, the mobile device 10, which includes a screen 10, keypad 1 2 and aerial 13, is operable to receive and display in real time a video originating from a computer (not shown) connected to the Internet 50, such as, for example, a video telephone call or a live streamed video broadcast.

In the present embodiment, the video broadcast, which originates from the Internet 50, is routed firstly to a video pre-processing server computer 40 which is connected to the Internet 50 via a Local Area Network (LAN) 30. The video pre-processing server computer 40 pre-processes the video broadcast to be transmitted to the mobile device 10 to account for the fact that the air interface 1 5 over which the video broadcast must travel to reach the mobile device 10 has only a limited amount of bandwidth (eg for a GPRS over GSM connection an average bandwidth of approximately 20kbps would be typical, whilst an average bandwidth of approximately 10kbps would be typical for a normal GSM connection (without GPRS)). In the present embodiment, the video broadcast is transmitted over the Internet 50 and across the air interface 1 5 using the MPEG-4 standard (see ISO/IEC 14496-2:2001 (E), 1 rue de Varembe, 1 21 1 Geneva 20 Switzerland, tel. + 41 22 749 01 1 1 , fax + 41 22 734 1079, internet: [email protected]. http://www.m4if.org/).

The MPEG-4 standard permits background frames to be transmitted separately from foreground frames. The video pre-processing server computer 40 in the present embodiment is used in cases where the background in the video broadcast is substantially static. The server computer 40 extracts a single background frame 60 (see Figure 4) which, together with a foreground frame in respect of each video frame contained in the original video broadcast, can be used to reconstruct, to a certain extent at least, the original video broadcast. This reconstruction is done in the present embodiment by the mobile device 10 which includes a video decoder compliant with the MPEG-4 standard. The present inventor has realised that in cases where the background is substantially static, very little notice is taken of the background by a user who is viewing a video transmission and thus there is very little perception of a lack of quality of the video transmission in cases where the background is kept exactly static because only a single background frame is employed.

The server computer 40 of the present embodiment is also employed in cases where a mask frame is available in respect of each frame of the original video broadcast. There are a number of circumstances in which such masks will be available. For example, the mask frames may be generated automatically from the original video frames using an alternative program running in the server computer 40. An example of a method which could be employed by the server computer 40 to generate the mask frames in this way is described in the present Applicant's co-pending European patent application No (IPD ref A30150). Alternative algorithms could however be employed, or the mask frames may have been received by the server computer together with the complete video frames as part of the received video broadcast.

The video pre-processing server computer 40 requires a sequence of complete video frames together with a corresponding sequence of respective mask frames. Such a sequence can be derived from the video broadcast by dividing the video broadcast up into a series of sequences each corresponding to a period of video transmission in the order of a few seconds in duration.

The steps performed by the server computer 40 are now described with reference to Figure 2. Upon commencement of the method, flow passes to step S5 in which the sequence of complete video frame and corresponding mask frames are received by the server computer 40.

Upon completion of step S5, flow passes to step S10. Step S10 is a subroutine, the steps of which are described below with reference to Figure 3, which processes the complete video frames and corresponding mask frames received in step S5 to produce a single background frame.

Upon completion of subroutine S10, flow passes to step S15 in which the single background frame extracted in subroutine S10 is transmitted from the server computer 40 to the mobile device 10. This transmission is done in the present embodiment via the LAN 30, base station 20 and air interface 1 5.

Upon completion of step S1 5, flow passes to step S20 in which a sequence of foreground video frames are transmitted from the server computer 40 to the mobile device 10 (also via the LAN 30, base station 20 and air interface 1 5). By transmitting only information about the pixels representing the foreground object, considerable bandwidth is spared. The extra bandwidth generated in this way can then be used to enhance the quality of the video information transmitted in respect of the foreground object thus improving the perceived quality of the received video transmission at the mobile device as a whole.

The detailed steps performed by the subroutine S10 are now described with reference to Figures 3, 4 and 5.

Figure 4 schematically illustrates the overall operation of the subroutine. A sequence of complete video frames V1 to Vn are compared with a corresponding sequence of mask frames M1 to Mn according to an automatic background extraction process 55 (which is the subroutine S10 whose steps are illustrated in Figure 3) to from a single background frame 60. It can be seen from Figure 4 that each mask frame M1 to Mn includes an opaque part 51 and a transparent part 52. Referring now to Figure 5, it can be seen that each frame comprises MxN pixels arranged in M columns 1 ,2, 3,...,M and N rows 1 ,2,3,...,N. Each frame can therefore be represented as a two dimensional array comprising MxN members. In the case of each video frame, each member of the array comprises a multi-value number or triplet of multi-value numbers indicating the brightness or brightness, chroma and hue (or equivalent values such as red, green and blue values). In the case of each mask frame, each member of the array is a binary number which can take either 0 to represent transparent pixels or 1 to represent opaque pixels. Where a pixel takes the value associated with transparency, it indicates that the corresponding pixel in the corresponding complete video frame is a pixel associated with the background. Similarly, where a pixel in a mask frame takes the value 1 corresponding to an opaque pixel, it indicates that the corresponding pixel in the corresponding complete video frame is a pixel associated with the foreground "object" (see the MPEG-4 standard).

Referring now to Figure 3, upon commencement of the sub-routine S10, flow passes to step S100 in which a reference background frame is set by masking the first complete video frame V1 with the first mask frame M1 . This essentially involves not setting any of the pixels in the reference background frame which correspond to opaque pixels in the mask frame M1 , whilst setting all of the pixels in the reference background frame which correspond to transparent pixels in the mask frame M1 to the same values as the corresponding pixels in the first complete video frame V1 .

Upon completion of step S100, flow passes to step S1 10 in which a reference mask frame which is exactly identical to the first mask frame M1 is created.

Upon completion of step S1 10, flow passes to step S120 in which the next video frame and mask frame are selected to form a current pair. On the first occurrence of this step (ie when step S1 20 has been reached directly from step S1 10 rather than step S1 70 for which see below) the next video frame is video frame V2 and the next mask frame is M2 and these are selected as the current pair.

Upon completion of step S120, flow passes to step S1 22 in which an attempt is made to detect a scene change. This is done in the present embodiment by taking the Sum of Absolute Difference (SAD) between the new pair and the reference pair, disregarding any pixels which correspond to either object pixels in the current pair or not yet filled in pixels in the reference background frame. This therefore contains only background pixels common to both pairs (current and reference). If this is zero or very close to zero (ie below some predefined low threshold), it is assumed that there is no scene change and flow proceeds to step S130. Otherwise, flow passes to step S124.

In step S1 24, an attempt is made to detect panning. This is done by attempting to determine if the background is largely unchanged except for having been translated by a certain amount. In order to do that, a well known technique used in standards such as MPEG-4 and H263 and referred to as motion estimation is employed. In the present embodiment, the motion estimation is carried out in the following manner: i) select sample box - first an attempt is made to find a square area of pixels which contains (set values of) background pixels in both the reference background frame and the current video frame and which has a border which contains set background pixel values in the reference background frame. Initially an attempt is made to find such a square of size (1 6) by (1 6) with a border of 16 pixels width (to permit the sample area of 1 6 by 1 6 pixels to be tried in positions up to 1 6 pixels translated in any direction away from the original position. If nor such square can be found, the search is carried out with progressively smaller sample areas and borders until one is found. ii) once a sample box has been found, a SAD is calculated for all possible translated positions of the pixels in the current video frame superimposed onto the reference background frame. The position giving the smallest SAD is selected, or, if there is more than one minimum SAD value, all such positions are selected. iii) the next step is to move the entire reference frame by the corresponding selected amounts and perform a SAD on all (set values of) background pixels in both the reference background frame and the current video frame (ignoring pixels where there is now no overlap between the reference and the current frames). If any of these result in a SAD below a minimum threshold value, then a match is considered to have been made such that it is determined that panning has been detected and the translation corresponding to the lowest calculated SAD is selected as the result of the panning and flow passes to step S 1 26, other wise flow passes to step S 1 28.

In step S1 26, in the present embodiment, the reference background frame is firstly output as a background frame for all of the frames processed up to this point, and a new reference frame for the post panning video frames is generated by moving the old reference frame in the selected direction (by the selected amount) and "filling as many gaps" as is possible from the background pixels in the current video frame. Upon completion of step S1 26, flow passes back to step S1 20.

In step S1 28, it is assumed that there has been a change of scene such that the old reference background frame will not be valid for video frames post the scene change. Therefore, the reference background frame is firstly output as a background frame for all of the frames processed up to this point (ie before the change of scene). Then we designate the current pair of video frame and mask frame (post the change of scene, as the new first video and mask frames and re-start the process by passing flow back to step S100.

If at step S1 22 no scene change is detected, flow passes to step S1 30 in which the mask frame in the current pair is compared with the reference mask frame to seek to detect any pixels which are transparent in the current mask frame (ie with value 0 in the present embodiment indicating that the corresponding pixel in the current video frame represents a background pixel) but which are set as opaque (ie with a value of 1 in the present embodiment) in the reference mask frame.

Upon completion of step S130, flow passes to step S 140 in which it is determined whether any pixels were detected which are transparent in the current mask frame but opaque in the reference mask frame. Such pixels represent newly uncovered background pixels which were previously obscured by the foreground object image, and which have therefore not yet been set to their appropriate background values in the reference background frame. If any such pixels are detected, flow passes to step S1 50.

In step S1 50, the pixels in the reference background frame which correspond to the newly uncovered pixels detected in step S130 are set to the values of the corresponding pixels in the current video frame.

Upon completion of step S1 50, flow passes to step S1 60 in which the pixels in the reference mask frame which correspond to the newly uncovered pixels identified in step S130 are set to the transparent value, to indicate that the corresponding pixels in the reference background frame have now been set and do not therefore need to be set again.

Upon completion of step S1 60, flow passes to step S170. Additionally, if it is determined in step S140 that no newly uncovered pixels were detected in step S130, then flow passes directly from step S140 to step S170 missing out steps S1 50 and S1 60.

In step S170 it is determined whether there are both more video frames remaining in the sequence which could be processed to attempt to uncover more background pixels and one or more pixels in the reference mask frame which have not yet been set to transparent, indicating that there are more background pixels to uncover.

Provided that both of these conditions are met, flow returns to step S120 and a new pair of video frame and mask frame are selected for comparison in step s130, etc.

If one of the above conditions is not met, flow passes to step S180 in which the reference background frame is set as the single background frame 60 which is the final output from the subroutine S10 as a whole. Upon completion of step S180, the subroutine S10 comes to an end and flow passes to step S1 5. although the embodiment has been described with special reference to the MPEG-4 standard, the present invention is, of course, suited for use with any video transmission standard which permits video information about foreground objects to be transmitted separately from video information about the background of a particular sequence of video frames.

Although the embodiment described above referred to a particular networked arrangement of devices, the present invention is of course applicable to any arrangement in which a transmitting unit transmits a sequence of video frames to a receiving unit using a transmission standard which permits video information about foreground objects to be transmitted separately from video information about the background of a particular sequence of video frames.

As an alternative to the described method of generating old and new reference background frames when there is a camera panning detected, the background frame could be extended and additional information transmitted to indicate which part of the larger background frame should be used for each of the transmitted video frames.

Claims

1 . A method of detecting camera panning occurring from one video frame to another, the method comprising the steps of comparing an input pair of pre- processing complete video frames with an input corresponding pair of comparison frames each of which indicates which of the pixels, in the corresponding input preprocessing complete video frame, represent object pixels, to thereby determine which of the pixels in each of the complete video frames represent background pixels and performing motion estimation in respect of the background pixels only, ignoring the object pixels.

2. A method of generating a single background frame for combining with a series of foreground object frames to create a series of post-processing complete video frames each of which consists of the respective foreground object frame superimposed onto the single background frame, the method comprising comparing an input series of pre-processing complete video frames with an input corresponding series of comparison frames each of which indicates which of the pixels, in the corresponding input pre-processing complete video frame, represent object pixels, to thereby derive. said single background video frame.

3. A method according to claim 2 wherein each of the comparison frames comprises a binary mask having a number of pixels each of which corresponds to a respective one of the pixels in each input pre-processing complete video frame and each of which is allocated a first or a second value in dependence upon whether or not the corresponding pixel in the corresponding input pre-processing complete video frame is a foreground object pixel or a background pixel.

4. A method according to any one of claims 2 or 3 wherein the comparing step comprises the substeps of: identifying a first pair of one of the input series of pre-processing complete video frames and the corresponding comparison frame; generating a reference background video frame by providing a plurality of pixels each of which corresponds to a respective one of the pixels in each of the input pre-processing complete video frames, setting each pixel corresponding to a pixel not indicated by the comparison frame of the first pair as being an object pixel to the value (for black-and-white) or triplet of values (for colour) associated with the corresponding pixel in the input pre-processing complete video frame of the first pair and marking each such pixel as set, whilst leaving each pixel corresponding to a pixel which is indicated by the comparison frame of the first pair as being an object pixel to the default value or triplet of values and marking such pixels as unset; identifying a next pair of a complete pre-processing video frame and its corresponding comparison frame to form a current pair; identifying unset pixels in the reference background video frame which correspond to pixels indicated by the comparison frame of the current pair as not corresponding to object pixels, and setting any thus identified pixels in the reference background frame to the value or values associated with the corresponding pixels in the complete pre-processing video frame of the current pair; and repeating the last two preceding steps until all of the input complete preprocessing video frames and comparison frames have been processed or there are no further unset pixels in the reference background frame.

5. A method as claimed in claim 4 wherein the marking of pixels as set or unset in the reference background frame is performed by generating a reference mask frame having a corresponding pixel for each pixel in the reference background frame and setting each pixel to one of two possible values depending on whether the corresponding pixel in the reference background frame is set or not.

6. A carrier medium carrying processor implementable instructions for causing a digital processor to carry out the method of any of claims 1 to 5 during implementation of the instructions.

7. A method of transmitting a sequence of video frame from a transmitting unit to a receiving unit including generating a single background frame from the sequence of video frames according to the method of any one of claims 2 to 5 and transmitting the single background frame to the receiving unit.

8. A method of any of claims 2 to 5 or 7 including the step of detecting camera panning as claimed in claim 1 and generating a first background frame in respect of video frames prior to the camera panning action, and generating a new background frame for video frames post the camera panning which new background frame incorporates some of the pixel values from the first background frame.

9. Apparatus for generating a single background frame for combining with a series of object frames to create a series of post-processing complete video frames each of which consists of the respective object frame superimposed onto the single background frame, the apparatus comprising means for comparing an input series of pre-processing complete video frames with an input corresponding series of comparison frames each of which indicates which of the pixels, in the corresponding input pre-processing complete video frame, represent object pixels, to thereby derive said single background video frame.

10. Apparatus according to claim 9 wherein each of the comparison frames comprises a binary mask having a number of pixels each of which corresponds to a respective one of the pixels in each input pre-processing complete video frame and each of which is allocated a first or a second value in dependence upon whether or not the corresponding pixel in the corresponding input pre-processing complete video frame is a foreground object pixel or a background pixel.

1 1 . Apparatus according to either claim 9 or claim 10 wherein the comparing means comprises: first identifying means for identifying a first pair of one of the input series of pre-processing complete video frames and the corresponding comparison frame; generating means for generating a reference background video frame by providing a plurality of pixels each of which corresponds to a respective one of the pixels in each of the input pre-processing complete video frames, setting each pixel corresponding to a pixel not indicated by the comparison frame of the first pair as being an object pixel to the value (for black-and-white) or triplet of values (for colour) associated with the corresponding pixel in the input pre-processing complete video frame of the first pair and marking each such pixel as set, whilst leaving each pixel corresponding to a pixel which is indicated by the comparison frame of the first pair as being an object pixel to the default value or triplet of values and marking such pixels as unset; second identifying means for identifying a next pair of a complete pre- processing video frame and its corresponding comparison frame to form a current pair; third identifying means for identifying unset pixels in the reference background video frame which correspond to pixels indicated by the comparison frame of the current pair as not corresponding to object pixels, and setting any thus identified pixels in the reference background frame to the value or values associated with the corresponding pixels in the complete pre-processing video frame of the current pair; and decision means for causing the^' second and third identifying means to repeatedly identify subsequent pairs of video and comparison frames and to identify and appropriately set previously unset pixels in the reference background frame until all of the input complete pre-processing video frames and comparison frames have been processed or there are no further unset pixels in the reference background frame.

1 2. Apparatus for detecting camera panning occurring from one video frame to another, the apparatus comprising comparison means for comparing an input pair of pre-processing complete video frames with an input corresponding pair of comparison frames each of which indicates which of the pixels, in the corresponding input preprocessing complete video frame, represent object pixels, to thereby determine which of the pixels in each of the complete video frames represent background pixels and motion estimation means for performing motion estimation in respect of the background pixels only, ignoring the object pixels.