GB2344714A

GB2344714A - Method and apparatus for creating real digital video effects

Info

Publication number: GB2344714A
Application number: GB9820502A
Authority: GB
Inventors: Tamir Ben-David
Original assignee: Orad Hi Tec Systems Ltd
Current assignee: Orad Hi Tec Systems Ltd
Priority date: 1998-09-22
Filing date: 1998-09-22
Publication date: 2000-06-14
Also published as: GB9820502D0

Abstract

A method and apparatus for creating real digital video effects (RDVE) by: creating an encoded pattern on an object on which the special effect is to be superimposed (or chromakeyed), using said coded pattern to follow changes in the position and shape of every basic point in the pattern, said basic point being defined as the basic subregion area in the picture pattern. This method/apparatus is capable of imitating a real 3D surface and displays image rolling and movement in 3D space from non-perpendicular angles.

Description

METHOD AND APPARATS FOR CREATING REAL DIGITAL VIDEO EFFECTS The present invention relates to a method and apparatus for creating real digital video effects (RDVE) and more particular for the automatic insertion of a digital video image into another video image.

In the present invention the inserted video image is placed onto a real object surface and in the present invention the real object can be either fixed or moving or may even, in a preferred embodiment, have a changeable surface.

Digital video effects (DVE) are widely used in feature films, commercials and TV programs.

In known conventional DVE techniques editors are enabled to play with the 2 dimensional shape of the total image. Effects include rolling the image, giving a 3D impression when looking from a perpendicular (straight on) angle.

A problem which exists with such conventional, known systems, is that until now there has not been any technique to imitate a real 3D surface. Also, there has been no way to display image rolling and movement in 3D space from non-perpendicular angles.

At present therefore the only implantations which are done are on flat surfaces and then editing is performed manually.

It is an object of the present invention to automatically create real digital video effects. The inventive principle is to use a special encoded pattern on the object. The encoded pattern enables the changes in the position and shape of every basic point on the pattern to be followed (a basic point is defined as the basic sub-region area in the pattern picture that is detected through the encoding method).

The present invention provides a method of creating real digital video effects comprising the steps of creating a specially encoded pattern on an object on which the video effect is to be superimposed, using said coded pattern to follow changes in the position and shape of every basic point on the pattern, said basic point being defined as the basic sub-region area in the pattern picture detected in said encoding method.

Preferably said pattern is selected in accordance with the complexity of the object.

In a first preferred embodiment the method comprises a pixel to pixel transform.

In a second preferred embodiment the method comprises a 2D projection of a matrix of flat sub-region polygons gathered together in a mosaic form.

The present invention also provides apparatus for automatically creating real digital video effects, said apparatus comprising means for creating a specially encoded grid pattern on an object in which the video effect is to be superimposed, means for detecting the parameters of movement of the object from the specially encoded grid pattern, means for identifying a relevant area for mapping of a video effect using a chroma key border, means for performing a perspective conversion of every pixel in the relevant area, means for replacing onto said object a pattern to provide said digital real video effect.

Embodiments of the present invention will now be described with reference to the accompany drawings in which: Figure 1 shows a flow diagram illustrating exemplary methods for creating real digital video effects on flat surfaces and complex surfaces; Figure 2 shows a hardware diagram for a general movement pattern; Figure 3 shows an exemplary pattern for a general movement pattern; and Figure 4 shows exemplary hardware for a pixel mapping pattern.

The present invention deals with several types of problems that can be discussed in two levels: 1. The complexity of the object : a) Flat planes, b) Simple objects with long space correlation between the surface points. For example, spherical and cylindrical planes, c) Complex surface shape, fixed in time. For example, all kind of rigid bodies, d) Complex surface shape that varies with time.

2. Keying a) the object is always seen at the foreground of the picture, b) the object can be seen also in the background.

For the case of la, a general pattern such as the conventional Orad grid, as described in (Virtual Studio 1I-PCT W097/09830) can be used, and the overall object movements can be traced. For this case a normal chroma keying can be applied.

For the case of lb a special pattern such as the pattern with special signs as shown Figure 3 can be used. Both the signs and the background have chroma key colour similar to the lines and background of the conventional ORAD grid as mentioned above. For this case a normal chroma keying method can be applied.

For the cases lc, d, a special encoded pattern is used, the pattern has the information on the identity (co-ordinates) of each point in the overall pattern image and his orientation and scaling compared to the initial picture conditions. The decoding is based on colours luminance variance in the encoding pattern. For the case of 2a a chroma key border is presented, and allows clear edge cutting. For the case of 2b, a chroma key net is presented on top of the encoded pattern. The net is used to evaluate the keying mate.

After the detection of the object changes in position and shape, a matching method is applied to introduce the external video clip into the master video film. The matching can be done either by using real time dedicated hardware or by appropriate software. For flat surfaces a single 2D projection is needed. For the other cases two methods can be applied : Pixel to pixel transform based on the encoding pattern, taking care for the right scaling and orientation transform.

2D projection of matrix of flat sub region polygons, gathering together in a mosaic form.

Examples for RDVE use: -For the advertising market the RDVE has the benefit of creating an advertising clip once, with the ability to change the label for advertising campaign for other languages, or using the same shooting for a different product. The pattern can be used as a sticker on top (or on the side) of the product, replacing the conventional sticker.

Putting video picture on real paper sheet, this can be used for television clips, advertising and book reading programs.

Putting video on fabrics, like flying flag and actor's clothes.

Specific embodiments of the present invention will now be described with reference to the drawings.

As stated above, the surface onto which the real digital video effect (RDVE) is to be placed may be either flat or of a complex shape. The sequence steps for performing the RDVE for both flat and other surfaces may be exemplified as follows.

For effecting an RDVE on an object, reference is made to figure 1 in which the flow diagram for the step is shown. The steps are: 1: flat surface = > 2.

Else =z > 3.

2a Putting a grid sticker with general movement pattern.

2b Video shooting.

2c Detecting the parameters of the object movement (position, orientation and scaling) from the changes of the grid pattern.

2d Perspective transformation (2D) of the external video film into the master video film.

3a Putting sticker with encoded pattern for pixels transformation.

3b The object is always in the foreground = > 4 Else = > 5 4a The encoding pattern has a chroma key border.

4b Video shooting.

4c Identifying the relevant area for mapping, by using the chroma key border 4d Perspective conversion of every pixel in the relevant area with the appropriate value deduced from the external image. The information on the pixel position scaling and orientation is encoded in the pattern.

5a The encoding pattern has a chroma key grid 5b Video shooting.

5c Keying the relevant area for mapping by using the chroma grid.

5d High quality matching + long processing time = > 6 Else = > 7 6a Dividing the pattern into sub regions built from few grid squares (depend on the object complexity and the scaling factor).

6b Perspective transformation (2D) for each sub region, gatherings together in a mosaic form.

7 Perspective conversion of every pixel in the relevant area with the appropriate value deduced from the external image. The grid pixels on the object are replaced by average of the nearest neighbour.

With reference to Figure 2, apparatus for the general movement pattern steps (2a to 2d). The apparatus comprises a camera 200, a digital store for non-real time image storage 202 and a first monitor 204, a chroma keyer 206 connected to receive the output of camera 200. The output of chroma keyer circuit 206 is fed to a processor 208 in which the object movement is processed. The output of processor 208 is connected to a further processor 210 which performs calculation of external video deformation.

A VCR 212 contains external video images and the VCR 212 is connected to a second monitor 214 and to the processor 210 to supply the external video image data which is deformed in the processor 210 to enable the external video image to be added to the initial image in a further processor 216 which also receives the initial image from the output of camera 200, processor 216 also receiving the output of camera 200.

The addition of the external and initial video images provides the combined output on output 218.

An example of the general movement pattern 30 is provided in figure 3. This comprises two chroma key colours (preferably both being blue). The background colour 32 in a preferred embodiment is a brighter blue and the reference signs 34 imposed thereon are a darker blue. The pattern shown is exemplary and may be changed both in respect of the number and shape of the signs 34 for different object shapes.

Exemplary circuitry for the pixel mapping pattern is shown in Figure 4.

A camera 400 is connected to a store 402 for non-real time storage and also to a first monitor 404. A chroma key circuit 406 is connected to receive the output of camera 400 and the output of chroma keyer 406 is connected to a processor 408.

A VCR 410 supplies the external video image to processor 408 which performs the pixel mapping operation providing a signal output on line 412. A second monitor 414 is provided to display the external video image from VCR 410.

Claims

CLAIMS 1. A method of creating real digital video effects comprising the steps of creating a specially encoded pattern on an object on which the video effect is to be superimposed, using said coded pattern to follow changes in the position and shape of every basic point on the pattern, said basic point being defined as the basic sub-region area in the pattern picture detected in said encoding method.
2. A method of creating real digital video effects as claimed in claim 1 in which said pattern is selected in accordance with the complexity of the object.
3. A method of creating real digital video effects as claimed in claim 2 in which the method comprises a pixel to pixel transform.
4. A method of creating real digital video effects as claimed in claim 1 or claim 2 in which the method comprises a 2D projection of a matrix of flat sub-region polygons gathered together in a mosaic form.
5. Apparatus for automatically creating real digital video effects, said apparatus comprising means for creating a specially encoded grid pattern on an object in which the video effect is to be superimposed, means for detecting the parameters of movement of the object from the specially encoded grid pattern, means for identifying a relevant area for mapping of a video effect using a chroma key border, means for performing a perspective conversion of every pixel in the relevant area, means for replacing onto said object a pattern to provide said digital real video effect.
6. Apparatus as claimed in claim 5 further comprising means for effecting a pixel to pixel transformation of said relevant area of said object.
7. Apparatus as claimed in claim 5 including means for effecting a 2D projection of a matrix of flat sub-region polygons gathered together in mosaic form.
8. Apparatus as claimed in claim 5, said apparatus comprising a first processor (208) for assessing object movement, a second processor (210) for calculation of external video deformation, means for providing an external video image and means for processing said external image to add an image from said region of said object to said external image to create a combined replacement image.