CN1221502A - Method and apparatus for integrating hyperlinks in video - Google Patents

Abstract

Hypervideo data is encoded with two distinct portions, a first portion which contains the video data and a second portion, typically muc smaller than the first, which contains hyperlink information assocated with the video data. Preferably, the first and second portions are stored in separate and distinct files. A hypervideo player determines the hot link regions in frames between key frames by interpolating the hot link regions inkey frames which sandwich those frames.

Description

The method and apparatus of integrating hyperlinks in video

The present invention relates to generate the system with editing video, particularly relate to integrated system with hyperlinked information of video.

Hypermedia is a term, in order to describe other two kinds of new technologies: the fusion of multimedia and hypertext.Multimedia refers to and includes text, image, figure, the message form of Voice ﹠ Video.Hypertext document is a kind ofly to have set up linked document by hyperlink and other documents.Hyperlink often appears in the hypertext literary composition retaining with a highlighted textual form, and the text is normally described the word or the phrase of the information that the user may further want.When the user uses Genius mouse that its click is activated this hyperlink typically, chained command is activated, the program at the place, address that feasible quilt links is performed, thereby upgrade user's visual field successively so that show that the document comprises the more information relevant with highlighting word or phrase in typical case by linked document.This information can be text, audio frequency, video, the form of two-dimensional image or three-dimensional image.Hyperlink makes the interleaving access of following between document become easy.The hypermedia document is the hypertext document with multimedia function.Activated hyperlink zone is called hot link on the screen.

Recently, a lot of people are familiar with using hypertext by using Genius mouse to click on the graphoscope from the hot link on WWW (database server Web) the www homepage.Data on the Web are located by URLs, and URL represents uniform resource locator, and it is the criteria for selection of specifying target on the Internet, in order to the position of indication access method and file.Document on the Web is write as with " mark is put language " that simply be called HTML, and HTML represents that hypertext mark puts language.The last document format data of Web is appointed as the MIME form, MIME represents that the example of multi-usage internet mail expansion (with reference to http://www.oac.uci.edu/indiv/ehood/MIME/MIME.html) last file layout of Web has .au (almost be General Audio form), .html (html file), .Jpg (image of JPEG coding), .mid (Midi music format), what mpg (video of mpeg encoded) and .ps (Postcript file) and present hypertext technology were the most general is text and visual medium, also begins to have occurred animation and video.

Hypervideo is the address that increases the video of hyperlink.Demonstration was done to the NewSbytes system of by name " video hyper-media system " by NEC Corporation, and such system will be the point of hypertext-hit function (point and click capabilities) to bring in the whole motion video.Being described in more detail of relevant hypervideo can be at " NEC ' s Video Hypertext System ", Newsbytes NewsNetwork July 31,1995.Find in one literary composition.

Hyper Cafe is the experimental hypermedia prototype as the illustration exploitation of general hypervideo system.This program is inserted a virtual coffee shop (cafe) to the user, and digital video montage (clips) formation of the performing artist in the session is mainly fabricated by being involved in coffee shop by this virtual coffee shop.Hypercafe allows the user to follow different sessions, and provides the dynamic interaction chance by the time, and space-time and link structure are to present account (alternativenarratives) alternately.Being described in more detail of relevant Hyper Cafe can be at Nitin " Nick Sawhne; the article of David Balcom and Ian Smith " " Hyper Cafe:Narrative andAesthetic Properties of HyperVideo ", ((March 20 for Recipient of the first Engelbant Best PaperAward at Hypertext ' 96 for Hypertext ' 96:Seventh ACMConference on Hypertext, 1996), http:/Silver.Skiles.gatech.edu/gallery/hypercafe/HT96, look among the Talk/ to.

VideoActive is in order to generate the instrument of writing of staggered form film.It adopts hypervideo technology mark hot link in digital video file.This instrument allows user's preparation to have the video clipping of hot link information, then with the media links of they and other type.Being described in more detail of VideoActive can be at " HyperVideo Authoring Tool (User Notes) ", http://ephyx.com/, and Pre-Release Version finds among the Feb.1996.

Progressive Networks company has comprised into " the video figure that can click (click-able Video maps) " in its Real Video technology.Genius mouse is clicked the demonstration that a part of video can cause the new video montage, inquires about in current montage, or sends a URL message.The RealPlayer that company provides it make this interactivity to realize.Being described in more detail of RealVideo can be found among the http://www.realandio.com/products/realvideo/overview/index.html at " the RealVideo Technical WhitePaper " of Progressive Network company.

The total characteristics of all these technology are enormous size of hypervideo.In essence, hypervideo information comprise the video of coding and hyperlinked information the two.In order to add hyperlinked information in video clipping, the essential file that generates greater than original video files.Even the take over party may have the local copy of original video files,, often be that the user has a video file on CD-ROM, and wants to add hyperlinked information to this video in order to transmit the hyperlinked information huge file of yet essential transmission.Adopt existing technology, the user will generate a file or download a file that has generated, and its size is big equally with source document at least.

When adopting existing technology,, generate the hypervideo content and make us dull because in the frame that each hypervideo exists, all will encode hot link information.

The present invention deals with problems by generating a small documents, and this document comprises pointer that points to corresponding video file and the hyperlinked information of encoding with effective and efficient manner.The hypervideo data are encoded with two character different pieces, and first comprises video data, and second portion is littler than first usually, and it comprises and relates to the sign of first and the information of hyperlinked information.Effectively generate the coding of hyperlinked information by the hot link district of key frame in the encoded video only and coding simple geometric shape.The hypervideo player is determined the hot link zone in these frames between the key frame according to the hot link district that inserts in the key frame that is clipped in these frames.

In a preferred embodiment, the hot link district is that its axle of rectangle is parallel to main shaft, and encode by listing its upper left and summit, bottom right, hot link district interpolation is linear between key frame, this scheme effect is fine because usually by roughly all being linear in the motion of continuous time at interval they each, can be similar to target is pressed the time interval in video motion well.It is the required data total amount of coding hot-zone information that this scheme had both reduced widely, has quickened the process of generation hypervideo content again significantly.

Fig. 1 is the functional block diagram of the computer processing system that can be utilized by the embodiment of the invention.

The Fig. 2 (A) and (B) form of video file of explanation mpeg encoded;

Fig. 3 (A)～(C) explanation is according to the form of hyperlink file of the present invention; And

Fig. 4 is a functional block diagram, illustrates to be used for video player that video file and hyperlinked information are integrated, and hyperlinked information wherein is stored in the hyperlink file shown in Fig. 3 (A)～(C).

Fig. 5 (A) and (B) be a functional block diagram illustrates the instrument that is stored in the hyperlinked information in the hyperlink file shown in Fig. 3 (A)～(C) in order to generation.

The graphic user interface example of Fig. 6 and 7 instrument with caption Fig. 5 (A) with (B).

The present invention can realize on any computer processing system, for example personal computer or workstation, as shown in Figure 1, can be generally included storer 101 by the computer processing system that the present invention utilizes, at least one CPU (central processing unit) (CPU) 103 (having shown one among the figure), and at least one user input apparatus 107 (as if keyboard, Genius mouse, operating rod, speech recognition system or hand-written discrimination system).In addition, computer processing system also comprises nonvolatile memory, as ROM and/or other non-volatile memory storage 108, fixed disk drive for example, the operating system that it is stored and one or more application program are loaded onto storer 101 and are carried out by CPU, in operating system and application program execution, CPU may use the data that are stored in non-volatile memory storage 108 and/or the storer 101.In addition, computer processing system also comprises graphics adapter 104, and it is connected CPU103 and as if between the such display device 105 of CRT monitor or LCD display.Application program and/or the operating system carried out by CPU103 produce graph command, for example, draw the order of a frame (or window), the order of display bitmap image, and three-dimensional modeling becomes the order of figure, or the order of display video file.This class order can be handled by application program/operating system that CPU103 carries out, perhaps application program/operating system associated working of being carried out by hardware and CPU103.Suitable pixel data is generated and display device 105 correspondingly is updated in this process.

In addition, computer processing system may comprise the communication linkage 109 (as network adapter, RF link or modulator-demodular unit) that is connected to CPU103, and it makes CPU103 on communication link, for example communicates by letter with other computer processing system on Internet.In executive operating system and the application program, CPU103 can receive the operating system part on communication linkage 109, application program part, or the data division that is used by CPU103.

It is to be noted that application program/operating system of being carried out by CPU103 can realize following illustrated method of the present invention.The hardware of application program/operating system associated working that illustrated part or all of method can be carried out with CPU103 in addition, is implemented.And, below said method also can implement by distributed processing system(DPS), thereby the part of these methods is distributed on two or more disposal systems, these systems are linked at together by communication linkage 109.

According to the present invention, corresponding to given video file and preferably independently and be different from second file (being hereinafter referred to as [thermovision is frequently] Hot Video data file) of corresponding video file, in order to identify the hot link in the corresponding video file.Hot link (or hot link district) is the pixel area in one or more frames of video file, in order to the hyperlink of expression relevant information.When the user typically used Genius mouse that its click is activated hyperlink, a chained command was activated.When chained command is activated, one or more operations that this chained command identification process system and this disposal system will be carried out.The disposal system that is identified by chained command can be to handle the same system of video file, or the remote system that is connected with the system of processing video file by communication link.The startup of chained command makes to be carried out by the disposal system that chained command identified by the operation of chained command sign, and representational is that the execution of chained command causes user's view update so that show the content that is linked.By the content that linked typically comprise with at the hot link place or near the relevant information of image of hot link place demonstration, this information can be text, audio frequency, video, the form of bidimensional image or three-dimensional image.The example of chained command can be the hypertext link by html format, and it makes remote computer download the disposal system that html page is given the user.Thermovision coded format frequently

Scheme below coding hot link district preferably adopts in specific video.At first, the hot link district has the simple geometric shape.In a preferred embodiment, the hot link district is that its each limit of rectangle is consistent with the main shaft of display, and represents with upper left in the given frame of video and the coordinate bottom right pixel.The second, the hot link district is not encoded in each frame, but the user writes down the hot link district in being called key frame (Key frames).Best, key frame should be chosen like this so that with the relevant target route in given hot link district, between this frame by the time interval near linear (just, not adding fastoperation).In the case, what can specify the hot link district by its position is this key frame, and playback reproducer can realize that linear interpolation determines the position in hot link district in the intermediate frame between this key frame.Being described in more detail of relevant this key frame will be stated below.

Video file is with various standard video formats, as AVI, and MPEG, original YUV and original RGB are encoded.For example, Le Gall and Didier J.At " MPEG videocompression algorithm ", Signal Process Image Commun V4 n2 Apr1992 p129-140, the middle form of describing the MPEG video.These video formats comprise header information, in order to some feature such as the frame per second and the file size of explanation video; The encoded pixel value of the color component of various frames in the video; Follow the coding audio data of video synchronously.Frame of video is a single width image; Video is made of continuous frame, like this (being typically per second 25 or 30 frames) satisfied motion video of vision consequently when these frames are play with sufficiently high frame per second.Frame in the video is first frame by number consecutively-frame 1 usually, and frame 2 is second frames or the like.The predicate of frame number refers to the number of frame in this continuous sequence.As example, Fig. 2 (A) and (B) provided the form of the video file of mpeg encoded.HEADER includes the information as the bit rate of system clock reference and video and audio component.Packet DATA (j), j=1,2 ..., N comprises actual encoded video and voice data, and DATA (j) has given explanation in Fig. 2 (B).The signal that special START sign indicating number begins as new data packets; The forthcoming data of AV (j) sign are audio frequency or video; TIME (j) provides the synchronous necessary timing information of video/audio; ENCODED-DATA (j) is actual audio or video data.The coding video data comprise with frame of video in the relevant information of chrominance component pixel value.In the MPEG video, for example ENCODED-DATA (j) is the binary stream of the Huffman coded run length of quantization DCT coefficient.The more detailed visible mpeg standard document of example, Draft International Standard ISO/DIS 11172 (ISO/IEC JTC1).All draw at this and to be reference.

Fig. 3 (A)-(C) provides the structure of second file, and it comprises hyperlinked information (i.e. Bian Ma thermovision audio data).In Fig. 3 (A), HD is a header information, and it identifies the corresponding relation between second file and the relevant video file.In this preferred embodiment, the prefix identical (for example, be called " title.mpg " if video file is a mpeg file, the name in header is " title ") of the name of second file and relevant video file name.Header information HD also can identify the character of relevant video file, for example, and the frame per second of size of file (with byte representation) and associated video file.Such information may be used to the video file of phase-detecting pass, school and the coupling of thermovision audio data file, because the name of file may change.Yet this is not a safest school detecting method, and it can run into a large amount of not matching.

OB (1) ..., OB (N) is a N thermal target.Each thermal target OB (j) is definition one hot link district in a frame of relevant video file or a plurality of frame.The explanation of its coding provides in Fig. 2 (B).The most handy identifier ON (j) of each target OB (j), a destination name, a relevant chained address OA (j), one in video its hot link district of sign list item L (j, kj) ^*And the tail tag T (j) that distinguishes the end-of-encode of j target encodes, and symbol Kj represents j object identifier sum.

Shown in Fig. 2 (C), identifier L (j, k) locus in hot link district in the indication one specific key frame.It preferably includes 4 information: and position B (j, whether k) indicate this key frame is end frame; Number N (j, k) the k key frame of expression j target; The coordinate of rectangle hot-zone left upper apex in the k key frame of j target (x (j, k, 1), y (j, k, 1); The coordinate on summit, bottom right, rectangle hot-zone in the k key frame of j target (x (j, k, 2), y (j, k, 2).(j, k) corresponding to frame number, they are key frame or end frame to number N.

Following table provides the explanation of each field of hyperlink file among Fig. 2 (A)-(C):

The corresponding video file of HD sign;

OB (j) is to j hot link target, according to the hot link that the user selects, the chained command that sign is carried out;

ON (j) is to j hot link target, interrelates with its name;

OA (j) identifies its chained address to j hot link target;

L (j, k) to j hot link target, the k key frame that uses during this target of identification code;

B (j, k) to j hot link target, identifying this k key frame is an end frame;

(j k) to j hot link target, identifies the frame number of k key frame to N;

X (j, k, 1) defines the Building X mark in the rectangle region upper left corner in hot link district in the k frame of j target;

The y coordinate in the rectangle region upper left corner in the hot link district that y (j, k, 1) defines in the k frame of j target;

X (j, k, 2) defines the Building X mark in the rectangle region lower right corner in hot link district in the k frame of j target;

Y (j, k, 2) defines the Building Y mark in the rectangle region lower right corner in hot link district in the k frame of j target.

As mentioned above, the hot link district preferably has rectangular geometry.Yet the present invention is not limited to this mode; Therefore, the hot link district can have random geometry.For example, the hot link district can have circular geometric configuration, and is defined by the position and the radius of a circle in the center of circle.Thermovision is player frequently

Fig. 3 illustrates thermovision player frequently, and it is read a video file and the hot link information that is included in the second relevant file is integrated with the video that is included in the video file, and these are exactly illustrated data in Fig. 2 and Fig. 3 (A)-(C).Thermovision frequency player can be application program and/or the operating system part by the CPU103 execution of Fig. 1.In addition, partly or entirely thermovision frequency player can be embedded in the hardware, with application program/operating system associated working of being carried out by CPU103.And the portion of hot video player is embedded in the distributed processing system(DPS), so as to the thermovision functional part of player frequently, is distributed on communication linkage 109 is linked at together two or more disposal systems.

Now referring to Fig. 4, according to header data HD, HVP determines which video file is relevant with thermovision frequency file (second file).It send a message to remove to open this video file to VIDEO PLAYER.If the character of this video file is matched with those descriptions in the header, then VIDEO PLAYER begins to play this video file.Otherwise VIDEO PLAYER prompting user goes for suitable video file.HVP also analyzes data and arranges its feasible inquiry to it of form to become simple.

VIDEO PLAYER begins displaying video.At each frame, VIDEO PLAYER transmits a value to the FRAME CHECK stage, indicates the frame number of current broadcast.Because of having frame losing, so frame number is not must be continuously, behind the decoding present frame, VIDEO PLAYER exports these frame image data to BUFFER.The operation more specifically of VIDEO PLAYER is not relevant with the present invention in the process of decoded video file frame, and therefore, the present invention can use any VIDEO PLAYER in order to decoded video file frame, and the output frame pictorial data is to non-volatile storage.The example of this VIDEO PLAYER is the Active Movie video player that Microsoft provides in Active Movie software development kit, describes visible http://www.microsoft.com/imdeia/activemovie/activem.htm in detail.Draw in full at this and to be reference.

The FRAME-CHECK stage determines whether that to each the target OB (j) in second file following two conditions are satisfied:

Be a key frame for the current frame number of this target ⅰ), perhaps current frame number is between two key frames of this target; And

ⅱ) first key frame is not the end frame of this target.

As satisfy last two conditions, then these operations of INTERPOLATE/OVERLAY/DISPLAY are performed.

INTERPOLATE manipulates from the coordinate data of two key frames that clip present frame and their frame numbers separately, and to present frame linear interpolation one rectangle region, this will be as its thermal region.The more details condition of INTERPOLATE operation will illustrate below.

OVERLAY then adopts this information and revises data among the BUFFER with sign hot link district.Preferably, OVERLAY revises the data among the BUFFER, becomes a constant color and identifies the hot link district by changing pixel value corresponding to the hot link zone boundary.As an alternative, the OVERLAY function can change this regional brightness and/or tone, or with other technology with sign hot link district.

DISPLAY then gets result data from BUFFER, and it is presented on the display of computing machine.

If in the FRAME CHECK stage, above-mentioned two conditions do not satisfy, and this target is no longer done any processing in present frame, and the part that this frame is used as video sequence is shown.

Illustrate the details of INTERPOLATE operation now.Suppose frame number F that VIDEO_PLAYER is showing be in two key frame N (j, k) and N (j, k+1) between.As mentioned above N (j k) is frame number to the k key frame of target OB (j), and N (j k+1) is frame number to the k+1 key frame of target OB (j).Suppose simultaneously frame N (j k) is not end frame to target OB (j), and in the case, some rectangle region is the hot link district in the F frame, and will point to address OA (j) corresponding to the hot link in this district, and the thermal region in frame F is determined by following:

If X (F, 1), Y (F, 1), X (F, 2), Y (F, 2) is according to representing respectively in the frame of frame number F upper left Building X, the rectangle hot-zone of OB (j) mark of storing in second file, the upper left Building Y mark of hot-zone, Building X, the bottom right mark of hot-zone and Building Y, the bottom right mark of hot-zone, then X (F, 1)=(1-Z) ^*X (j, k1)+Z ^*X (j, k+1,1),

Y(F,1)=(1-Z) ^＊Y(j,k,1)+Z ^＊Y(j,k+1,1),

X(F,2)=(1-Z) ^＊X(j,k,2)+Z ^＊X(j,k+1,2),

Y(F,2)=(1-Z) ^＊Y(j,k,2)+Z ^＊Y(j,k+1,2),

Wherein Z=(F-N (j, k))/(N (and j, k+1)-N (j, k)),

Here ^*Number expression is taken advantage of, and top 4 equations are preferably in and find the solution after turning to immediate integer again.

INTERPOLATE carries out the calculating by above-mentioned equation regulation, to form the coordinate of the rectangle of determining border, hot link district.

An example of following table explanation coding rectangle thermal region in the thermovision frequency file:

ObjectName

http：//www.object-address.html

0	65	(20,30)	(40,70)
				0	95	(20,30)	(40,70)
0	120	(70,130)	(90,170)
				1	220	(120,130)	(140,170)
0	300	(120,130)	(140,170)
				1	400	(120,130)	(140,230)

The target that is called ObjectName links with URL http://www.object-addres.html, and it is encoded into 6 key frames: 65,95,120,220,300 and 400.Frame 220 and 400 be the end points frame (by binary number 1 expression, those be designated as 0 be key frame, they are not the end points frames).In this video, the thermal region rectangle generates in frame 65 and has left upper apex (20,30) and summit, bottom right (40,70), so the size of this rectangle is 20 * 40 pixels (high by 20, wide by 40).

Hot rectangular area comprises the part target in its border.For example, if target is a people, the rectangular area wherein may comprise whole people, or people's head.Select rectangle to describe target and be not decided by that thermovision compiles the author frequently, the principle of deferring to is thermovision onlooker frequently according to the position of the relevant relatively rectangle of target recognition objective easily.Thermal region then is encoded in second key frame of frame number 95.This rectangle identical with in first key frame.

Thermovision frequently player is preferably explained this data like this, and for the whole frames (comprising 65 and 95 frames) between 65 and 95, the rectangle with specified coordinate will be identified as the hot link district.This hot link district is encoded in the 3rd key frame of frame number 120 again, is that present position has become, its left upper apex in (70,130) and its summit, bottom right in (90,170).The size of rectangle does not change, but has moved down 50 pixels and 100 pixels that moved right.Thermovision frequently player is preferably explained this data like this so that to the whole frames between 95 and 120, a size be bordering on the rectangle of 20 * 40 pixels and position at frame 95 and position certain position between frame 120, will be identified as the hot link district.Its definite size and determining positions are in the interpolation type of using.Thermal region is encoded in the 4th key frame of frame number 220 again, has the left upper apex of coordinate (120,130) and the summit, bottom right of coordinate (140,170).In addition, rectangle size does not change, this moment it to having moved down 50 pixels, and horizontal direction does not move fully.This data of the thermovision frequency best interpolation of player are to generate the thermal region between these key frames.Thermal region is encoded in the 5th key frame of frame number 300 once more, has coordinate (120 once more, 130) left upper apex and coordinate (140,170) summit, bottom right, because the 4th key frame (number 220) is an end points frame, thermovision player does not frequently produce thermal region to this target of frame 221 to 299.At last, thermal region is encoded in the 6th key frame of frame number 400 again, has used the left upper apex of coordinate (120,130) and the summit, bottom right of coordinate (140,230).This moment, the size of rectangle changed.In this case, thermovision is the such data of the best interpolation of player frequently, the frame between 300 and 400 is produced the hot rectangle of increased in size.This 6th key frame also is an end points frame.Thermovision is write instrument frequently

Fig. 5 (A) and (B) provide the functional block diagram of software application embodiment, this application program provides the ability that generates aforesaid thermovision frequency file to compiling the author.

OPEN-HVF starts this software application, opens the new thermovision frequency file of a thermovision frequency file that has existed or mark one simultaneously.OPEN VIDEO control HOTVIDEOPREVIEWER removes to open the video file relevant with the thermovision frequency file.Decode each frame of this video of HOTVIDEOPREVIEWER, be included in the thermovision audio data file information be included in the video file the frame image data integration together, and output result frame pictorial data is in order to show.HOTVIDEO PREVIEWER function class is similar to the HOTVIDEOPLAYER that states above.Yet, be different from the frame that is inserted in the use of HOTVIDEO PLAYER between the key frame and place the hot link district, HOTVIDEO PREVIEWER places the hot link district at its assigned position that is right after the front (at its last-defined position), is marked up to next key frame or end points frame.

In addition, application program has comprised the NAVIGATE function it provides the ability of browsing by each frame of video file to the user.For example, the function of NAVIGATE preferably includes PLAY, STOP, SEEK, STEP FORWARD and STEP BACKWARD function, PLAY functions control HOTVIDEO PREVIEWER playing video file (promptly progressively each frame by video file and output frame pictorial data in order to show).STOP functions control HOTVIDEO PREVIEWER stop playing video file (promptly stop stepping by the video file frame, and output corresponding to the frame image data of the frame that stops to ask in order to show).SEEK functions control HOTVIDEO PREVIEWER decoding and output, to show the particular video frequency frame, this frame can identify by the position of following said slider bar (slider bar) top shoe.This is to compiling (with course-grained control) the .STEP FORWARD of the control to HOTVIDEO PREVIEWER and STEP BACKWARD functions control HOTVIDEO PREVIEWER decoding and the output that the author provides characteristic, to show next subsequent frame.This provides the distinctive meticulous control of HOTVIDEO PREVIEWER (with fine-grained control) to compiling the author.

When the identification of ENCODE function compiles the author has been determined coding hot link district in the current frame that is just being shown by player.Identify at the ENCODE function and to compile the author when having determined in present frame coding one hot link district, call the DEFINE-HOT-LINK-REGION function, a reciprocal process at this moment, its allows to compile the author and goes to be identified at a zone that will become the hot link district in the present frame.For example, the hot link district can by frame number and as above said rectangle be upper left and the pixel coordinates summit, bottom right defines.In the case, current frame number is provided by HOTVIDEO PREVIEWER.The coordinate on rectangle summit can generate by click and towing (click-and-drag) processing of Windows, when cursor is clicked the Genius mouse left button during in left upper apex, then the user presses left button always and pulls Genius mouse till cursor is on the summit, bottom right, and the user decontrols this button that is pressed in this.

After definition hot link district, preferably call the ATTACH-0BJECT function, it provides the ability that the hot link district is connected with a target to compiling the author, this target may be already present target (target corresponding to the hot link district of front video file just), or a fresh target (promptly not corresponding with the hot link district of front video file target).This process or the name by this target of input or under the situation that target has been defined, are clicked the destination name that is listed on another window of this instrument by Genius mouse and are finished, and have listed the targets that all defined in above-mentioned window.

After the hot link zone, the ATTACH-LINK-COMMAND function is called at linking objective, and it connects a last chained command to this target, if fresh target, and the user ID one target chained command of being correlated with therewith then.If not fresh target, this target may have relevant chained command so.In the case, the ATACH-LINK-COMMAND function can be by bypass.As mentioned above, when chained command started, this chained command was determined a disposal system and the operation of being carried out by such disposal system.For example, chained command may be a URL address that will be linked when this hot link district is chosen by the user.

Application program also provides the MARK-END-FRAME function, and it provides sign one to the volume author is the ability of end points frame with the frame of thermovision audio data coding.Preferably determine that by compiling the author this key frame that just has been encoded is an end points frame or is not.If an end points frame, the user then with the MARK-END-FRAME function to indicate that this frame is an end points frame.Preferablely be, the user then has option with current state of preserving thermovision audio data file and the option that withdraws from application, if the user determines to continue and does not withdraw from, the user can continue to handle to repeat this, so as to browsing all videos file and coding hot link district in the frame of video file.

In addition, application preferably comprises like this, and some options restore the step that (undo) carried out to allow the compiling author, remove the geometric configuration in definition one key frame hot link district, remove a target (removing the full detail of relevant this target of coding), the standard compliation instrument that provides by operating system is provided.

Fig. 6 provides the example to Fig. 5 (A) and software application graphic user interface (B), in this embodiment, is a URL with giving the relevant chained command that sets the goal.Yet, the invention is not restricted to this mode, and can be any operation of carrying out with the disposal system that chained command is determined.Left window (VW) 501, above-mentioned therein video and hot link district will be shown and the video capability button of establishing criteria and having an effect: broadcast/stop button 503, step button 507 backward, step button 509 forward, stochastic searching slide block 511, and the ON/OFF hot link highlights button 505.Broadcast/stop button 503 provides control HOTVIDEO PREVIEWER playing video file (promptly to the user, stepping supplies to show through each frame and the output frame pictorial data of video file) and stop playing video file (promptly, stop stepping each frame, and output shows the frame image data corresponding to the frame that stops to ask by video file) function; Step button 507 forward and backward, 509 provide control HOTVIDEO PREVIEWER decoding and output to show the function of the next frame/former frame of present frame to the user, searching slide block 511 provides control HOTVIDEOPREVIEWER decoding and output to show the ability of the frame of video of appointment to the user, and this can determine by the position of slide block on slider bar.At last, the ON/OFF hot link highlights button 505 provides control HOTVIDEO PREVIEWER on/off switch in video window 501 to show the hot link district to the user.

The right is a target window (OW) 521, and the name and the position 523 of compiling the video file that the author encoding are wherein arranged; The URL524 of the thermovision frequency file that promising HOTVIDEO PLAYER sign finally will be visited searches relevant video file there; The tabulation 525 of one target is arranged, and these targets will be defined in their hot link district KEY frame and coordinate, also have one with the relevant URL list of links 527 of target that defines.

When starting software application, best tool window 531 is shown, if compile the author from graticule, chooses New from the File menu, and then target window also can occur.Open button on OW then is used to load the video file that will encode with hot link information.Then first frame at video area VW meeting and video occurs together.If the volume author wishes to open already present thermovision audio data file and works in the above, then select the Open button from the File menu, OW can manifest once more, and the information that all produced this moment are appearance thereupon all.In addition, the Open button among the OW is in order to load the video file that will encode with hot link information.

For first (or next) target of encoding, compile the author and browse all videos frame, be parked in the volume author and want on its frame with hot link information coding.Compile the author and can use stochastic searching slide block 511, or in VW513, import frame number in the suitable square frame and browse whole frames by press broadcast button 503.In addition, compiling the author can use and reach forward backward that 507,509 pairs one thermal targets of step button identify initial KEY frame.In case initial KEY frame is identified, compile the author for example defines hot link by the rectangle region of specifying a surrounding target zone.This also can adopt routine techniques to realize: cursor under giving directions in the upper left corner of desirable rectangle, pull it to the lower right corner and relieving exist, the rectangle of a dotted line occurs in the zone of streaking, and its four jiaos are highlighted.Below VW, preferably be shown about the coordinate information in hot link district and URL information pointed.Also have, the geometric configuration in hot link district preferably can be by edit-modify, and the left button by the device of clicking the mouse and pull Genius mouse when cursor is in rectangle for example perhaps also pulls when cursor is clicked Genius mouse when highlighting any top, angle for four left button.In addition, compile the hot link district that the author preferably can restore (undo) definition.For example, one menu will occur when pressing right mouse button simultaneously in cursor is in rectangle, choose Cut or delete from menu then.

In case the hot link district is defined, the regional same target that the volume author will define connects, and for example, in the time of in cursor is in the hot link district, compiles the author by the zone of definition can being connected with target by right mouse button.At this moment a menu that comprises " add to " button ejects, and clicks " Addto " and then will eject secondary menu.This new menu has the tabulation that whole defined targets add " NewObject (fresh target) ".If this zone will interrelate with already present target, identify this already present target by already present target is clicked the Genius mouse left button, otherwise then import the name of " New Object ".

If hot link district and already present target interrelate, software application will be imported fresh information suitable position in the 0W automatically.To fresh target, a fresh target window will eject and point out you input information: target name and the relevant chained command of target therewith, for example relevant with this target URL.The tabulation of already used chained command will be arranged in this window in addition.Specify in hope under the situation of an already used chained command, as long as click left button simply, be preferably on hot link district and the target association after, the display properties in hot link district is changed with notice volume author, for example, the border in change hot link district becomes solid line from dotted line.

Along with a KEY frame is encoded, compiles the author and can use the same step described above next key frame of encoding.Preferably, when the volume author browsed each frame of all videos, HOTVIDEOPREVIEWER showed the hot link district of last definition.When the volume author wishes to encode next KEY frame, compile the hot link district that the author can use last definition, perhaps draw and define a new hot link district.For using the hot link district of last definition, compiling the author as long as use the zone of last definition is that oneself just draws just as it.Will define a newly developed area if compile the author, preferably the zone of last definition still appears on the screen, so that the ability of the relative position in these two zones of comparison to be provided to the volume author.

After having finished the coding thermal target in the last KEY frame of volume author at specified time interval, compiling the author is this frame identification the END frame, thereby this preferably highlights this frame and realizes by the Object Box and the right button of clicking the mouse on the frame label of rolling mouse device in the OW, then comprise the menu box that " Markas End Frame " is labeled as End frame option to eject one, choose " Mark as End Frame " by right mouse button.As shown in Figure 7, KEY frame arrow 570 becomes END frame arrow 571 with distortion and tells that with this compiling the author changes and work.Preferably, the volume author can be made the END frame into the KEY frame, and (meaning is, not as the END frame), the target frame (Object box) that way is the rolling mouse device in the OW, and on the frame label 573, press the Genius mouse left button, then comprise the menu of " Mark as Key Frame is labeled as key frame " option with ejection, choose " Mark as Key Frame " by right button.END frame arrow 571 becomes KEY frame arrow 570 with distortion and works so as to prompting volume author change.Preferably, any key frame and end points frame can both be cancelled, way be move on the target frame of Genius mouse in the OW and on frame label 573 by left mouse button, then comprise the menu of deletion (" Delete ") option by right mouse button with ejection again, choose " Delete ".

Preferably, by once or even pressing twice mouse button, will make the video among the VW jump to this frame on any frame label 573 in the target frame of OW.And, click continuously for twice on any target label 575 in the target frame of OW, cause the disappearance of relevant key frame information, if they current be visible.On the contrary, if they current disappeared then can stretch manifest.Preferably, this also can rely on cursor to drop on the target label 575 by the Genius mouse right button, and the menu that comprises suitable commands with ejection obtains.

Preferably, compile the author and can previewing use the integrated hot link information of HOTVIDEO PREVIEWER to video file, for example, by by through broadcast button and use on-off button 505 to carry out previewing in the situation that has the hot link district to highlight or not have.

It is easily because compile definition hot link district in each frame that needn't there be the hot link district therein in the author that HOTVIDEO writes instrument, compiles only definition hot link district in key frame of author.Place the hot link district as being inserted in the top said HOTVIDEO PLAYER utilization in those frames between two key frames.The size that this has just reduced the thermovision frequency file is not that each frame all has coded data, only key frame is just had data.Secondly, the thermovision frequency file is highly susceptible to coding, because as long as encoded key frames has been quickened cataloged procedure.For example, if 40 frames are on average arranged between the key frame, the size of thermal information data file is about 1/40 of situation that each frame all need be encoded so, and encoding process is approximately also fast 40 times.

In addition, the present invention requires preferably to press independent and different file storage thermovision audio data and corresponding video file, and this provides very big dirigibility to the user.For example, someone can transmit the thermovision audio data and give a user, the local copy that he has had a video (promptly, someone has a video copy on CD, and from Web side's acquisition thermovision audio data), yet the present invention is not only limited to this mode, and in addition, thermovision audio data of the present invention can be stored on the public documents with relevant video data.

The people who is familiar with this area should see that the above method of the present invention and function can implement on the program storage device that can be read by machine, aforesaid the inventive method and function have positively been realized, and can in carrying out, be used instruction repertorie by machine, the present invention simultaneously is as being described in the preferred embodiment project, being familiar with these professional personnel will appreciate that, in the spirit and scope of appended claims, the present invention can make an amendment and be actually used.

Claims (66)

1. be used for representing being stored in the method for the hyperlink of the frame of video in first file, this method may further comprise the steps:

Canned data in being different from second file of said first file, these information definitions are stored at least one hot-zone in the said frame of video in said first file.

2. the process of claim 1 wherein that said video encodes with standard format.

3. the method for claim 2, wherein said standard format is AVI, MPEG, one of original YUV and original RGB.

4. the process of claim 1 wherein and said information in said a lot of frame of video, define the border of at least one hot-zone.

5. the method for claim 4, wherein said at least one thermal region is a polygon, it comprises a lot of summits, and wherein the border of said at least one thermal region is defined by said polygonal vertex position.

6. the method for claim 5, wherein said polygon is to have left upper apex, upper right summit, the rectangle on summit, lower-left and summit, bottom right, and wherein said square boundary is defined by the position on said left upper apex and summit, bottom right.

7. the method for claim 4, wherein said at least one thermal region is a circle with the center of circle and radius, and the border of said thermal region is defined by the position and the radius in the said center of circle.

8. the method for claim 1, wherein said information definition be stored at least one thermal region in the said key frame of video of said first file, one of them key frame is a start frame, a key frame is an end frame, and wherein said thermal region at least one frame between start frame and end frame is not defined.

9. the method for claim 8, wherein said key frame comprises at least one the middle key frame between said start frame and the end frame, wherein said thermal region has at least a frame not to be defined at said start frame with in the middle of at least one between the key frame, and is not defined in said thermal region some frames between said at least one middle key frame and end frame.

10. the method for claim 9, wherein said key frame comprises the first and second middle key frames between said start frame and end frame, and wherein said thermal region has at least a frame not to be defined between the key frame in the middle of said first and the said second middle key frame.

11. the method for claim 8 also comprises the following steps:

Input according to the user identifies said key frame; And

Input according to the user produces the said information that defines a thermal region at least.

12. the method for claim 1 also comprises following step:

According to user's input, said thermal region is connected with chained command at least; And

In said second file, represent the contact between said thermal region and said at least one chained command.

13. the method for claim 12, the contact between wherein said thermal region and at least one the chain order is to represent by the data organization that is stored in said second file.

14. the method for claim 13, the wherein said data that are stored in second file comprise the clauses and subclauses of said thermal region of sign and corresponding chained command.

15. the method for claim 1 is further comprising the steps of:

Object identifier is composed to the target corresponding to said thermal region;

Input according to the user connects said thermal region with object identifier;

In said second file, represent the contact between said thermal region and the said object identifier.

16. the method for claim 15, the contact between wherein said thermal region and the described object identifier are to represent by the data organization that is stored in said second file.

17. the method for claim 16, the wherein said data that are stored in described second file comprise the clauses and subclauses of said thermal region of sign and corresponding object identifier.

18. the method for claim 15 also comprises the following steps:

Input according to the user connects chained command at least with object identifier, thereby said at least one chained command is connected with said thermal region; And

In said second file, represent the contact between said chained command and the said object identifier.

19. the method for claim 18, the contact between wherein said chained command and the said object identifier are to represent by the data organization that is stored in said second file.

20. the method for claim 19, the wherein said data that are stored in second file comprise the clauses and subclauses that identify said object identifier, corresponding chained command and corresponding thermal region.

21. be used for representing the method for hyperlink in the frame of video, the method comprising the steps of:

Information at least one thermal region of definition in the key frame of said video is stored, one of them key frame is a start frame, a key frame is an end frame, and wherein said thermal region is not defined at least one frame between said start frame and end frame.

22. the method for claim 21: wherein said key frame comprises at least one middle key frame between said start frame and the end frame, wherein said thermal region is not defined at least one frame between the key frame at said start frame with in the middle of at least one, and is not defined in said thermal region some frames between said at least one middle key frame and described end frame.

23. the method for claim 22, wherein said key frame comprises the middle key frame of first and second between said start frame and the end frame, and wherein said thermal region is not defined at least one frame between the key frame in the middle of the key frame and said second in the middle of said first.

24. the method for claim 21 also comprises the following steps:

Input according to the user identifies said key frame; And

Produce the information of said at least one thermal region of definition according to user's input.

25. the method for claim 21, wherein said video is encoded with standard format.

One of 26. the method for claim 25, wherein said standard format is AVI, MPEG, and original YUV, and original RGB.

27. the method for claim 21, wherein said information, in said a lot of key frame of video, the border of said at least one thermal region of definition.

28. the method for claim 27, wherein said at least one thermal region is a polygon, and it comprises a lot of summits, and wherein the border of said at least one thermal region is defined by said polygonal vertex position.

29. the method for claim 28, wherein said polygon is to have left upper apex, upper right summit, and the rectangle on summit, lower-left and summit, bottom right, and wherein said square boundary is defined by said left upper apex and bottom right vertex position.

30. the method for claim 27, wherein said at least one thermal region are circles with the center of circle and radius, and the border of said thermal region is defined by the position and the radius in the said center of circle.

31. the method for claim 21 also comprises following step:

According to user's input, be said thermal region that chained command connects at least together; And

In said information, represent the contact between said thermal region and said at least one chain order.

32. the method for claim 31, the contact between wherein said thermal region and at least one the chain order is to represent by the tissue of said information.

33. the method for claim 32, wherein said information comprise the clauses and subclauses of said thermal region of sign and corresponding chained command.

34. the method for claim 21 also comprises the following steps:

Object identifier is composed to corresponding to said thermal region target;

Input according to the user connects said thermal region with object identifier;

In said information, represent the contact between said thermal region and the said object identifier.

35. the method for claim 34, the contact between wherein said thermal region and the said object identifier is to represent by the tissue of said information.

36. the method for claim 35, wherein said information comprise the clauses and subclauses of said thermal region of sign and corresponding object identifier.

37. the method for claim 34 also comprises the following steps:

According to user's input, being that chained command connects with object identifier at least, thereby at least one chained command is connected with said thermal region; And

In said information, represent the contact between said chained command and the said object identifier.

38. the method for claim 37, the contact between wherein said chained command and the said object identifier is to represent by the tissue of said information.

39. the method for claim 38, wherein said information comprises the clauses and subclauses that identify said object identifier, corresponding chained command and corresponding thermal region.

40. the method for claim 21, wherein said video is stored in first file, and said information is stored in second file that is different from said first file.

41. the method for claim 21, wherein said video and information are stored in the public documents.

42., comprise the following steps: hyperlink and the integrated method of frame of video

First file of stores video frames is provided;

Provide and be different from first file, second file of at least one thermal region information in the said frame of video of area definition;

Select at least one frame of video to be used for showing;

According to the rules, said second file is included in the information of definition one thermal region in the frame of being chosen, and sign reaches corresponding to a lot of pixels of the selected frame of said thermal region

The value of color of the pixel that modification is identified, thus to said thermal region in the said frame make the mark oneself.

43. the method for claim 24, wherein said video is encoded with standard format.

44. the method for claim 43, wherein said standard format is AVI, MPEG, one of original YUV and original RGB.

45. the method for claim 42, wherein said information in said a lot of frame of video, defines the border of at least one hot-zone; And middle sign comprises the step of the pixel of determining selected frame corresponding to the step of a lot of pixels of the selected frame of said thermal region, and this pixel is arranged in the boundary vicinity of said at least one thermal region of selected frame.

46. the method for claim 45, wherein at least one thermal region is a polygon, and it comprises a lot of summits, and wherein the border of said at least one thermal region is defined by said polygon vertex position; And

Wherein identify the step corresponding to a lot of pixels of the selected frame of said thermal region, comprise the step of the pixel of determining selected frame, this pixel is positioned near the line that links said summit.

47. the method for claim 46, wherein said polygon is to have left upper apex, upper right summit, and the rectangle on summit, lower-left and summit, bottom right, and wherein said square boundary is defined by the position on said left upper apex and summit, bottom right; And

Wherein identify the step corresponding to a lot of pixels of the selected frame of said thermal region, comprise the step of the pixel of determining selected frame, this pixel is positioned near the said square boundary.

48. the method for claim 45, wherein said at least one thermal region are circles with the center of circle and radius, and the border of said thermal region is defined by the position and the radius in the said center of circle; And

Wherein identify the step that comprises the pixel of determining selected frame corresponding to the step of a lot of pixels of the selected frame of said thermal region, this pixel is positioned at the boundary vicinity of said circle.

49. the method for claim 42, information definition wherein be stored at least one thermal region in the said key frame of video in said first file, one of them key frame is a start frame, a key frame is an end frame, and is not defined in wherein said thermal region at least one frame between start frame and end frame.

50. the method for claim 49, wherein said key frame comprises between said start frame and the end frame key frame in the middle of at least one: wherein said thermal region has at least in the frame between the key frame at said start frame with in the middle of at least one and is not defined, and said thermal region is not defined in some frames between key frame and the end frame in the middle of at least one said.

51. the method for claim 50, wherein said key frame comprises the middle key frame of first and second between said start frame and the end frame, and wherein said thermal region is not defined at least one frame between the key frame in the middle of the key frame and second in the middle of said first.

52. the method for claim 49 determines that wherein whether said second file comprises in the step of thermal region information in the selected frame of definition, comprises determining whether the step of selected framing bit between key frame; And

The step that wherein identifies corresponding to a lot of pixels of the selected frame of said thermal region comprises the following steps:

According to the position of thermal region in said key frame, the position of the said thermal region of interpolation in selecteed frame, and

According to the position of the thermal region that is inserted in said selected frame, sign is corresponding to the pixel of the thermal region of said selected frame.

53. the method for claim 42 also comprises following step:

Storage is with the chained command of said at least one thermal region contact; And

According to said at least one thermal region that the user selects, carry out said chained command.

54. hyperlink and the integrated method of frame of video are comprised the following steps:

Frame of video is provided;

Be provided at the information of at least one thermal region that defines in the said key frame of video, one of them key frame is a start frame, and a key frame is an end frame, and wherein said thermal region is not defined at least one frame between said start frame and end frame;

Select at least one frame of video to be used for showing;

Said according to the rules information defines a thermal region in selected frame, sign reaches the value of color of revising the pixel that is identified, thereby the said thermal region in the said frame is made mark corresponding to a lot of pixels of the selected frame of said thermal region.

55. the method for claim 54, wherein said key frame comprises at least one the middle key frame between said start frame and the end frame, wherein said thermal region is not defined at least one frame between the key frame at said start frame with in the middle of at least one, and is not defined in said thermal region some frames between said at least one middle key and end frame.

56. the method for claim 55, wherein said key frame comprises the middle key frame of first and second between said start frame and the end frame, and wherein said thermal region is not defined at least one frame between the key frame in the middle of the key frame and the 2nd in the middle of said first.

57. the method for claim 54 determines that wherein whether said second file comprises in the step of hot-zone information in the selected frame of definition, comprises determining whether the step of selected framing bit between key frame; And

The step that wherein identifies corresponding to a lot of pixels of the selected frame of said thermal region comprises following step:

According to the position of thermal region in said key frame, the position of the said thermal region of interpolation in selecteed frame, and

According to the position of the thermal region that is inserted in the said selected frame, sign is corresponding to the pixel of the thermal region of said selected frame.

58. the method for claim 54, wherein said video is encoded with standard format.

59. the method for claim 58, wherein said standard format is AVI, MPEG, one of original YUV and original RGB.

60. the method for claim 54, wherein said information, in said a lot of key frame of video, the border of said at least one thermal region of definition; And

Wherein identify the step that comprises the pixel of determining selected frame corresponding to the step of a lot of pixels of the selected frame of said thermal region, this pixel is arranged in the boundary vicinity of said at least one thermal region of selected frame.

61. the method for claim 60, wherein at least one thermal region is a polygon, and it comprises a lot of summits, and wherein the border of said at least one thermal region is defined by the position on said polygonal summit; And

Wherein identify the step that comprises the pixel of determining selected frame corresponding to the step of a lot of pixels of the selected frame of said thermal region, this pixel is positioned near the straight line that links said summit.

62. the method for claim 61, wherein said polygon is to have left upper apex, upper right summit, and the rectangle on summit, lower-left and summit, bottom right, and wherein said square boundary is defined by said left upper apex and bottom right vertex position; And

Wherein identify the step that comprises the pixel of determining selected frame corresponding to the step of a lot of pixels of the selected frame of said thermal region, this pixel is positioned near the said square boundary.

63. the method for claim 60, wherein said at least one thermal region are circles with the center of circle and radius, and the border of said thermal region is defined by the position and the radius in the said center of circle; And

Wherein identify the step that comprises the pixel of determining selected frame corresponding to the step of a lot of pixels of the selected frame of said thermal region, this pixel is positioned at the boundary vicinity of said circle.

64. the method for claim 54 also comprises the following steps:

Storage is with the chained command of said at least one thermal region contact; And

According to said at least one thermal region that the user selects, carry out said chained command.

65. the method for claim 54, wherein said video is stored in first file, and said information stores is in second file different with first file.

66. the method for claim 54, wherein said video and information are stored in the public documents.

Images