US10386993B2 - Technique for searching and viewing video material - Google Patents

Technique for searching and viewing video material Download PDF

Info

Publication number
US10386993B2
US10386993B2 US14/524,770 US201414524770A US10386993B2 US 10386993 B2 US10386993 B2 US 10386993B2 US 201414524770 A US201414524770 A US 201414524770A US 10386993 B2 US10386993 B2 US 10386993B2
Authority
US
United States
Prior art keywords
frames
thumbnail
frame
video
video file
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US14/524,770
Other versions
US20150153919A1 (en
Inventor
Justin Frank Matejka
George Fitzmaurice
Tovi Grossman
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Autodesk Inc
Original Assignee
Autodesk Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Autodesk Inc filed Critical Autodesk Inc
Priority to US14/524,770 priority Critical patent/US10386993B2/en
Assigned to AUTODESK, INC. reassignment AUTODESK, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GROSSMAN, TOVI, MATEJKA, JUSTIN FRANK, FITZMAURICE, GEORGE
Publication of US20150153919A1 publication Critical patent/US20150153919A1/en
Application granted granted Critical
Publication of US10386993B2 publication Critical patent/US10386993B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482Interaction with lists of selectable items, e.g. menus
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/32Monitoring with visual or acoustical indication of the functioning of the machine
    • G06F11/323Visualisation of programs or trace data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/74Browsing; Visualisation therefor
    • G06F16/743Browsing; Visualisation therefor a collection of video files or sequences
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/93Document management systems
    • G06F17/509
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/102Programmed access in sequence to addressed parts of tracks of operating record carriers
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/34Indicating arrangements 
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/12Discovery or management of network topologies
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14Network analysis or design
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/22Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/387Composing, repositioning or otherwise geometrically modifying originals
    • H04N1/3871Composing, repositioning or otherwise geometrically modifying originals the composed originals being of different kinds, e.g. low- and high-resolution originals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/387Composing, repositioning or otherwise geometrically modifying originals
    • H04N1/393Enlarging or reducing
    • H04N1/3935Enlarging or reducing with modification of image resolution, i.e. determining the values of picture elements at new relative positions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/472End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4728End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/482End-user interface for program selection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/482End-user interface for program selection
    • H04N21/4821End-user interface for program selection using a grid, e.g. sorted out by channel and broadcast time

Definitions

  • the present invention generally relates to computer graphics and, more specifically, to a technique for searching and viewing video material.
  • Video of movies and television shows are made up of multiple scenes. Often a viewer may want to watch a specific scene that occurs in the video. Current approaches to searching for a particular scene in a computer-based video player can be problematic. Locating a target scene can be difficult because the viewer may not know where the scene is located chronologically in the video. Considerable time and effort may be expended in searching through a video.
  • a viewer may simply play the video from the beginning and identify the scene whenever it occurs. This may be a time consuming approach, particularly when the target scene occurs late in the video. Alternatively the viewer may manipulate a timeline control, often available in video players, to search through the video. Other, more refined approaches provide a single thumbnail that displays a single still scene of the video and may be updated continuously with a timeline slider control.
  • network latency prevents a video player from updating the displayed frame until the seeking operation has ended, that is, until the slider stops moving.
  • This approach can work well if the user knows an exact timestamp of the portion of video to be reviewed. In the absence of this knowledge, however, the process becomes haphazard. Further, as only a single frame is visible at one time, large portions of the video may be missed due to insufficient resolution provided by the timeline control. In addition, the video must still be viewed sequentially, and the user may expend considerable effort to find the required content.
  • One embodiment of the invention includes a computer-implemented method for identifying a target scene within a video file including obtaining a first set of frames that reflect a first portion of the video file, obtaining a second set of frames that reflect a second portion of the video file, generating a graphical user interface (GUI) that displays a timeline control, causing the GUI to display a first frame included in the first set of frames and a first frame included in the second set of frames when the timeline control resides at a first position, and causing the GUI to display a second frame included in the first set of frames and a second frame included in the second set of frames when the timeline control resides at a second position.
  • GUI graphical user interface
  • One advantage of the disclosed technique is that the configurable interactive grid enables the user to rapidly locate a target scene within a video of unspecified length by scrubbing multiple video thumbnails simultaneously.
  • FIG. 1 illustrates a system configured to implement one or more aspects of the present invention
  • FIG. 2 is a screenshot depicting the first frame of a video, according to one embodiment of the present invention.
  • FIG. 3 is a screenshot depicting a grid of video thumbnails, according to one embodiment of the present invention.
  • FIG. 4 is a screenshot depicting a grid of video thumbnails scanned to subsequent frames by operation of a timeline control, according to one embodiment of the present invention
  • FIG. 5 is a screenshot depicting the selection of a target scene in a grid of video thumbnails scanned to subsequent frames, according to one embodiment of the present invention
  • FIG. 6 is a flow diagram of method steps for configuring a video processing engine to search through a video to find a desired scene, according to one embodiment of the present invention
  • FIG. 7 is a conceptual diagram showing various thumbnail densities, according to one embodiment of the present invention.
  • FIG. 8 is a flow diagram of method steps for preparation of video thumbnails, according to one embodiment of the present invention.
  • FIG. 1 illustrates a system 100 configured to implement one or more aspects of the present invention.
  • the system includes client machine 110 coupled to server machine 120 by network 140 .
  • Server machine 120 is also coupled to database 150 .
  • Client machine 110 represents a client computing device configured to interact with server machine 120 in order to produce video content that enables rapid searching of the video content to locate user specified target scenes.
  • Client machine 110 includes processing unit 112 coupled to input/output (I/O) devices 114 and to memory 103 .
  • Processing unit 112 may be a central processing unit (CPU), a graphics processing unit (GPU), or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU. In general, processing unit 112 may be any technically feasible hardware unit capable of processing data and/or executing software applications.
  • I/O devices 114 are also coupled to memory 103 and may include devices capable of receiving input, devices capable of producing output, as well as devices capable communicating via network 140 .
  • Client machine 110 is further coupled to display device 170 , keyboard 180 , and mouse 190 , which afford the end-user access to client machine 110 .
  • Memory 103 further includes video processing engine 116 and local video storage 118 .
  • Server machine 120 is a computing device that may reside within a data center remote from client machine 110 .
  • Server machine 120 includes processing unit 122 coupled to I/O devices 124 and to memory unit 126 .
  • Processing unit 122 may be a central processing unit (CPU), a graphics processing unit (GPU), or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU.
  • I/O devices 124 are also coupled to memory unit 126 and may include devices capable of receiving input such as a keyboard, mouse, or other input elements, devices capable of producing output such as a monitor or printer, as well as elements that enable communication via network 140 .
  • Memory 126 further includes video formatting engine 128 and video thumbnail storage 130 .
  • Server machine 120 is also coupled to database 150 , which includes video source files 160 .
  • Video source files 160 may be a collection of movies, television content, personal video content, or any other types of video files.
  • server machine 120 sequentially transmits video data in a frame-by-frame fashion, termed streaming.
  • client machine 110 initiates a request to server machine 120 to invoke a specific video from among video source files 160 .
  • Video formatting engine 128 when invoked by processing unit 112 , accesses the specific video within video source files 160 and modifies and reworks the content of the specific video into a complement of thumbnail video segments, where each thumbnail video segment is a unique subset of the specific video. Video formatting engine 128 then arranges the complement of thumbnails into a grid array and stores the grid array of thumbnail video segments in video thumbnail storage 130 in order to be available for streaming to client machine 110 .
  • Video formatting engine 128 then streams the specific video to client machine 110 via network 140 . If the target scene is not immediately evident upon displaying the specific video, the user may then invoke the thumbnail grid view, as described below in conjunction with FIG. 3 . Video formatting engine 128 then streams the thumbnail grid view to client machine 110 .
  • Video processing engine 116 processes the data provided by video formatting engine 128 , creates viewable data structures, and displays the grid of thumbnail video segments via display device 170 or other elements included in I/O devices 114 . Each thumbnail within the grid of thumbnails then displays active video of each unique video segment of the selected video.
  • the user may then examine the static thumbnail grid to identify a target scene. If the target scene is not evident in the grid of thumbnails, the user may manipulate a timeline control, as described below in conjunction with FIG. 3 , to scrub all thumbnails simultaneously. When the target scene is identified in a particular thumbnail, the user may select the particular thumbnail to view the scene in full-screen mode.
  • the client machine operates in a stand-alone mode with no server interaction.
  • a video file is stored locally in local video storage 118 .
  • Video processing engine 116 modifies and reworks the content of the video file into a complement of thumbnail video segments, where each thumbnail video segment is a unique subset of the selected video.
  • Video processing engine 116 then arranges the thumbnail video segments into a grid array and stores the grid array of thumbnail video segments in local video storage 118 .
  • Video processing engine 116 then displays the selected video via display device 170 or other elements included in I/O devices 114 . If the target scene is not immediately evident upon displaying the video that the end-user specifies from among video source files 160 , the user may then invoke the thumbnail grid view. Video processing engine 116 then displays the thumbnail grid view via display device 170 or other elements included in I/O devices 114 . As described above, the user may then examine the static thumbnail grid, and, if the target scene is not evident in the grid of thumbnails, manipulate the timeline control to scan all thumbnails simultaneously. When the target scene is identified in a particular thumbnail, the user may select that thumbnail to view the scene in full screen mode.
  • FIG. 2 is a screenshot 200 depicting the first frame of a video, according to one embodiment of the present invention.
  • Screenshot 200 may be captured from display device 170 .
  • screenshot 200 includes a video display area 210 , depicted as displaying the first frame of a video in full screen mode.
  • Screenshot 200 further includes a timeline control 220 , depicted as a conventional slider control.
  • Timeline control 220 affords scanning through a video by moving the slider, where the slider in the left-most position, as shown, queues the first frame of the video, and moving further to the right queues frames successively further towards the chronological end of the video.
  • the action of scanning, or synonymously, scrubbing, through a video is a chronological movement through the frames of the video in response to the changing position of the timeline control.
  • Scenes, or component frames within scenes, may be identified by their relative chronology within the totality of the video.
  • FIGS. 3 through 5 illustrate the search activity by depicting a progression of screenshots.
  • the screenshots proceed from the beginning of the video, illustrated in FIG. 3 , to an intermediate point, illustrated in FIG. 4 , to a final point, illustrated in FIG. 5 , at which a scene is identified and a final selection is made.
  • FIG. 3 is a screenshot 300 depicting a grid of video thumbnails, according to one embodiment of the present invention.
  • Screenshot 300 may be captured from display device 170 .
  • Screenshot 300 includes a grid of video thumbnail segments 310 ( 0 ) through 310 ( 19 ).
  • screenshot 300 depicts a five-by-five grid, persons skilled in the art will understand that a grid of any dimension or symmetry is within the scope of the present invention.
  • the dimension or symmetry of the grid of static video thumbnails may be selected based on the size, resolution, and aspect ratio of display device 170 or other elements included in I/O devices 114 ,
  • the grid of video thumbnail segments 310 illustrates an example of a video that includes one thousand frames divided into twenty-five equal segments of forty frames each.
  • Video thumbnail segment 310 ( 0 ) depicts the display of frame one, which is the first frame of the first segment, where the first segment includes frames one through forty.
  • Video thumbnail segment 310 ( 1 ) depicts the display of frame forty-one, which is the first frame of the second segment, where the second segment includes frames forty-one through eighty.
  • Successive video thumbnail segments follow displaying successive groups of frames until video thumbnail segment 310 ( 19 ) depicts frame nine hundred sixty one, which is the first frame of the twenty-fifth and final segment, where the final segment includes frames nine hundred sixty one through one thousand.
  • Screenshot 300 further includes a timeline control 320 .
  • Timeline control 320 is shown in the left-most position so that each thumbnail segment displays the first frame of each unique segment of the video.
  • Screenshot 300 depicts a static snapshot of thumbnails 310 based on the position of timeline control 320 .
  • video processing engine 116 may scroll the video starting at the chronological point determined by the position of timeline control 320 .
  • FIG. 4 is a screenshot 400 depicting the grid of video thumbnails scanned to subsequent frames by operation of the timeline control slider, according to one embodiment of the present invention.
  • Screenshot 400 may be captured from display device 170 .
  • screenshot 400 includes a grid of video thumbnail segments 410 ( 0 ) through 410 ( 19 ).
  • the grid of video thumbnail segments 410 shown illustrates the example of a video that includes one thousand frames, similar to that shown in FIG. 3 .
  • Video thumbnail segment 410 ( 0 ) depicts the display of frame twelve, which is the twelfth frame of the first segment, where the first segment includes frames one through forty.
  • Video thumbnail segment 410 ( 1 ) depicts the display of frame fifty-two, which is the twelfth frame of the second segment, where the second segment includes frames forty-one through eighty.
  • Successive video thumbnail segments follow, displaying successive groups of frames until video thumbnail segment 410 ( 19 ) depicts frame nine hundred seventy two, which is the twelfth frame of the twenty-fifth and final segment, where the final segment includes frames nine hundred sixty one through one thousand.
  • Screenshot 400 further includes a timeline control 420 .
  • the end-user moves timeline control 420 to a position, shown in the example as left of center, so that each thumbnail 410 scans to a further time stamp within the unique segment associated with the segment.
  • each segment then displays the twelfth frame of each unique segment content.
  • FIG. 4 then illustrates a scenario in which video processing engine 116 has scanned to a point chronologically further into the video but has not yet identified the target scene. Further scanning is needed to locate the target.
  • FIG. 5 is a screenshot 500 depicting the selection of a target scene in a grid of video thumbnails scanned to subsequent frames, according to one embodiment of the present invention.
  • Screenshot 500 may be captured from display device 170 .
  • screenshot 500 includes the grid of video thumbnail segments 510 ( 0 ) through 510 ( 19 ).
  • the grid of video thumbnail segments 510 shown illustrates the example of a video that includes one thousand frames, similar to that shown in FIGS. 3 and 4 .
  • Video thumbnail segment 510 ( 0 ) depicts the display of frame thirty, which is the thirtieth frame of the first segment, where the first segment includes frames one through forty.
  • Video thumbnail segment 510 ( 1 ) depicts the display of frame seventy, which is the seventieth frame of the second segment, where the second segment includes frames forty-one through eighty.
  • Successive video thumbnail segments follow displaying successive groups of frames until video thumbnail segment 510 ( 19 ) depicts frame nine hundred ninety, which is the nine hundred and ninetieth frame of the twenty-fifth and final segment, where the final segment includes frames nine hundred sixty one through one thousand.
  • Screenshot 500 further includes a timeline control 520 .
  • the end-user moves timeline control 520 to a position, shown in the example as right of center, so that each thumbnail scans to a further time stamp within the unique segment associated with the segment.
  • each segment then displays the thirtieth frame of each unique segment content.
  • screenshot 500 further illustrates, the end-user moves a curser 530 and selects video thumbnail segment 510 ( 17 ). Selecting segment 510 ( 17 ) causes the display to revert to the full screen view, as described above in conjunction with FIG. 2 , where video processing engine 116 displays the scene illustrated in video thumbnail segment 510 ( 17 ) in full-screen mode via display device 170 or other elements included in I/O devices 114 .
  • FIG. 6 is a flow diagram of method steps for configuring a video processing engine 116 to search through a video to find a desired scene, according to one embodiment of the present invention.
  • the method steps are described in conjunction with the system of FIG. 1 , persons skilled in the art will understand that any system configured to perform the method steps, in any order, is within the scope of the present invention.
  • a method 600 begins at step 602 , where video processing engine 116 , when invoked by processing unit 112 , initiates the search for a target scene by accessing a specific video.
  • the specific video may reside locally in local video storage 118 or may reside remotely within video source files 160 .
  • video processing engine 116 invokes the video search grid array as illustrated above in conjunction with FIG. 3 .
  • video processing engine 116 examines the selected video via end-user manipulation of the timeline control 420 to scrub through the multiple scenes as illustrated above in conjunction with FIG. 4 .
  • step 608 video processing engine 116 determines whether the desired target scene is among the video thumbnails included in the grid of video thumbnails. If video processing engine 116 determines that the desired target scene is not included in the grid of video thumbnails, then method 600 proceeds to step 610 . At step 610 , video processing engine 116 responds to timeline control 420 input by simultaneously scanning through each unique scene of each video thumbnail 410 , as illustrated above in conjunction with FIG. 4 . The method 600 then returns to step 606 .
  • step 608 video processing engine 116 determines that the desired target scene is among the video thumbnails displayed, then method 600 proceeds to step 612 where video processing engine 116 displays the selected target scene in full-screen mode via display device 170 or other elements included in I/O devices 114 , as described above in conjunction with FIG. 2 . The method then ends.
  • video processing engines 116 allows an end-user to quickly and efficiently scrub a video in order to locate a desired target scene.
  • FIG. 7 is a conceptual diagram showing various thumbnail densities, according to one embodiment of the present invention.
  • thumbnail segments 310 , 410 , and 510 each include a complement of frames of the subject video.
  • Thumbnail segments 310 , 410 , and 510 may include all sequential frames of each segment, so that the end-user may scrub through the multiple scenes and view all possible frames.
  • thumbnail segments 310 , 410 , and 510 may instead include a reduced number of the total frames within each segment. In this manner, video processing engine 116 may afford more efficient streaming operation while still providing the end-user an effective scrubbing tool.
  • thumbnail complement 710 illustrates an example of a typical thumbnail segment composed of 100 frames where thumbnail complement 710 includes all of those 100 frames. Thumbnail complement 710 then may be considered to include a thumbnail density of 100 percent. Thumbnail segments 310 , 410 , and 510 may be configured with a 100 percent thumbnail density.
  • a thumbnail complement 720 illustrates another example of a typical thumbnail segment composed of 100 frames where thumbnail complement 720 only includes every fourth frame. Thumbnail complement 720 then may be considered to include a thumbnail density of 25 percent. Thumbnail segments 310 , 410 , and 510 may be configured with a 25 percent thumbnail density.
  • a thumbnail complement 730 illustrates an example of the same typical thumbnail segment 310 composed of 100 frames where thumbnail complement 730 only includes every tenth frame. Thumbnail complement 730 then may be considered to include a thumbnail density of 10 percent. Thumbnail segments 310 , 410 , and 510 may be configured with a 10 percent thumbnail density.
  • a thumbnail complement 740 illustrates an example of the same typical thumbnail segment 310 composed of 100 frames where thumbnail complement 740 includes only two frames. Thumbnail complement 740 then may be considered to include a thumbnail density of 2 percent. Thumbnail segments 310 , 410 , and 510 may be configured with a 2 percent thumbnail density.
  • FIG. 7 depicts exemplary scenes with specific frame densities and specific first frames within each thumbnail 310 , persons skilled in the art will understand that a scene composed of any number of frames with any density reduction and any starting scene, is within the scope of the present invention.
  • FIG. 8 is a flow diagram of method steps for preparation of video thumbnails, according to one embodiment of the present invention. Although the method steps are described in conjunction with the system of FIG. 1 , persons skilled in the art will understand that any system configured to perform the method steps, in any order, is within the scope of the present invention.
  • a method 800 begins at step 802 , where video formatting engine 128 , when invoked by processing unit 122 , configures the dimension of the thumbnail grid array, as described in detail above in conjunction with FIGS. 3 through 5 .
  • video formatting engine 128 determines the distribution among the video thumbnail grid array 300 of the content of the video that the end-user specifies from among video source files 160 .
  • One example of a distribution may be to assign an equal number of frames to each video thumbnail 310 , as illustrated above in conjunction with FIG. 3 .
  • Another example of a distribution may be to assign preexisting chapters within the video to each video thumbnail 310 . Persons skilled in the art will understand that any distribution of frames, sequentially assigned to successive video thumbnails 310 , is within the scope of the present invention.
  • video formatting engine 128 determines the thumbnail density as described above in conjunction with FIG. 7 .
  • video formatting engine 128 creates the video thumbnail grid array.
  • video formatting engine 128 streams the thumbnail grid array to video processing engine 116 via network 140 . The method then ends.
  • a video processing engine is configured to generate a graphical user interface (GUI) that allows an end-user of the video processing engine to select a specific video and search through the specific video to detect a desired target scene.
  • GUI graphical user interface
  • the video processing engine provides a grid array of video thumbnails that are configured to each display a segment of the video so that multiple scenes may be visually scanned simultaneously.
  • the end-user may launch the content of the video thumbnail in full-screen mode to verify that the scene is in fact the desired target scene.
  • each thumbnail includes a reduced number of frames, that is, a reduced frame density, allowing the viewer to more efficiently view the essential content of a scene while minimizing the time required to stream the content.
  • the ability to rapidly select a target scene from among multiple thumbnails affords scrubbing in a manner that is efficient in the time and effort required.
  • One embodiment of the invention may be implemented as a program product for use with a computer system.
  • the program(s) of the program product define functions of the embodiments (including the methods described herein) and can be contained on a variety of computer-readable storage media.
  • Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, flash memory, ROM chips or any type of solid-state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., floppy disks within a diskette drive or hard-disk drive or any type of solid-state random-access semiconductor memory) on which alterable information is stored.
  • non-writable storage media e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, flash memory, ROM

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Quality & Reliability (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

A video processing engine is configured to generate a graphical user interface (GUI) that allows an end-user of the video processing engine to select a specific video and search through the specific video to detect a desired target scene. The video processing engine provides a grid array of video thumbnails that are configured to each display a segment of the video so that multiple scenes may be visually scanned simultaneously. When the end-user identifies a scene within a video thumbnail that may be the desired target scene, the end-user may launch the content of the video thumbnail in full-screen mode to verify that the scene is in fact the desired target scene. An advantage of the approach described herein is that the video processing engine provides a sampled overview of the video in its entirety, thus enabling the end-user to more effectively scrub the video for the desired target scene.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims the benefit of U.S. provisional patent application titled “Techniques for Viewing and Searching Different Types of Content,” filed on Dec. 3, 2013 and having Ser. No. 61/911,301. The subject matter of this related application is hereby incorporated herein by reference.
BACKGROUND OF THE INVENTION
Field of the Invention
The present invention generally relates to computer graphics and, more specifically, to a technique for searching and viewing video material.
Description of the Related Art
Videos of movies and television shows are made up of multiple scenes. Often a viewer may want to watch a specific scene that occurs in the video. Current approaches to searching for a particular scene in a computer-based video player can be problematic. Locating a target scene can be difficult because the viewer may not know where the scene is located chronologically in the video. Considerable time and effort may be expended in searching through a video.
To locate a specific scene, a viewer may simply play the video from the beginning and identify the scene whenever it occurs. This may be a time consuming approach, particularly when the target scene occurs late in the video. Alternatively the viewer may manipulate a timeline control, often available in video players, to search through the video. Other, more refined approaches provide a single thumbnail that displays a single still scene of the video and may be updated continuously with a timeline slider control. Typically, during the search operation, referred to as scrubbing, network latency prevents a video player from updating the displayed frame until the seeking operation has ended, that is, until the slider stops moving. This approach can work well if the user knows an exact timestamp of the portion of video to be reviewed. In the absence of this knowledge, however, the process becomes haphazard. Further, as only a single frame is visible at one time, large portions of the video may be missed due to insufficient resolution provided by the timeline control. In addition, the video must still be viewed sequentially, and the user may expend considerable effort to find the required content.
As the foregoing illustrates, what is needed in the art is a more effective way to search for and view a target scene in video-oriented material.
SUMMARY OF THE INVENTION
One embodiment of the invention includes a computer-implemented method for identifying a target scene within a video file including obtaining a first set of frames that reflect a first portion of the video file, obtaining a second set of frames that reflect a second portion of the video file, generating a graphical user interface (GUI) that displays a timeline control, causing the GUI to display a first frame included in the first set of frames and a first frame included in the second set of frames when the timeline control resides at a first position, and causing the GUI to display a second frame included in the first set of frames and a second frame included in the second set of frames when the timeline control resides at a second position.
One advantage of the disclosed technique is that the configurable interactive grid enables the user to rapidly locate a target scene within a video of unspecified length by scrubbing multiple video thumbnails simultaneously.
BRIEF DESCRIPTION OF THE DRAWINGS
So that the manner in which the above recited features of the invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments.
FIG. 1 illustrates a system configured to implement one or more aspects of the present invention;
FIG. 2 is a screenshot depicting the first frame of a video, according to one embodiment of the present invention;
FIG. 3 is a screenshot depicting a grid of video thumbnails, according to one embodiment of the present invention;
FIG. 4 is a screenshot depicting a grid of video thumbnails scanned to subsequent frames by operation of a timeline control, according to one embodiment of the present invention;
FIG. 5 is a screenshot depicting the selection of a target scene in a grid of video thumbnails scanned to subsequent frames, according to one embodiment of the present invention;
FIG. 6 is a flow diagram of method steps for configuring a video processing engine to search through a video to find a desired scene, according to one embodiment of the present invention;
FIG. 7 is a conceptual diagram showing various thumbnail densities, according to one embodiment of the present invention; and
FIG. 8 is a flow diagram of method steps for preparation of video thumbnails, according to one embodiment of the present invention.
DETAILED DESCRIPTION
In the following description, numerous specific details are set forth to provide a more thorough understanding of the present invention. However, it will be apparent to one of skill in the art that the present invention may be practiced without one or more of these specific details. In other instances, well-known features have not been described in order to avoid obscuring the present invention.
System Overview
FIG. 1 illustrates a system 100 configured to implement one or more aspects of the present invention. As shown, the system includes client machine 110 coupled to server machine 120 by network 140. Server machine 120 is also coupled to database 150. Client machine 110 represents a client computing device configured to interact with server machine 120 in order to produce video content that enables rapid searching of the video content to locate user specified target scenes.
Client machine 110 includes processing unit 112 coupled to input/output (I/O) devices 114 and to memory 103. Processing unit 112 may be a central processing unit (CPU), a graphics processing unit (GPU), or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU. In general, processing unit 112 may be any technically feasible hardware unit capable of processing data and/or executing software applications. I/O devices 114 are also coupled to memory 103 and may include devices capable of receiving input, devices capable of producing output, as well as devices capable communicating via network 140. Client machine 110 is further coupled to display device 170, keyboard 180, and mouse 190, which afford the end-user access to client machine 110. Memory 103 further includes video processing engine 116 and local video storage 118.
Server machine 120 is a computing device that may reside within a data center remote from client machine 110. Server machine 120 includes processing unit 122 coupled to I/O devices 124 and to memory unit 126. Processing unit 122 may be a central processing unit (CPU), a graphics processing unit (GPU), or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU. I/O devices 124 are also coupled to memory unit 126 and may include devices capable of receiving input such as a keyboard, mouse, or other input elements, devices capable of producing output such as a monitor or printer, as well as elements that enable communication via network 140. Memory 126 further includes video formatting engine 128 and video thumbnail storage 130. Server machine 120 is also coupled to database 150, which includes video source files 160. Video source files 160 may be a collection of movies, television content, personal video content, or any other types of video files.
In one embodiment, server machine 120 sequentially transmits video data in a frame-by-frame fashion, termed streaming. In the streaming mode, client machine 110 initiates a request to server machine 120 to invoke a specific video from among video source files 160.
Video formatting engine 128, when invoked by processing unit 112, accesses the specific video within video source files 160 and modifies and reworks the content of the specific video into a complement of thumbnail video segments, where each thumbnail video segment is a unique subset of the specific video. Video formatting engine 128 then arranges the complement of thumbnails into a grid array and stores the grid array of thumbnail video segments in video thumbnail storage 130 in order to be available for streaming to client machine 110.
Video formatting engine 128 then streams the specific video to client machine 110 via network 140. If the target scene is not immediately evident upon displaying the specific video, the user may then invoke the thumbnail grid view, as described below in conjunction with FIG. 3. Video formatting engine 128 then streams the thumbnail grid view to client machine 110. Video processing engine 116 processes the data provided by video formatting engine 128, creates viewable data structures, and displays the grid of thumbnail video segments via display device 170 or other elements included in I/O devices 114. Each thumbnail within the grid of thumbnails then displays active video of each unique video segment of the selected video.
The user may then examine the static thumbnail grid to identify a target scene. If the target scene is not evident in the grid of thumbnails, the user may manipulate a timeline control, as described below in conjunction with FIG. 3, to scrub all thumbnails simultaneously. When the target scene is identified in a particular thumbnail, the user may select the particular thumbnail to view the scene in full-screen mode.
In another embodiment, the client machine operates in a stand-alone mode with no server interaction. In the stand-alone mode, a video file is stored locally in local video storage 118. Video processing engine 116 modifies and reworks the content of the video file into a complement of thumbnail video segments, where each thumbnail video segment is a unique subset of the selected video. Video processing engine 116 then arranges the thumbnail video segments into a grid array and stores the grid array of thumbnail video segments in local video storage 118.
Video processing engine 116 then displays the selected video via display device 170 or other elements included in I/O devices 114. If the target scene is not immediately evident upon displaying the video that the end-user specifies from among video source files 160, the user may then invoke the thumbnail grid view. Video processing engine 116 then displays the thumbnail grid view via display device 170 or other elements included in I/O devices 114. As described above, the user may then examine the static thumbnail grid, and, if the target scene is not evident in the grid of thumbnails, manipulate the timeline control to scan all thumbnails simultaneously. When the target scene is identified in a particular thumbnail, the user may select that thumbnail to view the scene in full screen mode.
Exemplary Screenshots Illustrating a Technique for Scrubbing
FIG. 2 is a screenshot 200 depicting the first frame of a video, according to one embodiment of the present invention. Screenshot 200 may be captured from display device 170. As shown, screenshot 200 includes a video display area 210, depicted as displaying the first frame of a video in full screen mode. Screenshot 200 further includes a timeline control 220, depicted as a conventional slider control. Timeline control 220 affords scanning through a video by moving the slider, where the slider in the left-most position, as shown, queues the first frame of the video, and moving further to the right queues frames successively further towards the chronological end of the video. The action of scanning, or synonymously, scrubbing, through a video is a chronological movement through the frames of the video in response to the changing position of the timeline control. Scenes, or component frames within scenes, may be identified by their relative chronology within the totality of the video.
FIGS. 3 through 5 illustrate the search activity by depicting a progression of screenshots. The screenshots proceed from the beginning of the video, illustrated in FIG. 3, to an intermediate point, illustrated in FIG. 4, to a final point, illustrated in FIG. 5, at which a scene is identified and a final selection is made.
FIG. 3 is a screenshot 300 depicting a grid of video thumbnails, according to one embodiment of the present invention. Screenshot 300 may be captured from display device 170. Screenshot 300 includes a grid of video thumbnail segments 310(0) through 310(19). Although screenshot 300 depicts a five-by-five grid, persons skilled in the art will understand that a grid of any dimension or symmetry is within the scope of the present invention. The dimension or symmetry of the grid of static video thumbnails may be selected based on the size, resolution, and aspect ratio of display device 170 or other elements included in I/O devices 114,
The grid of video thumbnail segments 310 illustrates an example of a video that includes one thousand frames divided into twenty-five equal segments of forty frames each. Video thumbnail segment 310(0) depicts the display of frame one, which is the first frame of the first segment, where the first segment includes frames one through forty. Video thumbnail segment 310(1) depicts the display of frame forty-one, which is the first frame of the second segment, where the second segment includes frames forty-one through eighty. Successive video thumbnail segments follow displaying successive groups of frames until video thumbnail segment 310(19) depicts frame nine hundred sixty one, which is the first frame of the twenty-fifth and final segment, where the final segment includes frames nine hundred sixty one through one thousand.
Screenshot 300 further includes a timeline control 320. Timeline control 320 is shown in the left-most position so that each thumbnail segment displays the first frame of each unique segment of the video. Screenshot 300 depicts a static snapshot of thumbnails 310 based on the position of timeline control 320. In operation, video processing engine 116 may scroll the video starting at the chronological point determined by the position of timeline control 320.
FIG. 4 is a screenshot 400 depicting the grid of video thumbnails scanned to subsequent frames by operation of the timeline control slider, according to one embodiment of the present invention. Screenshot 400 may be captured from display device 170. As shown, screenshot 400 includes a grid of video thumbnail segments 410(0) through 410(19).
The grid of video thumbnail segments 410 shown illustrates the example of a video that includes one thousand frames, similar to that shown in FIG. 3. Video thumbnail segment 410(0) depicts the display of frame twelve, which is the twelfth frame of the first segment, where the first segment includes frames one through forty. Video thumbnail segment 410(1) depicts the display of frame fifty-two, which is the twelfth frame of the second segment, where the second segment includes frames forty-one through eighty. Successive video thumbnail segments follow, displaying successive groups of frames until video thumbnail segment 410(19) depicts frame nine hundred seventy two, which is the twelfth frame of the twenty-fifth and final segment, where the final segment includes frames nine hundred sixty one through one thousand.
Screenshot 400 further includes a timeline control 420. The end-user moves timeline control 420 to a position, shown in the example as left of center, so that each thumbnail 410 scans to a further time stamp within the unique segment associated with the segment. As illustrated, each segment then displays the twelfth frame of each unique segment content. FIG. 4 then illustrates a scenario in which video processing engine 116 has scanned to a point chronologically further into the video but has not yet identified the target scene. Further scanning is needed to locate the target.
FIG. 5 is a screenshot 500 depicting the selection of a target scene in a grid of video thumbnails scanned to subsequent frames, according to one embodiment of the present invention. Screenshot 500 may be captured from display device 170. As shown, screenshot 500 includes the grid of video thumbnail segments 510(0) through 510(19).
The grid of video thumbnail segments 510 shown illustrates the example of a video that includes one thousand frames, similar to that shown in FIGS. 3 and 4. Video thumbnail segment 510(0) depicts the display of frame thirty, which is the thirtieth frame of the first segment, where the first segment includes frames one through forty. Video thumbnail segment 510(1) depicts the display of frame seventy, which is the seventieth frame of the second segment, where the second segment includes frames forty-one through eighty. Successive video thumbnail segments follow displaying successive groups of frames until video thumbnail segment 510(19) depicts frame nine hundred ninety, which is the nine hundred and ninetieth frame of the twenty-fifth and final segment, where the final segment includes frames nine hundred sixty one through one thousand.
Screenshot 500 further includes a timeline control 520. The end-user moves timeline control 520 to a position, shown in the example as right of center, so that each thumbnail scans to a further time stamp within the unique segment associated with the segment. As illustrated, each segment then displays the thirtieth frame of each unique segment content.
As screenshot 500 further illustrates, the end-user moves a curser 530 and selects video thumbnail segment 510(17). Selecting segment 510(17) causes the display to revert to the full screen view, as described above in conjunction with FIG. 2, where video processing engine 116 displays the scene illustrated in video thumbnail segment 510(17) in full-screen mode via display device 170 or other elements included in I/O devices 114.
FIG. 6 is a flow diagram of method steps for configuring a video processing engine 116 to search through a video to find a desired scene, according to one embodiment of the present invention. Although the method steps are described in conjunction with the system of FIG. 1, persons skilled in the art will understand that any system configured to perform the method steps, in any order, is within the scope of the present invention.
As shown, a method 600 begins at step 602, where video processing engine 116, when invoked by processing unit 112, initiates the search for a target scene by accessing a specific video. The specific video may reside locally in local video storage 118 or may reside remotely within video source files 160. At step 604, video processing engine 116 invokes the video search grid array as illustrated above in conjunction with FIG. 3. At step 606, video processing engine 116 examines the selected video via end-user manipulation of the timeline control 420 to scrub through the multiple scenes as illustrated above in conjunction with FIG. 4.
At step 608, video processing engine 116 determines whether the desired target scene is among the video thumbnails included in the grid of video thumbnails. If video processing engine 116 determines that the desired target scene is not included in the grid of video thumbnails, then method 600 proceeds to step 610. At step 610, video processing engine 116 responds to timeline control 420 input by simultaneously scanning through each unique scene of each video thumbnail 410, as illustrated above in conjunction with FIG. 4. The method 600 then returns to step 606.
If, at step 608, video processing engine 116 determines that the desired target scene is among the video thumbnails displayed, then method 600 proceeds to step 612 where video processing engine 116 displays the selected target scene in full-screen mode via display device 170 or other elements included in I/O devices 114, as described above in conjunction with FIG. 2. The method then ends. By implementing the approach described above, video processing engines 116 allows an end-user to quickly and efficiently scrub a video in order to locate a desired target scene.
Techniques for Preparing Video Thumbnails
FIG. 7 is a conceptual diagram showing various thumbnail densities, according to one embodiment of the present invention. As described above in conjunction with FIGS. 3 through 5, thumbnail segments 310, 410, and 510 each include a complement of frames of the subject video. Thumbnail segments 310, 410, and 510 may include all sequential frames of each segment, so that the end-user may scrub through the multiple scenes and view all possible frames. Alternatively, thumbnail segments 310, 410, and 510 may instead include a reduced number of the total frames within each segment. In this manner, video processing engine 116 may afford more efficient streaming operation while still providing the end-user an effective scrubbing tool.
As shown, a thumbnail complement 710 illustrates an example of a typical thumbnail segment composed of 100 frames where thumbnail complement 710 includes all of those 100 frames. Thumbnail complement 710 then may be considered to include a thumbnail density of 100 percent. Thumbnail segments 310, 410, and 510 may be configured with a 100 percent thumbnail density.
A thumbnail complement 720 illustrates another example of a typical thumbnail segment composed of 100 frames where thumbnail complement 720 only includes every fourth frame. Thumbnail complement 720 then may be considered to include a thumbnail density of 25 percent. Thumbnail segments 310, 410, and 510 may be configured with a 25 percent thumbnail density.
A thumbnail complement 730 illustrates an example of the same typical thumbnail segment 310 composed of 100 frames where thumbnail complement 730 only includes every tenth frame. Thumbnail complement 730 then may be considered to include a thumbnail density of 10 percent. Thumbnail segments 310, 410, and 510 may be configured with a 10 percent thumbnail density.
A thumbnail complement 740 illustrates an example of the same typical thumbnail segment 310 composed of 100 frames where thumbnail complement 740 includes only two frames. Thumbnail complement 740 then may be considered to include a thumbnail density of 2 percent. Thumbnail segments 310, 410, and 510 may be configured with a 2 percent thumbnail density.
Although FIG. 7 depicts exemplary scenes with specific frame densities and specific first frames within each thumbnail 310, persons skilled in the art will understand that a scene composed of any number of frames with any density reduction and any starting scene, is within the scope of the present invention.
FIG. 8 is a flow diagram of method steps for preparation of video thumbnails, according to one embodiment of the present invention. Although the method steps are described in conjunction with the system of FIG. 1, persons skilled in the art will understand that any system configured to perform the method steps, in any order, is within the scope of the present invention.
As shown, a method 800 begins at step 802, where video formatting engine 128, when invoked by processing unit 122, configures the dimension of the thumbnail grid array, as described in detail above in conjunction with FIGS. 3 through 5. At step 804, video formatting engine 128 determines the distribution among the video thumbnail grid array 300 of the content of the video that the end-user specifies from among video source files 160. One example of a distribution may be to assign an equal number of frames to each video thumbnail 310, as illustrated above in conjunction with FIG. 3. Another example of a distribution may be to assign preexisting chapters within the video to each video thumbnail 310. Persons skilled in the art will understand that any distribution of frames, sequentially assigned to successive video thumbnails 310, is within the scope of the present invention.
At 806, video formatting engine 128 determines the thumbnail density as described above in conjunction with FIG. 7. At 808, video formatting engine 128 creates the video thumbnail grid array. At 810, video formatting engine 128 streams the thumbnail grid array to video processing engine 116 via network 140. The method then ends.
In sum, a video processing engine is configured to generate a graphical user interface (GUI) that allows an end-user of the video processing engine to select a specific video and search through the specific video to detect a desired target scene. The video processing engine provides a grid array of video thumbnails that are configured to each display a segment of the video so that multiple scenes may be visually scanned simultaneously. When the end-user identifies a scene within a video thumbnail that may be the desired target scene, the end-user may launch the content of the video thumbnail in full-screen mode to verify that the scene is in fact the desired target scene.
Advantageously, multiple thumbnail videos are displayed simultaneously thereby allowing the end-user to scrub through multiple scenes to more readily identify a desired target scene. Further, each thumbnail includes a reduced number of frames, that is, a reduced frame density, allowing the viewer to more efficiently view the essential content of a scene while minimizing the time required to stream the content. The ability to rapidly select a target scene from among multiple thumbnails affords scrubbing in a manner that is efficient in the time and effort required.
One embodiment of the invention may be implemented as a program product for use with a computer system. The program(s) of the program product define functions of the embodiments (including the methods described herein) and can be contained on a variety of computer-readable storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, flash memory, ROM chips or any type of solid-state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., floppy disks within a diskette drive or hard-disk drive or any type of solid-state random-access semiconductor memory) on which alterable information is stored.
The invention has been described above with reference to specific embodiments. Persons skilled in the art, however, will understand that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. The foregoing description and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Claims (16)

The invention claimed is:
1. A computer-implemented method for identifying a target scene within a video file, the method comprising:
receiving a first set of frames that are associated with a first portion of the video file, wherein the first portion includes the first set of frames, and the first set of frames is a subset of all of the frames included in the first portion;
receiving a second set of frames that are associated with a second portion of the video file, wherein the second portion includes the second set of frames, and the second set of frames is a subset of all of the frames included in the second portion;
generating a graphical user interface (GUI) that displays a timeline control and a thumbnail array, the thumbnail array comprising a first thumbnail at a first location in the thumbnail array and a second thumbnail at a second location in the thumbnail array;
causing the GUI to display a first frame included in the first set of frames in the first thumbnail and a second frame included in the second set of frames in the second thumbnail when the timeline control resides at a first position; and
causing the GUI to display a third frame included in the first set of frames in the first thumbnail and a fourth frame included in the second set of frames in the second thumbnail when the timeline control resides at a second position, wherein a first number of frames between the first frame and the third frame in the first set of frames equals a second number of frames between the second frame and the fourth frame in the second set of frames, and wherein the first thumbnail remains at the first location in the thumbnail array and the second thumbnail remains at the second location in the thumbnail array when the timeline control resides at the second position.
2. The computer-implemented method of claim 1, further comprising:
identifying a total number of frames to include in the first set of frames based on a first selection criterion; and
identifying a second total number of frames to include in the second set of frames based on a second selection criterion.
3. The computer-implemented method of claim 1, wherein the first frame in the first set of frames occurs chronologically before the third frame in the first set of frames.
4. The computer-implemented method of claim 1, wherein the second frame in the second set of frames occurs chronologically before the fourth frame in the second set of frames.
5. The computer-implemented method of claim 1, further comprising causing the GUI to display frames included in the first set of frames and frames included in the second set of frames based on a chronological ordering of frames in the video file.
6. The computer-implemented method of claim 5, further comprising:
causing the GUI to display frames at a first chronological location within the first portion of the video file based on a third position of the timeline control; and
causing the GUI to display frames at a second chronological location within the second portion of the video file based on the third position of the timeline control.
7. The computer-implemented method of claim 1, further comprising:
identifying that one of the first portion of the video file or the second portion of the video file is the target scene; and
causing the GUI to display one of the first portion of the video file or the second portion of the video file.
8. One or more non-transitory computer-readable media storing program instructions that, when executed by one or more processors, cause the one or more processors to facilitate identifying a target scene within a video file by performing the steps of:
receiving a first set of frames that are associated with a first portion of the video file, wherein the first portion includes the first set of frames, and the first set of frames is a subset of all of the frames included in the first portion;
receiving a second set of frames that are associated with a second portion of the video file, wherein the second portion includes the second set of frames, and the second set of frames is a subset of all of the frames included in the second portion;
generating a graphical user interface (GUI) that displays a timeline control and a thumbnail array, the thumbnail array comprising a first thumbnail at a first location in the thumbnail array and a second thumbnail at a second location in the thumbnail array;
causing the GUI to display a first frame included in the first set of frames in the first thumbnail and a second frame included in the second set of frames in the second thumbnail when the timeline control resides at a first position; and
causing the GUI to display a third frame included in the first set of frames in the first thumbnail and a fourth frame included in the second set of frames in the second thumbnail when the timeline control resides at a second position, wherein a first number of frames between the first frame and the third frame in the first set of frames equals a second number of frames between the second frame and the fourth frame in the second set of frames, and wherein the first thumbnail remains at the first location in the thumbnail array and the second thumbnail remains at the second location in the thumbnail array when the timeline control resides at the second position.
9. The one or more non-transitory computer-readable media of claim 8, further comprising:
identifying a total number of frames to include in the first set of frames based on a first selection criterion; and
identifying a second total number of frames to include in the second set of frames based on a second selection criterion.
10. The one or more non-transitory computer-readable media of claim 8, wherein the first frame in the first set of frames occurs chronologically before the third frame in the first set of frames.
11. The one or more non-transitory computer-readable media of claim 8, wherein the second frame in the second set of frames occurs chronologically before the fourth frame in the second set of frames.
12. The one or more non-transitory computer-readable media of claim 8, further comprising causing the GUI to display frames included in the first set of frames and frames included in the second set of frames based on a chronological ordering of frames in the video file.
13. The one or more non-transitory computer-readable media of claim 8, further comprising:
causing the GUI to display frames at a first chronological location within the first portion of the video file based on a third position of the timeline control; and
causing the GUI to display frames at a second chronological location within the second portion of the video file based on the third position of the timeline control.
14. A system configured to facilitate identifying a target scene within a video file, the system comprising:
a display device configured to display a graphical user interface (GUI);
a memory unit;
a processor coupled to the memory unit and configured to:
receive a first set of frames that reflect a first portion of the video file, wherein the first portion includes the first set of frames, and the first set of frames is a subset of all of the frames included in the first portion,
receive a second set of frames that reflect a second portion of the video file, wherein the second portion includes the second set of frames, and the second set of frames is a subset of all of the frames included in the second portion,
generate a GUI that displays a timeline control and a thumbnail array, the thumbnail array comprising a first thumbnail at a first location in the thumbnail array and a second thumbnail at a second location in the thumbnail array, and
cause the GUI to display a first frame included in the first set of frames in the first thumbnail and a second frame included in the second set of frames in the second thumbnail when the timeline control resides at a first position; and
cause the GUI to display a third frame included in the first set of frames in the first thumbnail and a fourth frame included in the second set of frames in the second thumbnail when the timeline control resides at a second position, wherein a first number of frames between the first frame and the third frame in the first set of frames equals a second number of frames between the second frame and the fourth frame in the second set of frames, and wherein the first thumbnail remains at the first location in the thumbnail array and the second thumbnail remains at the second location in the thumbnail array when the timeline control resides at the second position.
15. The system of claim 14, wherein the memory unit stores program instructions that, when executed by the processor, cause the processor to:
identify a total number of frames to include in the first set of frames based on a first selection criterion; and
identify a second total number of frames to include in the second set of frames based on a second selection criterion.
16. The system of claim 14, wherein the processor is further configured to:
identify that one of the first portion of the video file or the second portion of the video file is the target scene; and
cause the GUI to display one of the first portion of the video file or the second portion of the video file.
US14/524,770 2013-12-03 2014-10-27 Technique for searching and viewing video material Active 2035-10-14 US10386993B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/524,770 US10386993B2 (en) 2013-12-03 2014-10-27 Technique for searching and viewing video material

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201361911301P 2013-12-03 2013-12-03
US14/524,770 US10386993B2 (en) 2013-12-03 2014-10-27 Technique for searching and viewing video material

Publications (2)

Publication Number Publication Date
US20150153919A1 US20150153919A1 (en) 2015-06-04
US10386993B2 true US10386993B2 (en) 2019-08-20

Family

ID=53265344

Family Applications (3)

Application Number Title Priority Date Filing Date
US14/524,770 Active 2035-10-14 US10386993B2 (en) 2013-12-03 2014-10-27 Technique for searching and viewing video material
US14/542,402 Active 2036-04-08 US10694071B2 (en) 2013-12-03 2014-11-14 Techniques for viewing and searching documents from collections of documents
US14/542,390 Active 2037-12-09 US10761956B2 (en) 2013-12-03 2014-11-14 Techniques for visualizing dynamic datasets

Family Applications After (2)

Application Number Title Priority Date Filing Date
US14/542,402 Active 2036-04-08 US10694071B2 (en) 2013-12-03 2014-11-14 Techniques for viewing and searching documents from collections of documents
US14/542,390 Active 2037-12-09 US10761956B2 (en) 2013-12-03 2014-11-14 Techniques for visualizing dynamic datasets

Country Status (1)

Country Link
US (3) US10386993B2 (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9208051B2 (en) * 2012-12-26 2015-12-08 Bmc Software, Inc. Automatic creation of graph time layer of model of computer network objects and relationships
US10488911B2 (en) * 2016-11-01 2019-11-26 National Taiwan University Method and computing system of allocating registers
US20180350404A1 (en) * 2017-06-01 2018-12-06 Microsoft Technology Licensing, Llc Video splitter
US11068121B2 (en) * 2017-09-28 2021-07-20 Fujifilm Business Innovation Corp. System and method for visual exploration of subnetwork patterns in two-mode networks
US10460191B1 (en) * 2018-08-20 2019-10-29 Capital One Services, Llc Dynamically optimizing photo capture for multiple subjects
US11868402B2 (en) 2019-10-11 2024-01-09 Kinaxis Inc. Systems and methods of network visualization
US11501231B2 (en) * 2019-10-17 2022-11-15 Université De Lorraine Method for process analysis
CN113254680B (en) * 2020-02-10 2023-07-25 北京百度网讯科技有限公司 Cover map processing method of multimedia information, client and electronic equipment
US11425460B1 (en) * 2021-01-29 2022-08-23 Rovi Guides, Inc. Selective streaming based on dynamic parental rating of content

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060120624A1 (en) * 2004-12-08 2006-06-08 Microsoft Corporation System and method for video browsing using a cluster index
US20080184120A1 (en) * 2007-01-31 2008-07-31 Obrien-Strain Eamonn Concurrent presentation of video segments enabling rapid video file comprehension
US20100088646A1 (en) * 2008-10-07 2010-04-08 Yuichi Nishimori Information processing apparatus reproducing moving image and displaying thumbnails, and information processing method thereof
US20110197131A1 (en) * 2009-10-21 2011-08-11 Mod Systems Incorporated Contextual chapter navigation
US20130307792A1 (en) * 2012-05-16 2013-11-21 Google Inc. Gesture touch inputs for controlling video on a touchscreen

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6873343B2 (en) * 2000-05-11 2005-03-29 Zoran Corporation Scalable graphics image drawings on multiresolution image with/without image data re-usage
AU2002366924A1 (en) * 2001-12-12 2003-07-09 Nec Corporation Electronic document reading system and method
US7996786B2 (en) * 2007-03-05 2011-08-09 Microsoft Corporation Dynamically rendering visualizations of data sets
US7870224B1 (en) * 2007-06-08 2011-01-11 Adobe Systems Incorporated Managing online composite image content
JP4469885B2 (en) * 2007-09-07 2010-06-02 シャープ株式会社 Image collation apparatus, image collation method, image data output processing apparatus, program, and recording medium
US20090222412A1 (en) * 2008-02-29 2009-09-03 Microsoft Corporation Facet visualization
US8775424B2 (en) * 2010-01-26 2014-07-08 Xerox Corporation System for creative image navigation and exploration
US20110295879A1 (en) * 2010-05-27 2011-12-01 Neuone, Llc Systems and methods for document management
US20120188248A1 (en) * 2011-01-26 2012-07-26 The Boeing Company Image Management and Presentation
US9099161B2 (en) * 2011-01-28 2015-08-04 Apple Inc. Media-editing application with multiple resolution modes
US8850324B2 (en) * 2011-02-02 2014-09-30 Cisco Technology, Inc. Visualization of changes and trends over time in performance data over a network path
US8872849B2 (en) * 2012-01-27 2014-10-28 Microsoft Corporation Relational rendering of multi-faceted data
US10268662B2 (en) * 2012-09-10 2019-04-23 The Boeing Company Panoptic visualization of a document according to the structure thereof
EP2738704A1 (en) * 2012-12-03 2014-06-04 Dassault Systèmes A computer-implemented method for simulating, in a three-dimensional scene, the evolution of biological data
US20130232452A1 (en) * 2013-02-01 2013-09-05 Concurix Corporation Force Directed Graph with Time Series Data
US20140019879A1 (en) * 2013-02-01 2014-01-16 Concurix Corporation Dynamic Visualization of Message Passing Computation
US9471200B2 (en) * 2013-03-15 2016-10-18 Apple Inc. Device, method, and graphical user interface for organizing and presenting a collection of media items
JP6061273B2 (en) * 2013-03-29 2017-01-18 インターナショナル・ビジネス・マシーンズ・コーポレーションInternational Business Machines Corporation Display device, display method, and program
US9246773B2 (en) * 2013-07-30 2016-01-26 Draios Inc. System, method, and graphical user interface for application topology mapping in hosted computing environments
US9785317B2 (en) * 2013-09-24 2017-10-10 Palantir Technologies Inc. Presentation and analysis of user interaction data

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060120624A1 (en) * 2004-12-08 2006-06-08 Microsoft Corporation System and method for video browsing using a cluster index
US20080184120A1 (en) * 2007-01-31 2008-07-31 Obrien-Strain Eamonn Concurrent presentation of video segments enabling rapid video file comprehension
US20100088646A1 (en) * 2008-10-07 2010-04-08 Yuichi Nishimori Information processing apparatus reproducing moving image and displaying thumbnails, and information processing method thereof
US20110197131A1 (en) * 2009-10-21 2011-08-11 Mod Systems Incorporated Contextual chapter navigation
US20130307792A1 (en) * 2012-05-16 2013-11-21 Google Inc. Gesture touch inputs for controlling video on a touchscreen

Non-Patent Citations (48)

* Cited by examiner, † Cited by third party
Title
Alexander, et al., "Revisiting Read Wear: Analysis, Design, and Evaluation of a Footprints Scrollbar", CHI Apr. 8, 2009, Desktop Techniques, pp. 1665-1674.
André, et al., "Discovery Is Never by Chance: Designing for (Un)Serendipity", Creativity and Cognition, Oct. 26-30, 2009, 10 pages.
Aris, et al., "Visual Overviews for Discovering Key Papers and Influences Across Research Fronts", Journal of the American Society for Information Science and Technology, 60(11), 2009, pp. 2219-2228.
Bailer, et al., "Video Content Browsing Based on Iterative Feature Clustering for Rushes Exploration", TRECVID Workshop, May 2011, pp. 230-239.
Barnes, et al., "Video Tapestries with Continuous Temporal Zoom", ACM SIGGRAPH. Article 89, Jul. 2010, 9 pages.
Bederson, "Interfaces for Staying in the Flow", Ubiquity, Sep. 2004, 8 pages.
Bederson, et al., "Pad++: A Zooming Graphical Interface for Exploring Alternate Interface Physics", UIST, Jan. 1994, 10 pages.
Brandt, et al., "Example-Centric Programming: Integrating Web Search into the Development Environment", CHI 2010 End-User Programming I, Apr. 10-15, 2010, pp. 513-522.
Byrd, "A Scrollbar-based Visualization for Document Navigation", Proceedings of the Fourth ACM Conference on Digital Libraries, 1999, 8 pages.
Cao, et al., "FacetAtlas: Multifaceted Visualization for Rich Text Corpora", IEEE Transactions on Visualization and Computer Graphics, 16(6), 2010, pp. 1172-1181.
Card, et al., "The WebBook and the Web Forager: An Information Workspace for the World-Wide Web", Human factors in computing systems, 1996, 11 pages.
Divakaran, et al., "Augmenting Fast-Forward and Rewind for Personal Digital Video Recorders", IEEE ICCE, 2005, 5 pages.
Dunne, et al., "GraphTrail: Analyzing Large Multivariate, Heterogeneous Networks while Supporting Exploration History", CHI May 5-10, 2012, 10 pages.
Dunne, et al., "Rapid Understanding of Scientific Paper Collections: Integrating Statistics, Text Analytics, and Visualization", Computational Linguistics, 2011, 31 pages.
Giles, et al., "CiteSeer: An Automatic Citation Indexing System", Third ACM Conference on Digital Libraries, 1998, pp. 89-98.
Girgensohn, et al., "DocuBrowse: Faceted Searching, Browsing, and Recommendations in an Enterprise Context", Feb. 7-10, 2010, 10 pages.
Gove, et al., "Evaluating Visual and Statistical Exploration of Scientific Literature Networks", IEEE VLHCC, 2011, 8 pages.
Gray, et al., "Milliseconds Matter: An Introduction to Microstrategies and to Their Use in Describing and Predicting Interactive Behavior", Journal of Experimental Psych., 6(4), 2000, pp. 322-335.
Grossman, et al., "Chronicle: Capture, Exploration, and Playback of Document Workflow Histories", UIST, Oct. 3-6, 2010, pp. 143-152.
Hearst, "UIs for Faceted Navigation Recent Advances and Remaining Open Problems", Interfaces, 2008, 5 pages.
Hong, et al., "Turning Pages of 3D Electronic Books", IEEE Symposium on 3D User Interfaces, Mar. 25-26, 2006, pp. 159-165.
Howland, et al., "How Scholarly Is Google Scholar? A Comparison to Library Databases", College & Research Libraries, 70(3), 2009, pp. 227-234.
Hürst, "Interactive Audio-Visual Video Browsing", ACM MM, Oct. 23-27, 2006, 4 pages.
Hürst, et al., "Quantity versus Quality-The Role of Layout and Interaction Complexity in Thumbnail-based Video Retrieval Interfaces", ICMR, Jun. 5-8, 2012, 8 pages.
Hürst, et al., "Quantity versus Quality—The Role of Layout and Interaction Complexity in Thumbnail-based Video Retrieval Interfaces", ICMR, Jun. 5-8, 2012, 8 pages.
Jackson, et al., "Panopticon: A Parallel Video Overview System", ACM UIST, Oct. 8-11, 2013, pp. 8.
Janin, et al., "Joke-o-mat HD: Browsing Sitcoms with Human Derived Transcripts", ACM Multimedia, 2010, 3 pages.
Jeng, "What Is Usability in the Context of the Digital Library and How Can It Be Measured?" Information Technology and Libraries, 24(2), Jun. 2005, pp. 47-56.
Lee, et al., "Understanding Research Trends in Conferences using PaperLens", CHI Apr. 2-7, 2005, pp. 1969-1972.
Lee, et al., FacetLens: Exposing Trends and Relationships to Support Sensemaking within Faceted Datasets, CHI, Apr. 4-9, 2009, 10 pages.
Matejka, et al., "Citeology: Visualizing Paper Genealogy", CHI Extended Abstracts, May 5-10, 2012, 9 pages.
Matejka, et al., "Swift: Reducing the Effects of Latency in Online Video Scrubbing", ACM CHI, May 5-10, 2012, 10 pages.
Matejka, et al., "Swifter: Improved Online Video Scrubbing", ACM CHI, Apr. 27-May 2, 2013, pp. 1159-1168.
Nel, "Large Image Support in Digital Repositories", Master of Philosophy, Department of Computer Science, University of Cape Town, Jan. 2010, 92 pages.
Pietriga, "A Toolkit for Addressing HCI issues in Visual Language Environments", VLHCC Oct. 2005, 9 pages.
Pirolli, "Information Foraging in Information Access Environments", Creativity, Jan. 1995, 10 pages.
Plaisant, "The Challenge of Information Visualization Evaluation", AVI, 11(4), 2004, 8 pages.
Pongnumkul, "Content-Aware Dynamic Timeline for Video Browsing", ACM UIST, Oct. 3-6, 2010, pp. 139-142.
Ramos, et al., "Fluid Interaction Techniques for the Control and Annotation of Digital Video", ACM UIST, 2003, pp. 105-114.
Rooke, et al., "AppMap: Exploring User Interface Visualizations", Graphics Interface 2011, pp. 111-118.
Schoeffmann, et al., "The Video Explorer-A Tool for Navigation and Searching within a Single Video based on Fast Content Analysis", ACM SIGMM, Feb. 22-23, 2010, 12 pages.
Schoeffmann, et al., "The Video Explorer—A Tool for Navigation and Searching within a Single Video based on Fast Content Analysis", ACM SIGMM, Feb. 22-23, 2010, 12 pages.
Strobelt, et al., "Document Cards: A Top Trumps Visualization for Documents", InfoVis, Jul. 27, 2009, 8 pages.
Teevan, et al., "Visual Snippets: Summarizing Web Pages for Search and Revisitation", CHI, 2009, 11 pages.
Truong, et al., "Video Abstraction: A Systematic Review and Classification" ACM Transactions on Multimedia Computing Communications and Applications, vol. 3, No. 1, Article 3, Feb. 2007, 37 pages.
Tse, et al., "Dynamic Key Frame Presentation Techniques for Augmenting Video Browsing", ACM AVI, 1998, pp. 185-194.
Wexelblat, et al., "Footprints: History-Rich Tools for Information Foraging", CHI, 1999, 8 pages.
Woodruff, et al., "Using Thumbnails to Search the Web", SIGCHI Mar. 31-Apr. 4, 2001, 8 pages.

Also Published As

Publication number Publication date
US10694071B2 (en) 2020-06-23
US20150154188A1 (en) 2015-06-04
US20150153919A1 (en) 2015-06-04
US20150156076A1 (en) 2015-06-04
US10761956B2 (en) 2020-09-01

Similar Documents

Publication Publication Date Title
US10386993B2 (en) Technique for searching and viewing video material
US10909307B2 (en) Web-based system for capturing and sharing instructional material for a software application
US8701002B2 (en) Hierarchical display and navigation of document revision histories
US8533595B2 (en) Hierarchical display and navigation of document revision histories
US8533593B2 (en) Hierarchical display and navigation of document revision histories
US8533594B2 (en) Hierarchical display and navigation of document revision histories
US20120271867A1 (en) Hierarchical display and navigation of document revision histories
US20070266322A1 (en) Video browsing user interface
US9736526B2 (en) Real-time scrubbing of videos using a two-dimensional grid of thumbnail images
CN107005741A (en) Immersion scales interactive television
US20210004131A1 (en) Highlights video player
US20170109020A1 (en) Interactive presentation system
CN111095939A (en) Identifying previously streamed portions of a media item to avoid repeated playback
US10733925B2 (en) Display control apparatus, display control method, and non-transitory computer-readable storage medium
WO2015078257A1 (en) Search information display device and method
JP2009037282A (en) Image browsing device
JP4741808B2 (en) Information processing apparatus and program
JP2008293360A (en) Object information display device and method
KR102372181B1 (en) Display device and method for control thereof
US20150081765A1 (en) System and method for enhancing multi-module experience
US11397514B2 (en) Systems and methods for rotational video-like display of images of a gemstone in a viewer having a pan feature
US9582577B2 (en) Graphical diagram having elements that correspond to objects and that have locations in correspondence with appearance relationships of objects
JP2014229323A (en) Method and apparatus for interactive review of dataset
EP4360066A2 (en) Methods and systems for utilizing live embedded tracking data within a live sports video stream
CN114153342A (en) Visual information display method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
AS Assignment

Owner name: AUTODESK, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MATEJKA, JUSTIN FRANK;FITZMAURICE, GEORGE;GROSSMAN, TOVI;SIGNING DATES FROM 20141025 TO 20141105;REEL/FRAME:034641/0773

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4