US20200000529A1 - System and method for advanced data management with video enabled software tools for video broadcasting environments - Google Patents
System and method for advanced data management with video enabled software tools for video broadcasting environments Download PDFInfo
- Publication number
- US20200000529A1 US20200000529A1 US16/264,596 US201916264596A US2020000529A1 US 20200000529 A1 US20200000529 A1 US 20200000529A1 US 201916264596 A US201916264596 A US 201916264596A US 2020000529 A1 US2020000529 A1 US 2020000529A1
- Authority
- US
- United States
- Prior art keywords
- image stream
- image
- processing module
- video
- image processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims abstract description 23
- 238000013523 data management Methods 0.000 title description 3
- 238000012545 processing Methods 0.000 claims abstract description 42
- 230000008569 process Effects 0.000 claims description 10
- 239000000872 buffer Substances 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims 17
- 239000000203 mixture Substances 0.000 claims 2
- 239000000835 fiber Substances 0.000 claims 1
- 230000000007 visual effect Effects 0.000 abstract description 4
- 238000005516 engineering process Methods 0.000 description 22
- 238000010586 diagram Methods 0.000 description 12
- 230000009471 action Effects 0.000 description 7
- 230000008901 benefit Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000002716 delivery method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000002459 sustained effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000009966 trimming Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/61—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio
- H04L65/611—Network streaming of media packets for supporting one-way streaming services, e.g. Internet radio for multicast or broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/25—User interfaces for surgical systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
- A61B1/000095—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope for image enhancement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00011—Operational features of endoscopes characterised by signal transmission
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/313—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for introducing through surgical openings, e.g. laparoscopes
- A61B1/3132—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for introducing through surgical openings, e.g. laparoscopes for laparoscopy
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/361—Image-producing devices, e.g. surgical cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/752—Media network packet handling adapting media to network capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/762—Media network packet handling at the source
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
Definitions
- the technology disclosed herein relates to video editing software and, in particular, some embodiments relate to systems and methods for a software tools platform in a video editing or broadcasting environment, incorporating advanced data management.
- Video editing and broadcasting occurs with advanced audio-visual capturing technology. Still or video cameras may be used to capture image data.
- Video broadcast feeds are operated under a broad range of conditions and are equipped with tools to allow for streaming video, including editing tools that allow broadcasters to incorporate audio-visual content from a variety of sources.
- a broadcast stream may be equipped with a video camera that transmits live data and can operate under a broad range of conditions. Data from these different sources are typically displayed on various display devices within the video broadcasting station. These data streams are typically transmitted using a variety of protocols and formats according to the source device and purpose of the data stream. Accordingly, a video broadcast control station typically devotes separate resources to the separate data streams.
- large panel video and multi-screen displays are commonly employed in such contexts as Network Operations Centers (NOCs), entertainment, theaters, and retail venues. Such displays are often referred to as video walls, info walls, data walls, tiled displays, virtual walls, display walls, or virtual monitor walls.
- Large panel video and multi-screen displays provide groups of people with visual information that can be automatically updated, or are frequently used for enabling collaboration among viewers. Where a large panel video and multi-screen display is used for group collaboration, there is generally a requirement that the group has the ability to update and reconfigure the visual information displayed, which is usually facilitated through a video switch. Traditional video switches are controlled through a switch box and have only a single point for control access.
- Video broadcasts for sporting events can identify and transmit plays in real time to a sports team fan through the fan's mobile device.
- This includes video data such as highlight reels, JPEGS, and collated clips.
- This data is also used to produce web based video, and can also be used by teams for player training and evaluations. This allows for all teams have access to video archives online to do opposing team research and player evaluations.
- This technology also allows for video archiving and alternative audio overlay, with the ability to record proxy video in high quality.
- Various embodiments of the disclosed technology provide a software platform for video editing and recording that can turn around a high-end, professional product in real time. It provides the ability to distribute video from live feeds, such as major league baseball games, as the action is still happening.
- the platform uses specialized data management techniques to efficiently handle incoming video data streams, making it possible to ingest, edit, log, review and distribute multiple video data streams, in multiple formats more cost effectively using commodity hardware.
- broadcasting companies commonly take portions of video content and repurpose it for distribution on an Internet portal such as HBO GO or on social media platforms such as Facebook so they can reach more viewers.
- broadcasting companies utilize computer infrastructure that ingests live or file feeds and transcodes them into various formats—Main Screen or ABR Multi-screen-, various codec-MPEG2, H264, HEVC- and various resolutions—mobile, SD, HD, UHD.
- This type of real time processing typically requires vast computational resources, including a lot of expensive computer equipment.
- the disclosed technology includes a new approach to managing the incoming data streams which eliminates the need for temporary file storage by using an data pipe, specifically an in-memory ring buffer to feed an incoming stream to the various different encoders.
- these types of systems store a received data stream into a temporary file which is subsequently read by each of the various encoders. Since the process of writing and reading a file from a disk drive is relative much slower than accessing the information from computer memory, the use of the data pipe makes video stream processing many times faster.
- the use of the data pipe greatly reduces the network bandwidth requirements and disk requirements and enables sustained high data rates. It also adds the ability to record video content in software without using a physical recording card.
- the software recorder is designed to run in a virtual machine with limited resources. Virtual machines allow multiple recorders to run simultaneously on a physical computer. This physical computer can reside anywhere on a network, not just in the traditional down-link location.
- broadcasters the ability to run these software recorders anywhere such as on the cloud or on their own infrastructure.
- broadcasters typically must run recorders on blade servers which do not scale as well as cloud technology.
- a broadcaster can handle only a single feed per blade and run only a single encoder.
- using software recorders it is possible to run multiple encoders and multiple recorders per blade. This enables broadcasters to greatly expand the flexibility and the number of streams that they can record without having to vastly expand their hardware infrastructure.
- Various embodiments of the disclosed technology provide a configurable live digital video recorder.
- Input to the recorder is a file or more commonly an RTP/TS stream with optional Forward Error Correction.
- Output from the recorder is a configurable mix of H.264/TS, MXF wrapped H.264, Apple ProRes, HLS segmented H.264/TS, and SCC/TTML closed captioning. All outputs can be in live mode, with a minimum of latency.
- FIG. 1 is a block diagram illustrating an overview of an operating environment in which the present invention would typically be used.
- FIG. 2 is a block diagram illustrating an example of the video editing software tools platform in accordance with some embodiments of the technology described herein.
- FIG. 3 is a block diagram illustrating the Real-time Transport Protocol (RTP) stream recorder program.
- RTP Real-time Transport Protocol
- FIG. 4 is a block diagram illustrating the Digital Scissors Services Oriented Architecture.
- FIG. 1 is a block diagram illustrating an overview of an operating environment in which the present invention would typically be used, showing data streams incoming to the system and outgoing after having been processed by the system.
- data channels are able to transmit data to an archive 114 or content provider such as cable & direct-to-home (DTH) television 116 ; internet protocol television (IPTV) 118 ; and over-the-top (OTT) broadcasting 120 over both private and public channels.
- Live video data 108 is collected from live off-net 100 sources, digital internet protocol (IP) data 102 , and data from online IP links 104 transmitted over an online IP network.
- Mezzanine files 106 is processed by the transcoder 110 .
- the data from the live video channels 108 and the transcoder 110 is transmitted to the recorder 112 .
- the recorder 112 stores the data streams from all channels into a cloud based online accessible archive 114 , or streams the data directly to any or all of the multiple content delivery methods such as: cable & DTH 116 ; IPTV 118 ; and OTT 120 .
- FIG. 2 is a block diagram illustrating an example of the video editing software tools platform in accordance with some embodiments of the technology described herein.
- the video editing software tools platform 200 provides access to applications (apps) or widgets that can assist members of the video broadcasting team during a live event.
- the video editing software tools platform system 200 includes: an image stream interface module 202 ; a user interface overlay module 204 ; video editing software tools 206 ; a video editing device interface module 220 ; and an image stream processing system interface module 222 .
- the image stream interface module 202 may receive an image stream acquired by a video camera or the like.
- the image stream may be received directly from the video camera, or may be provided by way of one or more components, such as an image stream processing system.
- the image stream received from the image stream interface module 202 may vary in resolution, frame rate, format, and protocol according to the video camera or the image stream processing system providing the image stream.
- the user interface overlay module 204 may provide a user interface to the video editing software tools platform system 200 , which may include one or more graphical user interface (GUI) elements presented over the image stream received through the image stream interface module 202 .
- GUI graphical user interface
- the user interface comprises a bottom toolbar configured to be presented over the image stream, and configured to provide access to various video editing software tools 206 available through the video editing software tools platform system 200 .
- the video editing software tools 206 may include one or more video editing software tools, such as applications or widgets, which can be utilized with respect to the image stream being received through the image stream interface module 202 .
- the video editing software tools 206 platform includes but is not limited to: a video editing device control module 208 ; an image similarity search module 210 ; an image stream processing control module 212 ; an image stream tagging and tracking module 214 ; a timer module 216 ; and an image enhancement module 218 .
- the video editing device interface module 220 may facilitate communication between the video editing software tools platform system 200 , one or more of the video editing software tools 206 , and one or more various video editing devices utilized in video broadcasting.
- the image stream processing system interface module 222 may facilitate communication between the video editing software tools platform system 200 and an image stream processing system utilized to process an image stream acquired by a video camera or the like. Through the communication, the image stream processing system interface module 222 may transmit control data to an image stream processing system, or receive an image stream from a video camera as processed by the image stream processing system.
- the image stream processing system interface module 222 may include various data interfaces, including wired or wireless network interfaces and serial communication interfaces.
- FIG. 3 is a block diagram illustrating the Real-time Transport Protocol (RTP) stream recorder program.
- the RTP stream recorder is a computer program that receives an encoded digital video stream via standard internet protocols and stores and repurposes the stream in real-time on a “cloud” oriented device.
- the recorder can produce a wide variety of output products from the single input stream. Because the recorder has limited hardware requirements, the recorder can run on modest hardware, or multiple recorders can run simultaneously on more robust hardware.
- the RTP Stream Recorder reads the encoded input from an RTP broadcast source 300 with optional Forward Error Correction (FEC) and writes the packets to a Data Pipe 302 .
- FEC Forward Error Correction
- the receiver can run “anywhere”, not necessarily where the encoder is physically situated.
- the encoded source is an 100 Mb/s AVC-I/TS stream produced by a multichannel Ateme encoder.
- AVC-I is an archive quality, H.264 encoded, I-frame only (no P or B frames) digital video elementary stream.
- the encoded source also includes up to 16 channels of MP2 audio and 2 channels of closed captioning (EIA-608/708).
- a transport stream (TS) 300 is the container format used to enclose each of these elements into a single stream.
- the RTP broadcast is received via a unicast or multicast socket.
- the optional FEC is a 1D or 2D column and row broadcast, where each dimension is received on separate multicast ports.
- the data pipe 302 is a buffering mechanism that consumes the packets received by the RPT/TS 300 process and distributes them as needed, to each of the components in the recorder process.
- the data pipe removes the need for temporary disk space on the device and allows the recorder to run on limited capability hardware including virtual machines.
- the present invention uses data pipes, which are in-memory ring buffers, one for each of the recorder components. For example, if the recorder is configured to produce MXF 304 , HLS 308 and closed captioning 310 , the system would create 3 named pipes and copy the received transport stream to each. Because each of the recorder components is reading from its own data source, the component can run at its optimal rate, without having to wait for or coordinate with the other components.
- Other data pipes can include ProRes Recorder 306 and Future Recorder 312 data streams.
- the recorder can produce a wide variety of output products.
- Each of the output products has unique requirements and implementation of the specific recorder component is greatly simplified by being able to read its input from its “own” data source.
- HLS 318 requires transcoding and multiple output files 324
- MXF 304 only requires wrapping; no transcoding and a single output file 314 .
- Some examples of output products supported are: live and closed MXF 314 ; Apple ProRes 316 ; HLS with master and IFrame playlists 318 ; SCC and TTML closed captioning 320 ; and future recorded output 322 .
- FIG. 4 is a block diagram illustrating the Digital Scissors Services Oriented Architecture.
- Digital Scissors is a Service Oriented Architecture (SOA) that provides for the manipulation of digital video files. Commands are sent to the service requesting an action be performed upon a source and potentially producing an output.
- a command 400 is created by an external process requesting an action by the service.
- a command 400 can be an XML document describing the action, a REST API request or a direct library call.
- Commands include: Cut a timecode range from a source 408 to a destination 410 ; Grab 1 or more frames from a source 408 with a specified timecode signature to destination(s) 410 ; Transcode change the essence or container of a source 408 to a different essence or container.
- H.264/TS to MPEG4 Melt is the process of trimming the pre-amble and post-amble of a recording; Append a timecode range from a source file 408 onto a destination 410 ; Wrap a source essence with a container format to a destination 410 ; Metadata-Info provides the information about the underlying digital video: frame size, bitrate, format, color space and the like; and Archive 408 is the near-line storage of archived sources and clips.
- the commands 400 use SMPTE timecode to identify which frame or of range frames is being requested.
- Timecode is generally present in the digital video source material and make it possible for all observers to reference the same frame. Timecode is what makes frame accurate identification possible.
- An example XML command to request the cutting a source file to a destination using a timecode range is:
- the Job Builder 402 coalesce all requests into a queue of jobs that are executed in parallel but at a configured maximum rate. Invalid requests are immediately reported to the requestor. Requests are received from multiple clients and in multiple forms. Each request creates a new Job, the request is parsed and validated. The Job Builder 402 , then creates an internal list of tasks needed to accomplish the request and adds them to the Job. The Job is placed in a queue ready for execution by the Job Manager 404 .
- the Job Manager 404 processes the Job Queue and executes ready Jobs in parallel at a configured maximum rate.
- Jobs will have many tasks that are needed to accomplish the requested action.
- Tasks within the Job may be required to execute sequentially. Certain tasks may be able to run in parallel. For example, a task to transcode a clip that was created during a cutting task will have to execute sequentially.
- Each task is executed and monitored 406 for failure. When the Job is complete, the result is reported to the requestor.
- Each task is specialized and designed to accomplish a specific action by the task execution and monitor 406 .
- H.264/TS is much harder.
- the tail clip is the final frame-accurate clip and the process is finished.
- From the frame accurate cut point find the audio frame that synchronizes with the frame accurate cut point video frame.
- From the frame-accurate cut point seek REVERSE in the stream until an I-frame (REV IFRAME) encountered.
- FFMPEG transcodes the video-head-Itol clip into a raw YUV 4:2:2 video frame file (YUV RAW). This will give full video data for each frame in the video-head-Itol clip. Note that in this process, all video-head-Itol clip timing information is lost.
- YUV RAW raw YUV 4:2:2 video frame file
- YUV FA frame-accurate raw YUV 4:2:2 video frame file
- module does not imply that the components or functionality described or claimed as part of the module are all configured in a common package. Indeed, any or all of the various components of a module, whether control logic or other components, may be combined in a single package or separately maintained and can further be distributed in multiple groupings or packages or across multiple locations.
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- Multimedia (AREA)
- Heart & Thoracic Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Animal Behavior & Ethology (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Medical Informatics (AREA)
- Pathology (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Biophysics (AREA)
- Radiology & Medical Imaging (AREA)
- Optics & Photonics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Human Computer Interaction (AREA)
- Robotics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Television Signal Processing For Recording (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Video editing software tools platform utilizing a video display to provide access to specific video editing software tools, such as video oriented applications or widgets, that can assist those in a video broadcasting team, such as a camera operator or video editor, with a video broadcast feed. Various video editing software tools can provide features and functions that can add visual context to video data presented in the image stream from the video camera and provide archived information pertaining to the same. Various embodiments relate to systems and methods for simultaneously switching input image streams to output devices, while providing optional image processing functions on the image streams. Certain embodiments may enable multiple users/viewers to collaboratively control such systems and methods.
Description
- This application is a continuation of U.S. patent application Ser. No. 15/657,080, filed Jul. 21, 2017, which is a continuation in part of U.S. patent application Ser. No. 15/456,458, filed Mar. 10, 2017, which is a continuation in part of U.S. patent application Ser. No. 14/107,329, filed Dec. 16, 2013, and issued as U.S. Pat. No. 9,526,586 on Dec. 27, 2016, which claims the benefit of U.S. Provisional Patent Application Ser. No. 61/865,037, filed Aug. 12, 2013. This application is also a continuation in part of U.S. patent application Ser. No. 15/170,575, filed Jun. 1, 2016, and issued as U.S. Pat. No. 10,142,641 on Nov. 27, 2018, which is a continuation-in-part of U.S. patent application Ser. No. 12/776,048, filed May 7, 2010, and issued as U.S. Pat. No. 8,266,333 on Sep. 11, 2012, which claims the benefit of U.S. Provisional Patent Application Ser. No. 61/182,624, filed May 29, 2009, and 61/234,577, filed Aug. 17, 2009. Each of the foregoing applications is incorporated herein by reference.
- The technology disclosed herein relates to video editing software and, in particular, some embodiments relate to systems and methods for a software tools platform in a video editing or broadcasting environment, incorporating advanced data management.
- Video editing and broadcasting occurs with advanced audio-visual capturing technology. Still or video cameras may be used to capture image data. Video broadcast feeds are operated under a broad range of conditions and are equipped with tools to allow for streaming video, including editing tools that allow broadcasters to incorporate audio-visual content from a variety of sources. For example, a broadcast stream may be equipped with a video camera that transmits live data and can operate under a broad range of conditions. Data from these different sources are typically displayed on various display devices within the video broadcasting station. These data streams are typically transmitted using a variety of protocols and formats according to the source device and purpose of the data stream. Accordingly, a video broadcast control station typically devotes separate resources to the separate data streams.
- To display visual information from data streams, large panel video and multi-screen displays are commonly employed in such contexts as Network Operations Centers (NOCs), entertainment, theaters, and retail venues. Such displays are often referred to as video walls, info walls, data walls, tiled displays, virtual walls, display walls, or virtual monitor walls. Large panel video and multi-screen displays provide groups of people with visual information that can be automatically updated, or are frequently used for enabling collaboration among viewers. Where a large panel video and multi-screen display is used for group collaboration, there is generally a requirement that the group has the ability to update and reconfigure the visual information displayed, which is usually facilitated through a video switch. Traditional video switches are controlled through a switch box and have only a single point for control access.
- Video broadcasts for sporting events can identify and transmit plays in real time to a sports team fan through the fan's mobile device. This includes video data such as highlight reels, JPEGS, and collated clips. This data is also used to produce web based video, and can also be used by teams for player training and evaluations. This allows for all teams have access to video archives online to do opposing team research and player evaluations. This technology also allows for video archiving and alternative audio overlay, with the ability to record proxy video in high quality.
- Various embodiments of the disclosed technology provide a software platform for video editing and recording that can turn around a high-end, professional product in real time. It provides the ability to distribute video from live feeds, such as major league baseball games, as the action is still happening. The platform uses specialized data management techniques to efficiently handle incoming video data streams, making it possible to ingest, edit, log, review and distribute multiple video data streams, in multiple formats more cost effectively using commodity hardware.
- Commercial broadcasting companies commonly take portions of video content and repurpose it for distribution on an Internet portal such as HBO GO or on social media platforms such as Facebook so they can reach more viewers. To do this, broadcasting companies utilize computer infrastructure that ingests live or file feeds and transcodes them into various formats—Main Screen or ABR Multi-screen-, various codec-MPEG2, H264, HEVC- and various resolutions—mobile, SD, HD, UHD. This type of real time processing typically requires vast computational resources, including a lot of expensive computer equipment. The disclosed technology includes a new approach to managing the incoming data streams which eliminates the need for temporary file storage by using an data pipe, specifically an in-memory ring buffer to feed an incoming stream to the various different encoders. Typically, these types of systems store a received data stream into a temporary file which is subsequently read by each of the various encoders. Since the process of writing and reading a file from a disk drive is relative much slower than accessing the information from computer memory, the use of the data pipe makes video stream processing many times faster.
- The use of the data pipe greatly reduces the network bandwidth requirements and disk requirements and enables sustained high data rates. It also adds the ability to record video content in software without using a physical recording card. The software recorder is designed to run in a virtual machine with limited resources. Virtual machines allow multiple recorders to run simultaneously on a physical computer. This physical computer can reside anywhere on a network, not just in the traditional down-link location.
- This provides broadcasters the ability to run these software recorders anywhere such as on the cloud or on their own infrastructure. Today, broadcasters typically must run recorders on blade servers which do not scale as well as cloud technology. For example, typically, a broadcaster can handle only a single feed per blade and run only a single encoder. However, using software recorders it is possible to run multiple encoders and multiple recorders per blade. This enables broadcasters to greatly expand the flexibility and the number of streams that they can record without having to vastly expand their hardware infrastructure.
- Various embodiments of the disclosed technology provide a configurable live digital video recorder. Input to the recorder is a file or more commonly an RTP/TS stream with optional Forward Error Correction. Output from the recorder is a configurable mix of H.264/TS, MXF wrapped H.264, Apple ProRes, HLS segmented H.264/TS, and SCC/TTML closed captioning. All outputs can be in live mode, with a minimum of latency.
- Other embodiments provide for a computer readable storage medium having instructions embedded thereon to cause a processor to perform operations similar to those described above with respect to the various systems and methods in accordance with the present invention.
- These and other features, embodiments, and aspects of the present invention can be appreciated from the following drawing description and detailed description of a preferred embodiment. Other features and aspects of the disclosed technology will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the features in accordance with embodiments of the disclosed technology. The summary is not intended to limit the scope of any inventions described herein, which are defined solely by the claims attached hereto.
-
FIG. 1 is a block diagram illustrating an overview of an operating environment in which the present invention would typically be used. -
FIG. 2 is a block diagram illustrating an example of the video editing software tools platform in accordance with some embodiments of the technology described herein. -
FIG. 3 is a block diagram illustrating the Real-time Transport Protocol (RTP) stream recorder program. -
FIG. 4 is a block diagram illustrating the Digital Scissors Services Oriented Architecture. -
FIG. 1 is a block diagram illustrating an overview of an operating environment in which the present invention would typically be used, showing data streams incoming to the system and outgoing after having been processed by the system. In accordance with the preferred embodiment of the present invention, data channels are able to transmit data to anarchive 114 or content provider such as cable & direct-to-home (DTH)television 116; internet protocol television (IPTV) 118; and over-the-top (OTT) broadcasting 120 over both private and public channels.Live video data 108 is collected from live off-net 100 sources, digital internet protocol (IP)data 102, and data fromonline IP links 104 transmitted over an online IP network. Mezzanine files 106 is processed by thetranscoder 110. The data from thelive video channels 108 and thetranscoder 110 is transmitted to therecorder 112. Therecorder 112 stores the data streams from all channels into a cloud based onlineaccessible archive 114, or streams the data directly to any or all of the multiple content delivery methods such as: cable &DTH 116;IPTV 118; andOTT 120. -
FIG. 2 is a block diagram illustrating an example of the video editing software tools platform in accordance with some embodiments of the technology described herein. The video editingsoftware tools platform 200 provides access to applications (apps) or widgets that can assist members of the video broadcasting team during a live event. In accordance with the preferred embodiment of the present invention, the video editing softwaretools platform system 200 includes: an imagestream interface module 202; a userinterface overlay module 204; videoediting software tools 206; a video editingdevice interface module 220; and an image stream processingsystem interface module 222. The imagestream interface module 202 may receive an image stream acquired by a video camera or the like. Depending on the embodiment, the image stream may be received directly from the video camera, or may be provided by way of one or more components, such as an image stream processing system. The image stream received from the imagestream interface module 202 may vary in resolution, frame rate, format, and protocol according to the video camera or the image stream processing system providing the image stream. - The user
interface overlay module 204 may provide a user interface to the video editing softwaretools platform system 200, which may include one or more graphical user interface (GUI) elements presented over the image stream received through the imagestream interface module 202. For some embodiments, the user interface comprises a bottom toolbar configured to be presented over the image stream, and configured to provide access to various videoediting software tools 206 available through the video editing softwaretools platform system 200. - The video
editing software tools 206 may include one or more video editing software tools, such as applications or widgets, which can be utilized with respect to the image stream being received through the imagestream interface module 202. The videoediting software tools 206 platform includes but is not limited to: a video editingdevice control module 208; an imagesimilarity search module 210; an image streamprocessing control module 212; an image stream tagging andtracking module 214; atimer module 216; and animage enhancement module 218. - The video editing
device interface module 220 may facilitate communication between the video editing softwaretools platform system 200, one or more of the videoediting software tools 206, and one or more various video editing devices utilized in video broadcasting. The image stream processingsystem interface module 222 may facilitate communication between the video editing softwaretools platform system 200 and an image stream processing system utilized to process an image stream acquired by a video camera or the like. Through the communication, the image stream processingsystem interface module 222 may transmit control data to an image stream processing system, or receive an image stream from a video camera as processed by the image stream processing system. The image stream processingsystem interface module 222 may include various data interfaces, including wired or wireless network interfaces and serial communication interfaces. -
FIG. 3 is a block diagram illustrating the Real-time Transport Protocol (RTP) stream recorder program. In accordance with the preferred embodiment of the present invention, the RTP stream recorder is a computer program that receives an encoded digital video stream via standard internet protocols and stores and repurposes the stream in real-time on a “cloud” oriented device. The recorder can produce a wide variety of output products from the single input stream. Because the recorder has limited hardware requirements, the recorder can run on modest hardware, or multiple recorders can run simultaneously on more robust hardware. The RTP Stream Recorder reads the encoded input from anRTP broadcast source 300 with optional Forward Error Correction (FEC) and writes the packets to aData Pipe 302. Because multicast RTP is being received over the Internet, the receiver can run “anywhere”, not necessarily where the encoder is physically situated. For example, the encoded source is an 100 Mb/s AVC-I/TS stream produced by a multichannel Ateme encoder. AVC-I is an archive quality, H.264 encoded, I-frame only (no P or B frames) digital video elementary stream. The encoded source also includes up to 16 channels of MP2 audio and 2 channels of closed captioning (EIA-608/708). A transport stream (TS) 300 is the container format used to enclose each of these elements into a single stream. The RTP broadcast is received via a unicast or multicast socket. The optional FEC is a 1D or 2D column and row broadcast, where each dimension is received on separate multicast ports. - The
data pipe 302 is a buffering mechanism that consumes the packets received by the RPT/TS 300 process and distributes them as needed, to each of the components in the recorder process. The data pipe removes the need for temporary disk space on the device and allows the recorder to run on limited capability hardware including virtual machines. The present invention uses data pipes, which are in-memory ring buffers, one for each of the recorder components. For example, if the recorder is configured to produceMXF 304,HLS 308 andclosed captioning 310, the system would create 3 named pipes and copy the received transport stream to each. Because each of the recorder components is reading from its own data source, the component can run at its optimal rate, without having to wait for or coordinate with the other components. Other data pipes can includeProRes Recorder 306 andFuture Recorder 312 data streams. - The recorder can produce a wide variety of output products. Each of the output products has unique requirements and implementation of the specific recorder component is greatly simplified by being able to read its input from its “own” data source. For example,
HLS 318 requires transcoding andmultiple output files 324, whileMXF 304 only requires wrapping; no transcoding and asingle output file 314. Some examples of output products supported are: live and closedMXF 314;Apple ProRes 316; HLS with master andIFrame playlists 318; SCC and TTML closedcaptioning 320; and future recordedoutput 322. -
FIG. 4 is a block diagram illustrating the Digital Scissors Services Oriented Architecture. In accordance with the preferred embodiment of the present invention, Digital Scissors is a Service Oriented Architecture (SOA) that provides for the manipulation of digital video files. Commands are sent to the service requesting an action be performed upon a source and potentially producing an output. Acommand 400 is created by an external process requesting an action by the service. Acommand 400 can be an XML document describing the action, a REST API request or a direct library call. Commands include: Cut a timecode range from asource 408 to adestination 410; Grab 1 or more frames from asource 408 with a specified timecode signature to destination(s) 410; Transcode change the essence or container of asource 408 to a different essence or container. For example, H.264/TS to MPEG4; Melt is the process of trimming the pre-amble and post-amble of a recording; Append a timecode range from asource file 408 onto adestination 410; Wrap a source essence with a container format to adestination 410; Metadata-Info provides the information about the underlying digital video: frame size, bitrate, format, color space and the like; andArchive 408 is the near-line storage of archived sources and clips. - The
commands 400 use SMPTE timecode to identify which frame or of range frames is being requested. Timecode is generally present in the digital video source material and make it possible for all observers to reference the same frame. Timecode is what makes frame accurate identification possible. An example XML command to request the cutting a source file to a destination using a timecode range is: -
<cut> <in> <file>/damsamfs1/team1-team2.dv</file> <timecode> <start>10:20:22.00</start> <end>10:20:32.00</end> </timecode> <granularity>frame-accurate</granularity> </in> <out> <overwrite>false</overwrite> <append>false</append> <file>/damsamfs1/gclips/team1-team2-clip1.dv</file> </out> </cut> - The
Job Builder 402 coalesce all requests into a queue of jobs that are executed in parallel but at a configured maximum rate. Invalid requests are immediately reported to the requestor. Requests are received from multiple clients and in multiple forms. Each request creates a new Job, the request is parsed and validated. TheJob Builder 402, then creates an internal list of tasks needed to accomplish the request and adds them to the Job. The Job is placed in a queue ready for execution by theJob Manager 404. - The
Job Manager 404 processes the Job Queue and executes ready Jobs in parallel at a configured maximum rate. There are many design tradeoffs that are configurable within the Job Manager. For example, does the request want priority execution; should the manager try to minimize load, memory use or time. Generally, Jobs will have many tasks that are needed to accomplish the requested action. Tasks within the Job may be required to execute sequentially. Certain tasks may be able to run in parallel. For example, a task to transcode a clip that was created during a cutting task will have to execute sequentially. Each task is executed and monitored 406 for failure. When the Job is complete, the result is reported to the requestor. Each task is specialized and designed to accomplish a specific action by the task execution and monitor 406. For example, there are cutting, grabbing, metadata, transcode, and wrapping tasks. And each of those tasks as specific versions to handle the following formats: H.264, H.265; MPEG 1/2/4; DV 25/50/100; Apple ProRes; QuickTime; MPEG Transport Streams; MXF; HLS, M3U8; and SCC, TTML. Tasks are specific to the action being requested. Each of these tasks has specialized versions, depending on the input or output format being requested. A task that is cutting a DV100 file has different requirements from a task cutting an MP4 file. Searching for timecode is also different for each of the formats, H.264/TS has timecode encoded into every frame, where MXF just has a start timecode in the header of the file. - Some formats, like AVC-I/MXF, ProRes and DV100 directly support frame accurate cutting or grabbing. H.264/TS is much harder. As an example, this is an algorithm to support frame accurate cutting of H.264/TS: Find the TS packet at which the clip starting time-code occurs. This will be the frame-accurate cut point. From the frame accurate cut point, seek FORWARD in the stream until an I-frame (FWD (FRAME). If the clip starting time-code happens to fall on an I-frame then it is not necessary to seek forward. Create the tail clip that includes all video/audio data from this I-frame up to the frame-accurate frame that occurs at the clip ending time-code. If the clip starting time-code happened to fall on an I-frame, then the tail clip is the final frame-accurate clip and the process is finished. From the frame accurate cut point, find the audio frame that synchronizes with the frame accurate cut point video frame. Create the audio-only audio-head clip that contains all audio data from this point up to I-frame FWD IFRAME. From the frame-accurate cut point, seek REVERSE in the stream until an I-frame (REV IFRAME) encountered. Create the video-only video-head-Itol clip that contains all video data from this point up to I-frame FWD IFRAME. Note that this video-head-Itol clip should contain exactly 60 frames since this is what Inlet uses as the period (2 seconds) between successive I-frames. Using the FFMPEG application, transcode the video-head-Itol clip into a raw YUV 4:2:2 video frame file (YUV RAW). This will give full video data for each frame in the video-head-Itol clip. Note that in this process, all video-head-Itol clip timing information is lost. Calculate the number of frames to drop from the head of the YUV RAW file and create the frame-accurate raw YUV 4:2:2 video frame file (YUV FA). Using the FFMPEG application, transcode the YUV FA frame-accurate raw YUV 4:2:2 video frame file into a video-only H.264 transport stream video-head clip. Note that this video-head clip will contain <60 video frames. Change the PTS timestamps in the video-head clip so that they match the corresponding PTS timestamps as they appear in the source video. Recall that the transcoding from timestamped transport stream to YUV RAW removed all timing information. The transcoding of the YUV RAW to YUV FA reapplies timing information, however, this timing information is based at time zero whereas the associated audio clip is timestamped with the original timing data as it appears in the source video. Append the audio-head clip to the video-head clip to create the frame accurate head clip that includes all video/audio data from the frame-accurate cut point to FWD IFRAME. Append the tail clip to the head clip to create the final complete frame-accurate clip.
- While various embodiments of the disclosed technology have been described above, it should be understood that they have been presented by way of example only, and not of limitation. Likewise, the various diagrams may depict an example architectural or other configuration for the disclosed technology, which is done to aid in understanding the features and functionality that may be included in the disclosed technology. The disclosed technology is not restricted to the illustrated example architectures or configurations, but the desired features may be implemented using a variety of alternative architectures and configurations. Indeed, it will be apparent to one of skill in the art how alternative functional, logical or physical partitioning and configurations may be implemented to implement the desired features of the technology disclosed herein. Also, a multitude of different constituent module names other than those depicted herein may be applied to the various partitions. Additionally, with regard to flow diagrams, operational descriptions and method claims, the order in which the steps are presented herein shall not mandate that various embodiments be implemented to perform the recited functionality in the same order unless the context dictates otherwise.
- Although the disclosed technology is described above in terms of various exemplary embodiments and implementations, it should be understood that the various features, aspects and functionality described in one or more of the individual embodiments are not limited in their applicability to the particular embodiment with which they are described, but instead may be applied, alone or in various combinations, to one or more of the other embodiments of the disclosed technology, whether or not such embodiments are described and whether or not such features are presented as being a part of a described embodiment. Thus, the breadth and scope of the technology disclosed herein should not be limited by any of the above-described exemplary embodiments.
- Terms and phrases used in this document, and variations thereof, unless otherwise expressly stated, should be construed as open ended as opposed to limiting. As examples of the foregoing: the term “including” should be read as meaning “including, without limitation” or the like; the term “example” is used to provide exemplary instances of the item in discussion, not an exhaustive or limiting list thereof; the terms “a” or “an” should be read as meaning “at least one,” “one or more” or the like; and adjectives such as “conventional,” “traditional,” “normal,” “standard,” “known” and terms of similar meaning should not be construed as limiting the item described to a given time period or to an item available as of a given time, but instead should be read to encompass conventional, traditional, normal, or standard technologies that may be available or known now or at any time in the future. Likewise, where this document refers to technologies that would be apparent or known to one of ordinary skill in the art, such technologies encompass those apparent or known to the skilled artisan now or at any time in the future.
- The presence of broadening words and phrases such as “one or more,” “at least,” “but not limited to” or other like phrases in some instances shall not be read to mean that the narrower case is intended or required in instances where such broadening phrases may be absent. The use of the term “module” does not imply that the components or functionality described or claimed as part of the module are all configured in a common package. Indeed, any or all of the various components of a module, whether control logic or other components, may be combined in a single package or separately maintained and can further be distributed in multiple groupings or packages or across multiple locations.
- Additionally, the various embodiments set forth herein are described in terms of exemplary block diagrams, flow charts and other illustrations. As will become apparent to one of ordinary skill in the art after reading this document, the illustrated embodiments and their various alternatives may be implemented without confinement to the illustrated examples. For example, block diagrams and their accompanying description should not be construed as mandating a particular architecture or configuration.
Claims (25)
1. A system for switching an image stream input to an image stream output, comprising:
an image stream input interface;
an image stream output interface;
a first image processing module configured to accept an image stream from the image stream input interface or another image processing module, apply an image processing function to the image stream, and output a processed image stream;
a second image processing module; and
a switching matrix in communication with the image stream input interface, the image stream output interface, the first image processing module, and the second image processing module, wherein the switching matrix is configured to:
selectively map the image stream input interface to the image stream output interface or to the first image processing module,
selectively map the processed image stream from the first image processing module to the image stream output interface or to the second image processing module,
a ring buffer suitable for reducing network bandwidth requirements for recording said image stream without the use of a physical recording card and said ring buffer is sized sufficiently to facilitate recording said image stream for allowing multiple recorders to run simultaneously on a physical computer of a size wherein said size is minimized corresponding with said image stream requirements, and
a cloud or hosted local data platform providing broadcasters with a software recorder for recording said image stream without need for a non-software recorder corresponding with said software recorder enabling said broadcasters greater video handling capacity per a fixed amount of computer capacity.
2. The system of claim 1 , further comprising a data input interface, wherein the switching matrix is further in communication with the data input interface such that the switching matrix can further selectively map the data input interface to the image stream output interface or to the first image processing module.
3. The system of claim 2 , wherein the image stream input interface comprises the data input interface.
4. The system of claim 1 , wherein the switching matrix can selectively map the image stream input interface or the processed image stream in real-time.
5. The system of claim 1 , wherein the switching matrix can selectively map the image stream input interface to more than one image processing module or to more than one image stream output interface simultaneously.
6. The system of claim 1 , wherein the switching matrix can selectively map the processed image stream to more than one image processing module or to more than one image stream output interface simultaneously.
7. The system of claim 1 , wherein the switching matrix can selectively map the image stream input interface or the processed image stream based on a criterion.
8. The system of claim 7 , wherein the criterion relates to source or content of the image stream input interface or the processed image stream.
9. The system of claim 7 , wherein the criterion for the processed image stream relates to results of a preceding image processing module.
10. The system of claim 1 , wherein the image processing module can process a plurality of image streams in parallel.
11. The system of claim 1 , wherein the image stream input interface or image stream output interface has a Digital Video Interface (DVI) connector, High-definition Multimedia Interface (HDMI) connector, a Bayonet Neill-Concelman (BNC) connector, a fiber optic connector, a DisplayPort connector, a Universal Serial Bus (USB) connector, or a Firewire (IEEE1394) connector.
12. The system of claim 1 , wherein the image stream input interface is configured to convert from or the image stream output interface is configured to: a Digital Video Interface (DVI) standard, a High-definition Multimedia Interface (HDMI) compatible standard, a Red-Green-Blue (RGB) Analog standard, a Red-Green-Blue (RGB) Digital standard, a DisplayPort standard, a National Television System Committee (NTSC) standard, a Phase Alternate Line (PAL) standard, or a Serial Digital Interface (SDI) standard.
13. The system of claim 1 , wherein the system comprises a plurality of image processing modules.
14. The system of claim 1 , wherein the system comprises a plurality of image stream input interfaces.
15. The system of claim 1 , wherein the system comprises a plurality of image stream output interfaces.
16. The system of claim 1 , wherein the image processing function comprises an image stream mix function, an image stream scale function, an image stream blend function, an image stream encoding algorithm, or an image stream enhancement function.
17. The system of claim 1 wherein the image stream input interface is configured to receive the image stream from an image stream capture device, an image stream playback device, a computer system, or a sensor device.
18. The system of claim 1 , wherein the image stream output interface is configured to output to a display, a computer system, or recording device.
19. The system of claim 1 , wherein the system is configured to output an image stream through a virtual display.
20. The system of claim 1 , further comprising a control interface through which a user can operate the system, configure the system, or check status of the system.
21. A method for switching an image stream input to an image stream output, comprising:
receiving an original image stream from an image stream input interface;
receiving a first processed image stream from a first image processing module, wherein the first image processing module generates the first processed image stream by applying a first image processing function to an image stream;
using a switching matrix to selectively map the original image stream to:
a second image processing module, wherein the second image processing module is configured to receive the original image stream from the switching matrix, apply a second image processing function to the original image stream, and output a second processed image stream, or
an image stream output interface; and
using the switching matrix to selectively map the first processed image stream to:
the second image processing module, wherein the second image processing module is further configured to receive the first processed image stream from the switching matrix, apply the second image processing function to the first processed image stream, and output a third processed image stream, or the image stream output interface,
a ring buffer processor suitable for reducing network bandwidth requirements for recording said image stream without the use of a physical recording card and said ring buffer is sized sufficiently to facilitate recording said image stream for allowing multiple recorders to run simultaneously on a physical computer of a size wherein said size is minimized corresponding with said image stream requirements, and
a cloud or hosted local data platform processor providing broadcasters with a software recorder for recording said image stream without need for a non-software recorder corresponding with said software recorder enabling said broadcasters greater video handling capacity per a fixed amount of computer capacity.
22. The method of claim 21 , wherein when the switching matrix selectively maps the original image stream to the second image processing module, the method further comprises using the switching matrix to selectively map the second processed image stream to a third image processing module.
23. The method of claim 21 , wherein the switching matrix selectively maps the image stream input interface to more than one image processing module or to more than one image stream output interface simultaneously.
24. The method of claim 21 , wherein the switching matrix selectively maps the processed image stream to more than one image processing module or to more than one image stream output interface simultaneously.
25. The method of claim 21 , wherein the switching matrix selectively maps the image stream input interface or the processed image stream based on a criterion implemented through the control interface.
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/264,596 US20200000529A1 (en) | 2009-05-29 | 2019-01-31 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
US16/857,087 US20200360094A1 (en) | 2009-05-29 | 2020-04-23 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
US17/337,364 US11576731B2 (en) | 2009-05-29 | 2021-06-02 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
US18/108,267 US20230291779A1 (en) | 2009-05-29 | 2023-02-10 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
Applications Claiming Priority (9)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18262409P | 2009-05-29 | 2009-05-29 | |
US23457709P | 2009-08-17 | 2009-08-17 | |
US12/776,048 US8266333B1 (en) | 2009-05-29 | 2010-05-07 | System and method for parallel image processing and routing |
US201361865037P | 2013-08-12 | 2013-08-12 | |
US14/107,329 US9526586B2 (en) | 2013-08-12 | 2013-12-16 | Software tools platform for medical environments |
US15/170,575 US10142641B2 (en) | 2016-06-01 | 2016-06-01 | System and method for parallel image processing and routing |
US15/456,458 US10758314B2 (en) | 2011-12-12 | 2017-03-10 | Enhanced video enabled software tools for medical environments |
US15/657,080 US10226303B2 (en) | 2009-05-29 | 2017-07-21 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
US16/264,596 US20200000529A1 (en) | 2009-05-29 | 2019-01-31 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/657,080 Continuation US10226303B2 (en) | 2009-05-29 | 2017-07-21 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/857,087 Continuation US20200360094A1 (en) | 2009-05-29 | 2020-04-23 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
Publications (1)
Publication Number | Publication Date |
---|---|
US20200000529A1 true US20200000529A1 (en) | 2020-01-02 |
Family
ID=69007437
Family Applications (5)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/657,080 Active US10226303B2 (en) | 2009-05-29 | 2017-07-21 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
US16/264,596 Abandoned US20200000529A1 (en) | 2009-05-29 | 2019-01-31 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
US16/857,087 Abandoned US20200360094A1 (en) | 2009-05-29 | 2020-04-23 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
US17/337,364 Active US11576731B2 (en) | 2009-05-29 | 2021-06-02 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
US18/108,267 Pending US20230291779A1 (en) | 2009-05-29 | 2023-02-10 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/657,080 Active US10226303B2 (en) | 2009-05-29 | 2017-07-21 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
Family Applications After (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/857,087 Abandoned US20200360094A1 (en) | 2009-05-29 | 2020-04-23 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
US17/337,364 Active US11576731B2 (en) | 2009-05-29 | 2021-06-02 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
US18/108,267 Pending US20230291779A1 (en) | 2009-05-29 | 2023-02-10 | System and method for advanced data management with video enabled software tools for video broadcasting environments |
Country Status (1)
Country | Link |
---|---|
US (5) | US10226303B2 (en) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11025693B2 (en) | 2017-08-28 | 2021-06-01 | Banjo, Inc. | Event detection from signal data removing private information |
US10313413B2 (en) * | 2017-08-28 | 2019-06-04 | Banjo, Inc. | Detecting events from ingested communication signals |
US10581945B2 (en) | 2017-08-28 | 2020-03-03 | Banjo, Inc. | Detecting an event from signal data |
US10585724B2 (en) | 2018-04-13 | 2020-03-10 | Banjo, Inc. | Notifying entities of relevant events |
US11611784B2 (en) * | 2019-08-02 | 2023-03-21 | Dao Lab Limited | System and method for transferring large video files with reduced turnaround time |
US11627243B2 (en) * | 2021-07-23 | 2023-04-11 | Phaox LLC | Handheld wireless endoscope image streaming apparatus |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6271752B1 (en) * | 1998-10-02 | 2001-08-07 | Lucent Technologies, Inc. | Intelligent multi-access system |
US20020149617A1 (en) * | 2001-03-30 | 2002-10-17 | Becker David F. | Remote collaboration technology design and methodology |
US7039723B2 (en) * | 2001-08-31 | 2006-05-02 | Hinnovation, Inc. | On-line image processing and communication system |
US8063936B2 (en) * | 2004-06-01 | 2011-11-22 | L-3 Communications Corporation | Modular immersive surveillance processing system and method |
US8295649B2 (en) * | 2006-11-08 | 2012-10-23 | Nextgenid, Inc. | System and method for parallel processing of images from a large number of cameras |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5694150A (en) * | 1995-09-21 | 1997-12-02 | Elo Touchsystems, Inc. | Multiuser/multi pointing device graphical user interface system |
US7197513B2 (en) * | 2000-12-08 | 2007-03-27 | Aol Llc | Distributed image storage architecture |
US6880002B2 (en) * | 2001-09-05 | 2005-04-12 | Surgient, Inc. | Virtualized logical server cloud providing non-deterministic allocation of logical attributes of logical servers to physical resources |
US7539504B2 (en) * | 2001-12-05 | 2009-05-26 | Espre Solutions, Inc. | Wireless telepresence collaboration system |
US7219127B2 (en) * | 2003-03-13 | 2007-05-15 | Oracle International Corporation | Control unit operations in a real-time collaboration server |
JP4760572B2 (en) * | 2006-06-30 | 2011-08-31 | ソニー株式会社 | Editing apparatus, editing method, and program |
US9245020B2 (en) * | 2011-12-14 | 2016-01-26 | Microsoft Technology Licensing, Llc | Collaborative media sharing |
-
2017
- 2017-07-21 US US15/657,080 patent/US10226303B2/en active Active
-
2019
- 2019-01-31 US US16/264,596 patent/US20200000529A1/en not_active Abandoned
-
2020
- 2020-04-23 US US16/857,087 patent/US20200360094A1/en not_active Abandoned
-
2021
- 2021-06-02 US US17/337,364 patent/US11576731B2/en active Active
-
2023
- 2023-02-10 US US18/108,267 patent/US20230291779A1/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6271752B1 (en) * | 1998-10-02 | 2001-08-07 | Lucent Technologies, Inc. | Intelligent multi-access system |
US20020149617A1 (en) * | 2001-03-30 | 2002-10-17 | Becker David F. | Remote collaboration technology design and methodology |
US7039723B2 (en) * | 2001-08-31 | 2006-05-02 | Hinnovation, Inc. | On-line image processing and communication system |
US8063936B2 (en) * | 2004-06-01 | 2011-11-22 | L-3 Communications Corporation | Modular immersive surveillance processing system and method |
US8295649B2 (en) * | 2006-11-08 | 2012-10-23 | Nextgenid, Inc. | System and method for parallel processing of images from a large number of cameras |
Also Published As
Publication number | Publication date |
---|---|
US20180368926A1 (en) | 2018-12-27 |
US20230291779A1 (en) | 2023-09-14 |
US11576731B2 (en) | 2023-02-14 |
US10226303B2 (en) | 2019-03-12 |
US20200360094A1 (en) | 2020-11-19 |
US20220039880A1 (en) | 2022-02-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11576731B2 (en) | System and method for advanced data management with video enabled software tools for video broadcasting environments | |
US11330311B2 (en) | Transmission device, transmission method, receiving device, and receiving method for rendering a multi-image-arrangement distribution service | |
US8589992B2 (en) | Video workflow automation platform for publishing a video feed in multiple formats | |
US8918820B2 (en) | Video workflow automation platform | |
US10567765B2 (en) | Streaming multiple encodings with virtual stream identifiers | |
US6907081B2 (en) | MPEG encoder control protocol for on-line encoding and MPEG data storage | |
EP1239674B1 (en) | Recording broadcast data | |
US20120266203A1 (en) | Ingest-once write-many broadcast video production system | |
EP2602995A1 (en) | Content supplying device, content supplying method, content reproduction device, content reproduction method, program, and content viewing system | |
US11895352B2 (en) | System and method for operating a transmission network | |
EP2135100B1 (en) | Converting video data into video streams | |
US9008488B2 (en) | Video recording apparatus and camera recorder | |
KR101228942B1 (en) | Apparatus and method for managing contents based on ip | |
JP6178893B1 (en) | Unicast delivery apparatus, playback synchronization system, unicast delivery method, and unicast delivery program | |
KR101447190B1 (en) | Input-output System for eding and playing of UHD image contents | |
EP1534005A2 (en) | Method and apparatus for recording broadcast data | |
Raps et al. | Field-to-Air-to-Online-to-Mobile: Cross-Platform Integration and New Technology Approaches to News & Sports System Design |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |