WO2016082285A1 - Video processing apparatus and method and terminal device - Google Patents

Video processing apparatus and method and terminal device Download PDF

Info

Publication number
WO2016082285A1
WO2016082285A1 PCT/CN2014/095597 CN2014095597W WO2016082285A1 WO 2016082285 A1 WO2016082285 A1 WO 2016082285A1 CN 2014095597 W CN2014095597 W CN 2014095597W WO 2016082285 A1 WO2016082285 A1 WO 2016082285A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
notification
lock
capture device
copy
Prior art date
Application number
PCT/CN2014/095597
Other languages
French (fr)
Chinese (zh)
Inventor
张伟
刘清
Original Assignee
广州华多网络科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州华多网络科技有限公司 filed Critical 广州华多网络科技有限公司
Publication of WO2016082285A1 publication Critical patent/WO2016082285A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234309—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs

Definitions

  • the present invention relates to the field of video processing, and in particular, to a video processing apparatus, method, and terminal device.
  • the video stream live broadcast system includes a server and a client.
  • the client is divided into a host client and a viewer client.
  • the server sets a channel, or a virtual room, and the anchor can control and upload the live content in the channel.
  • the server will This content is pushed to the client and displayed on the client.
  • the channel is an information interaction platform provided by the server for the user.
  • the anchor client uploads the video stream to the server, and the server broadcasts the video stream uploaded by the anchor client in the channel.
  • the server sends the video to the audience client.
  • the video stream is pushed by the viewer, and the video stream is parsed by the viewer client.
  • the viewer client can interact with the anchor client within the channel and other viewer clients entering the channel.
  • the method for generating the video stream by the anchor client is as follows: the anchor client collects the video source by the video capture device to generate a video image, copies the captured video image to generate a copy, sends the copy to the video encoder for compression processing, and compresses After processing, a preset video stream is generated and the video stream is uploaded to the server.
  • the current video compression coding scheme stores the original video image in the buffer corresponding to the video collection device after the video capture device collects and generates the original video image, and the video encoder copies the original video image in the cache to the local. After the buffering, subsequent video compression encoding processing is performed. It can be seen that the original video image needs to be copied before compression coding. Since the amount of original video image data is very large, the copy operation will inevitably increase the CPU consumption and occupy a large amount of storage space.
  • a technical problem to be solved by embodiments of the present invention is to provide a video processing method, apparatus, and client.
  • the problem that the CPU resource consumption is large and the storage space is large in the video processing process in the prior art can be solved.
  • an embodiment of the present invention provides a video processing method, including:
  • the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image
  • an embodiment of the present invention provides a video processing apparatus, including:
  • a receiving module configured to receive a lock notification sent by the video capture device, where the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image;
  • a conversion module configured to determine whether the video encoder is in an idle state, and if yes, generate a copy of the video image, and convert the format of the copy into a preset format
  • An unlocking notification module configured to send an unlocking notification to the video collection device after the format conversion operation is completed, where the unlocking notification is used to instruct the video collection device to release the locking of the video image;
  • an encoding indication module configured to send a copy of the preset format to the video encoder, so that the video encoder performs compression encoding processing on the copy of the preset format.
  • an embodiment of the present invention provides a client, including:
  • Video capture device video encoder and video processing device
  • the video capture device generates a video image, locks the video image, and sends a lock notification to the video processing device after completing the locking operation;
  • the video processing device receives a lock notification sent by the video capture device; the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image;
  • the video processing device determines whether the video encoder is in an idle state, and if so, converts the video image into a copy of a preset format
  • the video processing device sends an unlock notification to the video collection device, where the unlock notification is used to instruct the video capture device to release the lock on the video image;
  • the video capture device After receiving the unlock notification, the video capture device releases the lock on the video image
  • the video encoder performs compression coding processing on the copy of the preset format.
  • Receiving a lock notification of the video capture device when determining that the video encoder is in an idle state, directly performing a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid copying the uncompressed video image. Effectively reduce CPU consumption and reduce storage space.
  • FIG. 1 is a structural diagram of a live broadcast network according to an embodiment of the present invention.
  • FIG. 2 is a schematic flowchart diagram of a video processing method according to a first embodiment of the present invention.
  • FIG. 3 is a schematic flowchart of a video processing method according to a second embodiment of the present invention.
  • FIG. 4 is a schematic flowchart of a video processing apparatus according to a first embodiment of the present invention.
  • FIG. 5 is a schematic flowchart of a video processing apparatus according to a second embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a client according to an embodiment of the present invention.
  • FIG. 1 is a schematic diagram of a live broadcast network architecture according to an embodiment of the present invention.
  • the live network architecture may include a service device and a plurality of clients.
  • the service device is also referred to as a background server, a component server, or the like, and the service device may communicate with multiple clients through the Internet.
  • the channel is simply a virtual space (or a virtual room) created by the service device, and the viewer client located in the virtual space can view not only the live content of the anchor client but also the anchor client. Voice or text interaction.
  • the anchor client not only has access control and kick-out control access to the rest of the viewer clients in its channel, but also has the right to broadcast live.
  • the anchor client and the viewer client can implement various functions including a group chat, a live video, a channel K song, an online game, and an online video.
  • the anchor client and the viewer client can run on, but not limited to, a mobile phone, a mobile computer, a tablet, a personal digital assistant (PDA), a media player, and a smart device.
  • a mobile phone a mobile computer
  • a tablet a personal digital assistant
  • TV smart watches, smart glasses, smart bracelets and other user devices.
  • FIG. 1 A video processing method according to an embodiment of the present invention is described in detail below with reference to FIG. 1 and FIG.
  • FIG. 1 is a schematic flowchart of a video processing method according to a first embodiment of the present invention.
  • the method includes:
  • S101 Receive a lock notification sent by the video capture device.
  • the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image.
  • the video capture device is configured to generate an uncompressed video image, where the generated video image includes multiple video frames, and the video capture device may use the optical sensor to collect and generate a video image from the external environment, for example, the video capture device is a camera.
  • the video capture device may also be an application software, such as screen recording software, that generates a video image by screen recording a designated area of the user interface of the user device.
  • the captured video image is stored in the buffer of the video capture device, and the video image in the cache is locked.
  • the lock operation means that the video processing device owns the video.
  • Exclusive operation rights of images except for video processing devices (such as devices, programs, and threads) that do not have video images
  • the operation rights while the video image in the cache is locked directly to the cache of the video processing device.
  • the video processing device After the video capture device completes the locking operation on the video image, the video processing device sends a lock notification to the video processing device, and the video processing device receives the lock notification to determine that the video capture device completes the locking operation on the video image, and the video processing device can use the locked video image to perform the lock operation. Subsequent processing.
  • S102 Determine whether the video encoder is in an idle state. If yes, convert the video image into a copy of a preset format.
  • the video encoder is configured to compress and encode the video image to remove the data information of the video image to reduce the file size of the video image.
  • the video processing device determines whether the video encoder is in an idle state, and the determining method can determine by querying a status identifier bit indicating an idle state or a busy state of the video encoder. If the video encoder is in an idle state, the video processing device directly utilizes the video.
  • the video image in the locked state of the capture device is formatted to generate a copy of the preset format. In this way, the video image in the buffer of the video capture device is avoided from being copied into the buffer of the video processing device, which reduces the consumption of hardware resources when the video image is copied.
  • the video capture device is notified to unlock the video image. After the lock is released, the video capture device can preempt the operation right of the video image.
  • S104 Send a copy of the preset format to the video encoder, so that the video encoder performs compression coding processing on the copy of the preset format.
  • the video encoder compresses and encodes the copy of the formatted video image to reduce information redundancy.
  • the compression encoding algorithm may be any compression coding algorithm in the prior art, for example, MPEG-4, JVT.
  • the present invention is not limited to H.264 and AVC.
  • An embodiment of the present invention receives a lock notification of a video capture device, and when determining that the video encoder is in an idle state, directly performs a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid uncompressed The video image is copied, which can effectively reduce the CPU consumption and reduce the storage space.
  • FIG. 2 is a schematic flowchart of a video processing method according to a second embodiment of the present invention.
  • the method includes:
  • the video capture device is configured to generate an uncompressed video image, where the generated video image includes multiple video frames, and the video capture device may use the optical sensor to collect and generate a video image from the external environment, for example, the video capture device is a camera.
  • the video capture device may also be an application software, such as screen recording software, that generates a video image by screen recording a designated area of the user interface of the user device.
  • the captured video image is saved into the cache, and the video image in the cache is locked.
  • the lock operation means that the video processing device has exclusive use of the video image. Operation rights, entities other than video processing devices (such as devices, programs, and threads) do not have operational rights to video images.
  • the video processing device After the video capture device completes the locking operation on the video image, the video processing device sends a lock notification to the video processing device, and the video processing device receives the lock notification from the video capture device to determine that the video capture device completes the lock operation on the video image, and the video processing device can utilize the video capture device.
  • the locked video image is processed for subsequent processing.
  • the number of video frames included in the video image may be determined according to the processing capability of the video encoder.
  • the video processing device initiates a detection thread and waits for a lock notification sent by the video capture device.
  • the video capture device can lock the captured video image by using a resource lock, where the resource lock is used to represent a mutual exclusion mechanism, and needs to be acquired before performing operations (such as a read operation or a write operation) on the target resource.
  • the resource lock corresponding to the target resource After the operation request object acquires the resource lock of the target resource, the other operation request object can no longer operate on the target resource.
  • the video collection device After the video collection device generates the video resource, the resource lock corresponding to the video image may be set to the occupied state, and the identity representation of the video processing device is added to the owner list of the resource lock, indicating that the video processing device has exclusive use of the video image. Operational authority.
  • the lock notification is received, and the lock notification is used to indicate that the video capture device completes the lock operation on the generated video image.
  • the lock notification further includes an address range of the video image, and the video processing device may determine the video image in the locked state on the video capture device according to the address range, so that the video processing device uses the locked video image for subsequent processing.
  • a corresponding coding thread and a video are generated in the operating system.
  • the processing device determines the encoding thread of the video encoder in the operating system, and queries the status identifier of the encoding thread.
  • the status identifier is used to indicate that the encoding thread is in an idle state or a busy state, and determines whether the video encoder is idle according to the queried status identifier. status. For example, a "0" indicates that the encoding thread is in an idle state, and a "1" indicates that the encoding thread is in a busy state.
  • the video processing device may query the status of the video encoder by using a status query interface provided by the video encoder, and query the status identifier indicating the status of the video encoder by the status query interface to determine that the video encoder is in an idle state or a busy state. . If the video encoder is in an idle state, S206 is executed, and if it is in a busy state, S204 is executed.
  • the video processing device determines that the video encoder is in a busy state, the video encoder cannot perform compression coding processing on the video image input by the video capture device, and the video processing device sends an unlock notification to the video capture device to notify the video capture device to release. After the video image is locked and unlocked, other entities can obtain the operation authority for the video image through competition.
  • the video processing device sends an unlock notification to the video capture device, the video capture device sets the state of the resource lock of the video image to an idle state, and sets the identity of the video capture device from the owner list. Removed in to remove the lock on the video image.
  • the time that the lock notification sent by the video collection device is not received is greater than a preset duration.
  • the video processing device calculates a time interval between the current time and the last time the lock notification sent by the video capture device is received. If the calculated time interval is greater than the preset duration, the video capture device is within the preset duration. Did not receive the lock notification, if you continue to wait for the input notification will cause a waste of hardware resources.
  • the video processing device detects whether a lock notification is received every one minute, and sets the preset duration to 5 minutes, that is, when the lock notification sent by the video capture device is still not received at the sixth detection period, S209 is executed. It can be understood that the detection period and the preset duration of the video processing device can be set according to actual needs, and the specific values are not limited by the present invention.
  • the video processing device performs a format conversion operation on the video image in the locked state, and generates a copy of the preset format after the format conversion to meet the requirements of the video encoder.
  • the video encoder requires that the format condition of the image to be encoded input is: resolution 1024*768, the color space is RGB, the resolution of the video image generated by the video capture device is 2048*1024, the color space is YIQ, the generated copy of the video image is the same as the attribute information of the video image, and the video processing device is based on the YIQ color space.
  • the format condition of the image to be encoded input is: resolution 1024*768, the color space is RGB, the resolution of the video image generated by the video capture device is 2048*1024, the color space is YIQ, the generated copy of the video image is the same as the attribute information of the video image, and the video processing device is based on the YIQ color space.
  • RGB color space converting YIQ color space into RGB color space, the corresponding relationship between them is:
  • the video processing device reduces the video image to meet the resolution requirements of the video encoder.
  • the video processing apparatus may perform other pre-processing on the copy, such as noise filtering, watermarking, or sampling processing, before compressing and encoding the copy of the preset format.
  • the video processing device sends an unlock notification to the video capture device to notify the video capture device to unlock the video image.
  • the video processing device sends an unlock notification to the video capture device to notify the video capture device to unlock the video image.
  • other entities may acquire the pair through competition. The operation rights of the video image.
  • the video processing device sends an unlock notification to the video capture device, the video capture device sets the state of the resource lock of the video image to an idle state, and sets the identity of the video capture device from the owner list. Removed in to remove the lock on the video image.
  • the video encoder performs compression coding processing on the copy of the formatted video image to reduce information redundancy
  • the compression encoding algorithm may be any compression coding algorithm in the prior art, for example, MPEG- 4. JVT, H.264, AVC, etc., and the invention is not limited.
  • the exit identifier is used to instruct the video processing processing device to stop working, and the exit identifier may be set by the user or set by the video collection device. For example, after the video capture device completes the acquisition operation of the video source, the video capture device does not generate a new video image, and the video capture device sets the exit identifier of the video processing device. When the video processing device detects that the exit identifier is set, the video processing device stops working and releases. Occupied hardware resources, hardware resources include memory resources and CPU resources. The video processing device determines whether the exit flag is set, and if so, executes S210, and if NO, executes S201.
  • the exit operation is performed, and the operating system correspondingly sets the exit identifier of the video processing device in the register, and when the video processing device detects that the exit identifier is set, the operation stops, and the occupied hardware resources are released. .
  • An embodiment of the present invention receives a lock notification of a video capture device, and when determining that the video encoder is in an idle state, directly performs a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid uncompressed The video image is copied, which can effectively reduce the CPU consumption and reduce the storage space.
  • FIG. 3 is a schematic structural diagram of a video processing apparatus according to a first embodiment of the present invention.
  • the video processing apparatus includes a receiving module 10, a converting module 11, an unlocking notification module 12, and an encoding indication.
  • Module 13 is a receiving module 10, a converting module 11, an unlocking notification module 12, and an encoding indication.
  • the receiving module 10 is configured to receive a lock notification sent by the video capture device, where the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image.
  • the conversion module 11 is configured to determine whether the video encoder is in an idle state, and if so, convert the video image into a copy of a preset format.
  • the unlocking notification module 12 is configured to send an unlocking notification to the video collection device after the format conversion operation is completed, where the unlocking notification is used to instruct the video collection device to release the locking of the video image.
  • the code indication module 13 is configured to send a copy of the preset format to the video encoder, so that the video encoder performs compression coding processing on the copy of the preset format.
  • An embodiment of the present invention receives a lock notification of a video capture device, and when determining that the video encoder is in an idle state, directly performs a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid uncompressed The video image is copied, which can effectively reduce the CPU consumption and reduce the storage space.
  • FIG. 5 is a schematic structural diagram of a video processing apparatus according to a second embodiment of the present invention.
  • the device includes a timeout processing module 14 and a state processing module 15 in addition to the receiving module 10, the converting module 11, the unlocking notification module 12, and the encoding indication module 13.
  • the timeout processing module 14 is configured to determine whether the exit identifier is set when the time for not receiving the lock notification sent by the video collection device is greater than a preset duration, and if yes, release the occupied hardware resource, where the hardware resource includes CPU resources and memory resources.
  • the state processing module 15 is configured to determine whether the video encoder is in an idle state, and if not, send an unlock notification to the video collection device; if the exit identifier is set, release the occupied hardware resource.
  • the conversion module 11 is configured to:
  • the preset format includes a preset color space and/or a preset size.
  • An embodiment of the present invention receives a lock notification of a video capture device, and when determining that the video encoder is in an idle state, directly performs a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid uncompressed The video image is copied, which can effectively reduce the CPU consumption and reduce the storage space.
  • FIG. 6 is a schematic structural diagram of a client according to an embodiment of the present invention.
  • the client 2 includes a video capture device 20, a video processing device 21, and a video encoder 22.
  • the client 2 may be a live broadcast.
  • the anchor client in the network architecture, the anchor client is used to compress and encode the captured video image and upload it to the server device, and then the service device sends the processed video image to multiple audience clients.
  • the working process of the client 2 is: the video capture device 20 generates a video image, locks the video image, and sends a lock notification to the video processing device 21 after completing the locking operation; the video processing device 21 receiving a lock notification sent by the video capture device 20; the lock notification is used to indicate that the video capture device 20 completes the lock operation on the generated video image; the video processing device 21 determines whether the video encoder is in an idle state, and if so, The video map Converting to a copy of the preset format; after the format conversion operation is completed, the video processing device 21 sends an unlock notification to the video capture device 20, the unlock notification is used to instruct the video capture device 20 to unlock the video image; After receiving the unlocking notification, the collecting device 20 releases the locking of the video image; the video processing device 21 sends a copy of the preset format to the video encoder 22; the video encoder 22 pairs the preset format The copy is subjected to compression encoding processing, and a copy of the compression-coded preset format is
  • An embodiment of the present invention receives a lock notification of a video capture device, and when determining that the video encoder is in an idle state, directly performs a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid uncompressed The video image is copied, which can effectively reduce the CPU consumption and reduce the storage space.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

Disclosed is a video processing method, which comprises: receiving a locking notification sent by a video collection apparatus, wherein the locking notification is used for indicating that the video collection apparatus completes locking operation on a generated video image; determining whether a video coder is in an idle state, and if yes, converting the video image into a copy of a preset format; sending an unlocking notification to the video collection apparatus after the format conversion operation is completed, wherein the unlocking notification is used for instructing the video collection apparatus to unlock the video image; and sending the copy of the preset format to the video coder to enable the video coder to compress and code the copy of the preset format. Also disclosed are a video processing apparatus and a client. By means of the present invention, the hardware resource consumption is reduced.

Description

一种视频处理装置、方法和终端设备Video processing device, method and terminal device
本发明要求2014年11月28日递交的发明名称为“一种视频处理装置、方法和终端设备”的申请号201410714669.7的在先申请优先权，上述在先申请的内容以引入的方式并入本文本中。The present invention claims the priority of the prior application entitled "A Video Processing Apparatus, Method, and Terminal Device", filed on Nov. 28, 2014, the disclosure of which is incorporated herein by reference. This.
技术领域Technical field
本发明涉及视频处理领域，尤其涉及一种视频处理装置、方法和终端设备。The present invention relates to the field of video processing, and in particular, to a video processing apparatus, method, and terminal device.
背景技术Background technique
视频流直播***包括服务器和客户端，客户端分为主播客户端和观众客户端，通过服务器来设置一个频道，或者说是虚拟房间，主播可以在频道内进行直播内容的控制和上传，服务器将这些内容推送至客户端，在客户端进行展示。The video stream live broadcast system includes a server and a client. The client is divided into a host client and a viewer client. The server sets a channel, or a virtual room, and the anchor can control and upload the live content in the channel. The server will This content is pushed to the client and displayed on the client.
频道是服务器为用户提供的一种信息交互平台，主播客户端将视频流上传至服务器，服务器在频道内广播主播客户端上传的视频流，观众客户端进入主播客户端的频道后，服务器向观众客户端推送视频流，由观众客户端对视频流进行解析。同时，观众客户端在频道内可以与该频道内的主播客户端以及进入该频道的其他观众客户端之间进行信息交互。The channel is an information interaction platform provided by the server for the user. The anchor client uploads the video stream to the server, and the server broadcasts the video stream uploaded by the anchor client in the channel. After the viewer client enters the channel of the anchor client, the server sends the video to the audience client. The video stream is pushed by the viewer, and the video stream is parsed by the viewer client. At the same time, the viewer client can interact with the anchor client within the channel and other viewer clients entering the channel.
目前主播客户端生成视频流的方法为：主播客户端通过视频采集装置对视频源进行采集生成视频图像，将采集到的视频图像进行复制生成副本，将副本送往视频编码器进行压缩处理，压缩处理后生成预设的视频码流以及将视频码流上传至服务器。申请人发现，目前的视频压缩编码方案在视频采集装置在采集生成原始视频图像后，将原始视频图像存放至视频采集装置对应的缓存中，视频编码器将该缓存中的原始视频图像拷贝至本地的缓存后再进行后续的视频压缩编码处理。由此可见，在压缩编码前需要对原始视频图像进行复制，由于原始视频图像数据量十分巨大，复制操作势必会增加CPU的消耗和占用大量的存储空间。At present, the method for generating the video stream by the anchor client is as follows: the anchor client collects the video source by the video capture device to generate a video image, copies the captured video image to generate a copy, sends the copy to the video encoder for compression processing, and compresses After processing, a preset video stream is generated and the video stream is uploaded to the server. The applicant finds that the current video compression coding scheme stores the original video image in the buffer corresponding to the video collection device after the video capture device collects and generates the original video image, and the video encoder copies the original video image in the cache to the local. After the buffering, subsequent video compression encoding processing is performed. It can be seen that the original video image needs to be copied before compression coding. Since the amount of original video image data is very large, the copy operation will inevitably increase the CPU consumption and occupy a large amount of storage space.
发明内容Summary of the invention
本发明实施例所要解决的技术问题在于，提供一种视频处理方法、装置和客户端。可解决现有技术中视频处理过程中CPU资源消耗大和占用存储空间大的问题。A technical problem to be solved by embodiments of the present invention is to provide a video processing method, apparatus, and client. The problem that the CPU resource consumption is large and the storage space is large in the video processing process in the prior art can be solved.
为了解决上述技术问题，本发明实施例提供了一种视频处理方法，包括：In order to solve the above technical problem, an embodiment of the present invention provides a video processing method, including:
接收视频采集装置发送的锁定通知；所述锁定通知用于表示所述视频采集装置完成对生成的视频图像的锁定操作；Receiving a lock notification sent by the video capture device; the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image;
判断视频编码器是否处于空闲状态，若为是，将所述视频图像转换成预设格式的副本；Determining whether the video encoder is in an idle state, and if so, converting the video image into a copy of a preset format;
当格式转换操作完成后，向所述视频采集装置发送解锁通知，所述解锁通知用于指示所述视频采集装置解除对所述视频图像的锁定；After the format conversion operation is completed, sending an unlock notification to the video collection device, where the unlock notification is used to instruct the video capture device to release the lock on the video image;
将所述预设格式的副本发送至所述视频编码器，以使所述视频编码器对所述预设格式的副本进行压缩编码处理。Sending a copy of the preset format to the video encoder, so that the video encoder performs compression coding processing on a copy of the preset format.
相应的，本发明实施例提供了一种视频处理装置，包括：Correspondingly, an embodiment of the present invention provides a video processing apparatus, including:
接收模块，用于接收视频采集装置发送的锁定通知；所述锁定通知用于表示所述视频采集装置完成对生成的视频图像的锁定操作；a receiving module, configured to receive a lock notification sent by the video capture device, where the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image;
转换模块，用于判断视频编码器是否处于空闲状态，若为是，生成所述视频图像的副本，并将所述副本的格式转换成预设格式；a conversion module, configured to determine whether the video encoder is in an idle state, and if yes, generate a copy of the video image, and convert the format of the copy into a preset format;
解锁通知模块，用于当格式转换操作完成后，向所述视频采集装置发送解锁通知，所述解锁通知用于指示所述视频采集装置解除对所述视频图像的锁定；An unlocking notification module, configured to send an unlocking notification to the video collection device after the format conversion operation is completed, where the unlocking notification is used to instruct the video collection device to release the locking of the video image;
编码指示模块，用于将所述预设格式的副本发送至所述视频编码器，以使所述视频编码器对所述预设格式的副本进行压缩编码处理。And an encoding indication module, configured to send a copy of the preset format to the video encoder, so that the video encoder performs compression encoding processing on the copy of the preset format.
相应的，本发明实施例提供了一种客户端，包括：Correspondingly, an embodiment of the present invention provides a client, including:
视频采集装置、视频编码器和视频处理装置；Video capture device, video encoder and video processing device;
所述视频采集装置生成视频图像，将所述视频图像进行锁定，以及在完成锁定操作后，向所述视频处理装置发送锁定通知；The video capture device generates a video image, locks the video image, and sends a lock notification to the video processing device after completing the locking operation;
所述视频处理装置接收视频采集装置发送的锁定通知；所述锁定通知用于表示所述视频采集装置完成对生成的视频图像的锁定操作；The video processing device receives a lock notification sent by the video capture device; the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image;
所述视频处理装置判断视频编码器是否处于空闲状态，若为是，将所述视频图像转换成预设格式的副本； The video processing device determines whether the video encoder is in an idle state, and if so, converts the video image into a copy of a preset format;
当格式转换操作完成后，所述视频处理装置向所述视频采集装置发送解锁通知，所述解锁通知用于指示所述视频采集装置解除对所述视频图像的锁定；After the format conversion operation is completed, the video processing device sends an unlock notification to the video collection device, where the unlock notification is used to instruct the video capture device to release the lock on the video image;
所述视频采集装置接收到所述解锁通知后，解除对所述视频图像的锁定；After receiving the unlock notification, the video capture device releases the lock on the video image;
所述视频处理装置将所述预设格式的副本发送至所述视频编码器；Transmitting, by the video processing device, a copy of the preset format to the video encoder;
所述视频编码器对所述预设格式的副本进行压缩编码处理。The video encoder performs compression coding processing on the copy of the preset format.
实施本发明实施例，具有如下有益效果：Embodiments of the present invention have the following beneficial effects:
接收到视频采集装置的锁定通知，在判断视频编码器处于空闲状态时，对视频采集装置生成的锁定状态的视频图像直接进行后续的格式转换处理，避免对未压缩的视频图像进行复制操作，可有效的减少了CPU的消耗和减少了存储空间的占用。Receiving a lock notification of the video capture device, when determining that the video encoder is in an idle state, directly performing a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid copying the uncompressed video image. Effectively reduce CPU consumption and reduce storage space.
附图说明DRAWINGS
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and other drawings can be obtained from those skilled in the art without any creative work.
图1是本发明实施例提供的一种直播网络结构图；1 is a structural diagram of a live broadcast network according to an embodiment of the present invention;
图2是本发明第一实施例提供的一种视频处理方法的流程示意图。FIG. 2 is a schematic flowchart diagram of a video processing method according to a first embodiment of the present invention.
图3是本发明第二实施例提供的一种视频处理方法的流程示意图；3 is a schematic flowchart of a video processing method according to a second embodiment of the present invention;
图4是本发明第一实施例提供的一种视频处理装置的流程示意图；4 is a schematic flowchart of a video processing apparatus according to a first embodiment of the present invention;
图5是本发明第二实施例提供的一种视频处理装置的流程示意图；FIG. 5 is a schematic flowchart of a video processing apparatus according to a second embodiment of the present invention; FIG.
图6是本发明实施例提供的一种客户端的结构示意图。FIG. 6 is a schematic structural diagram of a client according to an embodiment of the present invention.
具体实施方式detailed description
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
为了更好理解本发明实施例公开的一种视频处理方法、装置和客户端，下 面先对本发明实施例适用的直播网络构架进行描述。请参阅图1，图1是本发明实施例公开的一种直播网络构架示意图。如图1所示，该直播网络构架可以包括服务设备以及多个客户端，其中，服务设备也被称后台服务器、组件服务器等，服务设备可以通过互联网与多个客户端进行通信连接。In order to better understand a video processing method, apparatus, and client disclosed in the embodiments of the present invention, The live network architecture to which the embodiments of the present invention are applied is described first. Please refer to FIG. 1. FIG. 1 is a schematic diagram of a live broadcast network architecture according to an embodiment of the present invention. As shown in FIG. 1 , the live network architecture may include a service device and a plurality of clients. The service device is also referred to as a background server, a component server, or the like, and the service device may communicate with multiple clients through the Internet.
在图1所示的直播网络构架中，多个客户端中的部分客户端可以作为主播客户端，而其余客户端可以作为观众客户端。本发明实施例中，频道简单来讲就是服务设备创建的一种虚拟空间(或虚拟房间)，位于该虚拟空间内的观众客户端可以不仅观看主播客户端的直播内容，还可以和主播客户端进行语音或文字的交互。主播客户端对接入其频道内的其余观众客户端不仅具有接入、踢出控制权，还具有直播主动权。本发明实施例中，主播客户端、观众客户端可以实现包括集合群聊、视频直播、频道K歌、在线游戏、在线影视等多种功能。在图1所示的直播网络构架中，主播客户端、观众客户端可以运行在包括但不限于移动电话、移动电脑、平板电脑、个人数字助理(Personal Digital Assistant,PDA)、媒体播放器、智能电视、智能手表、智能眼镜、智能手环等用户设备上。In the live network architecture shown in FIG. 1, some of the plurality of clients can serve as the anchor client, and the remaining clients can serve as the viewer client. In the embodiment of the present invention, the channel is simply a virtual space (or a virtual room) created by the service device, and the viewer client located in the virtual space can view not only the live content of the anchor client but also the anchor client. Voice or text interaction. The anchor client not only has access control and kick-out control access to the rest of the viewer clients in its channel, but also has the right to broadcast live. In the embodiment of the present invention, the anchor client and the viewer client can implement various functions including a group chat, a live video, a channel K song, an online game, and an online video. In the live network architecture shown in FIG. 1, the anchor client and the viewer client can run on, but not limited to, a mobile phone, a mobile computer, a tablet, a personal digital assistant (PDA), a media player, and a smart device. TV, smart watches, smart glasses, smart bracelets and other user devices.
下面结合图1和图2.，对本发明实施例提供的一种视频处理方法进行详细介绍。A video processing method according to an embodiment of the present invention is described in detail below with reference to FIG. 1 and FIG.
参见图1，为本发明第一实施例提供的一种视频处理方法的流程示意图，在本发明实施例中，所述方法包括：FIG. 1 is a schematic flowchart of a video processing method according to a first embodiment of the present invention. In an embodiment of the present invention, the method includes:
S101、接收视频采集装置发送的锁定通知；所述锁定通知用于表示所述视频采集装置完成对生成的视频图像的锁定操作。S101. Receive a lock notification sent by the video capture device. The lock notification is used to indicate that the video capture device completes a lock operation on the generated video image.
具体的，视频采集装置用于生成未经压缩的视频图像，生成的视频图像包括多个视频帧，视频采集装置可以利用光传感器对外部环境进行采集生成的视频图像，例如，视频采集装置为摄像头、录像机或照相机等；视频采集装置也可以一种应用软件，例如屏幕录像软件，通过对用户设备的用户界面的指定区域进行屏幕录像生成的视频图像。视频采集装置采集并生成未经压缩的视频图像后，将采集到的视频图像存储至视频采集装置的缓存中，并将缓存中的视频图像进行锁定操作，锁定操作是指视频处理装置拥有对视频图像的独占的操作权限，除视频处理装置之外的实体(例如装置、程序和线程)没有对视频图像 的操作权限，同时缓存中处于锁定状态的视频图像直接映射到视频处理装置的缓存。视频采集装置对视频图像完成锁定操作后，向视频处理装置发送锁定通知，视频处理装置接收到锁定通知即可确定视频采集装置完成对视频图像的锁定操作，视频处理装置可以利用锁定的视频图像进行后续的处理。Specifically, the video capture device is configured to generate an uncompressed video image, where the generated video image includes multiple video frames, and the video capture device may use the optical sensor to collect and generate a video image from the external environment, for example, the video capture device is a camera. The video capture device may also be an application software, such as screen recording software, that generates a video image by screen recording a designated area of the user interface of the user device. After the video capture device collects and generates the uncompressed video image, the captured video image is stored in the buffer of the video capture device, and the video image in the cache is locked. The lock operation means that the video processing device owns the video. Exclusive operation rights of images, except for video processing devices (such as devices, programs, and threads) that do not have video images The operation rights, while the video image in the cache is locked directly to the cache of the video processing device. After the video capture device completes the locking operation on the video image, the video processing device sends a lock notification to the video processing device, and the video processing device receives the lock notification to determine that the video capture device completes the locking operation on the video image, and the video processing device can use the locked video image to perform the lock operation. Subsequent processing.
S102、判断视频编码器是否处于空闲状态，若为是，将所述视频图像转换成预设格式的副本。S102. Determine whether the video encoder is in an idle state. If yes, convert the video image into a copy of a preset format.
具体的，视频编码器用于对视频图像进行压缩编码，去除视频图像的数据信息，以减少视频图像的文件大小。视频处理装置判断视频编码器是否处于空闲状态，判断的方法可以通过查询表示视频编码器的空闲状态或忙碌状态的状态标识位来进行判断，若视频编码器处于空闲状态，视频处理装置直接利用视频采集装置的缓存中处于锁定状态的视频图像进行进行格式转换，生成预设格式的副本。这样，避免将视频采集装置的缓存中的视频图像拷贝到视频处理装置的缓冲中，降低了视频图像拷贝时硬件资源的消耗。Specifically, the video encoder is configured to compress and encode the video image to remove the data information of the video image to reduce the file size of the video image. The video processing device determines whether the video encoder is in an idle state, and the determining method can determine by querying a status identifier bit indicating an idle state or a busy state of the video encoder. If the video encoder is in an idle state, the video processing device directly utilizes the video. The video image in the locked state of the capture device is formatted to generate a copy of the preset format. In this way, the video image in the buffer of the video capture device is avoided from being copied into the buffer of the video processing device, which reduces the consumption of hardware resources when the video image is copied.
S103、当格式转换操作完成后，向所述视频采集装置发送解锁通知，所述解锁通知用于指示所述视频采集装置解除对所述视频图像的锁定。S103. After the format conversion operation is completed, send an unlock notification to the video collection device, where the unlock notification is used to instruct the video collection device to release the lock on the video image.
具体的，视频处理装置将副本的格式转换成预设格式的操作完成后，通知视频采集装置解除对视频图像的锁定，解除锁定后，视频采集装置可以抢占视频图像的操作权限。Specifically, after the video processing device converts the format of the copy into the preset format, the video capture device is notified to unlock the video image. After the lock is released, the video capture device can preempt the operation right of the video image.
S104、将所述预设格式的副本发送至所述视频编码器，以使所述视频编码器对所述预设格式的副本进行压缩编码处理。S104. Send a copy of the preset format to the video encoder, so that the video encoder performs compression coding processing on the copy of the preset format.
具体的，视频编码器对格式转换后的视频图像的副本进行压缩编码处理，以减少信息冗余，压缩编码算法可以是现有技术中的任意一种压缩编码算法，例如，MPEG-4、JVT、H.264和AVC等，本发明不作限制。Specifically, the video encoder compresses and encodes the copy of the formatted video image to reduce information redundancy. The compression encoding algorithm may be any compression coding algorithm in the prior art, for example, MPEG-4, JVT. The present invention is not limited to H.264 and AVC.
实施本发明的实施例，接收到视频采集装置的锁定通知，在判断视频编码器处于空闲状态时，对视频采集装置生成的锁定状态的视频图像直接进行后续的格式转换处理，避免对未压缩的视频图像进行复制操作，可有效的减少了CPU的消耗和减少了存储空间的占用。An embodiment of the present invention receives a lock notification of a video capture device, and when determining that the video encoder is in an idle state, directly performs a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid uncompressed The video image is copied, which can effectively reduce the CPU consumption and reduce the storage space.
参见图2，为本发明第二实施例提供的一种视频处理方法的流程示意图，在本发明实施例中，所述方法包括： FIG. 2 is a schematic flowchart of a video processing method according to a second embodiment of the present invention. In the embodiment of the present invention, the method includes:
S201、检测视频采集装置发送的锁定通知。S201. Detect a lock notification sent by the video collection device.
具体的，视频采集装置用于生成未经压缩的视频图像，生成的视频图像包括多个视频帧，视频采集装置可以利用光传感器对外部环境进行采集生成的视频图像，例如，视频采集装置为摄像头、录像机或照相机等；视频采集装置也可以一种应用软件，例如屏幕录像软件，通过对用户设备的用户界面的指定区域进行屏幕录像生成的视频图像。视频采集装置采集并生成未经压缩的视频图像后，将采集到的视频图像保存到缓存中，并将缓存中的视频图像进行锁定操作，锁定操作是指视频处理装置拥有对视频图像的独占的操作权限，除视频处理装置之外的实体(例如装置、程序和线程)没有对视频图像的操作权限。视频采集装置对视频图像完成锁定操作后，向视频处理装置发送锁定通知，视频处理装置接收到视频采集装置接收到锁定通知即可确定视频采集装置完成对视频图像的锁定操作，视频处理装置可以利用锁定的视频图像进行后续的处理。其中，视频图像中包含的视频帧的数量可以根据视频编码器的处理能力来确定。视频处理装置启动检测线程，等待视频采集装置发送的锁定通知。Specifically, the video capture device is configured to generate an uncompressed video image, where the generated video image includes multiple video frames, and the video capture device may use the optical sensor to collect and generate a video image from the external environment, for example, the video capture device is a camera. The video capture device may also be an application software, such as screen recording software, that generates a video image by screen recording a designated area of the user interface of the user device. After the video capture device collects and generates the uncompressed video image, the captured video image is saved into the cache, and the video image in the cache is locked. The lock operation means that the video processing device has exclusive use of the video image. Operation rights, entities other than video processing devices (such as devices, programs, and threads) do not have operational rights to video images. After the video capture device completes the locking operation on the video image, the video processing device sends a lock notification to the video processing device, and the video processing device receives the lock notification from the video capture device to determine that the video capture device completes the lock operation on the video image, and the video processing device can utilize the video capture device. The locked video image is processed for subsequent processing. The number of video frames included in the video image may be determined according to the processing capability of the video encoder. The video processing device initiates a detection thread and waits for a lock notification sent by the video capture device.
可选的，视频采集装置可以通过资源锁对采集到的视频图像进行锁定，资源锁用于表征一种互斥机制，在对目标资源进行操作(例如读操作或写操作)之前，需要获取到目标资源对应的资源锁，操作请求对象获取到目标资源的资源锁后，其他操作请求对象不能再对目标资源进行操作。视频采集装置生成视频资源后，可以将视频图像对应的资源锁设置为占用状态，并将视频处理装置的身份表示添加至该资源锁的拥有者列表中，表示视频处理装置拥有对视频图像独占的操作权限。Optionally, the video capture device can lock the captured video image by using a resource lock, where the resource lock is used to represent a mutual exclusion mechanism, and needs to be acquired before performing operations (such as a read operation or a write operation) on the target resource. The resource lock corresponding to the target resource. After the operation request object acquires the resource lock of the target resource, the other operation request object can no longer operate on the target resource. After the video collection device generates the video resource, the resource lock corresponding to the video image may be set to the occupied state, and the identity representation of the video processing device is added to the owner list of the resource lock, indicating that the video processing device has exclusive use of the video image. Operational authority.
S202、接收视频采集装置发送的锁定通知。S202. Receive a lock notification sent by the video collection device.
具体的，视频处理装置的检测线程检测到视频采集装置发送的锁定通知后，接收该锁定通知，锁定通知用于表示视频采集装置完成对生成的视频图像的锁定操作。其中，锁定通知中还包括视频图像的地址范围，视频处理装置可以根据地址范围确定视频采集装置上处于锁定状态的视频图像，以便于视频处理装置利用锁定的视频图像进行后续的处理。Specifically, after the detection thread of the video processing device detects the lock notification sent by the video capture device, the lock notification is received, and the lock notification is used to indicate that the video capture device completes the lock operation on the generated video image. The lock notification further includes an address range of the video image, and the video processing device may determine the video image in the locked state on the video capture device according to the address range, so that the video processing device uses the locked video image for subsequent processing.
S203、判断视频编码器是否处于空闲状态。S203. Determine whether the video encoder is in an idle state.
具体的，视频编码器启动后，会在操作***中生成对应的编码线程，视频 处理装置确定视频编码器进驻在操作***中的编码线程，并查询编码线程的状态标识，状态标识用于表示编码线程为空闲状态或忙碌状态，根据查询到的状态标识判断视频编码器是否处于空闲状态。例如，用“0”表示编码线程为空闲状态，“1”标识编码线程为忙碌状态。Specifically, after the video encoder is started, a corresponding coding thread and a video are generated in the operating system. The processing device determines the encoding thread of the video encoder in the operating system, and queries the status identifier of the encoding thread. The status identifier is used to indicate that the encoding thread is in an idle state or a busy state, and determines whether the video encoder is idle according to the queried status identifier. status. For example, a "0" indicates that the encoding thread is in an idle state, and a "1" indicates that the encoding thread is in a busy state.
可选的，视频处理装置可以通过视频编码器提供的状态查询接口查询视频编码器的状态，通过状态查询接口查询表示视频编码器的状态的状态标识来确定视频编码器的处于空闲状态或忙碌状态。若视频编码器处于空闲状态，执行S206，若处于忙碌状态，执行S204。Optionally, the video processing device may query the status of the video encoder by using a status query interface provided by the video encoder, and query the status identifier indicating the status of the video encoder by the status query interface to determine that the video encoder is in an idle state or a busy state. . If the video encoder is in an idle state, S206 is executed, and if it is in a busy state, S204 is executed.
S204、向所述视频采集装置发送解锁通知。S204. Send an unlock notification to the video collection device.
具体的，视频处理装置在确定视频编码器处于忙碌状态时，视频编码器无法对视频采集装置输入的视频图像进行压缩编码处理，视频处理装置向视频采集装置发送解锁通知，以通知视频采集装置解除对视频图像的锁定，解除锁定后，其他实体可以通过竞争的方式获取对视频图像的操作权限。Specifically, when the video processing device determines that the video encoder is in a busy state, the video encoder cannot perform compression coding processing on the video image input by the video capture device, and the video processing device sends an unlock notification to the video capture device to notify the video capture device to release. After the video image is locked and unlocked, other entities can obtain the operation authority for the video image through competition.
示例性的，在资源锁的场景中，视频处理装置向视频采集装置发送解锁通知，视频采集装置将视频图像的资源锁的状态设置为空闲状态，并将视频采集装置的身份标识从拥有者列表中删除，以解除对视频图像的锁定。Exemplarily, in the scenario of resource lock, the video processing device sends an unlock notification to the video capture device, the video capture device sets the state of the resource lock of the video image to an idle state, and sets the identity of the video capture device from the owner list. Removed in to remove the lock on the video image.
S205、未接收到所述视频采集装置发送的锁定通知的时间大于预设时长。S205. The time that the lock notification sent by the video collection device is not received is greater than a preset duration.
具体的，视频处理装置计算当前时刻和上一次接收到视频采集装置发送的锁定通知的时刻之间的时间间隔，若计算得到的时间间隔大于预设时长，则表明视频采集装置在预设时长内都没有接收到锁定通知，如果继续等待输入通知就会造成硬件资源的浪费。Specifically, the video processing device calculates a time interval between the current time and the last time the lock notification sent by the video capture device is received. If the calculated time interval is greater than the preset duration, the video capture device is within the preset duration. Did not receive the lock notification, if you continue to wait for the input notification will cause a waste of hardware resources.
示例性的，视频处理装置每隔1分钟检测是否接收到锁定通知，设置预设时长为5分钟，即在第6个检测周期时仍然没有接收到视频采集装置发送的锁定通知时，执行S209。可以理解的是，视频处理装置的检测周期和预设时长可以根据实际需要进行设定，本发明不对具体数值做限定。Exemplarily, the video processing device detects whether a lock notification is received every one minute, and sets the preset duration to 5 minutes, that is, when the lock notification sent by the video capture device is still not received at the sixth detection period, S209 is executed. It can be understood that the detection period and the preset duration of the video processing device can be set according to actual needs, and the specific values are not limited by the present invention.
S206、将所述视频图像转换成预设格式的副本。S206. Convert the video image into a copy of a preset format.
具体的，视频处理装置将处于锁定状态的视频图像进行格式转换操作，格式转换后生成预设格式的副本，以满足视频编码器的要求。Specifically, the video processing device performs a format conversion operation on the video image in the locked state, and generates a copy of the preset format after the format conversion to meet the requirements of the video encoder.
示例性的，视频编码器要求输入的待编码图像的格式条件为：分辨率 1024*768，颜色空间为RGB，视频采集装置生成的视频图像的分辨率为2048*1024，颜色空间为YIQ，生成的视频图像的副本和视频图像的属性信息相同，视频处理装置根据YIQ颜色空间和RGB颜色空间的对应关系，将YIQ颜色空间转换为RGB颜色空间，二者的对应关系为：Exemplarily, the video encoder requires that the format condition of the image to be encoded input is: resolution 1024*768, the color space is RGB, the resolution of the video image generated by the video capture device is 2048*1024, the color space is YIQ, the generated copy of the video image is the same as the attribute information of the video image, and the video processing device is based on the YIQ color space. Corresponding relationship with RGB color space, converting YIQ color space into RGB color space, the corresponding relationship between them is:
Y＝0.299R+0.587G+0.114BY=0.299R+0.587G+0.114B
I＝0.596R-0.275G-0.321BI=0.596R-0.275G-0.321B
Q＝0.212R-0.523G+0.311BQ=0.212R-0.523G+0.311B
同时，视频处理装置将视频图像进行缩小处理，以符合视频编码器的分辨率的要求。在本发明的实施例中，，在对预设格式的副本进行压缩编码之前，视频处理装置还可以对副本进行其他的预处理，例如，噪声滤除、加水印或抽样处理等。At the same time, the video processing device reduces the video image to meet the resolution requirements of the video encoder. In an embodiment of the present invention, the video processing apparatus may perform other pre-processing on the copy, such as noise filtering, watermarking, or sampling processing, before compressing and encoding the copy of the preset format.
S207、当格式转换操作完成后，向所述视频采集装置发送解锁通知。S207. After the format conversion operation is completed, send an unlock notification to the video collection device.
具体的，当视频处理装置完成格式转换的处理后，Specifically, when the video processing device completes the process of format conversion,
具体的，视频处理装置完成副本的格式转换处理后，视频处理装置向视频采集装置发送解锁通知，以通知视频采集装置解除对视频图像的锁定，解除锁定后，其他实体可以通过竞争的方式获取对视频图像的操作权限。Specifically, after the video processing device completes the format conversion process of the copy, the video processing device sends an unlock notification to the video capture device to notify the video capture device to unlock the video image. After the lock is released, other entities may acquire the pair through competition. The operation rights of the video image.
示例性的，在资源锁的场景中，视频处理装置向视频采集装置发送解锁通知，视频采集装置将视频图像的资源锁的状态设置为空闲状态，并将视频采集装置的身份标识从拥有者列表中删除，以解除对视频图像的锁定。Exemplarily, in the scenario of resource lock, the video processing device sends an unlock notification to the video capture device, the video capture device sets the state of the resource lock of the video image to an idle state, and sets the identity of the video capture device from the owner list. Removed in to remove the lock on the video image.
S208、通过所述视频编码器对所述预设格式的视频图像副本进行压缩编码处理。S208. Perform compression coding processing on the video image copy of the preset format by using the video encoder.
具体的，具体的，视频编码器对格式转换后的视频图像的副本进行压缩编码处理，以减少信息冗余，压缩编码算法可以是现有技术中的任意一种压缩编码算法，例如，MPEG-4、JVT、H.264和AVC等，本发明不作限制。Specifically, specifically, the video encoder performs compression coding processing on the copy of the formatted video image to reduce information redundancy, and the compression encoding algorithm may be any compression coding algorithm in the prior art, for example, MPEG- 4. JVT, H.264, AVC, etc., and the invention is not limited.
S209、退出标识是否被设置。S209. Whether the exit identifier is set.
具体的，所述退出标识用于指示视频处理处理装置停止工作，退出标识可以由用户进行设置或由视频采集装置进行设置。例如，视频采集装置完成对视频源的采集操作后，后续不会在生成新的视频图像，视频采集装置设置视频处理装置的退出标识，视频处理装置检测到退出标识被设置时，停止工作并释放 占用的硬件资源，硬件资源包括内存资源和CPU资源。视频处理装置判断退出标识是否被设置，若为是，执行S210，若为否，执行S201。Specifically, the exit identifier is used to instruct the video processing processing device to stop working, and the exit identifier may be set by the user or set by the video collection device. For example, after the video capture device completes the acquisition operation of the video source, the video capture device does not generate a new video image, and the video capture device sets the exit identifier of the video processing device. When the video processing device detects that the exit identifier is set, the video processing device stops working and releases. Occupied hardware resources, hardware resources include memory resources and CPU resources. The video processing device determines whether the exit flag is set, and if so, executes S210, and if NO, executes S201.
又例如，用户不需要进行视频压缩处理时，执行退出操作，操作***相应的在寄存器中设置视频处理装置的退出标识，视频处理装置检测到退出标识被设置时，停止工作，释放占用的硬件资源。For another example, when the user does not need to perform video compression processing, the exit operation is performed, and the operating system correspondingly sets the exit identifier of the video processing device in the register, and when the video processing device detects that the exit identifier is set, the operation stops, and the occupied hardware resources are released. .
S210、释放占用的硬件资源。S210. Release the occupied hardware resources.
实施本发明的实施例，接收到视频采集装置的锁定通知，在判断视频编码器处于空闲状态时，对视频采集装置生成的锁定状态的视频图像直接进行后续的格式转换处理，避免对未压缩的视频图像进行复制操作，可有效的减少了CPU的消耗和减少了存储空间的占用。An embodiment of the present invention receives a lock notification of a video capture device, and when determining that the video encoder is in an idle state, directly performs a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid uncompressed The video image is copied, which can effectively reduce the CPU consumption and reduce the storage space.
参见图3，为本发明第一实施例提供的一种视频处理装置的结构示意图，在本发明实施例中，所述视频处理装置包括接收模块10、转换模块11、解锁通知模块12和编码指示模块13。FIG. 3 is a schematic structural diagram of a video processing apparatus according to a first embodiment of the present invention. In the embodiment of the present invention, the video processing apparatus includes a receiving module 10, a converting module 11, an unlocking notification module 12, and an encoding indication. Module 13.
接收模块10，用于接收视频采集装置发送的锁定通知；所述锁定通知用于表示所述视频采集装置完成对生成的视频图像的锁定操作。The receiving module 10 is configured to receive a lock notification sent by the video capture device, where the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image.
转换模块11，用于判断视频编码器是否处于空闲状态，若为是，将所述视频图像转换成预设格式的副本。The conversion module 11 is configured to determine whether the video encoder is in an idle state, and if so, convert the video image into a copy of a preset format.
解锁通知模块12，用于当格式转换操作完成后，向所述视频采集装置发送解锁通知，所述解锁通知用于指示所述视频采集装置解除对所述视频图像的锁定。The unlocking notification module 12 is configured to send an unlocking notification to the video collection device after the format conversion operation is completed, where the unlocking notification is used to instruct the video collection device to release the locking of the video image.
编码指示模块13，用于将所述预设格式的副本发送至所述视频编码器，以使所述视频编码器对所述预设格式的副本进行压缩编码处理。The code indication module 13 is configured to send a copy of the preset format to the video encoder, so that the video encoder performs compression coding processing on the copy of the preset format.
本发明实施例和方法实施例一基于同一构思，其带来的技术效果也相同，具体请参照方法实施例一的描述，此处不再描述。The embodiment of the present invention and the method embodiment 1 are based on the same concept, and the technical effects thereof are also the same. For details, refer to the description of the method embodiment 1, which is not described herein.
实施本发明的实施例，接收到视频采集装置的锁定通知，在判断视频编码器处于空闲状态时，对视频采集装置生成的锁定状态的视频图像直接进行后续的格式转换处理，避免对未压缩的视频图像进行复制操作，可有效的减少了CPU的消耗和减少了存储空间的占用。An embodiment of the present invention receives a lock notification of a video capture device, and when determining that the video encoder is in an idle state, directly performs a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid uncompressed The video image is copied, which can effectively reduce the CPU consumption and reduce the storage space.
参见图5，为本发明第二实施例提供的一种视频处理装置的结构示意图， 在本发明实施例中，所述装置除包括接收模块10、转换模块11、解锁通知模块12和编码指示模块13之外，还包括超时处理模块14和状态处理模块15。FIG. 5 is a schematic structural diagram of a video processing apparatus according to a second embodiment of the present invention. In the embodiment of the present invention, the device includes a timeout processing module 14 and a state processing module 15 in addition to the receiving module 10, the converting module 11, the unlocking notification module 12, and the encoding indication module 13.
超时处理模块14，用于当未接收到所述视频采集装置发送的锁定通知的时间大于预设时长时，判断退出标识是否被设置，若为是，释放占用的硬件资源，所述硬件资源包括CPU资源和内存资源。The timeout processing module 14 is configured to determine whether the exit identifier is set when the time for not receiving the lock notification sent by the video collection device is greater than a preset duration, and if yes, release the occupied hardware resource, where the hardware resource includes CPU resources and memory resources.
状态处理模块15，用于判断所述视频编码器是否处于空闲状态，若为否，向所述视频采集装置发送解锁通知；若所述退出标识被设置，释放占用的硬件资源。The state processing module 15 is configured to determine whether the video encoder is in an idle state, and if not, send an unlock notification to the video collection device; if the exit identifier is set, release the occupied hardware resource.
可选的，转换模块11用于：Optionally, the conversion module 11 is configured to:
确定所述视频编码器位于操作***中的编码线程，并查询所述编码线程的状态标识；其中，所述状态标识用于表示所述编码线程为空闲状态或忙碌状态；Determining, by the video encoder, an encoding thread in an operating system, and querying a status identifier of the encoding thread; wherein the status identifier is used to indicate that the encoding thread is in an idle state or a busy state;
根据所述状态标识判断所述视频编码器是否处于空闲状态。Determining whether the video encoder is in an idle state according to the status identifier.
可选的，所述预设格式包括预设颜色空间和/或预设尺寸。Optionally, the preset format includes a preset color space and/or a preset size.
本发明实施例和方法实施例二基于同一构思，其带来的技术效果也相同，具体过程请参照方法实施例二的描述，此处不再赘述。The embodiment of the present invention and the method embodiment 2 are based on the same concept, and the technical effects thereof are also the same. For the specific process, refer to the description of the second embodiment of the method, and details are not described herein again.
实施本发明的实施例，接收到视频采集装置的锁定通知，在判断视频编码器处于空闲状态时，对视频采集装置生成的锁定状态的视频图像直接进行后续的格式转换处理，避免对未压缩的视频图像进行复制操作，可有效的减少了CPU的消耗和减少了存储空间的占用。An embodiment of the present invention receives a lock notification of a video capture device, and when determining that the video encoder is in an idle state, directly performs a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid uncompressed The video image is copied, which can effectively reduce the CPU consumption and reduce the storage space.
参见图6，为本发明实施例提供的一种客户端的结构示意图，在本发明实施例中，客户端2包括视频采集装置20、视频处理装置21和视频编码器22，客户端2可以是直播网络架构中的主播客户端，主播客户端用于将采集到的视频图像进行压缩编码处理后上传至服务器设备，然后由服务设备将处理后的视频图像发送给多个观众客户端。FIG. 6 is a schematic structural diagram of a client according to an embodiment of the present invention. In the embodiment of the present invention, the client 2 includes a video capture device 20, a video processing device 21, and a video encoder 22. The client 2 may be a live broadcast. The anchor client in the network architecture, the anchor client is used to compress and encode the captured video image and upload it to the server device, and then the service device sends the processed video image to multiple audience clients.
在本发明实施例中，客户端2的工作过程为：视频采集装置20生成视频图像，将所述视频图像进行锁定，以及在完成锁定操作后，向视频处理装置21发送锁定通知；视频处理装置21接收视频采集装置20发送的锁定通知；所述锁定通知用于表示视频采集装置20完成对生成的视频图像的锁定操作；视频处理装置21判断视频编码器是否处于空闲状态，若为是，将所述视频图 像转换成预设格式的副本；当格式转换操作完成后，视频处理装置21向视频采集装置20发送解锁通知，所述解锁通知用于指示视频采集装置20解除对所述视频图像的锁定；视频采集装置20接收到所述解锁通知后，解除对所述视频图像的锁定；视频处理装置21将所述预设格式的副本发送至视频编码器22；视频编码器22对所述预设格式的副本进行压缩编码处理，并将压缩编码处理后的预设格式的副本上传至服务器。In the embodiment of the present invention, the working process of the client 2 is: the video capture device 20 generates a video image, locks the video image, and sends a lock notification to the video processing device 21 after completing the locking operation; the video processing device 21 receiving a lock notification sent by the video capture device 20; the lock notification is used to indicate that the video capture device 20 completes the lock operation on the generated video image; the video processing device 21 determines whether the video encoder is in an idle state, and if so, The video map Converting to a copy of the preset format; after the format conversion operation is completed, the video processing device 21 sends an unlock notification to the video capture device 20, the unlock notification is used to instruct the video capture device 20 to unlock the video image; After receiving the unlocking notification, the collecting device 20 releases the locking of the video image; the video processing device 21 sends a copy of the preset format to the video encoder 22; the video encoder 22 pairs the preset format The copy is subjected to compression encoding processing, and a copy of the compression-coded preset format is uploaded to the server.
本发明实施例中涉及的具体过程可以参见方法实施例二的描述，此处不再赘述。For the specific process involved in the embodiment of the present invention, refer to the description of the method embodiment 2, and details are not described herein again.
实施本发明的实施例，接收到视频采集装置的锁定通知，在判断视频编码器处于空闲状态时，对视频采集装置生成的锁定状态的视频图像直接进行后续的格式转换处理，避免对未压缩的视频图像进行复制操作，可有效的减少了CPU的消耗和减少了存储空间的占用。An embodiment of the present invention receives a lock notification of a video capture device, and when determining that the video encoder is in an idle state, directly performs a subsequent format conversion process on the video image of the locked state generated by the video capture device to avoid uncompressed The video image is copied, which can effectively reduce the CPU consumption and reduce the storage space.
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)或随机存储记忆体(Random Access Memory，RAM)等。One of ordinary skill in the art can understand that all or part of the process of implementing the foregoing embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. When executed, the flow of an embodiment of the methods as described above may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
以上所揭露的仅为本发明一种较佳实施例而已，当然不能以此来限定本发明之权利范围，本领域普通技术人员可以理解实现上述实施例的全部或部分流程，并依本发明权利要求所作的等同变化，仍属于发明所涵盖的范围。 The above disclosure is only a preferred embodiment of the present invention, and of course, the scope of the present invention is not limited thereto, and those skilled in the art can understand all or part of the process of implementing the above embodiments, and according to the present invention. The equivalent changes required are still within the scope of the invention.

Claims (11)

  1. 一种视频处理方法，其特征在于，包括：A video processing method, comprising:
    接收视频采集装置发送的锁定通知；所述锁定通知用于表示所述视频采集装置完成对生成的视频图像的锁定操作；Receiving a lock notification sent by the video capture device; the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image;
    判断视频编码器是否处于空闲状态，若为是，将所述视频图像转换成预设格式的副本；Determining whether the video encoder is in an idle state, and if so, converting the video image into a copy of a preset format;
    当格式转换操作完成后，向所述视频采集装置发送解锁通知，所述解锁通知用于指示所述视频采集装置解除对所述视频图像的锁定；After the format conversion operation is completed, sending an unlock notification to the video collection device, where the unlock notification is used to instruct the video capture device to release the lock on the video image;
    将所述预设格式的副本发送至所述视频编码器，以使所述视频编码器对所述预设格式的副本进行压缩编码处理。Sending a copy of the preset format to the video encoder, so that the video encoder performs compression coding processing on a copy of the preset format.
  2. 如权利要求1所述的方法，其特征在于，还用于：The method of claim 1 further comprising:
    当未接收到所述视频采集装置发送的锁定通知的时间大于预设时长时，判断退出标识是否被设置，若为是，释放占用的硬件资源，所述硬件资源包括CPU资源和内存资源。When the time for not receiving the lock notification sent by the video collection device is greater than the preset duration, it is determined whether the exit identifier is set. If yes, the occupied hardware resources are released, and the hardware resources include CPU resources and memory resources.
  3. 如权利要求1或2所述的方法，其特征在于，所述判断视频编码器是否处于空闲状态包括：The method according to claim 1 or 2, wherein the determining whether the video encoder is in an idle state comprises:
    确定所述视频编码器进驻在操作***中的编码线程，并查询所述编码线程的状态标识；其中，所述状态标识用于表示所述编码线程为空闲状态或忙碌状态；Determining, by the video encoder, an encoding thread in the operating system, and querying a status identifier of the encoding thread; wherein the status identifier is used to indicate that the encoding thread is in an idle state or a busy state;
    根据查询到的状态标识判断所述视频编码器是否处于空闲状态。Whether the video encoder is in an idle state is determined according to the queried status identifier.
  4. 如权利要求3所述的方法，其特征在于，所述将所述副本的格式转换成预设格式中，所述预设格式包括预设颜色空间和/或预设尺寸。The method of claim 3, wherein the converting the format of the copy into a preset format comprises a preset color space and/or a preset size.
  5. 如权利要求1-4任意一项所述的方法，其特征在于，还包括： The method of any of claims 1-4, further comprising:
    判断所述视频编码器是否处于空闲状态，若为否，向所述视频采集装置发送解锁通知；若所述退出标识被设置，释放占用的硬件资源。Determining whether the video encoder is in an idle state, and if not, sending an unlock notification to the video collection device; if the exit identifier is set, releasing the occupied hardware resources.
  6. 一种视频处理装置，其特征在于，包括：A video processing device, comprising:
    接收模块，用于接收视频采集装置发送的锁定通知；所述锁定通知用于表示所述视频采集装置完成对生成的视频图像的锁定操作；a receiving module, configured to receive a lock notification sent by the video capture device, where the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image;
    转换模块，用于判断视频编码器是否处于空闲状态，若为是，将所述视频图像转换成预设格式的副本；a conversion module, configured to determine whether the video encoder is in an idle state, and if so, converting the video image into a copy in a preset format;
    解锁通知模块，用于当格式转换操作完成后，向所述视频采集装置发送解锁通知，所述解锁通知用于指示所述视频采集装置解除对所述视频图像的锁定；An unlocking notification module, configured to send an unlocking notification to the video collection device after the format conversion operation is completed, where the unlocking notification is used to instruct the video collection device to release the locking of the video image;
    编码指示模块，用于将所述预设格式的副本发送至所述视频编码器，以使所述视频编码器对所述预设格式的副本进行压缩编码处理。And an encoding indication module, configured to send a copy of the preset format to the video encoder, so that the video encoder performs compression encoding processing on the copy of the preset format.
  7. 如权利要求6所述的装置，其特征在于，还包括：The device of claim 6 further comprising:
    超时处理模块，用于当未接收到所述视频采集装置发送的锁定通知的时间大于预设时长时，判断退出标识是否被设置，若为是，释放占用的硬件资源，所述硬件资源包括CPU资源和内存资源。a timeout processing module, configured to determine whether the exit identifier is set when the time for not receiving the lock notification sent by the video collection device is greater than a preset duration, and if yes, releasing the occupied hardware resource, where the hardware resource includes a CPU Resources and memory resources.
  8. 如权利要求6或7所述的装置，其特征在于，所述转换模块用于Apparatus according to claim 6 or claim 7 wherein said conversion module is for
    确定所述视频编码器位于操作***中的编码线程，并查询所述编码线程的状态标识；其中，所述状态标识用于表示所述编码线程为空闲状态或忙碌状态；Determining, by the video encoder, an encoding thread in an operating system, and querying a status identifier of the encoding thread; wherein the status identifier is used to indicate that the encoding thread is in an idle state or a busy state;
    根据所述状态标识判断所述视频编码器是否处于空闲状态。Determining whether the video encoder is in an idle state according to the status identifier.
  9. 如权利要求8所述的装置，其特征在于，所述预设格式包括预设颜色空间和/或预设尺寸。The device according to claim 8, wherein the preset format comprises a preset color space and/or a preset size.
  10. 如权利要求6-9任意一项所述的装置，其特征在于，还包括：The device according to any one of claims 6-9, further comprising:
    状态处理模块，用于判断所述视频编码器是否处于空闲状态，若为否，向所述视频采集装置发送解锁通知；若所述退出标识被设置，释放占用的硬件资 源。a state processing module, configured to determine whether the video encoder is in an idle state, and if not, send an unlock notification to the video collection device; if the exit identifier is set, release the occupied hardware resource source.
  11. 一种客户端，其特征在于，包括视频采集装置、视频编码器和视频处理装置；A client, comprising: a video capture device, a video encoder, and a video processing device;
    所述视频采集装置生成视频图像，将所述视频图像进行锁定，以及在完成锁定操作后，向所述视频处理装置发送锁定通知；The video capture device generates a video image, locks the video image, and sends a lock notification to the video processing device after completing the locking operation;
    所述视频处理装置接收视频采集装置发送的锁定通知；所述锁定通知用于表示所述视频采集装置完成对生成的视频图像的锁定操作；The video processing device receives a lock notification sent by the video capture device; the lock notification is used to indicate that the video capture device completes a lock operation on the generated video image;
    所述视频处理装置判断视频编码器是否处于空闲状态，若为是，将所述视频图像转换成预设格式的副本；The video processing device determines whether the video encoder is in an idle state, and if so, converts the video image into a copy of a preset format;
    当格式转换操作完成后，所述视频处理装置向所述视频采集装置发送解锁通知，所述解锁通知用于指示所述视频采集装置解除对所述视频图像的锁定；After the format conversion operation is completed, the video processing device sends an unlock notification to the video collection device, where the unlock notification is used to instruct the video capture device to release the lock on the video image;
    所述视频采集装置接收到所述解锁通知后，解除对所述视频图像的锁定；After receiving the unlock notification, the video capture device releases the lock on the video image;
    所述视频处理装置将所述预设格式的副本发送至所述视频编码器；Transmitting, by the video processing device, a copy of the preset format to the video encoder;
    所述视频编码器对所述预设格式的副本进行压缩编码处理。 The video encoder performs compression coding processing on the copy of the preset format.
PCT/CN2014/095597 2014-11-28 2014-12-30 Video processing apparatus and method and terminal device WO2016082285A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410714669.7A CN104394420B (en) 2014-11-28 2014-11-28 A kind of video process apparatus, method and terminal device
CN201410714669.7 2014-11-28

Publications (1)

Publication Number Publication Date
WO2016082285A1 true WO2016082285A1 (en) 2016-06-02

Family

ID=52612248

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2014/095597 WO2016082285A1 (en) 2014-11-28 2014-12-30 Video processing apparatus and method and terminal device

Country Status (2)

Country Link
CN (1) CN104394420B (en)
WO (1) WO2016082285A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104901820B (en) * 2015-06-29 2018-11-23 广州华多网络科技有限公司 A kind of wheat sequence controlling method, device and system
CN106846032A (en) * 2016-11-24 2017-06-13 北京小米移动软件有限公司 Interactive approach, apparatus and system in electric business application program
CN106791895B (en) * 2016-11-29 2020-07-03 北京小米移动软件有限公司 Interaction method and device in E-commerce application program

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003060954A (en) * 2001-08-16 2003-02-28 Fuji Photo Film Co Ltd Digital camera
CN101977322A (en) * 2010-11-10 2011-02-16 上海交通大学 Screen coding system based on universal video coding standard
CN102016912A (en) * 2008-05-02 2011-04-13 诺基亚公司 Methods, computer program products and apparatus providing improved image capturing
CN103024510A (en) * 2012-12-17 2013-04-03 四川长虹电器股份有限公司 Method capturing intelligent television pictures
CN104053014A (en) * 2013-03-13 2014-09-17 腾讯科技（北京）有限公司 Live broadcast system and method based on mobile terminal, and mobile terminal

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4709657B2 (en) * 2006-02-20 2011-06-22 株式会社東芝 Broadcast recording apparatus and broadcast recording method
CN101076106B (en) * 2006-05-19 2011-07-27 孟智平 Interdynamic video system of IPTV two-dimensional frame marked information

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003060954A (en) * 2001-08-16 2003-02-28 Fuji Photo Film Co Ltd Digital camera
CN102016912A (en) * 2008-05-02 2011-04-13 诺基亚公司 Methods, computer program products and apparatus providing improved image capturing
CN101977322A (en) * 2010-11-10 2011-02-16 上海交通大学 Screen coding system based on universal video coding standard
CN103024510A (en) * 2012-12-17 2013-04-03 四川长虹电器股份有限公司 Method capturing intelligent television pictures
CN104053014A (en) * 2013-03-13 2014-09-17 腾讯科技（北京）有限公司 Live broadcast system and method based on mobile terminal, and mobile terminal

Also Published As

Publication number Publication date
CN104394420A (en) 2015-03-04
CN104394420B (en) 2017-09-12

Similar Documents

Publication Publication Date Title
WO2021008290A1 (en) Video stream decoding method and apparatus, terminal device and storage medium
AU2014373838B2 (en) Content-adaptive chunking for distributed transcoding
WO2017193821A1 (en) Cloud desktop image processing method, server, client and computer storage medium
JP2021524176A (en) Methods, equipment and computer programs for video coding
US20230300410A1 (en) Re-encoding predicted picture frames in live video stream applications
US9609338B2 (en) Layered video encoding and decoding
US9667982B2 (en) Techniques for transform based transcoding
TWI637631B (en) Image processing device, video subsystem and video pipeline
WO2016082285A1 (en) Video processing apparatus and method and terminal device
WO2013189374A2 (en) Method for transmitting media data and virtual desktop server
JP2022539299A (en) Interruptible video transcoding
CN106412588B (en) Image frame processing method
US20180270360A1 (en) Audio data processing method, server, client and server, and storage medium
JPWO2020049958A1 (en) Information processing system, information processing method, terminal device and information processing device
WO2024061316A1 (en) Data compression method, data decompression method, and related device
CN106101830A (en) A kind of video flow detection method combined based on region detection and applying detection
US20210092403A1 (en) Object manipulation video conference compression
WO2022110870A1 (en) Image encoding and decoding method, encoding and decoding apparatus, encoder, and decoder
JP2023503592A (en) Signaling timeouts and completion data entry in cloud workflows
US20190260817A1 (en) Providing apparatus, data providing method, and storage medium
CN117435112B (en) Data processing method, system and device, electronic equipment and storage medium
WO2023130893A1 (en) Streaming media based transmission method and apparatus, electronic device and computer-readable storage medium
US12028549B1 (en) Enhanced video streaming and reference frame synchronization
JP7515546B2 (en) Interruptible Video Transcoding
CN118283298A (en) Video transmission method, processing method, apparatus, device, medium, and program product

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14907032

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 13/10/17)

122 Ep: pct application non-entry in european phase

Ref document number: 14907032

Country of ref document: EP

Kind code of ref document: A1