WO2012042701A1 - Multi-stream encoding control device and camera system - Google Patents
Multi-stream encoding control device and camera system Download PDFInfo
- Publication number
- WO2012042701A1 WO2012042701A1 PCT/JP2011/002916 JP2011002916W WO2012042701A1 WO 2012042701 A1 WO2012042701 A1 WO 2012042701A1 JP 2011002916 W JP2011002916 W JP 2011002916W WO 2012042701 A1 WO2012042701 A1 WO 2012042701A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- encoding
- unit
- stream
- output
- code amount
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
Definitions
- the present invention is a multi-stream code that provides high-quality moving images in a limited output band when a multi-stream image is distributed in real time in an image distribution or TV conference system mainly as a Web camera.
- the present invention relates to a computerized control device.
- a technique has been advanced in which a video signal captured by a camera is processed as a digital signal inside the camera and a digital video signal is distributed.
- the encoded video signal handled on the user side differs depending on the user environment, and it is necessary to perform encoding corresponding to the environment on the distribution side.
- FIG. 1 is a block diagram showing a system for inputting video signals from a camera and encoding and distributing desired resolution data.
- an image input unit 101 inputs a video image from a camera.
- the YC generation / resize processing unit 102 generates YC from the input video signal and resizes it to a desired resolution.
- the encoding unit 103 performs encoding by the encoding processing unit 105 according to the state of the distribution destination, and the output unit 104 performs the encoding processing according to an external output transmission rate via an external interface such as Ether or USB.
- the code amount is controlled by the unit 105 and the encoded video is distributed.
- output data from the output unit 104 is only one system, but the encoding unit 103 has a plurality of encoding processing units 105 and encodes them in two or more different encoding formats.
- the code data output rate of the output unit 104 will be exceeded.
- the amount of code to be encoded in each encoding processing unit is limited to a small amount of code in consideration of the maximum code amount in other encoding processing units, and a plurality of encoding processes It is also assumed that the total amount of codes encoded by the unit will be extremely small. In this case, the total code amount per unit time is extremely small as compared with the output rate of the output unit, and there is a problem that image quality deterioration occurs.
- An object of the present invention is to provide a plurality of encoding processing units that perform encoding in two or more different encoding formats in a multi-stream encoding control apparatus, and code data encoded by the plurality of encoding processes. Is output simultaneously from the output unit, image quality deterioration is suppressed and a high-quality image can be transmitted in real time.
- the total code amount per unit time encoded by a plurality of encoding processing units does not exceed the code data output rate of the output unit and is close to the code data output rate.
- the multi-stream encoding control apparatus includes an image input unit that inputs a video signal and two or more different encodings of the video signal input to the image input unit.
- An encoding unit that performs encoding in accordance with a scheme; an output unit that outputs two or more types of encoded data encoded by the encoding unit; and the encoding unit based on the encoding result of the encoding unit.
- An encoding prediction unit that determines the next encoding setting in the encoding unit, and the encoding prediction unit is set at the time of encoding the encoding results in two or more different encoding methods
- the total code amount in the two or more different encoding schemes is controlled based on the encoding parameters.
- the invention according to claim 2 is the multi-stream encoding control apparatus according to claim 1, wherein the two or more different encoding schemes are two or more different moving image encoding schemes, and the encoding prediction
- the unit for each of the two or more different encoding schemes, the encoding amount and encoding parameters after encoding, and the encoding amount and encoding parameters at the previous encoding than the encoding,
- the next encoding parameter is determined according to one or more of a frame rate to be encoded and an output rate of the output unit.
- the invention according to claim 3 is the multi-stream encoding control apparatus according to claim 1, wherein the two or more different encoding methods are one or more still image encoding methods and one or more moving image codes.
- the encoding prediction unit for each of the two or more different encoding methods, the encoding amount after encoding and the encoding parameter, and the encoding time before the encoding time.
- the next encoding parameter is determined according to any one or more of the following code amount and encoding parameter, the encoding frame rate, and the output rate of the output unit.
- the two or more different moving image encoding methods have different encoding processes for each frame of the video signal
- the encoding prediction unit predicts that the encoding amount varies due to different encoding processing for each frame of the video signal in each encoding method, and controls the encoding amount to be encoded next.
- the multi-stream encoding control apparatus further includes a dividing unit that divides a still image encoding result in the still image encoding method into a plurality of divided results. Then, the output unit notifies the encoded prediction unit to the encoded prediction unit, and the encoded prediction unit determines the moving image code based on the output rate of the output unit and the divided data size. The code amount of the moving image encoding process in the encoding method is controlled.
- the coding prediction unit holds information in which the code amount fluctuates periodically for the control of the code amount to be coded next.
- the control is performed so that frames with a large amount of codes do not overlap.
- a camera system comprising the multi-stream encoding control apparatus according to any one of the first to sixth aspects.
- the sum per unit time of the amount of code encoded by each of the plurality of encoding methods can be controlled in accordance with the code data output speed of the output unit, so that the sum of the code amount per unit time can always be controlled to the maximum sum according to the transmission rate of the output unit.
- High-definition images can be distributed while suppressing image quality degradation associated with conversion.
- a still image can be output in accordance with the characteristics of the moving image encoding method, it is possible to output a still image while delivering a high-definition image while suppressing deterioration of the image quality of the moving image. It is.
- the total code amount per unit time is obtained from the output unit. It is possible to prevent exceeding the code amount corresponding to the code data output speed, and to suppress deterioration in image quality.
- a still image can be output in accordance with the characteristics of the moving image coding system, the output of the still image can be output while delivering a high-definition image while suppressing the deterioration of the image quality of the moving image. Is possible.
- H Since it can be controlled so that pictures with a large code amount do not overlap in a moving image encoding method such as H.264, the total code amount per unit time can be prevented from exceeding the code amount according to the code data output speed of the output unit, It is possible to suppress image quality deterioration.
- the amount of code encoded by each of these plural encoding methods per unit time Since the sum can be controlled in accordance with the transmission rate of the output unit, the sum of the code amount per unit time can always be controlled to the maximum sum according to the transmission rate of the output unit, and a high-definition image can be controlled. Can be delivered.
- FIG. 1 is a block diagram of a conventional encoded distribution system.
- FIG. 2 is a block diagram showing the overall configuration of the multi-stream coding control apparatus according to the first embodiment of the present invention.
- FIG. 3 is a flowchart showing the operation of the multi-stream coding control apparatus.
- FIG. 4 is a block diagram showing the overall configuration of a Web camera equipped with the multi-stream encoding control apparatus according to the second embodiment of the present invention.
- FIG. 5A is a flowchart showing the operation of the Web camera
- FIG. 5B is a flowchart showing details of steps S504 and S506 in the operation flowchart of FIG. FIG.
- FIG. 6 is a block diagram showing the overall configuration of a Web camera equipped with the multi-stream encoding control apparatus according to the third embodiment of the present invention.
- FIG. 7A is a flowchart showing the operation of the Web camera
- FIG. 7B is a flowchart showing details of steps S704 and S706 of the operation flowchart of FIG.
- FIG. 2 shows the overall configuration of the multi-stream coding control apparatus according to the first embodiment of the present invention.
- the multi-stream encoding control apparatus shown in FIG. 1 includes an input unit 201 that inputs a video signal to be encoded from the outside, and an encoding unit 202 that encodes the input video signal in accordance with a predetermined encoding method. Is provided.
- the encoding unit 202 includes encoding processing units 205 and 206 that encode an input video signal using two different encoding methods.
- the multi-stream encoding control apparatus shown in FIG. 1 includes an output unit 203 that outputs code data encoded by the encoding unit 202, and an encoding prediction unit 204.
- the encoding prediction unit 204 inputs the encoded data output speed of the output unit 203 and the encoding results of the two encoding processing units 205 and 206 of the encoding unit 202, and performs next encoding. In this case, the total code amount in the encoding processing units 205 and 206 is controlled.
- step S301 of FIG. 3 an image input from the outside is acquired by the input unit 201.
- step S302 the acquired video signal is input to the encoding unit 202, and whether the desired encoding process is encoding in one encoding processing unit 205 or encoding in the other encoding processing unit 206 is performed. If the encoding processing unit 205 performs encoding, the process proceeds to step S303. If the encoding processing unit 206 performs encoding, the process proceeds to step S305.
- Steps S303 and S304 and steps S305 and S306 respectively show the processing in the encoding processing units 205 and 206.
- the encoding prediction unit 204 outputs code data from the output unit 203.
- the encoding processing units 205 and 206 determine the encoding parameters for the next encoding.
- steps S304 and S306 the input video signal is encoded according to the encoding method in the encoding processing units 205 and 206, respectively. Thereafter, in step S307, the result of encoding by the encoding processing units 205 and 206 is notified to the encoding prediction unit 204, and the encoding prediction unit 204 updates the held information based on the notification information. To do.
- step S ⁇ b> 308 each code data encoded by each encoding processing unit 205, 206 is output from the output unit 203.
- FIG. 4 shows a second embodiment of the present invention.
- Figure 4 shows that two different video encoding formats are H.264.
- 2 shows a configuration of a Web camera equipped with a H.264 and MPEG4 multi-stream encoding control device.
- the Web camera in FIG. 1 includes an image input unit 401 that acquires a video signal read from an image sensor that converts input video light into an electrical signal, YC generation and resizing to a desired image size. Part 402 and the obtained YC data.
- An encoding unit 403 including an encoding processing unit 406 that performs H.264 encoding and an encoding processing unit 407 that performs MPEG4 encoding, an output unit 404 that outputs the encoded code data to USB, and codes of the output unit 404
- An encoding prediction unit 405 that inputs the data output speed and the encoding results of the encoding processing units 406 and 407 and controls the amount of code when the encoding processing units 406 and 407 perform encoding next time; And transmitting the code data from the output unit 404 to the connection destination host PC 408 via the USB cable.
- step S501 a video signal read from an image sensor that converts video light input from the outside into an electrical signal in the image input unit 401 is acquired.
- step S502 the YC generation / resizing unit 402 regenerates the acquired video signal to YC generation and a desired image size.
- step S503 the YC generated and resized image is input to the encoding unit 403, and the desired encoding processing unit 406 or 407 is selected.
- Steps S504 and S505 and steps S506 and S507 indicate the encoding processes in the encoding processing units 406 and 407, respectively.
- the encoding prediction unit 405 based on the encoded data output speed of the USB output unit 404 and the encoding processing results of the encoding processing units 406 and 407, performs the encoding processing unit.
- encoding parameters for the next encoding are determined.
- the encoding processing units 406 and 407 respectively encode the input image from the YC generation / resizing unit 402 in accordance with its own encoding method.
- the results encoded by the encoding processing units 406 and 407 are notified to the encoding prediction unit 405, and the encoding prediction unit 405 determines the information held based on the notified information. Update.
- the respective code data encoded by the encoding processing units 406 and 407 are output from the USB output unit 404 to the host PC 408.
- FIG. 5B is a flowchart showing details of the encoding prediction process in steps S504 and S506 in FIG.
- step S510 for example, the previous H.264 processing in the encoding processing unit 406 held in step S508 of FIG.
- the H.264 encoding result, the code amount, the Qp value (quantization parameter) that is the encoding parameter, and the next picture type are read out.
- step S511 and step S512 the previous encoding result (code amount, Qp value, next picture type) of the encoding processing unit 407 in the MPEG4 encoding process which is another encoding format is acquired.
- step S513 the output bit rate and the total sum of output code amounts per unit time in the encoding processing units 406 and 407 are acquired as output information of the USB output unit 404.
- step S514 in order to determine the encoding parameter to be encoded next in each of the code processing units 406 and 407, the obtained output bit rate of the USB output unit 404, the total output code amount per unit time, Based on the above, the output possible code amount is calculated and, for example, the encoding processing unit 406 next outputs the H.264 code.
- the encoding processing unit 407 selects the picture type to be MPEG4 encoded next as P or B picture, and then the previous code amount and the next code amount in the encoding processing unit 407 are selected.
- the next encoding parameter Qp value is determined so as to perform control to increase the Qp value compared to the previous encoding.
- FIG. 6 shows a third embodiment of the present invention.
- the figure shows the video coding format H.264. 1 shows a configuration of a Web camera (camera system) equipped with a multi-stream encoding control device that uses H.264 and JPEG as a still image encoding format in two different encoding formats.
- the web camera shown in FIG. 6 includes an image input unit 601 that acquires a video signal read from an image sensor that converts input video light into an electrical signal, and YC generation that resizes the image to a desired image size.
- the resizing unit 602 and the obtained YC data are converted into H.264.
- An encoding unit 603 including an encoding processing unit 606 that performs H.264 encoding, an encoding processing unit 607 that performs JPEG encoding, a USB output unit 604 that outputs the encoded code data via USB, and a USB output unit Encoding prediction unit for controlling the amount of code when the encoding data output speed of 604 and the encoding results of the encoding processing units 606 and 607 are input and the encoding processing units 606 and 607 perform encoding next time 605 and a dividing unit 608 that divides still image JPEG-encoded code data into fixed lengths and outputs the divided data to the output unit 604 at a fixed period.
- the USB output unit 604 sends a USB to the connection destination host PC 609. In this configuration, data is transmitted via a cable.
- step S701 a video signal read from an image sensor that converts video light input from the outside into an electrical signal in the image input unit 601 is acquired.
- step S702 the YC generation / resizing unit 602 generates YC and resizes the acquired video signal to a desired image size.
- step S703 the YC generated and resized image is input to the encoding unit 603, and a desired encoding process target is selected.
- Steps S704 and S705 and steps S706 and S707 indicate processing in the encoding processing units 606 and 607, respectively.
- the encoding prediction unit 605 next performs each encoding process based on the encoded data output speed of the USB output unit 604 and the encoding results of the encoding processing units 606 and 607.
- Each encoding parameter at the time of encoding by the units 606 and 607 is determined.
- step S705 and S707 the input image signal is encoded in accordance with the encoding method in each of the encoding processing units 606 and 607.
- step S708 the encoding prediction unit 605 is notified of the encoding results of the encoding processing units 606 and 607, and the encoding prediction unit 605 updates the held information based on the notified information.
- step S707 information for dividing the JPEG-encoded data into a fixed length is notified to the dividing unit 608 at a fixed period.
- step S709 the encoded data encoded by the encoding processing units 606 and 607 is output from the USB output unit 604 to the host PC 609.
- the still image code data encoded by the encoding processing unit 607 is divided into fixed lengths so as to be transferred within a desired time by the dividing unit 608, and is output to the USB output unit 604 at a constant cycle. .
- FIG. 7B is a flowchart showing details of steps S704 and S706 in the flowchart of FIG.
- step S710 the H.D. stored in step S708 of FIG.
- step S711 and step S712 the previous encoding result (code amount, quantization table, scale factor) of the JPEG encoding format in the encoding processing unit 607 is acquired.
- step S713 the output bit rate and the total output code amount per unit time of both encoding processing units 606 and 607 are acquired as output information of the USB output unit 604.
- step S714 in order to determine each encoding parameter to be encoded next in each encoding processing unit 606, 607, the acquired output bit rate, the total output code amount per unit time, and a constant JPEG Based on the output code amount of the period, the code amount that can be output is calculated.
- the H.264 encoding processing unit 606 calculates the code amount per frame to be encoded next, compares the previous code amount per frame with the code amount per frame to be encoded next, If the previous code amount is small, the Qp value is decreased. If the previous code amount is large, the next encoding parameter is determined so as to increase the Qp value.
- the JPEG encoding processing unit 607 calculates fixed length information used by the dividing unit 608 based on the previous code amount.
- the multi-stream coding control apparatus of the present invention uses the output transfer speed of the output unit to use the full output band range, and suppresses the deterioration of the image quality of the moving image, thereby moving the high-quality moving image. Since images can be output in real time, it is suitable for use in, for example, network cameras such as surveillance cameras and WEB cameras, in-vehicle cameras, camera mounted on vehicles such as drive recorders, TV conference systems and TV phones connected to TVs and PCs, etc. It is.
Landscapes
- Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
In a multi-stream encoding control device, when each video data is encoded by two or more mutually different coding systems, an encode prediction unit (204), in the encoding by each coding system in an encoding unit (202), determines the setting in the next encoding on the basis of any one or more of an code amount and an encode parameter after encoding, the code amount and encode parameter at encoding before the encoding, the frame rate to be encoded, and the output rate of an output unit (203). Thus, the total of code amount per unit time encoded respectively by a plurality of coding systems can be controlled in accordance with the transmission rate of the output unit 203. Hereby, when code data encoded respectively by a plurality of coding systems is outputted at the same time, the deterioration of image quality is suppressed and a high-resolution image can be delivered.
Description
本発明は、主にWebカメラとしてTVやPCなどへの画像配信やTV会議システムにおいて、画像をリアルタイムにマルチストリーム配信する際に、限られた出力帯域で高画質な動画を提供するマルチストリーム符号化制御装置に関するものである。
The present invention is a multi-stream code that provides high-quality moving images in a limited output band when a multi-stream image is distributed in real time in an image distribution or TV conference system mainly as a Web camera. The present invention relates to a computerized control device.
近年、一般家庭でADSLや光ファイバ、CATV網などの専用線を使った広帯域で高速なデータ伝送が可能なIP常時接続が当り前になってきた。これにより、家庭やオフィス端末に対してインターネットが常時接続されるようになり、静止画や動画、音声、その他のマルチメディア情報の伝送を行うテレビ電話システムやテレビ会議システムなどの通信手段の実現が可能となった。
In recent years, it has become commonplace for general households to use IP always-on connections capable of broadband and high-speed data transmission using dedicated lines such as ADSL, optical fiber, and CATV network. As a result, the Internet is always connected to home and office terminals, and communication means such as videophone systems and videoconferencing systems that transmit multimedia information such as still images, videos, audio, etc. are realized. It has become possible.
上記のテレビ電話システム等での通信端末における映像入力機器として、カメラ撮影した映像信号をそのカメラ内部でデジタル信号として信号処理を実施し、デジタル映像信号を配信する技術が進んできた。
As a video input device in a communication terminal in the above videophone system or the like, a technique has been advanced in which a video signal captured by a camera is processed as a digital signal inside the camera and a digital video signal is distributed.
これにより、カメラで撮影した映像信号をインターネットなどを介して複数のユーザーへ配信することが可能となり、配信する映像信号のデータは配信先の出力伝送速度に応じて符号化を行い、撮影した映像をリアルタイムに提供する。
This makes it possible to distribute the video signal captured by the camera to multiple users via the Internet, etc., and the video signal data to be distributed is encoded according to the output transmission speed of the distribution destination, and the captured video In real time.
このとき、ユーザー側で扱う符号化された映像信号は、ユーザー環境に応じて異なっており、配信側で環境に応じた符号化を実施する必要性がある。
At this time, the encoded video signal handled on the user side differs depending on the user environment, and it is necessary to perform encoding corresponding to the environment on the distribution side.
図1は、カメラからの映像信号を入力して所望の解像データを符号化し配信するシステムを示すブロック図である。
FIG. 1 is a block diagram showing a system for inputting video signals from a camera and encoding and distributing desired resolution data.
図1において、画像入力部101は、カメラからの映像画像を入力する。YC生成リサイズ処理部102は、入力された映像信号からYC生成して所望の解像にリサイズする。符号化部103は、配信先の状態に応じた符号化処理部105にて符号化し、出力部104は、EtherやUSBなど外部インタフェースを経由して外部の出力伝送速度に応じて前記符号化処理部105にて符号量を制御し且つ符号化した映像を配信している。
In FIG. 1, an image input unit 101 inputs a video image from a camera. The YC generation / resize processing unit 102 generates YC from the input video signal and resizes it to a desired resolution. The encoding unit 103 performs encoding by the encoding processing unit 105 according to the state of the distribution destination, and the output unit 104 performs the encoding processing according to an external output transmission rate via an external interface such as Ether or USB. The code amount is controlled by the unit 105 and the encoded video is distributed.
しかしながら、上記のような構成は、出力部104からの出力データは1系統のみであるが、符号化部103が複数の符号化処理部105を持って2種以上の異なる符号化形式で符号化し、それ等の複数系統の符号データを出力部104から同時出力しようとする場合には、出力部104の符号データ出力レートを越えてしまうことが想定される。
However, in the configuration as described above, output data from the output unit 104 is only one system, but the encoding unit 103 has a plurality of encoding processing units 105 and encodes them in two or more different encoding formats. When it is attempted to simultaneously output such multiple types of code data from the output unit 104, it is assumed that the code data output rate of the output unit 104 will be exceeded.
そこで、例えば、複数の符号化処理部での符号結果の一部又は全部を再符号化により調整し、全符号量を抑制することが考えられるが、この場合には、再符号化する際にメモリアクセスが増加し、また出力部からの符号データの出力遅延を招き、更にはバッファリングによるメモリ増加が発生する課題が生じる。
Thus, for example, it is conceivable to adjust a part or all of the code results in a plurality of encoding processing units by re-encoding to suppress the total code amount. In this case, when re-encoding is performed. There is a problem that the memory access increases, the output of the code data from the output unit is delayed, and further the memory increases due to buffering.
更に、複数の符号化処理部での総符号量が出力部の出力レートに応じた最大総符号量を超えないように対処する場合に、複数の符号化処理部での各々の符号化処理での経時的な符号量変化を考慮すると、各符号化処理部で符号化する符号量は他の符号化処理部での最大符号量を考慮した少ない符号量に制限されて、複数の符号化処理部で符号化した符号量の総和は極めて少なくなってしまうことも想定される。そして、この場合には、出力部の出力レートに比して単位時間当たりの総和符号量が極めて少ない状況となって、画質劣化が発生する課題がある。
Further, when dealing with the total code amount in the plurality of encoding processing units so as not to exceed the maximum total code amount corresponding to the output rate of the output unit, in each encoding process in the plurality of encoding processing units In consideration of the change in the amount of code over time, the amount of code to be encoded in each encoding processing unit is limited to a small amount of code in consideration of the maximum code amount in other encoding processing units, and a plurality of encoding processes It is also assumed that the total amount of codes encoded by the unit will be extremely small. In this case, the total code amount per unit time is extremely small as compared with the output rate of the output unit, and there is a problem that image quality deterioration occurs.
本発明の目的は、マルチストリーム符号化制御装置において、2種以上の異なる符号化形式で符号化する複数の符号化処理部を備えて、それ等の複数の符号化処理で符号化した符号データを出力部から同時出力する場合に、画質劣化を抑制して、高画質画像をリアルタイムで伝送できるようにすることにある。
An object of the present invention is to provide a plurality of encoding processing units that perform encoding in two or more different encoding formats in a multi-stream encoding control apparatus, and code data encoded by the plurality of encoding processes. Is output simultaneously from the output unit, image quality deterioration is suppressed and a high-quality image can be transmitted in real time.
前記目的を達成するため、本発明では、複数の符号化処理部で符号化した単位時間当たりの総和符号量が、出力部の符号データ出力レートを越えず、且つその符号データ出力レートに近いように制御する。
In order to achieve the above object, in the present invention, the total code amount per unit time encoded by a plurality of encoding processing units does not exceed the code data output rate of the output unit and is close to the code data output rate. To control.
具体的に、請求項1記載の発明のマルチストリーム符号化制御装置は、映像信号を入力する画像入力部と、前記画像入力部に入力された前記映像信号を、2種以上の互いに異なる符号化方式に合わせて符号化する符号化部と、前記符号化部で符号化された2種以上の符号化データを出力する出力部と、前記符号化部での符号化結果に基づいて、前記符号化部での次の符号化設定を決定する符号化予測部とを備え、前記符号化予測部は、2種以上の異なる符号化方式での符号化結果と、それ等の符号化時に設定された符号化パラメータとに基づいて、前記2種以上の異なる符号化方式での総符号量を制御することを特徴とする。
Specifically, the multi-stream encoding control apparatus according to the first aspect of the present invention includes an image input unit that inputs a video signal and two or more different encodings of the video signal input to the image input unit. An encoding unit that performs encoding in accordance with a scheme; an output unit that outputs two or more types of encoded data encoded by the encoding unit; and the encoding unit based on the encoding result of the encoding unit. An encoding prediction unit that determines the next encoding setting in the encoding unit, and the encoding prediction unit is set at the time of encoding the encoding results in two or more different encoding methods The total code amount in the two or more different encoding schemes is controlled based on the encoding parameters.
請求項2記載の発明は、前記請求項1記載のマルチストリーム符号化制御装置において、前記2種以上の互いに異なる符号化方式は、2種以上の異なる動画符号化方式であり、前記符号化予測部は、前記2種以上の互いに異なる符号化方式の各々について、符号化後の符号量及び符号化パラメータと、前記符号化時よりも前回の符号化時での符号量及び符号化パラメータと、符号化するフレームレートと、前記出力部の出力レートとの何れか1つ以上によって、次の符号化パラメータを決定することを特徴とする。
The invention according to claim 2 is the multi-stream encoding control apparatus according to claim 1, wherein the two or more different encoding schemes are two or more different moving image encoding schemes, and the encoding prediction The unit, for each of the two or more different encoding schemes, the encoding amount and encoding parameters after encoding, and the encoding amount and encoding parameters at the previous encoding than the encoding, The next encoding parameter is determined according to one or more of a frame rate to be encoded and an output rate of the output unit.
請求項3記載の発明は、前記請求項1記載のマルチストリーム符号化制御装置において、前記2種以上の互いに異なる符号化方式は、1つ以上の静止画符号化方式と1つ以上の動画符号化方式であり、前記符号化予測部は、前記2種以上の互いに異なる符号化方式の各々について、符号化後の符号量及び符号化パラメータと、前記符号化時よりも前回の符号化時での符号量及び符号化パラメータと、符号化するフレームレートと、前記出力部の出力レートとの何れか1つ以上によって、次の符号化パラメータを決定することを特徴とする。
The invention according to claim 3 is the multi-stream encoding control apparatus according to claim 1, wherein the two or more different encoding methods are one or more still image encoding methods and one or more moving image codes. The encoding prediction unit, for each of the two or more different encoding methods, the encoding amount after encoding and the encoding parameter, and the encoding time before the encoding time. The next encoding parameter is determined according to any one or more of the following code amount and encoding parameter, the encoding frame rate, and the output rate of the output unit.
請求項4記載の発明は、前記請求項2記載のマルチストリーム符号化制御装置において、前記2つ以上の互いに異なる動画符号化方式は、各々、映像信号の各フレーム毎に符号化処理が異なり、前記符号化予測部は、前記各符号化方式において映像信号の各フレーム毎に符号化処理が異なって符号量が変動することを予測して、次に符号化する符号量を制御することを特徴とする。
According to a fourth aspect of the present invention, in the multi-stream encoding control apparatus according to the second aspect, the two or more different moving image encoding methods have different encoding processes for each frame of the video signal, The encoding prediction unit predicts that the encoding amount varies due to different encoding processing for each frame of the video signal in each encoding method, and controls the encoding amount to be encoded next. And
請求項5記載の発明は、前記請求項3記載のマルチストリーム符号化制御装置において、静止画符号化方式での静止画の符号化結果を複数に分割して前記出力部へ渡す分割部を有し、前記出力部は、分割されたデータサイズを前記符号化予測部に通知し、前記符号化予測部は、前記出力部の出力レートと、前記分割されたデータサイズとに基づいて、動画符号化方式での動画符号化処理の符号量を制御をすることを特徴とする。
According to a fifth aspect of the present invention, the multi-stream encoding control apparatus according to the third aspect further includes a dividing unit that divides a still image encoding result in the still image encoding method into a plurality of divided results. Then, the output unit notifies the encoded prediction unit to the encoded prediction unit, and the encoded prediction unit determines the moving image code based on the output rate of the output unit and the divided data size. The code amount of the moving image encoding process in the encoding method is controlled.
請求項6記載の発明は、請求項4記載のマルチストリーム符号化制御装置において、前記符号化予測部は、次に符号化する符号量の制御について、周期的に符号量が変動する情報を保持し、符号量が多いフレームが重ならないように制御することを特徴とする。
According to a sixth aspect of the present invention, in the multi-stream coding control apparatus according to the fourth aspect of the invention, the coding prediction unit holds information in which the code amount fluctuates periodically for the control of the code amount to be coded next. In addition, the control is performed so that frames with a large amount of codes do not overlap.
請求項7記載の発明のカメラシステムは、前記請求項1~6の何れか1項に記載のマルチストリーム符号化制御装置を備えたことを特徴とする。
According to a seventh aspect of the present invention, there is provided a camera system comprising the multi-stream encoding control apparatus according to any one of the first to sixth aspects.
以上により、請求項1記載の発明では、2種以上の異なる符号化方式の映像信号を同時に出力する際に、それ等複数の符号化方式で各々符号化された符号量の単位時間当たりの総和を、出力部の符号データ出力速度に合わせて制御することが可能であるので、常に、その単位時間当たりの符号量の総和を出力部の伝送速度に応じた最大総和に制御でき、よって、符号化に伴う画質劣化を抑制して、高精細な画像を配信できる。
As described above, according to the first aspect of the present invention, when video signals of two or more different encoding methods are output simultaneously, the sum per unit time of the amount of code encoded by each of the plurality of encoding methods. Can be controlled in accordance with the code data output speed of the output unit, so that the sum of the code amount per unit time can always be controlled to the maximum sum according to the transmission rate of the output unit. High-definition images can be distributed while suppressing image quality degradation associated with conversion.
しかも、各符号化方式で符号化した符号量を再符号化する必要がないので、遅延なく高精細な画像を配信することが可能である。
In addition, since it is not necessary to re-encode the code amount encoded by each encoding method, it is possible to deliver a high-definition image without delay.
また、請求項3記載の発明では、動画符号化方式の特徴に合わせて静止画像を出力できるので、動画像の画質劣化を抑制して高精細な画像を配信しつつ、静止画の出力が可能である。
Further, in the invention according to claim 3, since a still image can be output in accordance with the characteristics of the moving image encoding method, it is possible to output a still image while delivering a high-definition image while suppressing deterioration of the image quality of the moving image. It is.
更に、請求項4記載の発明では、複数の符号化方式で各々符号化する際に、符号量が大きくなるピクチャ同士が重ならないように制御できるので、単位時間当たりの符号量総和が出力部の符号データ出力速度に応じた符号量を越えることを防止でき、画質劣化を抑制することが可能である。
Furthermore, in the invention according to claim 4, since it is possible to control so that the pictures having large code amounts do not overlap each other when encoding by a plurality of encoding methods, the total code amount per unit time is obtained from the output unit. It is possible to prevent exceeding the code amount corresponding to the code data output speed, and to suppress deterioration in image quality.
加えて、請求項5記載の発明では、動画符号化方式の特徴に合わせて静止画像を出力できるので、動画像の画質劣化を抑制して高精細な画像を配信しつつ、静止画の出力が可能である。
In addition, in the invention according to claim 5, since a still image can be output in accordance with the characteristics of the moving image coding system, the output of the still image can be output while delivering a high-definition image while suppressing the deterioration of the image quality of the moving image. Is possible.
また、請求項6記載の発明では、H.264などの動画符号化方式において符号量が大きくなるピクチャが重ならないように制御できるので、単位時間当たりの符号量総和が出力部の符号データ出力速度に応じた符号量を越えることを防止でき、画質劣化を抑制することが可能である。
In the invention according to claim 6, H. Since it can be controlled so that pictures with a large code amount do not overlap in a moving image encoding method such as H.264, the total code amount per unit time can be prevented from exceeding the code amount according to the code data output speed of the output unit, It is possible to suppress image quality deterioration.
以上説明したように、本発明によれば、2種以上の異なる符号化方式の映像信号を同時に出力する際に、それ等複数の符号化方式で各々符号化された符号量の単位時間当たりの総和を、出力部の伝送速度に合わせて制御することが可能であるので、常に、その単位時間当たりの符号量の総和を出力部の伝送速度に応じた最大総和に制御でき、高精細な画像を配信できる。
As described above, according to the present invention, when video signals of two or more different encoding methods are output simultaneously, the amount of code encoded by each of these plural encoding methods per unit time Since the sum can be controlled in accordance with the transmission rate of the output unit, the sum of the code amount per unit time can always be controlled to the maximum sum according to the transmission rate of the output unit, and a high-definition image can be controlled. Can be delivered.
しかも、各符号化方式で符号化した符号量を再符号化する必要がないので、遅延なく高精細な画像を配信することができる。
In addition, since it is not necessary to re-encode the code amount encoded by each encoding method, a high-definition image can be distributed without delay.
以下、本発明を実施形態について図面を参照しながら説明する
(実施形態1)
図2は、本発明の実施形態1に関わるマルチストリーム符号化制御装置の全体構成を示す。 DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, embodiments of the invention will be described with reference to the drawings (Embodiment 1).
FIG. 2 shows the overall configuration of the multi-stream coding control apparatus according to the first embodiment of the present invention.
(実施形態1)
図2は、本発明の実施形態1に関わるマルチストリーム符号化制御装置の全体構成を示す。 DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, embodiments of the invention will be described with reference to the drawings (Embodiment 1).
FIG. 2 shows the overall configuration of the multi-stream coding control apparatus according to the first embodiment of the present invention.
図1に示したマルチストリーム符号化制御装置は、外部から符号化したい映像信号を入力する入力部201と、入力された映像信号を所定の符号化方式に合わせて符号化する符号化部202とを備える。前記符号化部202は、互いに異なる2つの符号化方式で入力映像信号を符号化する符号化処理部205、206を有する。
The multi-stream encoding control apparatus shown in FIG. 1 includes an input unit 201 that inputs a video signal to be encoded from the outside, and an encoding unit 202 that encodes the input video signal in accordance with a predetermined encoding method. Is provided. The encoding unit 202 includes encoding processing units 205 and 206 that encode an input video signal using two different encoding methods.
更に、図1に示したマルチストリーム符号化制御装置は、前記符号化部202で符号化された符号データを出力する出力部203と、符号化予測部204とを具備する。前記符号化予測部204は、前記出力部203の符号データ出力速度と、前記符号化部202の2つの各符号化処理部205、206での符号化結果とを入力して、次回符号化する場合の前記符号化処理部205、206での総符号量を制御する。
Further, the multi-stream encoding control apparatus shown in FIG. 1 includes an output unit 203 that outputs code data encoded by the encoding unit 202, and an encoding prediction unit 204. The encoding prediction unit 204 inputs the encoded data output speed of the output unit 203 and the encoding results of the two encoding processing units 205 and 206 of the encoding unit 202, and performs next encoding. In this case, the total code amount in the encoding processing units 205 and 206 is controlled.
次に、映像信号が外部から入力された場合の動作について図2の構成図と図3のフローチャートに基づいて説明する。
Next, the operation when a video signal is input from the outside will be described based on the configuration diagram of FIG. 2 and the flowchart of FIG.
図3のステップS301では、入力部201にて外部から入力された画像を取得する。次に、ステップS302では、符号化部202へ取得した映像信号を入力し、所望の符号化処理が一方の符号化処理部205での符号化か他方の符号化処理部206での符号化かを判断し、符号化処理部205での符号化の場合はステップS303に進み、符号化処理部206での符号化の場合はステップS305に進む。
In step S301 of FIG. 3, an image input from the outside is acquired by the input unit 201. Next, in step S302, the acquired video signal is input to the encoding unit 202, and whether the desired encoding process is encoding in one encoding processing unit 205 or encoding in the other encoding processing unit 206 is performed. If the encoding processing unit 205 performs encoding, the process proceeds to step S303. If the encoding processing unit 206 performs encoding, the process proceeds to step S305.
ステップS303、S304及びステップS305、S306は、各々、前記符号化処理部205、206での処理を示しており、ステップS303、S305では、符号化予測部204にて、出力部203の符号データ出力速度と、単位時間当たりの各符号化処理部205、206で符号化処理した2つの符号量の総和と、その各符号化処理でのフレームレートと、その各符号化処理でのビットレートと、前回の符号化時に設定された符号化パラメータとに基づいて、各符号化処理部205、206にて次に符号化する場合の各符号化パラメータを決定する。
Steps S303 and S304 and steps S305 and S306 respectively show the processing in the encoding processing units 205 and 206. In steps S303 and S305, the encoding prediction unit 204 outputs code data from the output unit 203. The speed, the sum of the two code amounts encoded by each encoding processing unit 205, 206 per unit time, the frame rate in each encoding process, the bit rate in each encoding process, Based on the encoding parameters set at the time of the previous encoding, the encoding processing units 205 and 206 determine the encoding parameters for the next encoding.
ステップS304、S306では、各符号化処理部205、206での符号化方式に則って入力映像信号を各々符号化する。その後、ステップS307では、各符号化処理部205、206で符号化した結果を前記符号化予測部204へ通知し、符号化予測部204はその保持している情報を前記通知情報に基づいて更新する。ステップS308では、各符号化処理部205、206で符号化された各符号データを出力部203から出力処理する。
In steps S304 and S306, the input video signal is encoded according to the encoding method in the encoding processing units 205 and 206, respectively. Thereafter, in step S307, the result of encoding by the encoding processing units 205 and 206 is notified to the encoding prediction unit 204, and the encoding prediction unit 204 updates the held information based on the notification information. To do. In step S <b> 308, each code data encoded by each encoding processing unit 205, 206 is output from the output unit 203.
(実施形態2)
図4は本発明の第2の実施形態を示す。 (Embodiment 2)
FIG. 4 shows a second embodiment of the present invention.
図4は本発明の第2の実施形態を示す。 (Embodiment 2)
FIG. 4 shows a second embodiment of the present invention.
図4は、2つの異なる動画符号化形式がH.264とMPEG4であるマルチストリーム符号化制御装置を搭載したWebカメラの構成を示す。
Figure 4 shows that two different video encoding formats are H.264. 2 shows a configuration of a Web camera equipped with a H.264 and MPEG4 multi-stream encoding control device.
同図のWebカメラは、入力された映像光を電気信号に変換する撮像素子から読み出された映像信号を取得する画像入力部401と、YC生成と所望の画像サイズにリサイズするYC生成・リサイズ部402と、得られたYCデータについてH.264符号化をする符号化処理部406及びMPEG4符号化をする符号化処理部407からなる符号化部403と、その符号化した符号データをUSB出力する出力部404と、前記出力部404の符号データ出力速度及び前記各符号化処理部406、407の符号化結果を入力して次回に前記各符号化処理部406、407で符号化する場合の符号量の制御を行う符号化予測部405とを備え、前記出力部404から接続先のホストPC408へUSBケーブル経由で符号データの送信を行う構成である。
The Web camera in FIG. 1 includes an image input unit 401 that acquires a video signal read from an image sensor that converts input video light into an electrical signal, YC generation and resizing to a desired image size. Part 402 and the obtained YC data. An encoding unit 403 including an encoding processing unit 406 that performs H.264 encoding and an encoding processing unit 407 that performs MPEG4 encoding, an output unit 404 that outputs the encoded code data to USB, and codes of the output unit 404 An encoding prediction unit 405 that inputs the data output speed and the encoding results of the encoding processing units 406 and 407 and controls the amount of code when the encoding processing units 406 and 407 perform encoding next time; And transmitting the code data from the output unit 404 to the connection destination host PC 408 via the USB cable.
次に、映像信号が外部から入力された場合の動作について図4の構成図と図5のフローチャートより説明する。
Next, the operation when a video signal is input from the outside will be described with reference to the configuration diagram of FIG. 4 and the flowchart of FIG.
図5(a)において、ステップS501では、画像入力部401にて外部から入力された映像光を電気信号に変換する撮像素子から読み出された映像信号を取得する。ステップS502では、YC生成・リサイズ部402において、前記取得された映像信号についてYC生成及び所望の画像サイズにリサイズする。次のステップS503では、前記YC生成・リサイズされた画像を符号化部403へ入力して、所望の符号化処理対象の符号化処理部406又は407を選択する。
5A, in step S501, a video signal read from an image sensor that converts video light input from the outside into an electrical signal in the image input unit 401 is acquired. In step S502, the YC generation / resizing unit 402 regenerates the acquired video signal to YC generation and a desired image size. In the next step S503, the YC generated and resized image is input to the encoding unit 403, and the desired encoding processing unit 406 or 407 is selected.
ステップS504、S505及びステップS506、S507は、各々、符号化処理部406、407での符号化処理を示している。ステップS504及びS506では、符号化予測部405にて、USB出力部404の符号データ出力速度と、各符号化処理部406、407での符号化処理結果とに基づいて、これ等符号化処理部406、407で各々次に符号化する場合の符号化パラメータを決定する。
Steps S504 and S505 and steps S506 and S507 indicate the encoding processes in the encoding processing units 406 and 407, respectively. In steps S504 and S506, the encoding prediction unit 405, based on the encoded data output speed of the USB output unit 404 and the encoding processing results of the encoding processing units 406 and 407, performs the encoding processing unit. In 406 and 407, encoding parameters for the next encoding are determined.
ステップS505とS507では、各符号化処理部406、407において、各々、自己の符号化方式に則り、前記YC生成・リサイズ部402からの入力画像を符号化する。ステップS508では、前記各符号化処理部406、407で符号化した各々の結果を符号化予測部405へ通知し、符号化予測部405は保持している情報をその通知された情報に基づいて更新する。ステップS509では、前記符号化処理部406、407で符号化された各々の符号データをUSB出力部404からホストPC408へ出力する。
In steps S505 and S507, the encoding processing units 406 and 407 respectively encode the input image from the YC generation / resizing unit 402 in accordance with its own encoding method. In step S508, the results encoded by the encoding processing units 406 and 407 are notified to the encoding prediction unit 405, and the encoding prediction unit 405 determines the information held based on the notified information. Update. In step S509, the respective code data encoded by the encoding processing units 406 and 407 are output from the USB output unit 404 to the host PC 408.
図5(b)は、前記図5(a)のステップS504とS506での符号化予測処理の詳細を示したフローチャートである。
FIG. 5B is a flowchart showing details of the encoding prediction process in steps S504 and S506 in FIG.
同図(b)のフローチャートにおいて、ステップS510では、前記図5(a)のステップS508で保持した例えば符号化処理部406での前回のH.264符号化結果であり、符号量、符号化パラメータであるQp値(量子化パラメータ)、次のピクチャタイプを読み出す。
In the flowchart of FIG. 5B, in step S510, for example, the previous H.264 processing in the encoding processing unit 406 held in step S508 of FIG. The H.264 encoding result, the code amount, the Qp value (quantization parameter) that is the encoding parameter, and the next picture type are read out.
ステップS511とステップS512では、他の符号化形式であるMPEG4符号化処理での符号処理部407の前回の符号化結果(符号量、Qp値、次のピクチャタイプ)を取得する。また、ステップS513では、USB出力部404の出力情報として出力ビットレートと、符号化処理部406、407での単位時間当たりの各出力符号量の総和を取得する。
In step S511 and step S512, the previous encoding result (code amount, Qp value, next picture type) of the encoding processing unit 407 in the MPEG4 encoding process which is another encoding format is acquired. In step S513, the output bit rate and the total sum of output code amounts per unit time in the encoding processing units 406 and 407 are acquired as output information of the USB output unit 404.
そして、ステップS514では、各符号処理部406、407で次に符号化する符号化パラメータを決定するために、前記取得したUSB出力部404の出力ビットレートと、単位時間当たりの出力符号量総和とに基づいて、出力可能符号量を算出すると共に、例えば符号化処理部406で次にH.264符号化する1フレーム当たりの符号量を算出し、この次のピクチャタイプが符号量の大きいIピクチャである場合に、符号化処理部407で次にMPEG4符号化処理もIピクチャであるときには、単位時間当たりの符号量が大きくなるため、符号化処理部407で次にMPEG4符号化処理するピクチャタイプをP又はBピクチャに選択し、その後、符号化処理部407での前回の符号量と次にMPEG4符号化する1フレーム当たりの符号量とを比較し、同等の場合は前回の符号化時と同じQp値、前回の符号量の方が小さい場合には前回の符号化時よりもQp値を減らし、逆に前回の符号量の方が大きい場合には前回の符号化時よりもQp値を増やす制御を行うように、次の符号化パラメータQp値を決定する。
In step S514, in order to determine the encoding parameter to be encoded next in each of the code processing units 406 and 407, the obtained output bit rate of the USB output unit 404, the total output code amount per unit time, Based on the above, the output possible code amount is calculated and, for example, the encoding processing unit 406 next outputs the H.264 code. When the amount of code per frame to be H.264 encoded is calculated and the next picture type is an I picture with a large amount of code, when the MPEG4 encoding process is also an I picture next in the encoding processing unit 407, Since the code amount per unit time increases, the encoding processing unit 407 selects the picture type to be MPEG4 encoded next as P or B picture, and then the previous code amount and the next code amount in the encoding processing unit 407 are selected. Are compared with the code amount per frame to be MPEG4 encoded, and if equal, the same Qp value as in the previous encoding, and if the previous code amount is smaller, the Qp value than in the previous encoding On the contrary, when the previous code amount is larger, the next encoding parameter Qp value is determined so as to perform control to increase the Qp value compared to the previous encoding.
(実施形態3)
図6は本発明の第3の実施形態を示す。 (Embodiment 3)
FIG. 6 shows a third embodiment of the present invention.
図6は本発明の第3の実施形態を示す。 (Embodiment 3)
FIG. 6 shows a third embodiment of the present invention.
同図は、動画符号化形式であるH.264と静止画符号化形式であるJPEGとを2種の異なる符号化形式とするマルチストリーム符号化制御装置を搭載したWebカメラ(カメラシステム)の構成を示す。
The figure shows the video coding format H.264. 1 shows a configuration of a Web camera (camera system) equipped with a multi-stream encoding control device that uses H.264 and JPEG as a still image encoding format in two different encoding formats.
図6に示したWebカメラは、入力された映像光を電気信号に変換する撮像素子から読み出された映像信号を取得する画像入力部601と、YC生成と所望の画像サイズにリサイズするYC生成・リサイズ部602と、得られたYCデータをH.264符号化する符号化処理部606と、JPEG符号化をする符号化処理部607とからなる符号化部603と、この符号化された符号データをUSB出力するUSB出力部604と、USB出力部604の符号データ出力速度及び各符号化処理部606、607の符号化結果を入力して各符号化処理部606、607で次回に符号化する場合の各々の符号量を制御する符号化予測部605と、静止画用のJPEG符号化された符号データを固定長に分割して一定周期で出力部604へ出力する分割部608とを備え、前記USB出力部604から接続先のホストPC609へUSBケーブル経由でデータの送信を行う構成である。
The web camera shown in FIG. 6 includes an image input unit 601 that acquires a video signal read from an image sensor that converts input video light into an electrical signal, and YC generation that resizes the image to a desired image size. The resizing unit 602 and the obtained YC data are converted into H.264. An encoding unit 603 including an encoding processing unit 606 that performs H.264 encoding, an encoding processing unit 607 that performs JPEG encoding, a USB output unit 604 that outputs the encoded code data via USB, and a USB output unit Encoding prediction unit for controlling the amount of code when the encoding data output speed of 604 and the encoding results of the encoding processing units 606 and 607 are input and the encoding processing units 606 and 607 perform encoding next time 605 and a dividing unit 608 that divides still image JPEG-encoded code data into fixed lengths and outputs the divided data to the output unit 604 at a fixed period. The USB output unit 604 sends a USB to the connection destination host PC 609. In this configuration, data is transmitted via a cable.
次に、映像信号が外部から入力された場合の動作について、図6の構成図と図7のフローチャートより説明する。
Next, the operation when a video signal is input from the outside will be described with reference to the configuration diagram of FIG. 6 and the flowchart of FIG.
図7(a)において、ステップS701では画像入力部601にて外部から入力された映像光を電気信号に変換する撮像素子から読み出された映像信号を取得する。ステップS702では、YC生成・リサイズ部602において、取得した映像信号をYC生成及び所望の画像サイズにリサイズする。その後、ステップS703では、YC生成・リサイズした画像を符号化部603へ入力し、所望の符号化処理対象を選択する。
7A, in step S701, a video signal read from an image sensor that converts video light input from the outside into an electrical signal in the image input unit 601 is acquired. In step S702, the YC generation / resizing unit 602 generates YC and resizes the acquired video signal to a desired image size. Thereafter, in step S703, the YC generated and resized image is input to the encoding unit 603, and a desired encoding process target is selected.
ステップS704、S705及びステップS706、S707は、各々、符号化処理部606、607での処理を示している。ステップS704とS706では、符号化予測部605にて、USB出力部604の符号データ出力速度と、各符号化処理部606、607での各符号化結果とに基づいて、次に各符号化処理部606、607で符号化する際の各々の符号化パラメータを決定する。
Steps S704 and S705 and steps S706 and S707 indicate processing in the encoding processing units 606 and 607, respectively. In steps S <b> 704 and S <b> 706, the encoding prediction unit 605 next performs each encoding process based on the encoded data output speed of the USB output unit 604 and the encoding results of the encoding processing units 606 and 607. Each encoding parameter at the time of encoding by the units 606 and 607 is determined.
ステップS705とS707では、各符号化処理部606、607での符号化方式に則って入力画像信号を符号化する。ステップS708では、各符号化処理部606、607での符号化結果を符号化予測部605へ通知し、符号化予測部605は、保持している情報を前記通知された情報に基づいて更新する。また、ステップS707では、JPEG符号化されたデータを固定長に分割する情報を一定周期で分割部608へ通知する。
In steps S705 and S707, the input image signal is encoded in accordance with the encoding method in each of the encoding processing units 606 and 607. In step S708, the encoding prediction unit 605 is notified of the encoding results of the encoding processing units 606 and 607, and the encoding prediction unit 605 updates the held information based on the notified information. . In step S707, information for dividing the JPEG-encoded data into a fixed length is notified to the dividing unit 608 at a fixed period.
その後、ステップS709では、各符号化処理部606、607で符号化された各々の符号データをUSB出力部604からホストPC609へ出力する。
Thereafter, in step S709, the encoded data encoded by the encoding processing units 606 and 607 is output from the USB output unit 604 to the host PC 609.
このとき、符号化処理部607で符号化された静止画用符号データは、分割部608にて所望の時間内に転送できるように固定長に分割され、一定周期でUSB出力部604へ出力する。
At this time, the still image code data encoded by the encoding processing unit 607 is divided into fixed lengths so as to be transferred within a desired time by the dividing unit 608, and is output to the USB output unit 604 at a constant cycle. .
図7(b)は、前記図7(a)のフローチャートのステップS704とS706の詳細を示すフローチャートである。同図において、ステップS710では、図7(a)のステップS708で保持したH.264符合化処理の前回の符号化結果である符号量、Qp値、次のピクチャタイプを読み出す。
FIG. 7B is a flowchart showing details of steps S704 and S706 in the flowchart of FIG. In the figure, in step S710, the H.D. stored in step S708 of FIG. The code amount, the Qp value, and the next picture type, which are the previous encoding results of the H.264 encoding process, are read out.
その後、ステップS711とステップS712では、符号化処理部607でのJPEG符号化形式の前回の符号化結果(符号量、量子化テーブル、スケールファクタ)を取得する。続くステップS713では、USB出力部604の出力情報として出力ビットレート、両符号化処理部606、607の単位時間当たりの出力符号量総和を取得する。
Thereafter, in step S711 and step S712, the previous encoding result (code amount, quantization table, scale factor) of the JPEG encoding format in the encoding processing unit 607 is acquired. In subsequent step S713, the output bit rate and the total output code amount per unit time of both encoding processing units 606 and 607 are acquired as output information of the USB output unit 604.
ステップS714では、各符号化処理部606、607で次に符号化する各々の符号化パラメータを決定するために、前記取得した出力ビットレートと、単位時間当たりの出力符号量総和と、JPEGの一定周期の出力符号量とに基づいて、出力可能符号量を算出すると共に、H.264符合化処理部606で次に符号化する1フレーム当たりの符号量を算出して、前回の1フレーム当たりの符号量と次に符号化する1フレーム当たりの符号量とを比較し、同等の場合は同じQp値に、前回の符号量が小さい場合はQp値を減らし、前回の符号量が大きい場合はQp値を増やす制御を行うよう、次の符号化パラメータを決定する。JPEG符号化処理部607は、前回の符号量に基づいて、分割部608で使用する固定長の情報を算出する。
In step S714, in order to determine each encoding parameter to be encoded next in each encoding processing unit 606, 607, the acquired output bit rate, the total output code amount per unit time, and a constant JPEG Based on the output code amount of the period, the code amount that can be output is calculated. The H.264 encoding processing unit 606 calculates the code amount per frame to be encoded next, compares the previous code amount per frame with the code amount per frame to be encoded next, If the previous code amount is small, the Qp value is decreased. If the previous code amount is large, the next encoding parameter is determined so as to increase the Qp value. The JPEG encoding processing unit 607 calculates fixed length information used by the dividing unit 608 based on the previous code amount.
以上説明したように、本発明のマルチストリーム符号化制御装置は、出力部の出力転送速度を考慮してその出力帯域範囲一杯に使用しつつ、動画像の画質劣化を抑制して高画質の動画像をリアルタイムに出力できるので、例えば、監視カメラ、WEBカメラなどのネットワークカメラや、車載カメラ、ドライブレコーダなどの自動車搭載用カメラ、TVやPC接続によるTV会議システムやTV電話などに利用すると、好適である。
As described above, the multi-stream coding control apparatus of the present invention uses the output transfer speed of the output unit to use the full output band range, and suppresses the deterioration of the image quality of the moving image, thereby moving the high-quality moving image. Since images can be output in real time, it is suitable for use in, for example, network cameras such as surveillance cameras and WEB cameras, in-vehicle cameras, camera mounted on vehicles such as drive recorders, TV conference systems and TV phones connected to TVs and PCs, etc. It is.
201 入力部
202 符号化部
203 出力部
204 符号化予測部
205、206 符号化処理部
401、601 画像入力部
402、602 YC生成・リサイズ部
403、603 符号化部
404、604 USB出力部
405、605 符号化予測部
406、606 H.264符号化処理部
407 MPEG4符号化処理部
408、609 ホストPC
607 JPEG符号化処理部
608 分割部 201Input unit 202 Encoding unit 203 Output unit 204 Encoding prediction unit 205, 206 Encoding processing unit 401, 601 Image input unit 402, 602 YC generation / resizing unit 403, 603 Encoding unit 404, 604 USB output unit 405, 605 encoding prediction units 406 and 606 H.264. H.264 encoding processing unit 407 MPEG4 encoding processing unit 408, 609 Host PC
607 JPEGencoding processing unit 608 dividing unit
202 符号化部
203 出力部
204 符号化予測部
205、206 符号化処理部
401、601 画像入力部
402、602 YC生成・リサイズ部
403、603 符号化部
404、604 USB出力部
405、605 符号化予測部
406、606 H.264符号化処理部
407 MPEG4符号化処理部
408、609 ホストPC
607 JPEG符号化処理部
608 分割部 201
607 JPEG
Claims (7)
- 映像信号を入力する画像入力部と、
前記画像入力部に入力された前記映像信号を、2種以上の互いに異なる符号化方式に合わせて符号化する符号化部と、
前記符号化部で符号化された2種以上の符号化データを出力する出力部と、
前記符号化部での符号化結果に基づいて、前記符号化部での次の符号化設定を決定する符号化予測部とを備え、
前記符号化予測部は、
2種以上の異なる符号化方式での符号化結果と、それ等の符号化時に設定された符号化パラメータとに基づいて、前記2種以上の異なる符号化方式での総符号量を制御する
ことを特徴とするマルチストリーム符号化制御装置。 An image input unit for inputting a video signal;
An encoding unit that encodes the video signal input to the image input unit in accordance with two or more different encoding methods;
An output unit that outputs two or more types of encoded data encoded by the encoding unit;
An encoding prediction unit that determines a next encoding setting in the encoding unit based on an encoding result in the encoding unit;
The encoding prediction unit
Controlling the total amount of codes in the two or more different encoding schemes based on the encoding results in the two or more different encoding schemes and the encoding parameters set at the time of the encoding. A multi-stream coding control apparatus characterized by the above. - 前記請求項1記載のマルチストリーム符号化制御装置において、
前記2種以上の互いに異なる符号化方式は、2種以上の異なる動画符号化方式であり、
前記符号化予測部は、
前記2種以上の互いに異なる符号化方式の各々について、
符号化後の符号量及び符号化パラメータと、
前記符号化時よりも前回の符号化時での符号量及び符号化パラメータと、
符号化するフレームレートと、
前記出力部の出力レートとの何れか1つ以上によって、
次の符号化パラメータを決定する
ことを特徴とするマルチストリーム符号化制御装置。 The multi-stream encoding control apparatus according to claim 1, wherein
The two or more different encoding schemes are two or more different video encoding schemes,
The encoding prediction unit
For each of the two or more different encoding schemes,
Code amount after encoding and encoding parameters;
Code amount and encoding parameter at the time of previous encoding than at the time of encoding, and
Encoding frame rate;
According to any one or more of the output rates of the output unit,
A multi-stream coding control apparatus characterized by determining a next coding parameter. - 前記請求項1記載のマルチストリーム符号化制御装置において、
前記2種以上の互いに異なる符号化方式は、1つ以上の静止画符号化方式と1つ以上の動画符号化方式であり、
前記符号化予測部は、
前記2種以上の互いに異なる符号化方式の各々について、
符号化後の符号量及び符号化パラメータと、
前記符号化時よりも前回の符号化時での符号量及び符号化パラメータと、
符号化するフレームレートと、
前記出力部の出力レートとの何れか1つ以上によって、
次の符号化パラメータを決定する
ことを特徴とするマルチストリーム符号化制御装置。 The multi-stream encoding control apparatus according to claim 1, wherein
The two or more different encoding methods are one or more still image encoding methods and one or more moving image encoding methods,
The encoding prediction unit
For each of the two or more different encoding schemes,
Code amount after encoding and encoding parameters;
Code amount and encoding parameter at the time of previous encoding than at the time of encoding, and
Encoding frame rate;
According to any one or more of the output rates of the output unit,
A multi-stream coding control apparatus characterized by determining a next coding parameter. - 前記請求項2記載のマルチストリーム符号化制御装置において、
前記2つ以上の互いに異なる動画符号化方式は、各々、映像信号の各フレーム毎に符号化処理が異なり、
前記符号化予測部は、
前記各符号化方式において映像信号の各フレーム毎に符号化処理が異なって符号量が変動することを予測して、次に符号化する符号量を制御する
ことを特徴とするマルチストリーム符号化制御装置。 The multi-stream encoding control apparatus according to claim 2, wherein
Each of the two or more different video encoding methods has different encoding processing for each frame of the video signal,
The encoding prediction unit
Multi-stream coding control characterized by predicting that the coding amount varies due to different coding processing for each frame of the video signal in each coding method, and controlling the coding amount to be coded next apparatus. - 前記請求項3記載のマルチストリーム符号化制御装置において、
静止画符号化方式での静止画の符号化結果を複数に分割して前記出力部へ渡す分割部を有し、
前記出力部は、分割されたデータサイズを前記符号化予測部に通知し、
前記符号化予測部は、
前記出力部の出力レートと、前記分割されたデータサイズとに基づいて、動画符号化方式での動画符号化処理の符号量を制御をする
ことを特徴とするマルチストリーム符号化制御装置。 The multi-stream encoding control apparatus according to claim 3, wherein
A dividing unit that divides the encoding result of the still image in the still image encoding method into a plurality of divisions,
The output unit notifies the encoded prediction unit of the divided data size,
The encoding prediction unit
A multi-stream encoding control apparatus that controls a code amount of a moving image encoding process in a moving image encoding method based on an output rate of the output unit and the divided data size. - 請求項4記載のマルチストリーム符号化制御装置において、
前記符号化予測部は、
次に符号化する符号量の制御について、周期的に符号量が変動する情報を保持し、符号量が多いフレームが重ならないように制御する
ことを特徴とするマルチストリーム符号化制御装置。 The multi-stream encoding control apparatus according to claim 4,
The encoding prediction unit
Next, with respect to control of the code amount to be encoded, the multi-stream encoding control apparatus is characterized by holding information in which the code amount fluctuates periodically so that frames with a large code amount do not overlap. - 前記請求項1~6の何れか1項に記載のマルチストリーム符号化制御装置を備えた
ことを特徴とするカメラシステム。 A camera system comprising the multi-stream encoding control device according to any one of claims 1 to 6.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2010217221A JP2012074844A (en) | 2010-09-28 | 2010-09-28 | Multi-stream encoding controller and camera system |
JP2010-217221 | 2010-09-28 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012042701A1 true WO2012042701A1 (en) | 2012-04-05 |
Family
ID=45892200
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2011/002916 WO2012042701A1 (en) | 2010-09-28 | 2011-05-25 | Multi-stream encoding control device and camera system |
Country Status (2)
Country | Link |
---|---|
JP (1) | JP2012074844A (en) |
WO (1) | WO2012042701A1 (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015126362A (en) * | 2013-12-26 | 2015-07-06 | アイホン株式会社 | Nurse call system |
JP6399758B2 (en) * | 2014-01-21 | 2018-10-03 | キヤノン株式会社 | IMAGING DEVICE AND IMAGING DEVICE CONTROL METHOD |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH05328293A (en) * | 1992-05-20 | 1993-12-10 | Hitachi Ltd | Image compression recorder with plural recording channels |
JPH10136379A (en) * | 1996-10-29 | 1998-05-22 | Toshiba Corp | Moving image coding method and its device |
JP2004140651A (en) * | 2002-10-18 | 2004-05-13 | Fujitsu Ltd | Device and method for data transmission |
JP2004320707A (en) * | 2003-02-26 | 2004-11-11 | Kddi Corp | Apparatus for simultaneously encoding multiple video information |
JP2004343576A (en) * | 2003-05-19 | 2004-12-02 | Hitachi Ltd | Coding system, video camera apparatus using same, and coding method |
JP2006303987A (en) * | 2005-04-21 | 2006-11-02 | Matsushita Electric Ind Co Ltd | Image encoder and image encoding method |
WO2010035378A1 (en) * | 2008-09-29 | 2010-04-01 | パナソニック株式会社 | Image encoding device, image encoding method, and imaging system |
WO2010041389A1 (en) * | 2008-10-07 | 2010-04-15 | パナソニック株式会社 | Video signal encoding device and imaging device |
-
2010
- 2010-09-28 JP JP2010217221A patent/JP2012074844A/en not_active Withdrawn
-
2011
- 2011-05-25 WO PCT/JP2011/002916 patent/WO2012042701A1/en active Application Filing
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH05328293A (en) * | 1992-05-20 | 1993-12-10 | Hitachi Ltd | Image compression recorder with plural recording channels |
JPH10136379A (en) * | 1996-10-29 | 1998-05-22 | Toshiba Corp | Moving image coding method and its device |
JP2004140651A (en) * | 2002-10-18 | 2004-05-13 | Fujitsu Ltd | Device and method for data transmission |
JP2004320707A (en) * | 2003-02-26 | 2004-11-11 | Kddi Corp | Apparatus for simultaneously encoding multiple video information |
JP2004343576A (en) * | 2003-05-19 | 2004-12-02 | Hitachi Ltd | Coding system, video camera apparatus using same, and coding method |
JP2006303987A (en) * | 2005-04-21 | 2006-11-02 | Matsushita Electric Ind Co Ltd | Image encoder and image encoding method |
WO2010035378A1 (en) * | 2008-09-29 | 2010-04-01 | パナソニック株式会社 | Image encoding device, image encoding method, and imaging system |
WO2010041389A1 (en) * | 2008-10-07 | 2010-04-15 | パナソニック株式会社 | Video signal encoding device and imaging device |
Also Published As
Publication number | Publication date |
---|---|
JP2012074844A (en) | 2012-04-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP5161130B2 (en) | Adaptive bandwidth footprint matching for multiple compressed video streams in fixed bandwidth networks | |
JP7068501B2 (en) | Methods, equipment and computer programs for intramode coding | |
US11546596B2 (en) | Dynamic codec adaptation | |
US20120057629A1 (en) | Rho-domain Metrics | |
US9826260B2 (en) | Video encoding device and video encoding method | |
US20130022116A1 (en) | Camera tap transcoder architecture with feed forward encode data | |
Zhang et al. | Constant quality constrained rate allocation for FGS-coded video | |
CN108476220A (en) | Use the peripheral bus video communication of Internet protocol | |
US10205763B2 (en) | Method and apparatus for the single input multiple output (SIMO) media adaptation | |
JP2005260935A (en) | Method and apparatus for increasing average image refresh rate in compressed video bitstream | |
CN112738516B (en) | Encoding method, encoding device, storage medium and electronic equipment | |
US9560356B2 (en) | Image processing apparatus, image processing method, and image processing system | |
US9226003B2 (en) | Method for transmitting video signals from an application on a server over an IP network to a client device | |
WO2012042701A1 (en) | Multi-stream encoding control device and camera system | |
CN118140477A (en) | Bilateral matching for compound reference patterns | |
JP2017175429A (en) | Encoding device having video switching device and encoding method including video switching detection method | |
JP2011192229A (en) | Server device and information processing method | |
CN116962613A (en) | Data transmission method and device, computer equipment and storage medium | |
US20120062753A1 (en) | Image encoding integrated circuit and encoded image data transmission method thereof | |
KR102312668B1 (en) | Video transcoding system | |
KR100899666B1 (en) | Dispersed multistreaming transmission apparatus | |
JP2005045666A (en) | Transcoder | |
WO2012027892A1 (en) | Rho-domain metrics | |
US20160014417A1 (en) | Methods and apparatuses for stripe-based temporal and spatial video processing | |
US20200213602A1 (en) | Transcoding apparatus, transcoding method, and transcoding program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11828271 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11828271 Country of ref document: EP Kind code of ref document: A1 |