WO2006004065A1 - Video image encoding system, video image encoding or decoding system, video image encoding method, and video image encoding or decoding method - Google Patents

Video image encoding system, video image encoding or decoding system, video image encoding method, and video image encoding or decoding method Download PDF

Info

Publication number
WO2006004065A1
WO2006004065A1 PCT/JP2005/012250 JP2005012250W WO2006004065A1 WO 2006004065 A1 WO2006004065 A1 WO 2006004065A1 JP 2005012250 W JP2005012250 W JP 2005012250W WO 2006004065 A1 WO2006004065 A1 WO 2006004065A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
encoding
decoding
amount
moving image
Prior art date
Application number
PCT/JP2005/012250
Other languages
French (fr)
Japanese (ja)
Other versions
WO2006004065A8 (en
Inventor
Miwako Kanamori
Kentaro Kawakami
Yasuhiro Morita
Hideo Ohira
Masahiko Yoshimoto
Original Assignee
Kanazawa University Technology Licensing Organization Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kanazawa University Technology Licensing Organization Ltd. filed Critical Kanazawa University Technology Licensing Organization Ltd.
Priority to JP2006528865A priority Critical patent/JPWO2006004065A1/en
Publication of WO2006004065A1 publication Critical patent/WO2006004065A1/en
Publication of WO2006004065A8 publication Critical patent/WO2006004065A8/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation

Definitions

  • Moving picture coding processing system moving picture coding or decoding processing system, moving picture coding processing method, and moving picture coding or decoding processing method
  • the present invention uses a processor that can change the operating frequency, the operating power supply voltage and the Z or substrate bias voltage, and sequentially encodes a moving image composed of a plurality of frames in units of frames.
  • the present invention relates to a moving image encoding processing system, a moving image encoding processing or decoding processing system, a moving image encoding processing method, and a moving image encoding or decoding processing method.
  • Non-Patent Document 1 Proceedings of IEEE International Symposium on Circuits and System 2001 (May, 2001) pp918-921 "An LSI for Vdd- Hopping and MPEG4 System Based on the Chip, (H. Kawaguchi, G. Zhang , S. Lee, and T. Sakurai)
  • FIG. 22 is a diagram showing a conventional technique for reducing power consumption for the moving image (moving image) encoding processing system shown in Non-Patent Document 1. Note that the same means for reducing power consumption is applied to the moving picture decoding processing system.
  • Non-Patent Document 1 an operation power supply for reducing power consumption when processing moving image coding (particularly MPEG) on a processor that can dynamically change the operation power supply voltage and operation frequency. Demonstrate how to control voltage and operating frequency!
  • Non-Patent Document 1 discloses that, as shown in FIG. 23, when moving picture encoding is performed, calculation of moving picture code key or decoding key is performed in units of frames depending on the intensity of motion in the moving picture. Focusing on the difference in amount, the operating frequency and operating power supply voltage of the processor are controlled to reduce power consumption.
  • the processing time of one frame is limited to the time Tf by the definition of the encoding method (MPEG, etc.), and the encoding process of one frame is completed within the processing time Tf. That power S is needed.
  • R be the number of moving image blocks processed in one time slot Tslot (moving image is processed in units of blocks) (ie, RXN is the number of blocks in one frame), and (RX i) TaccG + l) is the time spent in block processing (that is, the time actually processed for the block group to be processed from time slot Tslotl to time slot Tslot). Trd is the time until the operating power supply voltage and operating frequency stabilize when the voltage is changed. Note that the real time slot RTsloti is a time slot. G Indicates the actual processing time required for processing to be completed in Tsloti. In FIG.
  • the clock frequency Fmax that can be processed sufficiently in the time slots Tsloti and Tslot2 even when the load is maximum. Operate with. If the time Tacc 3 that was powerful in the processing is Tacc3 (Tf—TR2), that is, if the assigned block group has completed processing in the time slots Tslotl and Tslot2, it is assigned to the next time slot Tslot3.
  • Tacc3 Tf—Tacc3—TR3-Trd
  • the processing of the block group assigned to Tslot3 should be completed within this processing time Ttar3.
  • the block group is operated at a lower operating frequency.
  • the processing times Tfl, T12, and ⁇ in Fig. 22 indicate the processing times when operating at each operating frequency Fl, F2, F3 when the load is maximum in time slot Tslot3.
  • the minimum operation frequency can be selected from among the operation frequencies that can process a predetermined number of block groups within a predetermined time. Lowering the operating frequency and operating power supply voltage, and controlling the voltage according to the required processing, lower power consumption.
  • Ff KfZTf
  • Non-Patent Document 1 the operating power supply voltage and the operating frequency are changed a maximum of N times within one frame even though the unit of synchronization of the processing time Tf is one frame. Therefore, low power consumption was not enough.
  • the low power consumption of video encoding or decoding processing in a processor that can control the operating power supply voltage and operating frequency in multiple stages as in the conventional example is achieved by operating power supply many times during the processing of one frame. It was necessary to change the voltage and operating frequency.
  • the unit of processing time constraint is a frame, it is preferable to control at the minimum fixed frequency that enables processing during processing of one frame. Therefore, in this conventional example in which the power supply voltage and the operating frequency are changed at most N times during processing of one frame, the power consumption cannot be sufficiently reduced.
  • the inventors of the present application completed the invention of Patent Document 1 below.
  • the power consumption is reduced by performing the encoding process or the decoding process while operating the processor 1 at a constant operating power supply voltage / operating frequency.
  • Patent Document 1 Japanese Patent Application 2003—48535
  • the subthreshold leakage current is a minute current that flows when the gate voltage of the MOS transistor formed on the semiconductor substrate is lower than the threshold voltage.
  • the power consumption due to this subthreshold leakage current tends to dominate as MOS transistors become finer, and a video signal that uses a processor in which MOS transistors are integrated on a semiconductor substrate is used. This is one of the factors that hinder the reduction of power consumption in a video code key decoding system or decoding key system.
  • This sub-threshold leakage current is obtained at a constant operating frequency Ff at a constant operating frequency Ff as compared with the case where the operating frequency Ff of the processor is varied many times within the processing time Tf of one frame. It is reduced by operating, and the power consumption of the processor can be reduced.
  • the invention of Non-Patent Document 1 described above is effective even when the unit for synchronizing the processing time Tf is one frame. Regardless, the operating frequency has been changed up to N times within one frame, and not only the operating power supply voltage but also the power of subthreshold leakage current is favorable.
  • MOS transistors it is known that the subthreshold leakage current can be controlled by controlling the substrate bias voltage in the semiconductor region in which the MOS transistor is formed.
  • the present invention calculates the amount of computation required for encoding or decoding (hereinafter referred to as the necessary amount of computation) for each frame, and calculates a constant substrate bias voltage, an operating frequency, or a constant substrate bias voltage.
  • the power consumption can be reduced by performing the encoding / decoding process while operating the processor 1 at the voltage / operating frequency.
  • Patent Document 2 Japanese Patent Application 2003—409641
  • Patent Documents 1 and 2 described above when the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the calculation amount actually required for the encoding or decoding process There was a possibility that the failure of the encoding process or the decoding process could not be completed within the time Te allocated for the processing of one frame.
  • Each patent document 1 and 2 also disclosed failure avoidance means for avoiding the failure phenomenon.
  • several failure avoidance means are conceivable, and it is strongly desired to improve the deterioration of image quality caused by the failure avoidance means. .
  • the present invention is for solving the above-mentioned problems, and while realizing low power consumption by controlling the substrate bias voltage, the operating power supply voltage, and the operating frequency, the failure phenomenon is solved.
  • the present invention proposes a video encoding processing system, a video encoding or decoding processing system, a video encoding processing method, and a video encoding or decoding processing method that can be avoided. Means for solving the problem
  • the moving image encoding processing system of the present invention includes a processor functioning as a moving image encoding means for encoding a moving image composed of a plurality of continuous frames in units of frames. Prepared,
  • Necessary calculation amount calculation means for calculating the necessary calculation amount Kp necessary for encoding one frame, and encoding the necessary calculation amount Kp within the time Te previously allocated to the encoding process of the one frame
  • Operation determining means for determining a processing frequency that can be processed, and the processor is pre-allocated! /, The operation frequency determined by the operation determining means within the time period Te and an operation suitable for the operation frequency.
  • a moving image encoding processing system that performs encoding processing of the one frame by the moving image encoding means while operating at a power supply voltage and a Z or substrate bias voltage;
  • It is characterized by comprising first failure avoiding means for reducing the actual calculation amount of the codeh processing at a predetermined timing.
  • the moving image encoding processing method of the present invention includes a moving image encoding step for encoding a moving image composed of a plurality of consecutive frames using a processor in units of frames, and encoding of one frame.
  • Necessary computation amount calculation step for calculating the necessary computation amount Kp, and operation that can signify the required computation amount Kp within the time Te allocated in advance for the signing processing of the one frame
  • An operation determining step for determining the frequency
  • a first failure avoidance step is provided for reducing an actual calculation amount of the sign key processing at a predetermined timing.
  • the first failure avoiding means and the first failure avoiding step reduce the actual amount of computation by changing a part of the sign process to a processing with a smaller amount of computation.
  • the “encoding process” means the process of the operation determining means ⁇ the operation determining step and the encoding process means ⁇ the process of the encoding process step.
  • the “decoding process” is the operation determining process.
  • Means ⁇ Decoding processing means performed after the processing of the action determination step ⁇ Decoding processing step.
  • the necessary calculation amount Kp necessary for the coding process of one frame to be completed within the time Te is calculated,
  • the operating frequency required for the mouth sensor in the time Te is set as an operating frequency that can be processed, and the minimum operating frequency (or close to the minimum) that allows one frame to be processed in the time Te, and Since the encoding process is performed with the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency, power consumption can be reduced.
  • the required calculation amount Kp calculated by the above-described required calculation amount calculation means is smaller than the actually required calculation amount, the actual calculation amount of the sign ⁇ process is the first failure avoidance means or the first Since it is reduced by 1 failure avoidance step, the failure phenomenon can be avoided.
  • the first failure avoiding means is characterized in that a block having color difference signal information is processed as an invalid block among macroblocks in which the sign is not finished.
  • luminance block a block having luminance signal information
  • color difference block a block having color difference signal information
  • the first failure avoiding means performs an intra-frame encoding process on a macroblock for which the encoding has not ended.
  • the intraframe code processing has a smaller amount of computation than the interframe code processing.
  • the necessary amount of calculation is calculated by the necessary amount of calculation means by performing the intra-frame code processing on the macroblock for which the code processing has not been completed. It can be set to a value smaller than or closer to the required calculation amount Kp. This makes it possible to avoid bankruptcy.
  • the first failure avoiding means performs inter-frame code key processing with a motion vector set to 0 without performing motion vector detection for a macroblock for which the code key has not ended.
  • the amount of computation required for the motion vector detection processing is an element that occupies most of the amount of computation in the encoding processing.
  • the calculation amount is larger than that of the normal interframe code processing that performs the motion vector detection processing.
  • the amount of computation of the actual sign process is smaller than the required amount of computation Kp or close to the required amount of computation Kp. Thereby, it becomes possible to avoid the failure phenomenon.
  • the first failure avoidance means is characterized in that a code step is performed with a larger quantum step size for a macroblock that has not been finished.
  • the number of effective blocks generated in a macroblock is increased by performing the encoding process with the quantum step size being increased for a macroblock for which the code is not finished.
  • the number of effective coefficients is suppressed, and the number of executions of processes performed on effective blocks and effective coefficients (for example, when MPEG4 is used as a moving image encoding method, IDCT processing, IQ processing, VLC processing) is reduced.
  • the actual calculation amount of the encoding process can be made smaller than the required calculation amount Kp or close to the required calculation amount Kp. This makes it possible to avoid the bankruptcy phenomenon.
  • the moving image encoding processing system of the present invention has an effect given to the encoding process of the subsequent frame encoded after the first frame by the first failure avoidance means.
  • a mitigating means for mitigating is provided.
  • the first failure avoidance means described above performs the failure avoidance by changing the processing content of the encoding process that should normally be performed, the first failure avoidance means is executed in one frame. If this happens, the image quality of the subsequent frame will be affected by the processing by the first failure avoidance method.
  • the mitigation processing is performed by the mitigation means that mitigates the influence on the encoding processing of the subsequent frame. Degradation of image quality due to encoding processing can be suppressed.
  • the mitigation means applies a quantization step to the macroblock of the subsequent frame corresponding to the macroblock for which the processing by the first failure avoidance means has been executed in the one frame.
  • the feature is to reduce the size of the tape.
  • the present invention by reducing the quantization step size, it is possible to improve the image quality by increasing the code amount allocated to the corresponding macroblock, and the failure due to the first failure avoiding means. Even when the avoidance process is performed, the influence on the subsequent frame encoded after the one frame on which the failure avoidance process is performed can be reduced.
  • the necessary calculation of the subsequent frame to be encoded after the one frame is characterized by increasing the amount. For example, by adding m times (m is a real number greater than or equal to 1) or a real number n greater than 0, the subsequent frame is added to the element used for calculating the necessary calculation amount or the necessary calculation amount of the subsequent frame. Increase the required amount of computation.
  • the required calculation amount Kp is likely to be smaller than the actually required calculation amount for the subsequent frame.
  • the necessary amount of computation for the subsequent frame satisfies the actual amount of computation in order to increase the necessary amount of computation for the subsequent frame. The possibility increases and the failure phenomenon in subsequent frames can be avoided.
  • the moving image encoding or decoding processing system of the present invention functions as a moving image encoding or decoding means for encoding or decoding a moving image composed of a plurality of consecutive frames in units of frames.
  • Necessary amount of computation required for encoding or decoding one frame Necessary amount of computation for calculating Kp, and within the time Te previously assigned to the encoding / decoding processing of the one frame.
  • a moving image encoding or decoding processing system that performs encoding or decoding processing of the one frame by the ⁇ or decoding means
  • the second failure avoiding means is provided.
  • the moving image encoding or decoding processing method of the present invention encodes or decodes a moving image composed of a plurality of consecutive frame forces using a processor in units of frames.
  • Step a necessary computation amount calculating step for calculating a necessary computation amount Kp required for encoding or decoding one frame, and a time Te previously allocated to the encoding or decoding processing of the one frame.
  • An operation determining step for determining an operating frequency capable of encoding or decoding the necessary computation amount Kp, and the processor determines the operation determining step within the time period Te /! While operating with the obtained operating frequency and the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency, the video coding key or decoding step
  • the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount, the required calculation amount of the subsequent frame that is encoded or decoded after the one frame is increased.
  • the second failure avoidance step is provided.
  • the required calculation amount Kp is smaller than the actually required calculation amount.
  • the necessary calculation amount Kp is also smaller than the actually required calculation amount for the subsequent frames.
  • the second failure avoiding means and the second failure avoiding step increase the necessary amount of computation for the subsequent frame, the necessary amount of computation for the subsequent frame may satisfy the actual amount of computation. It becomes higher and the failure phenomenon in the subsequent frame can be avoided.
  • the processing of the second failure avoiding means 'step is performed when the required calculation amount Kp of the predetermined frame is smaller than the actual calculation amount regardless of the presence or absence of the first failure avoiding means' step.
  • the second failure avoiding means and the second failure avoiding step may be performed by multiplying the necessary calculation amount of the subsequent frame or an element used for calculating the required calculation amount by m times (m is 1 or more). Real number) or a real number n greater than 0 is added.
  • the value of the required calculation amount calculated by the required calculation amount calculation means in the processing of the subsequent frame or the value of the element used for calculating the required calculation amount is m times (m is By adding a real number n greater than 1) or a real number n greater than 0, it is possible to increase the necessary amount of computation Kp of the subsequent frame and avoid the occurrence of a failure phenomenon in the subsequent frame.
  • m is By adding a real number n greater than 1) or a real number n greater than 1, it is possible to increase the necessary amount of computation Kp of the subsequent frame and avoid the occurrence of a failure phenomenon in the subsequent frame.
  • elements to be increased, and values of m and ⁇ it is preferable to derive appropriate elements from the experiment beforehand and set them in the system.
  • the moving image encoding or decoding processing system of the present invention functions as moving image encoding or decoding means for encoding or decoding a moving image composed of a plurality of consecutive frames in units of frames.
  • Necessary amount of computation required for encoding or decoding one frame Necessary amount of computation for calculating Kp, and within the time Te previously assigned to the encoding / decoding processing of the one frame.
  • the moving image encoding code is operated while operating at the operating frequency determined by the operation determining means and the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency.
  • it is a moving image encoding or decoding processing system that performs encoding or decoding processing of the one frame by decoding decoding means, and includes third failure avoiding means that extends the time Te at a predetermined timing. It is characterized by having.
  • the moving image encoding or decoding processing method of the present invention includes a moving image encoding step of encoding or decoding a moving image composed of a plurality of continuous frame forces using a processor in units of frames.
  • the necessary calculation amount calculation step for calculating the necessary calculation amount KP required for encoding or decoding one frame, and the time Te allocated in advance for the encoding or decoding processing of the one frame.
  • An operation determining step for determining an operating frequency capable of encoding or decoding the necessary calculation amount Kp, and
  • a third failure avoidance step is provided which extends the time Te at a predetermined timing.
  • the present invention when the encoding / decoding processing of one frame cannot be completed at the time Te allocated in advance, the time allocated to the processing of the next subsequent frame is used.
  • the processing time Te of one frame is extended, and the encoding / decoding processing of the one frame is completed. As a result, it is possible to avoid a failure phenomenon that does not deteriorate the image quality without reducing the actual processing amount of encoding or decoding.
  • the third failure avoiding means when extending the time Te previously assigned to the encoding process of the one frame, is adapted to return the code after the one frame. For the subsequent frame to be signaled, the time Te that is assigned in advance to the encoding process or the decoding process of the subsequent frame is changed.
  • the time allocated to the subsequent frame is shortened. !, The time obtained by subtracting the extended processing time assigned to the above one frame from the time assigned to the encoding or decoding processing of the subsequent frame, The operation of the processor that can process the coding or decoding of the subsequent frame within the time changed in accordance with the extension time to change the time Te, such as the time Te of decoding or decoding processing
  • the frequency is determined, and the processor operates at the operating frequency and the appropriate substrate bias voltage or operating power supply voltage. Therefore, low power consumption is realized in the encoding or decoding processing of subsequent frames. Even if the time Te assigned to the succeeding frame is shortened by extending the time Te assigned to one frame, changing the time Te of the succeeding frame may cause a failure in the processing of the succeeding frame. Can be lowered.
  • the third failure avoiding means stores an input frame for storing a frame to be encoded next when the time Te previously allocated to the encoding process of the one frame is extended. It is characterized in that the frame write destination is changed with respect to the memory.
  • the sequentially input frames are stored by changing the write destination without overwriting one frame. Therefore, the one frame can be held in the input frame memory, and the encoding process can be continued and completed.
  • the third failure avoiding means is provided for an input frame memory that stores a frame to be encoded next when the time Te previously allocated to the encoding process of the one frame is extended. The writing is impossible.
  • the input frame memory cannot write the frame, so that the subsequent frame is not overwritten.
  • the frame can be held, and the sign key process can be continued and completed.
  • the time Te Encoding processing can be performed on the same frame as when the power is not extended.
  • the moving image encoding system when performing a moving image encoding process, assuming that a frame to be encoded before the one frame is a previous frame.
  • the necessary calculation amount calculating means includes a motion amount between the predetermined frame and the previous frame, an amount of activity of the predetermined frame, an amount of activity of the previous frame, an average value of a quantization step size of the previous frame, and a quantization step of the previous frame.
  • the difference between the average value of the size and the average value of the quantization step size of the previous frame, the number of macroblock matching of the previous frame, the number of effective blocks of the previous frame, the number of effective coefficients of the previous frame, the sign of the previous frame The amount of computation actually required for the key, the number of bits generated in the previous frame, the sign key rate of the predetermined frame, and the predetermined frame.
  • the required calculation amount calculation means includes the number of bits of encoded data of a predetermined frame, the type of whether the predetermined frame is intra-frame encoded or inter-frame encoded, predetermined frame Or the average value of the motion vector size of the previous frame, the variance of the motion vector size of the predetermined frame or the previous frame, the number of effective blocks of the predetermined frame or the previous frame, the effective coefficient of the predetermined frame or the previous frame Number, the bit rate of the predetermined frame or the previous frame, the code amount of the predetermined frame or the previous frame, the average value of the quantization step size of the predetermined frame or the previous frame, the difference of the average value of the quantization step size (1 Difference in quantization step size of previous frame, or quantization step size of previous frame Difference between the quantization step size of the previous frame and the previous frame), the amount of calculation actually required for
  • Each of the plurality of elements is an element that affects the required amount of computation Kp in the encoding or decoding process.
  • the required amount of calculation Kp calculated by the required amount of calculation means is The value is closer to the amount of computation when encoding or decoding is actually performed. Therefore, it is less likely that the calculated required calculation amount Kp is too large compared to the actual calculation amount and the reduction in power consumption is hindered. Also, the required calculation amount Kp is smaller than the actual calculation amount and the sign It is difficult for the failure phenomenon that the defect or decryption process is not completed in time.
  • the amount of computation required for the encoding process of one frame is determined by the number of executions of element processing such as macroblock matching.
  • Element processing is classified into processing that is repeatedly executed for the number of times of macroblock matching, processing that is repeatedly executed for the number of effective blocks, and processing that is repeatedly executed for the number of effective coefficients.
  • the amount of computation required for one frame of code processing is the maximum (worst).
  • the number of macroblock matching, the number of effective blocks, and the number of effective coefficients fluctuate below the maximum value for each frame, so the amount of computation required for code processing must vary from frame to frame. It becomes.
  • the number of macroblock matching, the number of effective blocks, the number of effective coefficients, and the amount of computation required for encoding processing are close to each other between frames that are close in time.
  • the number of macroblock matching and the number of effective blocks of the previous frame that has already completed the encoding process at the time of prediction By adopting the number of effective coefficients and the amount of computation required for the encoding process, the required amount of computation K for each frame can be approximated to the amount of computation when the encoding process is actually performed. .
  • the amount of computation required for the decoding process for one frame is also determined by the number of executions of the element process of the decoding process.
  • Element processing is classified into processing that is repeatedly executed by the number of effective blocks, and processing that is repeatedly executed by the number of effective coefficients. When all of the number of effective blocks and the number of effective coefficients are maximized, one-frame decoding is performed. ⁇
  • the amount of computation required for processing is the maximum (worst). In the actual decoding process, the number of effective blocks and the number of effective coefficients fluctuate below the maximum value for each frame, so the amount of computation required for the decoding process varies for each frame.
  • the amount of computation required for the decoding process varies from frame to frame
  • the parameters used to predict the required computational complexity of the decoding process the number of effective blocks, the number of effective coefficients, and the number of effective coefficients of the previous frame that have already been decoded at the time of prediction
  • the required amount of computation K for each frame can be approximated to the amount of computation when decoding is actually performed.
  • the video encoding processing system For one frame to be encoded or decoded (frame to be encoded or decoded in the future), a calculation for predicting the necessary amount of computation Kp required for the code or decoding is performed, and the predetermined value is calculated.
  • the time Te allocated for frame processing is the minimum or close operating frequency required to process the required amount of computation K p, and the operating frequency and operating power supply voltage or operating frequency and substrate bias voltage for each frame. Or operating frequency
  • the substrate bias voltage and the operating power supply voltage are dynamically controlled, low power consumption can be realized.
  • the failure avoiding means since the failure avoiding means is provided, it is possible to avoid the failure phenomenon that occurs when the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount. Alternatively, it is possible to prevent the moving image subjected to the decoding process from being deteriorated. In addition, when processing by the failure avoidance means is executed, mitigation processing is performed by mitigation means that mitigates the effect on subsequent frame encoding processing, thereby suppressing image quality deterioration due to subsequent frame encoding processing. Can do.
  • FIG. 1 is a schematic block diagram showing the operation of the moving image code processing system according to the first embodiment of the present invention.
  • FIG. 2 is a diagram showing an implementation example of a moving image code key processing system according to the first embodiment of the present invention.
  • FIG. 3 is a diagram showing a schematic flowchart of a moving image encoding processing program that causes a computer to function as the moving image encoding process system of the embodiment.
  • FIG. 4 is a cross-sectional view showing a triple tool structure.
  • FIG. 5 is a conceptual diagram showing the operation power supply voltage / substrate bias voltage / operation frequency of the processor used in the moving image code processing system of the embodiment.
  • FIG. 6 is an explanatory diagram for explaining that power consumption can be reduced by keeping the operating power supply voltage and operating frequency constant.
  • FIG. 7 is an explanatory diagram for explaining the relationship between the time for performing an interrupt and the remaining calculation amount in the embodiment.
  • FIG. 8 is a schematic block diagram showing the operation of the moving image code processing system of the second embodiment of the present invention.
  • FIG. 9 is a diagram showing a schematic flowchart of a moving image encoding processing program that causes a computer to function as the moving image encoding process system of the embodiment.
  • FIG. 10 is a schematic block diagram showing an operation of a video decoding process system according to a third embodiment of the present invention.
  • FIG. 11 causes a computer to function as the moving image code processing system of the above embodiment.
  • FIG. 12 is a graph showing a processing time transition of normal encoding processing.
  • FIG. 13 is a diagram showing a transition of processing time of sign key processing in the third embodiment.
  • FIG. 14 is a diagram showing a processing time transition of the sign key processing in the third embodiment.
  • FIG. 15 is a diagram showing a transition of processing time of sign key processing in the third embodiment.
  • FIG. 16 is a schematic block diagram showing the operation of the moving image decoding system according to the fourth embodiment of the present invention.
  • FIG. 17 is a schematic block diagram showing the operation of the moving picture code processing system according to the fifth embodiment of the present invention.
  • FIG. 18 is a conceptual diagram illustrating the relationship between the operation power supply voltage and the operation frequency of the processor used in the moving image code processing system of the above embodiment.
  • FIG. 19 is a schematic block diagram showing an operation of a moving image code processing system according to a sixth embodiment of the present invention.
  • FIG. 20 is a conceptual diagram for explaining a relationship between a substrate bias voltage and an operating frequency of a processor used in the moving image code processing system of the embodiment.
  • FIG. 21 is a diagram showing an example of the relationship between the operating frequency of the processor, the operating power supply voltage, and the substrate bias voltage in the example.
  • FIG. 22 is a diagram showing a conventional technique for reducing power consumption in a moving image code processing system.
  • FIG. 23 is a conceptual diagram showing a state in which the amount of calculation of a moving image code key or decoding key varies from frame to frame.
  • Video coding means 4, 54, 64 Operation control means 5 Video coding means
  • the moving image encoding / decoding system of the present invention is such that a processor 1 described later performs moving image encoding processing and moving image decoding processing.
  • moving image encoding processing is performed. It functions as a system and functions as a video decoding processing system when performing video decoding.
  • the moving image encoding / decoding processing system of the present invention may be one that performs encoding or decoding in units of frames or in units of time, or only decoding processing or only encoding processing. It may be something that performs.
  • processor 1 is a computing unit dedicated to moving image processing (command) for efficiently performing moving image coding processing or moving image decoding processing with a smaller number of cycles, smaller power consumption, and smaller program code amount.
  • arithmetic unit (instruction) dedicated to moving image processing a product-sum arithmetic unit (product-sum operation instruction) used in matrix operations such as discrete cosine transform processing, motion vectors
  • the difference absolute value sum operator (difference absolute value sum command) used in operations such as block matching operations in the detection process is given below.
  • the case where decoding is performed is a moving image decoding system, which will be described in detail separately for moving image encoding processing and moving image decoding processing.
  • the moving image coding processing system S1 has a predetermined frame that occurs when the required calculation amount Kp calculated by the required calculation amount calculation means 2 is smaller than the actual calculation amount of the predetermined frame.
  • the code of the predetermined frame is allocated in advance to the encoding process of the predetermined frame.
  • the first failure avoiding means 9 is provided to perform processing to avoid the failure phenomenon, and the operating frequency, the base bias voltage, and the operating power supply voltage are controlled to be constant for each frame.
  • This system S1 is realized by a computer which is an information terminal device such as a mobile phone or a personal computer with a built-in microcomputer, for example, and is particularly a system that functions as a part of a multimedia signal processing unit in the computer. There is a system that sequentially encodes a moving image composed of a predetermined number of frame forces in units of frames.
  • Fig. 1 is a schematic block diagram showing the operation of the video code key processing system S1 of the present embodiment
  • Fig. 3 shows a video code key processing method realized by the system S1.
  • the moving image coding processing system S1 is variable in operating power supply voltage Vdd and substrate bias voltage Vbn, Vb P and operating frequency F force S r (r is an integer of 2 or more) (ie, r operating power supply) Voltage Vdd and substrate bias voltage Vbn, Vbp, and operating frequency F), and the operating power supply voltage and substrate bias voltage and operating frequency can be changed by the program, and the operating power supply voltage and substrate bias of processor 1
  • a computer in particular, a computer having at least an operation control means 4 for controlling a voltage and an operating frequency, and a local decoding frame memory 6, an input frame memory 7, and an element memory 8 which are storage areas for storing predetermined data.
  • Vbn is the substrate bias voltage of the n-channel MOS transistor and Vbp is the p-channel MOS transistor.
  • Substrate bias voltage In the local decoding memory 6 and the input frame memory 7 and the like, the operation voltage “operation frequency” and the substrate bias voltage may be controlled by the operation control means 4 in the same manner as the processor 1.
  • the elements included in control area CA indicated by dotted lines The operating frequency, operating voltage, and substrate bias voltage are controlled.
  • the processor 1 is a semiconductor element having a triple-well structure, and the substrate bias voltage can be controlled for both the nMOS transistor and the pMOS transistor.
  • the local decoding memory 6 and the input frame memory 7 are semiconductor memory elements, and the operation control means 4 controls the operation power supply voltage / substrate bias voltage / operation frequency in the same manner as the processor 1.
  • FIG. 4 is a partial cross-sectional view of the processor 1 having a triple-well structure.
  • the processor 1 has a triple-well structure by forming an n-type well n-well on a p-type semiconductor substrate p-sub and further forming a p-type well p-well on an n-type well n-well. is there.
  • an n-channel MOS transistor and a p-type well contact layer p-Contact are formed in the p-type well p-well.
  • the n-channel MOS transistor has a source Z drain layer S, D composed of an n-type impurity layer, and a gate electrode G.
  • n-type n-well In the n-type n-well, a p-channel MOS transistor and an n-type contact layer n-contact are formed.
  • the n-channel MOS transistor has a source Z drain layer S, D having a p-type impurity layer force, and a gate electrode G.
  • a substrate bias voltage V bn is applied to a p-type well p-well, which is a semiconductor region where an n-channel MOS transistor is formed, via a p-type contact contact layer p-Contact.
  • a substrate bias voltage Vb p is applied to an n- type well n — well, which is a semiconductor region where a p-channel MOS transistor is formed, via an n- type well contact layer n — Contact.
  • the operation control means 4 includes an operation power supply voltage control means 4c having a DC-DC converter, etc., a substrate bias voltage Vbn generation means 4d for controlling the substrate bias voltage of the n-channel MOS transistor, p-channel Substrate bias voltage Vbp generation means 4e for controlling the substrate bias voltage of MOS transistors 4e, operating frequency control means 4b with PLL, etc.
  • a substrate bias voltage Vbn generation means 4d for controlling the substrate bias voltage of the n-channel MOS transistor
  • p-channel Substrate bias voltage Vbp generation means 4e for controlling the substrate bias voltage of MOS transistors 4e
  • operating frequency control means 4b with PLL etc.
  • each element of the operation control means 4 may exist outside the moving image encoding processing system SI, and may control the operating power supply voltage, the substrate bias voltage, or the operating frequency from outside the moving image encoding processing system S1.
  • the processor 1, the memories 6 and 7, and the operation control means 4 are connected to each other via wiring.
  • the processor 1 includes necessary calculation amount calculation means 2, operation determination means 3, moving picture encoding means 5, and first failure avoidance means 9 as means that operate on the processor 1.
  • Reference numeral 101 is input image data
  • reference numeral 102 is an operation power supply voltage and substrate bias voltage and operation frequency instruction
  • reference numeral 103 is local decoded data of the previous frame
  • reference numeral 105 is operation power supply voltage 'base bias voltage' operation frequency Supply
  • code 106 is the encoded data of the frame
  • code 107 is information on the average value of the quantization step size of the previous frame
  • code 108 is a force inter-frame code that is an intra-frame code for each frame.
  • the code 109 is the information on the video code bit rate
  • the code 110 is the activity amount of the previous frame
  • the code 111 is the number of macroblock matching in the previous frame
  • the code 112 is the number of effective blocks in the previous frame
  • the code 113 Is the number of effective coefficients of the previous frame
  • reference numeral 114 is the average of the quantization step size of the previous frame and the average of the quantization step size of the previous frame. Difference between the values, symbols 115 are code processing amount actually required to I spoon, reference numeral 116 necessary calculation amount of the previous frame calculated by necessary calculation amount calculations means 2 of the previous frame.
  • the element memory 8 is a part of a plurality of elements used in the required amount-of-computation calculation means 2 described later (intra-frame code ⁇ type 108 or code ⁇ . This is a storage area for storing the bit rate 109, the amount of frame activity 110, and the required amount of calculation 116) calculated by the required amount of calculation calculation means 2.
  • the processed macroblock number register 10 is a register that temporarily stores information on the number of macroblocks 117 that have been subjected to sign processing.
  • MPEG-4 is used as the video encoding method for moving image encoding means 5, but other encoding schemes such as H.26X, MPEG-1 and MPEG 2 can be used! ,.
  • FIG. 2 shows an implementation example of the video code key processing system S1.
  • System S1 mainly consists of processor 1, various memories MR, 7a, 7b as peripheral devices, various interfaces CI, DI, BI, operation control circuit 4a, PLL4b, DC-DC converter 4c, substrate bias voltage generation circuit Realized by hardware equipped with 4d, 4e, etc.
  • Each of the above components includes buses Bl, B2, etc. It becomes possible to communicate with each other via!
  • the processor 1 includes a processor core la, an instruction cache memory lb, a data cache memory lc, and a bus controller BC.
  • Necessary calculation amount calculation means 2, motion determination means 3, video encoding means 5, failure avoidance means 9, and 11 are executed by executing a program stored in the memory MR as necessary on the processor core 1a. Realized.
  • the instruction cache memory lb and the data cache memory lc are cache memories provided for high-speed processing of programs executed on the processor core la.
  • the local decoding frame memory 6, the element memory 8, and the processed macroblock number register 10 are aggregated in the memory MR of Fig. 2 and the average value of the quantization step size of the previous frame 107, for each frame.
  • the actual amount of processing required for the sign of the previous frame 115, the required amount of previous frame 116 calculated by the required amount calculation means, and the number of processed macro blocks 117 are stored in the memory MR. It is stored in.
  • Locally decoded data 103 is transmitted and received as signals 100j, 100k, and 1001 between memory MR and processor core la via bus controller BC.
  • the two input frame memories 7a and 7b correspond to the input frame memory 7 of FIG.
  • Video data (input image data 101) input from the camera interface CI is input to the input frame memory 7a (or input frame memory 7b) via the bus B2.
  • the input frame memory (# 0) 7a and the input frame memory (# 1) 7b are switched in use every time one frame is processed. In other words, in the processing of the i-th frame, the input image data is written to the input frame memory (# 1) 7b by the signal 100h, and the input frame memory is input by the signal 100 ⁇ for the encoding process by the moving image encoding processing means.
  • the input image data is read from (# 0) 7a
  • the input image data is written to the input frame memory (# 0) 7a by the signal 100i in the processing of the (i + 1) -th frame, and the moving image is encoded.
  • input image data is input from the input frame memory (# 1) 7b by the signal 100p. Data is read. Therefore, the signal ⁇ does not occur when the input image data is written to the input frame memory (# 1) 7b by the signal lOOh, and conversely, the signal 100h does not occur when the image is read by the signal ⁇ . .
  • the input frame memory (# 0) 7a is used for the processing of the i-th frame
  • the input frame memory (# 1) 7b is used for the processing of the (i + 1) -th frame.
  • Voltage and operating voltage are subject to control.
  • the input frame memory is prepared for two frames, and each operation frequency can be set independently, so that the input image data is always written from the camera interface CI at a constant operation frequency. As a result, it is possible to execute the reading operation of the input image data whose operating frequency varies based on the calculated value of the necessary calculation amount without interfering with each other.
  • the operation control circuit 4a can transmit and receive signals to and from the PLL 4b, the DC-DC converter 4c, and the substrate bias voltage generation circuits 4d and 4e, and these function as the operation control means 4.
  • the operation control circuit 4a receives the operation power supply voltage 'substrate bias voltage' operation frequency instruction 102 by the signal lOOe from the processor core la, and transmits the signal lOOu to the PL L4b based on the instruction 102, and the DC-DC converter Signal ⁇ is transmitted to 4c, signal lOOw is transmitted to substrate bias voltage generation circuit 4d, and signal ⁇ is transmitted to substrate bias voltage generation circuit 4e.
  • the PLL 4b transmits the operating frequency signal 100a based on the signal lOOu
  • the DC-DC converter 4c supplies the operating power voltage 100b based on the signal ⁇
  • the substrate bias voltage generating circuit 4d generates nM based on the signal lOOw.
  • the OS substrate bias voltage 100c is supplied, and the substrate bias voltage generation circuit 4e supplies the pMOS substrate bias voltage 100d based on the signal ⁇ .
  • Signals 100e, lOOj, 100k, 1001, 100m, 100 ⁇ , lOOp, 10 Oq, lOOr, 100s are the operating frequency signals output by PLL4b 100a, DC—DC comparator Power supply voltage output 100b output from the substrate 4c, nM OS substrate bias voltage 100c output from the substrate bias voltage generation circuit 4d, pMOS substrate output from the substrate bias voltage generation circuit 4e
  • the frequency and signal level depend on the value of the bias voltage 100d Change.
  • Code key data 10 6 after being encoded by the moving image code key means 5 operating on the processor 1 is transmitted as a signal 100m to the bit stream interface BI via the bus B1 and output as a signal 100 ⁇ .
  • the local decoded data 106 generated in the process of the code decoding process is transmitted as a signal 100j to the memory MR functioning as the local decoded frame memory 6.
  • image data and the like are read out from the memory as a signal 100q via the bus B1 and transmitted to the display interface DI.
  • the signal 100q received by the display interface DI is output as video data by the signal 100t.
  • Video data is output and displayed as moving images via a monitor connected to the display interface DI.
  • the operation control circuit 4a, the display interface DI, and the bit stream interface BI always operate at a constant operation power supply voltage, but signals 100e, 100q, and 100m transmitted and received between them are elements included in the control area CA.
  • the signal level fluctuates according to changes in the substrate bias voltage and operating power supply voltage (processor 1, memory MR, input frame memory 7a, 7b, etc.).
  • the operation control circuit 4a, the display interface DI, and the bit stream interface BI include a level converter that corrects the signal level of the received signals 100e, 100q, and 100m.
  • the moving image code processing system S1 is realized by causing a computer (in particular, a multimedia signal processing unit in the computer) to function as the following predetermined means by the moving image code processing program Prgl. Further, the system S1 realizes the moving picture coding method of the present invention having the following step force.
  • a predetermined frame that is, a frame to be encoded next when a certain frame is encoded.
  • the code is not processed yet, and the frame that is scheduled to be processed in the future (also called the current frame) is encoded before the predetermined frame.
  • Taichi no frame past The process of encoding a predetermined frame with the previous frame as a previous frame
  • the same process is performed for V and shifted frames.
  • the moving image encoding processing program Prgl causes the computer to function as follows in steps 1 to 5 described later.
  • Step 1 Input image information of a predetermined frame into the input frame memory 7.
  • Step 2 The required calculation amount Kp for a predetermined frame is calculated.
  • Step 3 It is made to function as the operation determining means 3 for determining the operating frequency F, the operating power supply voltage Vdd, and the substrate bias voltage Vb n, Vbp of the processor according to the calculated necessary calculation amount ⁇ .
  • Step 4 It is made to function as the operation control means 4 for controlling the operation of the processor 1 with the calculated operation frequency F, operation power supply voltage Vdd, and substrate bias voltage Vbn, Vbp.
  • Step 5 It is made to function as the moving image encoding means 5 for encoding the image information of a predetermined frame. As described above, the processing of step 1 and step 5 is performed for all frames in the order of frames input to the input frame memory 7 (that is, the order of encoding), thereby encoding the moving image. Details will be described below.
  • Step 1 The inputted input image data is stored in the input frame memory 7 which is a storage area for temporarily storing the frames in order to synchronize the frames.
  • Necessary computation calculation means 2 accesses the input frame memory 7 to acquire the input image data 101 of the predetermined frame, and is necessary for the encoding processing of the predetermined frame. Calculate the necessary amount of computation Kp.
  • There are various methods for calculating the required amount of computation ⁇ For example, it is desirable to calculate by using one or more elements that affect the amount of computation of the sign ⁇ processing of a predetermined frame. As an element, for example, in the case of moving image code processing, when the processing content is motion compensation, the amount of computation is large for a video with a lot of motion, while the amount of computation is small for a video with little motion.
  • the distortion value calculated by the sum of absolute differences as the amount of motion between the predetermined frame and the previous frame the value calculated by the sum of absolute differences of adjacent pixels as the amount of activity of each frame, and the macro
  • the number of effective blocks, the number of effective coefficients, the encoding bit rate, the number of generated bits, the calculation amount actually required for the sign of the previous frame, and the required calculation amount calculation means 2 The required amount of calculation of the previous frame calculated can be mentioned.
  • each element Assuming that only the value of one element changes and the value of the other element does not change for each, the required amount of computation is relatively smaller when the value of that one element is large than when it is small. The value of one element is small!
  • the required amount of computation is relatively small compared to the case.
  • the intra-frame code ⁇ is required when the required calculation amount Kp is relatively small compared to the case of an inter-frame code ⁇ .
  • the required amount of computation ⁇ is made relatively large. That is, since these multiple elements are elements that affect the required amount of calculation necessary for the sign processing of a predetermined frame, the required amount of calculation calculation means 2 determines the required amount of calculation ⁇ according to these elements. By calculating so as to increase or decrease (cycle), the required calculation amount ⁇ calculated by the required calculation amount calculation means 2 becomes a value closer to the calculation amount when the actual sign processing is performed.
  • calculation is performed using the function G, and the input image data 101 of a predetermined frame stored in the input frame memory 7 and the local decoding frame memory 6 are stored.
  • the motion magnitude of the input image is predicted (calculated).
  • the local decoded data 103 of the previous frame is encoded by the previous frame formed by encoding the previous frame in accordance with the code key processing of the previous frame in which the code is performed before the predetermined frame.
  • the data 106 is formed by decoding with a local decoder and stored in the local decoding frame memory 6. For example, the sum of absolute differences is used as an example of the prediction (calculation) of the magnitude of movement.
  • the local decoded data 106 decoded by the local decoder after encoding may be used, or the input image data of the input previous frame may be used as it is. good.
  • Input image data 101 of a predetermined frame stored in the input frame memory 7 is represented by X (i, j) (i is a horizontal coordinate of the image, j is a vertical coordinate), and a local decoded frame memory 6 to be described later.
  • the local decoding data 103 of the previous frame stored in is Y (i, j) (where i is the horizontal coordinate of the image and j is the vertical coordinate)
  • the amount of motion between the predetermined frame and the previous frame is the difference
  • Z ⁇ IX (i, j) -Y (i, j) I for all (or sampled) pixels. Let Z be the value of the sum of absolute differences.
  • the value of the adjacent pixel difference absolute value sum (that is, the activity amount of each frame) is set to w.
  • the sum of absolute differences is Z
  • the amount of activity of the predetermined frame is Wa
  • the amount of activity of the previous frame past frame
  • Wb the amount of activity of the previous frame
  • Wb the amount of activity of the previous frame
  • the average quantization step size of the previous frame (average value of quantization step size) Qprev
  • M for the number of macroblock matching in the previous frame
  • B for the number of effective blocks for the previous frame
  • C for the number of effective coefficients for the previous frame
  • S for the amount of processing actually required for the sign of the previous frame
  • BR is the frame code bit rate
  • a Qprev is the difference between the average quantization step size of the previous frame and the average quantization step size of the previous frame, and the actual number of bits generated in the previous frame.
  • Kp G (Z, Wa, Wb, Qprev, M, B, C, S, BR, ⁇ Qprev, D, Kp,)
  • G is Z, Wa, Wb, Qprev, M, B, C, S, BR, ⁇ Qprev, D,
  • a function derived from one or more elements of Kp ' is derived from one or more elements of Kp '.
  • type I is used as to whether the predetermined frame is a force inter-frame code ⁇ that is an intra-frame code ⁇ .
  • the function G will be described.
  • the image change between the previous frame and the predetermined frame is large (small), that is, when the sum of absolute differences Z is large (small)
  • the number of macroblock matching performed in the predetermined frame becomes large (small).
  • the amount of computation required for motion detection processing for a given frame (depending on the number of macroblock matching operations to be performed Becomes larger (smaller).
  • the activity amount Wa of the predetermined frame is large (small), it means that the predetermined frame includes a lot (small) of high frequency components of the image.
  • the function G is configured so that Kp is set large (small) when parameters such as Z and Wa are large (small).
  • the above function G is configured to set Kp large (small) when parameters such as M, B, C, S, Wb, Kp 'are large (small).
  • the value of the quantization step size is set small (large).
  • the number of effective blocks and the number of effective coefficients generated in the encoding process are large ( Small).
  • the quantization step size value for a given frame is set small (large), which is an effective value generated in the encoding process.
  • the number of blocks and the number of effective coefficients become smaller (larger).
  • the above-mentioned function G has a larger number of generated bits D in the previous frame compared to BR so that Kp is set to be large (small) when the code key bit rate BR of the predetermined frame is large (small). If (small ⁇ ), configure Kp to be small (large). Furthermore, the difference between the average quantization step size Qprev of the previous frame and the average quantization step size of the previous frame and the average quantization step size of the previous frame. By considering A Qprev, Kp calculated by the above function G can be made close to the amount of computation required to actually encode a predetermined frame.
  • the required amount of calculation 116 of the previous frame calculated by the calculation amount calculation means is stored in advance in the element memory 8, which is a storage area for storing elements, and is read into the calculation amount calculation means 2 when calculating the required calculation amount Kp. Used.
  • Average value of quantization step size of previous frame 107, number of macroblock matchings of previous frame 111, number of effective blocks of previous frame 112, number of effective coefficients of previous frame 113, average value of quantization step size of previous frame The difference between the average value of the quantization step size of the previous frame and the average value 114 of the previous frame and the amount of processing 115 actually required for the previous frame code It is fed back from the image code key means 5 to the necessary calculation amount calculation means 2.
  • the necessary calculation amount calculation means 2 only one of these elements may be used, or a plurality of elements may be used in combination.
  • Step 3 Operation Determination Step
  • the operation determination means 3 performs calculation for predicting the operation frequency Fe (cycle Z seconds) for the processing of a predetermined frame based on the value of the necessary calculation amount Kp.
  • the time Te allocated to the code processing for a given frame is the time Tp for predicting the amount of computation for a given frame from the time limit Tf for processing one frame and the operating frequency of the processor 'operating power supply voltage ⁇ board
  • the time to change the bias voltage is the time minus Ts.
  • the operating power supply voltage, substrate bias voltage, and operating frequency supported by peripheral devices including processor 1 and / or local decoding memory 6 are changed in r stages (r is an integer of 2 or more).
  • the operation determining means 3 performs an operation in which F (n)> Fe and Fe> F (n— 1).
  • Calculation is performed to select the frequency F (n) as the operating frequency for encoding the predetermined frame, and the operating power supply voltage Vdd (n) and the substrate bias voltage Vbn, Vbp (n) suitable for the operating frequency F (n) are calculated. Perform calculations to select and operate peripheral devices including processor 1 and / or local decoding memory 6 etc. at operating frequency F (n), operating power supply voltage Vdd (n), and substrate bias voltage Vbn, Vbp (n) The operation control means 4 is instructed to supply the operation power supply voltage 'substrate bias voltage' to the operation frequency (reference numeral 102). N is an integer between 1 and r
  • FIG. 5 is a diagram showing the relationship between the operating frequency “operating power supply voltage” and the substrate bias voltage.
  • the operation power supply voltage 'substrate is set so that the current consumed by the peripheral device including the processor 1 or the processor 1 and the local decoding memory 6 is equal to or less than a predetermined value for each operating frequency.
  • a combination of bias voltages is preset. For example, based on the relationship between the subthreshold leakage current 1st and the charge / discharge current led, the operating power supply voltage Vdd and the substrate bias voltage Vbn, Vbp that minimize the power consumption P for each operating frequency are obtained by experiments and calculations. A combination of power supply voltage Vdd and substrate bias voltage Vbn, Vbp is desirable.
  • the substrate bias voltage may be automatically calculated with respect to the operating power supply voltage corresponding to the operating frequency by hardware and / or a program built in the operation determining means 3. Further, the operating power supply voltage and the substrate bias voltage may be calculated with respect to the operating frequency by hardware and / or a program built in the operation determining means 3.
  • the operation control means 4 receives the operation power supply voltage V dd (n), the substrate bias voltages Vbn (n) and Vbp (n) and the operation frequency F (n) received from the operation determination means 3 Is supplied to peripheral devices including processor 1 and / or local decoding memory 6 (symbol 105), and its operating power supply voltage Vdd (n) and substrate bias voltage Vbn (n), Vbp (n)
  • the processor 1 and the peripheral device are controlled to operate at a constant frequency F (n).
  • the peripheral device including the processor 1 and / or the local decoding memory 6 and the like has a constant operating power supply voltage Vdd (n), a substrate bias voltage Vbn (n), Vbp (n), and an operating frequency F (n). Will work.
  • the operation power supply voltage control means 4c built in the operation control means 4 is constant.
  • the processor 1 is controlled to operate at the operating power supply voltage Vdd (n), and the processor 1 is controlled to operate at the constant substrate bias voltage Vbn (n) for the n-channel MOS transistor by the substrate bias voltage Vbn control means 4d. ⁇
  • the substrate bias voltage Vbp control means 4b controls the processor 1 to operate at a constant substrate bias voltage Vbp (n) for the p-channel MOS transistor, and the operating frequency control means (PLL) 4b controls the constant operating frequency F.
  • PLL operating frequency control means
  • the processor 1 is controlled to operate constantly.
  • Substrate bias voltage control applies p-channel MOS transistor by applying appropriate substrate bias voltage Vbn (n) to n-channel MOS transistor for substrate bias voltages Vbn (n) and Vbp (n). Is applied by applying an appropriate substrate bias voltage Vbp (n).
  • the potential difference between the substrate bias voltage Vbn (n) for the n-channel MOS transistor and the ground potential Vss is Vbbn (n)
  • the substrate bias voltage Vbp (n) and the operating power supply voltage Vdd for the p-channel MOS transistor is Vbbp (n). That is,
  • Vbn (n) Vbbn (n) + Vss
  • Vbp (n) Vbbp (n) + Vdd (n)
  • Vbbn (n) and Vbbp (n) and the operating power supply voltage Vdd (n) can be set independently.
  • Vbbn (n) is the voltage applied to the drain-source pn junction of the n-channel MOS transistor, and this voltage must not exceed the diffusion potential ⁇ ⁇
  • Vbb p (n) is the p-channel This is the voltage applied to the pn junction between the drain and source of the transistor, so that this voltage does not fall below the diffusion potential V ⁇ .
  • the diffusion potential V ⁇ is usually 0.6 V.
  • Step 5 moving picture coding step
  • the moving picture coding means 5 is a means realized on the processor 1 of the computer by the moving picture coding processing program Prgl. This is a means for accessing the input image data stored in the frame memory 7 in units of moving image encoding and performing the encoding process. That is, the moving image encoding means 5 acquires the input image data 101 of a predetermined frame from the input frame memory 7 and encodes it to generate the encoded data 106.
  • the peripheral devices including the processor 1 and / or the local decoding memory 6 and the like operate at a constant operating voltage supplied from the operation control means 4.
  • the operation control means 4 While operating peripheral devices including the processor 1 and / or the local decoding memory 6 at the operating frequency F (n), the operating power supply voltage Vdd (n), and the substrate bias voltage Vbn (n), Vbp (n) Therefore, the moving image code key means 5 that performs the code key using the processor 1 performs the key key code.
  • peripheral devices including the processor 1 and / or the local decoding memory 6 etc. are constantly operated at a high frequency for an image (input image data 101 of a predetermined frame).
  • the moving picture encoding unit 5 includes a local decoder having a function of decoding the code key data 106.
  • the code key data 106 of a predetermined frame is decoded by the local decoder and is then decoded in the local decoding frame memory 6 Is stored as locally decoded data 103.
  • the local decoded data 103 of the predetermined frame is used when calculating the necessary calculation amount Kp for the frame encoded next to the predetermined frame.
  • the encoded data 106 of a predetermined frame is transmitted through a transmission path or stored in a storage medium.
  • the operation power supply voltage and the substrate bias voltage are controlled according to the operation frequency F by the operation power supply voltage Vdd, the substrate bias voltage Vbp of the p-channel MOS transistor, and the n-channel MOS transistor substrate bias voltage Vbn. Only at least one voltage may be controlled.
  • the operation frequency 'operating power supply voltage' used for the processing of one frame, and the operation determining means 3 that only requires the substrate bias voltage to operate at a constant level is set to N sets of operating frequency 'operating power supply voltage' substrate bias voltage ( N may be a positive integer).
  • N may be a positive integer.
  • the operating time of each of N sets of operating frequencies “operating power supply voltage” and substrate bias voltage is calculated, and processor 1 is operated.
  • the code processing system S1 includes first failure avoiding means for avoiding the failure phenomenon, and the first failure avoiding step is executed by interrupting the moving image encoding step.
  • the first bankruptcy avoiding means is within the time Te when the code processing of a predetermined frame is allocated.
  • the process is switched to a process capable of reducing the amount of calculation of the normal code process so that the code process can be completed within the allocated time Te to avoid the failure phenomenon.
  • the timing Ti at which the process is switched is calculated by interrupting the sign process and temporarily interrupting the sign process.
  • the timing Ti may be updated according to the encoded processing.
  • Figure 7 shows the relationship between the time for interrupting and the remaining calculation.
  • Ks be the amount of computation required for processing.
  • the processor 1 reads from the processed macroblock number register 10 the number of macroblocks MBi (reference numeral 117) for which the sign process has been completed, and calculates the switching timing Ti using the following equation.
  • Example of First Failure Avoiding Means 1 The first failure avoiding means 91 of Example 1 is the same as the first failure avoiding means 91 in step 5, in which the moving picture code key means 5 is a code key processing routine for the input image data 101 of a predetermined frame.
  • the sign ⁇ is not finished at the timing Ti determined above.
  • a process of forcibly making only the color difference block an invalid block is performed on the black block. Only the color difference block of the macro block is forcibly converted to an invalid block, and the luminance block is processed in the same manner as normal, thereby preventing the image quality from being deteriorated. The image quality can be improved rather than forcibly making the block invalid and avoiding the failure.
  • Example 1 of first failure avoiding means 2 is the timing Ti determined as described above when executing the code processing routine of a predetermined frame.
  • step (3) intra-frame code processing is forcibly performed on macroblocks that have not been completed. Compared with the case where inter-frame coding is performed, since the motion vector detection process which occupies the most processing amount is not performed, the actual calculation amount can be reduced. In addition, since the sign key processing is performed up to the last macro block, the number of corresponding macro blocks increases compared to Example 1, and the image quality can be further improved.
  • Example 1 of first failure avoiding means 3 performs the timing Ti determined as described above when executing the code processing routine of a predetermined frame.
  • the inter-frame encoding process is performed forcibly with the size of the motion vector set to 0 for macroblocks that have not been completed.
  • Example 3 since the interframe coding process is performed without performing the motion vector detection process, the amount of calculation reduction is small and the number of corresponding macroblocks increases. The effect on the coding process of the next frame is Compared to Example 2, the smaller image quality can be further improved.
  • Example 4 of first failure avoiding means The first failure avoiding means 94 of Example 4 compulsorily applies to a macroblock whose code is not finished at the timing Ti determined above.
  • the quantization step size is increased and the sign key processing is performed.
  • the occurrence of the number of effective blocks and the number of effective coefficients for the macroblock is reduced, and among the encoding processes, the processes performed on the effective blocks and the variable length encoding performed on the effective coefficients. The number of processes can be reduced.
  • Each example of the first failure avoidance means described above omits the contents of the normal encoding process, forces the motion vector to 0, and suppresses the number of effective blocks and generation of effective coefficients. The amount is reduced. For this reason, the failure avoidance process by the first failure avoiding means in a predetermined frame is performed. If the interframe coding processing is performed on the macroblock of the next frame corresponding to the macroblock that has been subjected to the failure avoidance processing on the predetermined frame in the processing of the next frame, the image quality is affected. there is a possibility. For this reason, there is a possibility that the image quality of the next frame is deteriorated even though the image quality deterioration is prevented by the failure avoidance process at the predetermined frame.
  • the quantization is performed on the macroblock.
  • Relaxation processing is performed by the relaxation means 95 that performs the sign key processing by reducing the step size. This process increases the amount of generated code assigned to the corresponding macroblock, and increases the information obtained from the macroblock power to improve image quality.
  • the present invention can further reduce the power consumption due to the subthreshold leakage current as compared with the prior art in which one frame is encoded while changing the operating frequency of the processor a plurality of times.
  • the substrate bias voltage and operating frequency of processor 1 are variable in r steps as shown in Fig. 5, the required amount of computation for any one frame is Kt, and the time allotted for processing of that frame is Tt. .
  • the substrate bias voltages are Vbp and Vbn, and the thresholds are suitable for the substrate bias voltages Vbp and Vbn.
  • the substrate bias voltage is Vbp2 and Vbn2
  • the threshold voltage suitable for the substrate bias voltage Vbp2 and Vbn2 is Vt2
  • the required amount of computation at time T1 + T2 Case 2 is when the Kt processing ends.
  • Case2 consider the case of spoon codes I said arbitrary one frame. However, the threshold voltage is Vtl>Vt> Vt2, and it depends on the subthreshold leakage current. Power consumption is
  • Vdd Operating power supply voltage
  • Vgs Gate-source voltage
  • Vt Shiki! / Variable voltage
  • S Subthreshold swing
  • Pstl 10 " 2 : (10 _3 / 3 + 10" 1 X 2/3)
  • the present invention can further reduce the power consumption as compared with the prior art in which one frame is encoded while changing the operating power supply voltage and operating frequency of the processor a plurality of times. For example, when performing a certain amount of computation Kt at a certain time Tt Control at the same frequency during that particular time and the frequency Ft
  • the operating power supply voltage and operating frequency of processor 1 are variable in r steps as shown in Fig. 5, the required amount of computation for any one frame is Kt, and the time allotted for processing of that frame is Tt. .
  • the operating frequency is set to Ft
  • the operating power supply voltage when operating processor 1 at the operating frequency Ft is Vdd
  • the processing of the required amount of computation Kt is completed at time Tt. (I.e., when the operating frequency is constant) is set to Casel
  • the initial operating frequency is set to h * Ft, and the processor is operated at the operating frequency h * Ft.
  • the operating power supply voltage is VI
  • the processor operating frequency is changed to h * FtZ2 when time T1 has elapsed
  • the operating power supply voltage when operating processor 1 at the operating frequency h * FtZ2 is V2
  • the time Tl + Case 2 where the processing of the required amount of computation Kt ends at T2 (that is, when the operating frequency is switched once) is Case 2
  • power consumption is
  • an operating frequency that is greater than or equal to the predicted operating frequency and an operating frequency that is lower than the predicted operating frequency are selected, an interrupt is performed at a predetermined timing, and two steps are performed.
  • the operating frequency may be controlled.
  • the remaining amount of the required calculation amount of the predetermined frame calculated by the required calculation amount calculation means is actually used for the sign or decoding process of the predetermined frame by the code or processing unit. If it is smaller than the amount of computation required for the operation, the switching timing may be advanced so that the time for operating at a high V and operating frequency can be lengthened.
  • the moving image coding processing system S1 performs code coding processing after a predetermined frame when the calculation amount of code coding processing for a predetermined frame is reduced by the first failure avoidance means. It is also possible to increase the amount of calculation required for subsequent frames to be processed. This process of increasing the required amount of calculation for the subsequent frame is performed by the required calculation amount calculation means 2 for the subsequent frame in which the encoding process is performed after the predetermined frame. This is a process for increasing by a predetermined value, and is executed in the necessary calculation amount calculation step (Step 2) of the subsequent frame.
  • the final calculation is Necessary calculation amount Kp is
  • the required amount of calculation is calculated within the required amount of calculation calculation step for the subsequent frame. Multiply the coefficient of any element used in the function G (Z) by m times (m is a real number greater than 1). For example, if the macroblock matching count M has become smaller due to the execution of failure avoidance processing, using the above example, the required amount of computation Kp calculated in the next frame is
  • Kp j + (a + ⁇ ) XM + j8B + yC + ⁇ + ⁇ AQprev
  • Equation 1 (Equation 2), where all elements used in the function can be multiplied by m (m is a real number greater than or equal to 1), or real numbers n (n is a real number greater than or equal to 0) can be added to all elements. ), (Equation 3) may be increased by a fixed value regardless of the value of the required amount of computation.
  • FIG. 8 is a schematic block diagram of the video code key processing system S2 of the present embodiment
  • FIG. 9 is a schematic flowchart for explaining the video code key method realized by the system S2.
  • the moving image code processing system S2 of the present embodiment includes second failure avoiding means 96.
  • the second failure avoiding means 96 is actually required for the required calculation amount Kp calculated by the required calculation amount calculating means for the predetermined frame and the code amount processing for the predetermined frame after the sign key processing for the predetermined frame is completed. Compared with the calculated amount of computation, if “required amount of computation Kp ⁇ actual amount of computation required”, the subsequent frame in which encoding processing is performed after a predetermined frame is performed.
  • the video image code processing system S2 may include both the first failure avoiding means' step and the second failure avoiding means' step, or only the second failure avoiding means' step. Also good.
  • the second failure avoiding means 96 is a means that functions as a part of the required amount-of-computation calculation means 2, and is executed in the required amount-of-computation calculation step (Step 2). Specifically, the second failure avoiding means 96 compares the required calculation amount Kp calculated by the required calculation amount calculation means in the predetermined frame with the calculation amount actually required for the sign key processing of the predetermined frame. When “Required amount of computation Kp ⁇ actual amount of computation required”, the required computation amount of the succeeding frame is calculated according to the processing of the necessary computation amount calculating means for the subsequent frame in which the sign process is performed after the predetermined frame. Increase the amount Kp.
  • the required calculation amount Kp finally calculated is
  • the required amount of computation is calculated within the required computation amount calculation step for the subsequent frame. Multiply the coefficient of any element used in the function G (Z) by m times (m is a real number greater than 1). For example, macro block matching due to execution of bankruptcy avoidance processing If the number of times M has become small, using the above example, the required amount of computation Kp calculated in the next frame is
  • Kp j + (a + ⁇ ) XM + j8B + yC + ⁇ + ⁇ AQprev
  • Equation 1 (Equation 2), where all elements used in the function can be multiplied by m (m is a real number greater than or equal to 1), or real numbers n (n is a real number greater than or equal to 0) can be added to all elements. ), (Equation 3) may be increased by a fixed value regardless of the value of the required amount of computation.
  • the moving image code key processing system S3 replaces the first failure avoiding means with the sign code when the time allotted to the code key processing for a predetermined frame has elapsed.
  • a third failure avoiding means is provided for performing processing to avoid the failure phenomenon when the processing is not completed. The other points are the same as in the second embodiment.
  • FIG. 10 is a schematic block diagram of the video code processing system S3 including the third failure avoiding means 97
  • FIG. 11 is a schematic diagram for explaining the video encoding processing method realized by the system S3. It is a flowchart.
  • the third failure avoiding means 97 executes the encoding process when the allocated time Te elapses when the moving image code receiving means 5 executes the code processing routine of the input image data 101 of the predetermined frame in step 5. It is determined whether the ⁇ process has been completed or not, and if the sign process has not been completed, the time allocated to the next frame is used until the code process is completed. By extending the processing time of a predetermined frame, the encoding process cannot be completed! / And the failure phenomenon can be avoided.
  • the third failure avoiding means 97 will be specifically described below.
  • the input image data is stored in the input frame memory for each frame at the frame rate interval, and the input frame memory data force S is updated frame by frame at the input image frame rate interval.
  • the interval at which the data in the input frame memory is updated varies depending on the frame rate, and the time for processing the code for one frame is limited by the frame rate of the input image.
  • the input frame memory 7 is a memory that can store data for multiple frames (here, 2 frames), and the start address of the data is specified by the memory address, and the data is not specified. Address power shall be stored sequentially.
  • Fig. 12 is a diagram showing the transition of processing time in the case of normal encoding processing.
  • Fig. 13, Fig. 14 and Fig. 15 are diagrams of the code processing where the third failure avoiding means is executed by the third failure avoiding means. It is a figure which shows the processing time transition in a case. For example, in real-time processing, if 30 image data are input per second, the data in the external memory is updated every 1Z30 seconds. When encoding 10 images per second with the moving image encoding processing system, the input image is encoded at a rate of 1 out of 3 images.
  • the third failure avoiding means is for a case where the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount Km in the predetermined frame, that is, the time allocated to the sign key processing. In spite of the elapse of time, the encoding process is not completed.In contrast, the processing time Te of the predetermined frame is extended by using the encoding processing time assigned to the next frame! This is a means for controlling the status signal of the input frame memory in order to continue the encoding process and complete the process. Examples 3 and 2 below are examples of third means of avoiding bankruptcy.
  • the third failure avoiding means 97 has a function of executing Step 1 to Step 4 described later.
  • (Procedure 1) In order to extend the time Te allocated to the encoding process in a predetermined frame, the state of the input frame memory is controlled and the encoding process of the predetermined frame is continued.
  • (Procedure 2) When the encoding process for a given frame is completed, The time for which the frame sign can be processed is calculated.
  • (Procedure 3) Control switching of input frame memory status. Steps 1 to 3 are described in detail below.
  • FIG. 13 shows the transition of the sign key processing time by the failure avoiding means 97.
  • # 4 frame data is stored in the input frame memory 7b
  • # 7 frame data is stored in the input frame memory 7b.
  • the state of the input frame memory is controlled, the input frame memory 7b is controlled to be in a writable state, the input frame memory 7a is maintained in a writable state, and the frame data write destination of # 8 is set to the input frame. Change to memory 7a. Since the input frame memory 7b can hold the frame data of # 4 and the encoding process of # 4 can be continued, the time can be extended until the encoding process of # 4 is completed.
  • Step 2 Calculation of the next frame processing time
  • the required amount of calculation is calculated by the required calculation amount calculation means for the next frame to be encoded, and the operation frequency / operation power supply voltage and substrate bias voltage are calculated by the operation determination means.
  • the time T1 may be calculated when executing the necessary calculation amount calculation means for the next frame.
  • the sign key processing time T1 of the next frame it is possible to determine the optimum operating frequency for the sign key of the next frame, that is, the operating power supply voltage and the substrate bias voltage. The power consumption can be reduced efficiently. Note that the calculation of the time T1 may be performed when the necessary calculation amount calculating means for the encoding step of the next encoding target frame is executed.
  • Example of third failure avoiding means 2 The third failure avoiding means 98 is different from the third failure avoiding means 97 only in the state control method of the input frame memory. This is the same as the third bankruptcy avoidance measure 97.
  • FIG. 15 shows the time transition of the sign key processing by the third bankruptcy avoiding means 98.
  • the third failure avoiding means 98 when controlling the state of the input frame memory, if the processing cannot be completed in the encoding processing time Te assigned to # 4, at least the extension time To. During this time, the input frame memories 7a and 7b are controlled so that they cannot be written, and the time Te is extended by the time To.
  • the input frame memory holds the # 4 frame data stored in the input frame memory 7b and the # 7 frame data stored in the input frame memory 7a.
  • the failure phenomenon is avoided by extending the time Te.
  • the # 8 frame input during the extension time To is not written to the input frame memory.
  • the input frame memory 7b controlled to the writing state can write the input frame data.
  • the frame data # 7 is read out from the input frame memory 7a, and the frames # 9 and # 10 are sequentially written into the input frame memory 7b.
  • the encoding process of the # 7 and # 10 frame data is sequentially performed. Since the same frame number as that of the normal encoding process is encoded, an image quality equivalent to that of the normal encoding process can be obtained.
  • the encoded frames are # L # 2, # 3, # 4, # 5, # 6 , # 7, • ⁇ ', and the range in which the sign key processing time can be extended is the maximum Tf. # 4 encoding process If the extension time exceeds Tf, you may sign # 6 as the next frame.
  • the moving image decoding processing system S4 is a system for decoding an encoded moving image.
  • FIG. 16 is a schematic block diagram showing the operation of the video decoding processing system S4.
  • the operation power supply voltage, the substrate bias voltage, and the operation frequency are prepared in r stages (r is an integer of 2 or more), and the operation power supply voltage and the substrate bias are programmed.
  • the processor 1 capable of changing the voltage and the operating frequency, the operation control means 4 for controlling the operating power supply voltage, the substrate bias voltage and the operating frequency of the processor 1, and the local decoding frame memory 46 for storing the decoding data of the previous frame 46 With.
  • the operation power supply voltage, the substrate bias voltage, and the operation frequency may be controlled by the operation control means 4 in the same manner as the processor 1.
  • the processor 1 includes necessary calculation amount calculating means 42, operation determining means 3, moving picture decoding means 35, and third failure avoiding means 97 as means that operate on the processor 1.
  • Reference numeral 401 is input encoded data
  • reference numeral 102 is operating power supply voltage'substrate bias voltage / operating frequency instruction
  • reference numeral 105 is operating power supply voltage / substrate bias voltage / operating frequency supply
  • reference numeral 406 is decoding key data
  • the same reference numerals as those in the first embodiment are parts having the same function or equivalent functions.
  • the point of performing decoding key instead of code key is the same as that of the third embodiment except for the following points.
  • one of the frames that is sequentially decoded is arbitrarily decoded (that is, the frame that is decoded next on the basis of the time when a certain frame is decoded, in other words, At that time, a frame that has not been decoded yet and is scheduled to be decoded in the future) is a predetermined frame, and one frame that has been decoded before the predetermined frame (decoded in the past).
  • the moving picture decoding processing program Prg4 that causes the computer to function as the moving picture decoding processing system S4 is substantially the same as the moving picture code processing program Prgl.
  • Video decoding means 45 for decoding The computer (specifically, the processor 1 built in the computer) is made to function.
  • the input encoded data 401 input to the moving image decoding processing system S4 is input to the necessary calculation amount calculation means 42.
  • the required calculation amount calculation means 42 calculates the amount of information (number of bits) FB generated for one frame of the encoded data 401 (that is, the encoded data 401 of the predetermined frame), and predicts the required calculation amount Kp. (Required calculation step).
  • the required amount of computation ⁇ is
  • Kp G (FB, MVa, MVv, B, C, BR, Q, A Q, I, E, P)
  • FB is the number of bits of the code data of the predetermined frame or the previous frame
  • MVa is the average value of the motion vector of the predetermined frame or the previous frame
  • MVv is the size of the motion vector of the predetermined frame or the previous frame
  • B is the number of effective blocks of the specified frame or previous frame
  • C is the number of effective coefficients of the specified frame or previous frame
  • BR is the bit rate of the specified frame or previous frame
  • Q is the quantization of the specified frame or previous frame
  • the average step size ⁇ Q is the difference between the average quantization step sizes of the predetermined frame and the previous frame, or the average difference between the quantization step sizes of the previous frame and the previous frame
  • I is the predetermined frame power picture.
  • E is the performance required for decoding the previous frame.
  • the amount, P is represents a necessary calculation amount of the previous frame calculated by the required calculation amount calculating means.
  • the function G will be described below.
  • the amount of computation required for decoding a predetermined frame depends on the number of times ID CT processing, IQ processing, and VLD processing are executed in decoding the predetermined frame.
  • the number of executions of IDCT processing depends on the number of effective blocks included in the predetermined frame
  • the number of executions of IQ processing and VLD processing depends on the number of effective coefficients included in the predetermined frame. That is, when the number of effective blocks and the number of effective coefficients included in a predetermined frame is large (small), the amount of calculation required for the decoding process is large (small). Therefore, the above function G is configured so that Kp is set large (small) when B and C are large (small V).
  • the above function G is configured to set Kp small when the predetermined frame is an I picture.
  • the above function G is configured so that Kp is set large (small) when FB and BR are large (small). Also, since the quantization step size is changed when the bit rate is controlled, the above function G can be calculated by taking into consideration the average value Q of the quantization step size and the difference ⁇ Q of the average value of the quantization step size.
  • the Kp calculated by can be a value close to the amount of computation required to actually decode the predetermined frame.
  • the amount of computation required for decoding the predetermined frame is close to the amount of computation E actually required for the decoding processing of the previous frame. It becomes. Furthermore, if the predicted amount of computation calculated by the necessary amount-of-computation calculation means is close to the amount of computation required for the actual decoding process, P ⁇ E. Therefore, considering E and P, the Kp calculated by the above function G is necessary to actually decode the predetermined frame. It can be a value close to the amount of computation.
  • FB is the amount of generated information (number of bits) for one frame.
  • the function G is a function derived by using one or more elements FB, MVa, MVv, B, C, BR, D, Q, ⁇ Qprev, I, E, and P.
  • the required amount of computation Kp is the computation performance (frequency, cycle) that is predicted to be necessary for a given frame. The value is high when the number of bits FB in a given frame is large, and is low when the number of bits FB is small.
  • the required calculation amount Kp when the predetermined frame is the intraframe code ⁇ is small, and the required calculation amount Kp when the predetermined frame is the interframe code ⁇ is a large value.
  • the required calculation amount Kp is the average value of the motion vector size (for the frame to be decoded or the previous frame) MVA, the variance of the size of the motion vector (for the frame to be decoded or the previous frame) MVv, number of effective blocks (for frame to be decoded or previous frame) B, number of effective coefficients (for frame to be decoded or previous frame) C, bit rate ( The frame to be decoded or the previous frame) BR, the amount of generated information (the frame to be decoded or the previous frame) FB, the average quantization step size (this power of the frame to be decoded) Q, the difference between the average quantization step sizes (this is the difference between the Q of the frame to be decoded and the Q of the previous frame) , Or the difference between the Q of the previous frame and the Q of the second previous frame) AQ, I picture power ⁇ Picture power ⁇ Picture type I, actually decoding the previous frame The amount of computation required ⁇ and the predicted value of the amount of computation required for decoding the previous frame
  • the average motion vector size (from the frame to be decoded or from the previous frame) MVa the variance of the motion vector size (from the frame to be decoded or from the previous frame) MVv ,
  • the number of effective blocks (from the frame to be decoded or from the previous frame) B the number of effective coefficients (from the frame to be decoded or from the previous frame) C
  • the value of the element is large.
  • p is relatively large, and the element value is small !, large in some cases, and the required amount of computation Kp is relatively small compared to the case.
  • the necessary calculation amount calculation means 42 only one of these elements may be used or a plurality of elements may be used in combination. In other words, these multiple elements are elements that affect the required amount of calculation necessary for the decoding process of a predetermined frame, and therefore the required calculation amount calculating means 42 needs to calculate the necessary amount of calculation according to these elements. By calculating so as to increase or decrease Kp (cycle), the required calculation amount Kp calculated by the required calculation amount calculating means 42 becomes closer to the calculation amount when the decoding process is actually performed.
  • the operation determining means 3 (operation determining step) and the operation control means 4 are the same as in the first embodiment.
  • the moving picture decoding means 45 decodes the input code key data 401 of a predetermined frame to generate decoded data 406 (moving picture decoding step).
  • the operation control means 4 performs the decoding process while the processor 1 is operated by the operation control means 4 at a constant operating power supply voltage, substrate bias voltage and operating frequency. .
  • the required amount of computation required before decoding the frame is calculated, and the processor is operated while operating at a constant operating frequency, operating power supply voltage, and substrate bias voltage according to the required amount of computation. Since frame decoding is performed, low power consumption can be achieved.
  • the decrypted data 406 is displayed as a moving image on an image display unit of a mobile phone or a personal computer, or stored in a storage medium such as a hard disk.
  • a storage medium such as a hard disk.
  • the moving image decoding process system S4 also includes the third failure avoiding means 97, 98.
  • Each failure avoiding means is almost the same as in the third embodiment, but differs in that it judges not only the amount of computation of the encoding process but the amount of computation of the decoding process.
  • the third failure prevention means 97, 98 can avoid the failure phenomenon.
  • the moving image code processing system of the present invention includes first failure avoiding means 91, 92, 93, 94, first failure avoiding means with mitigation means 95, second failure avoiding means 96, Third bankruptcy avoider
  • the steps 97 and 98 may be provided independently, or each means may be provided in appropriate combination.
  • the decryption processing system may be provided with each second failure avoiding means 96 and third failure avoiding means 97 and 98, or may be provided with an appropriate combination of each means. For example, in the case of a sign key processing system, any one of the first failure avoidance means 91, 92, 93, 94, the first failure avoidance means with the mitigation means 95, and the second failure avoidance means 96 is used.
  • the first failure avoiding means 91 performs invalid block processing only on the color difference block or
  • the intra-frame code processing is performed by the first failure avoiding means 92, or the inter-frame coding process is performed by setting the motion vector size to 0 by the first failure avoiding means 93, or the first failure
  • the avoidance means 94 may increase the quantization step size and perform the code processing.
  • the moving image encoding or decoding processing program may realize low power consumption by controlling the operating frequency, the substrate bias voltage, and the operating voltage in two stages, and has the same function as the program. It may be realized by hardware.
  • the first to fourth embodiments control the operating power supply voltage, the substrate bias voltage, and the operating frequency.
  • the present embodiment controls the operating power supply voltage and the operating frequency. Thus, low power consumption is achieved.
  • this embodiment is not limited to a system integrated in a semiconductor.
  • FIG. 17 is a schematic block diagram showing the operation of the video encoding system S5 of the present embodiment
  • FIG. 18 is a conceptual diagram showing the relationship between the operating power supply voltage and the operating frequency of the processor 51.
  • the moving image coding processing system S5 of this embodiment is variable to the operating power supply voltage Vdd and the operating frequency force Sr stage (r is an integer on 2) instead of the processor 1 of the first embodiment.
  • the operation control means 54 controls the operation power supply voltage and the operation frequency of the processor 1.
  • the operation power supply voltage and the operation frequency of the processor 51, or the processor 1 and peripheral devices are controlled by the operation control means 52.
  • the operation determining unit 53 calculates F (n)> Fe and selects an operation frequency F (n) satisfying Fe> F (n-1) as an operation frequency for performing encoding processing of a predetermined frame.
  • the operation control means 54 sends the values of the operation frequency F (n) and the operation power supply voltage Vdd (n) received from the operation determination means 53 to peripheral devices including the processor 1 and / or the local decoding memory 6 and the like. (Symbol 505), and the processor 1 is controlled at a constant operating frequency F (n) and operating power supply voltage Vdd (n).
  • peripheral devices including the processor 1 and / or the local decoding memory 6 operate at the operating power supply voltage Vdd (n) and the operating frequency F (n).
  • the other points are almost the same as in the first embodiment.
  • the first failure avoiding means 91, 92, 93, 94, 95, the second failure avoiding means 96, or the third failure avoidance Preferably means 97, 98 are provided.
  • the first to fourth embodiments control the operating power supply voltage, the substrate bias voltage, and the operating frequency.
  • the substrate bias voltage and the operating frequency are controlled. By doing so, the power consumption can be reduced.
  • FIG. 19 is a schematic block diagram showing the operation of the moving image encoding system S6 of the present embodiment
  • FIG. 20 is a conceptual diagram showing the relationship between the substrate bias voltage and the operating frequency of the processor 61.
  • the substrate bias voltage Vbn, Vbp and operating frequency power S r step (r is an integer of 2 or more)
  • the plugging device 61 is variable (that is, can operate at r-stage substrate bias voltages Vbn, Vbp and operating frequency) and can change the substrate bias voltage and operating frequency by a program.
  • the operation control means 64 controls the substrate bias voltage and the operating frequency of the processor 1.
  • the processor 61 or the processor 61 and peripheral devices (such as the local decoding memory 6 and the input frame memory 7) have their substrate bias voltage and operating frequency controlled by the operation control means 64.
  • the operation determination unit 63 calculates F (n)> Fe and selects an operation frequency F (n) satisfying Fe> F (n-1) as an operation frequency for performing encoding processing of a predetermined frame. Calculating the substrate bias voltage Vbn (n), Vbp (n) suitable for the operating frequency F (n)! ⁇ , including the processor 1 and / or the local decoding memory 6 etc.
  • the operation control means 64 is instructed to operate the peripheral device at the operating frequency F (n) and the substrate bias voltages Vbn (n) and Vbp (n) (reference numeral 602).
  • the operation control means 64 receives the values of the operating frequency F (n) and the substrate bias voltages Vbn (n) and Vbp (n) received from the operation determining means 63 from the processor 61 and / or the local decoding memory 6 and the like. Supplied to the included peripheral device (reference numeral 605), and controls the processor 61 to operate constantly at its operating frequency F (n) and substrate bias voltages Vbn (n) and Vbp (n). As a result, peripheral devices including the processor 61 and / or the local decoding memory 6 and the like operate at a constant substrate bias voltage Vbn (n), Vbp (n) and an operating frequency F (n). The other points are almost the same as in the first embodiment.
  • the first failure avoiding means 91, 92, 93, 94, 95 or the third failure avoiding means 97, 98, or the second failure avoiding means are provided.
  • the operation power supply voltage suitable for the operation frequency instead of controlling the operation power supply voltage and the substrate bias voltage suitable for the operation frequency, the operation power supply voltage suitable for the operation frequency, Alternatively, the substrate bias voltage can be controlled.
  • the present example is an example of a system in which the moving image coding system 2 of the second embodiment includes the first failure avoiding means 91 and the second failure avoiding means.
  • the 35th frame will be described as an example of the encoded frame.
  • Each frame is composed of a 144 ⁇ 176 pixel array.
  • MPEG-4 is used as the encoding process.
  • FIG. 21 shows an example of the relationship between the operating frequency, the operating power supply voltage, and the substrate bias voltage in the processor 1 of the video encoding system S2.
  • the processor 1 of the video code system S 2 has an operating frequency of 50 MHz to 250 MHz, an operating power supply voltage of 0.5 V to 1. OV,
  • the first voltage is 1.0V to 0.5V, and is variable in 5 steps.
  • the moving image coding system S2 accesses the input frame memory 7, obtains the 35th frame, and calculates the required calculation amount Kp of the frame by the required calculation amount calculation means 2. Specifically, the required amount of computation Kp first calculates the sum of absolute differences Z using the following formula using the 34th frame as the previous frame.
  • the actual number of bits generated in the previous frame, D 56797, is obtained.
  • the required amount of computation Kp is calculated using the following formula using each element.
  • the switching timing time is calculated from the following formula.
  • Ks is the number of cycles required to process only the color difference block as an invalid block in one macroblock.
  • Kw X (MB-MBi) and F X (Te-Ti) are determined, and it is determined that there is a sufficient amount of calculation, and the switching timing is updated.
  • the 35th frame step ends, and the next 36th frame step is entered.
  • the second failure avoiding means 96 is used to predict the required amount of computation. Increase the process.
  • the number M of macroblock matching, the number B of effective blocks, and the number C of effective coefficients are multiplied by 1.1, and the necessary calculation amount Kp is calculated by the following formula.
  • Kp j + XI. L) M + (j8 XI. 1) ⁇ + ( ⁇ XI. L) C + ⁇ + ⁇ AQprev
  • the quantization step size is determined in the macroblock of the predetermined frame corresponding to the macroblock for which the failure avoidance processing has been executed in the previous frame by the first failure avoiding means 95.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

[PROBLEMS] To provide a video image encoding or decoding system and video image encoding or decoding method enabling avoidance of failure phenomenon while low power consumption is achieved. [MEANS FOR SOLVING PROBLEMS] This invention comprises required computational complexity calculating means (2) for calculating computational complexity required to encode or decode a predetermined frame, operation determining means (3) for calculating the operating power supply voltage, substrate bias voltage and operating frequency with which the encoding or decoding is possible within a time Te previously allocated for encoding or decoding the predetermined frame. During the time Te, a processor (1) encodes or decodes the predetermined frame with the calculated constant operating frequency, operating power supply voltage, and substrate bias voltage. The invention further comprises failure avoiding means for avoiding a failure phenomenon that occurs when the required computational complexity calculated by the required computational complexity calculating means is smaller than the computational complexity actually required.

Description

動画像符号化処理システム、動画像符号化又は復号化処理システム、動 画像符号化処理方法、及び、動画像符号化又は復号化処理方法  Moving picture coding processing system, moving picture coding or decoding processing system, moving picture coding processing method, and moving picture coding or decoding processing method
技術分野  Technical field
[oooi] 本発明は、動作周波数、及び、動作電源電圧及び Z又は基板バイアス電圧が変更 可能なプロセッサを使用して、連続する複数のフレームから構成される動画像をフレ ーム単位で順次符号化や復号化を行う動画像符号化処理システム、動画像符号ィ匕 又は復号化処理システム、動画像符号化処理方法、及び、動画像符号化又は復号 化処理方法に関する。  [oooi] The present invention uses a processor that can change the operating frequency, the operating power supply voltage and the Z or substrate bias voltage, and sequentially encodes a moving image composed of a plurality of frames in units of frames. The present invention relates to a moving image encoding processing system, a moving image encoding processing or decoding processing system, a moving image encoding processing method, and a moving image encoding or decoding processing method.
背景技術  Background art
[0002] 近年、伝送路を通じて動画像の送受信を行うことや、動画像を蓄積メディアに蓄積 することが可能となっている。一般に、動画像は情報量が大きいため、伝送ビットレー トの限られた伝送路を用いて動画像を伝送する場合、あるいは蓄積容量の限られた 蓄積メディアに動画像を蓄積する場合には、動画像を符号化'復号化する技術が必 要不可欠である。動画像の符号化'復号ィ匕方式として、 ISO/IECが標準化を進めて いる MPEG(Moving Picture Experts Group)や H.26Xがある。これらは動画像を構成 する経時的に連続した複数のフレームの符号化又は復号化を行うものであり、動画 像の時間的相関、空間的相関を利用した冗長性の削減を行うことにより動画像の情 報量を減らして符号化し、符号化された動画像を再度元の動画像に復号化する技術 である。  [0002] In recent years, it has become possible to transmit and receive moving images through a transmission path and to store moving images on a storage medium. In general, since moving images have a large amount of information, when transmitting moving images using a transmission path with a limited transmission bit rate, or when storing moving images on a storage medium with limited storage capacity, Techniques for encoding and decoding images are indispensable. There are MPEG (Moving Picture Experts Group) and H.26X, which are being standardized by ISO / IEC. These are used to encode or decode a plurality of consecutive frames that make up a moving image, and reduce the redundancy by using temporal correlation and spatial correlation of moving images. This is a technology that reduces the amount of information to be encoded, and decodes the encoded moving image back to the original moving image.
[0003] 力かる符号化'復号ィ匕技術はパーソナルコンピュータやマイクロコンピュータを内蔵 する携帯電話等の情報端末機器等に適用されており、符号化'復号化の手段を記述 したプログラムに基づいてコンピュータのプロセッサ等を動作させることにより、動画 像を送信等する場合は動画像符号化処理システムとして、動画像を受信等する場合 は動画像復号ィ匕処理システムとして機能させている。しカゝしながら、カゝかる動画像符 号ィ匕又は復号ィ匕処理は比較的に演算量が多いため消費電力が大きくなる傾向にあ り、ハードウェアよりも汎用性の高いソフトウェアを使用して、符号化'復号化処理に おける低消費電力化を図ることが大きな課題となっている。 [0003] Powerful encoding / decoding technology has been applied to information terminal devices such as mobile phones with built-in personal computers and microcomputers, and computers based on programs that describe encoding / decoding means. By operating this processor, etc., it functions as a moving image encoding processing system when transmitting moving images, and as a moving image decoding processing system when receiving moving images. However, the moving image code decoding or decoding decoding processing tends to increase power consumption due to the relatively large amount of computation, and software that is more versatile than hardware is used. And then encode to the decoding process Reducing power consumption is a major issue.
[0004] 以下に、ソフトウェアを使用した動画像符号ィ匕又は復号ィ匕システムにおける従来の 低消費電力化の手段を説明する。従来の低消費電力化の手段としては、例えば下 記の非特許文献 1に開示されて 、る。  [0004] Hereinafter, conventional means for reducing power consumption in a moving image encoding / decoding system using software will be described. Conventional means for reducing power consumption is disclosed in Non-Patent Document 1 below, for example.
[0005] 非特許文献 1： IEEE International Symposium on Circuits and System 2001(May,2001 )の予稿集 pp918- 921 " An LSI for Vdd- Hopping and MPEG4 System Based on the Chip，(H. Kawaguchi, G. Zhang, S. Lee, and T. Sakurai)  [0005] Non-Patent Document 1: Proceedings of IEEE International Symposium on Circuits and System 2001 (May, 2001) pp918-921 "An LSI for Vdd- Hopping and MPEG4 System Based on the Chip, (H. Kawaguchi, G. Zhang , S. Lee, and T. Sakurai)
[0006] 図 22は、非特許文献 1で示された、動画像 (動画像)符号化処理システムにつ!/ヽて 従来の低消費電力化を行う手法を示した図である。なお、低消費電力化の手段は、 動画像復号化処理システムにお ヽても同様である。  FIG. 22 is a diagram showing a conventional technique for reducing power consumption for the moving image (moving image) encoding processing system shown in Non-Patent Document 1. Note that the same means for reducing power consumption is applied to the moving picture decoding processing system.
[0007] 非特許文献 1では、動的に動作電源電圧及び動作周波数を変更可能なプロセッサ 上で、動画像符号化 (特に MPEG)を処理する場合の低消費電力化を行うための動 作電源電圧及び動作周波数の制御方法を示して!/ヽる。すなわち非特許文献 1の発 明は、図 23に示すように、動画像符号化を行う場合に、動画像内の動きの激しさなど によりフレーム単位に動画像符号ィ匕又は復号ィ匕の演算量が異なることに注目し、プ 口セッサの動作周波数及び動作電源電圧を制御して低消費電力化を図るものである  [0007] In Non-Patent Document 1, an operation power supply for reducing power consumption when processing moving image coding (particularly MPEG) on a processor that can dynamically change the operation power supply voltage and operation frequency. Demonstrate how to control voltage and operating frequency! In other words, Non-Patent Document 1 discloses that, as shown in FIG. 23, when moving picture encoding is performed, calculation of moving picture code key or decoding key is performed in units of frames depending on the intensity of motion in the moving picture. Focusing on the difference in amount, the operating frequency and operating power supply voltage of the processor are controlled to reduce power consumption.
[0008] 符号化処理は、 1フレームの処理時間が符号ィ匕方式 (MPEGなど）の規定などにより 時間 Tfに制約されており、その処理時間 Tf内に 1フレームの符号ィ匕処理が完了する こと力 S必要とされる。 1フレームの処理時間 Tf (秒）に対して、それを一定間隔に N個 に分割し、一つ一つの間隔（時間）をタイムスロット Tslot (Tslot=TfZN)と定義し、ま た、タイムスロット Tslotl力もタイムスロット Tslotiが終了した時点の残時間 TRiを TRi =Tf Tslot X iと定義する。一つのタイムスロット Tslotで処理する動画像のブロック 数（動画像の符号化はブロック単位に処理が行われる）を R (すなわち R X Nが 1フレ ームのブロック数となる）とし、（R X i)ブロック処理に力かった時間（すなわちタイムス ロット Tslotlからタイムスロット Tslotほでに処理すべきブロック群に対して実際に処理 にかかった時間)を TaccG+l)とする。電圧変更した場合に動作電源電圧及び動作周 波数が安定するまでの時間を Trdとする。なお、実タイムスロット RTslotiはタイムスロッ ト Tsloti内に完了されるべき処理に対して実際に要した処理時間を示す。図 22では 、まずタイムスロット Tsloti及びタイムスロット Tslot2に割り当てられたブロック群の処 理に対しては、負荷が最大の場合でもそのタイムスロット Tsloti, Tslot2内に十分に 処理が完了可能なクロック周波数 Fmaxで動作させる。その処理に力かった時間 Tacc 3が Tacc3く（Tf— TR2)である場合、すなわち、割り当てられたブロック群がタイムス ロット Tslotl,Tslot2内で処理が完了した場合、次のタイムスロット Tslot3に割り当てら れたブロック群の処理に使用可能な処理時間 Ttar3は Ttar3 =Tf— Tacc3— TR3 - Trdであり、この処理時間 Ttar3内に Tslot3に割り当てられたブロック群の処理が完 結すればよいので、このブロック群に対しては動作周波数を下げて動作させる。図 2 2の処理時間 Tfl, T12, Τβは、タイムスロット Tslot3において負荷が最大の場合に、 各動作周波数 Fl, F2, F3で動作させたときの処理時間を示す。動作周波数として は、図 22において F2=FmaxZ2の動作周波数を選択すれば、負荷が最大の場合 でもタイムスロット Tslotiからタイムスロット Tslot3までに完了されるべき処理時間が（ Tf TR3)以内である、次のタイムスロット Tslot4に処理が入り込むことはない。一方 、動作周波数 F3 = FmaxZ3を選択した場合は、処理時間 Tf3が処理時間 Ttar3を超 えてしまう。したがって、このタイムスロット Tslot3で処理すべきブロック群に対しては F 2=FmaxZ2の動作周波数及びその動作周波数に適する動作電源電圧で動作させ る。同様にして、タイムスロット Tslot毎にこの処理を行う。 [0008] In the encoding process, the processing time of one frame is limited to the time Tf by the definition of the encoding method (MPEG, etc.), and the encoding process of one frame is completed within the processing time Tf. That power S is needed. The processing time Tf (second) of one frame is divided into N at regular intervals, and each interval (time) is defined as a time slot Tslot (Tslot = TfZN). Tslotl force is also defined as TRi = Tf Tslot X i as the remaining time TRi when the time slot Tsloti ends. Let R be the number of moving image blocks processed in one time slot Tslot (moving image is processed in units of blocks) (ie, RXN is the number of blocks in one frame), and (RX i) TaccG + l) is the time spent in block processing (that is, the time actually processed for the block group to be processed from time slot Tslotl to time slot Tslot). Trd is the time until the operating power supply voltage and operating frequency stabilize when the voltage is changed. Note that the real time slot RTsloti is a time slot. G Indicates the actual processing time required for processing to be completed in Tsloti. In FIG. 22, for the processing of the block group assigned to time slot Tsloti and time slot Tslot2, first, the clock frequency Fmax that can be processed sufficiently in the time slots Tsloti and Tslot2 even when the load is maximum. Operate with. If the time Tacc 3 that was powerful in the processing is Tacc3 (Tf—TR2), that is, if the assigned block group has completed processing in the time slots Tslotl and Tslot2, it is assigned to the next time slot Tslot3. The processing time Ttar3 that can be used for the processing of the selected block group is Ttar3 = Tf—Tacc3—TR3-Trd, and the processing of the block group assigned to Tslot3 should be completed within this processing time Ttar3. The block group is operated at a lower operating frequency. The processing times Tfl, T12, and Τβ in Fig. 22 indicate the processing times when operating at each operating frequency Fl, F2, F3 when the load is maximum in time slot Tslot3. As the operating frequency, if the operating frequency of F2 = FmaxZ2 is selected in Fig. 22, the processing time to be completed from time slot Tsloti to time slot Tslot3 is within (Tf TR3) even when the load is maximum. No processing enters Tslot4. On the other hand, when the operating frequency F3 = FmaxZ3 is selected, the processing time Tf3 exceeds the processing time Ttar3. Therefore, the block group to be processed in this time slot Tslot3 is operated at an operating frequency of F 2 = FmaxZ2 and an operating power supply voltage suitable for the operating frequency. Similarly, this process is performed for each time slot Tslot.
[0009] これにより、動的に動作クロック周波数及び動作電源電圧を変更するに際し、所定 時間内に所定数のブロック群を処理可能な動作周波数のうち最小の動作周波数を 選択することにより、総合的に動作周波数及び動作電源電圧を下げて動作させ、必 要処理に応じて電圧を制御することにより、低消費電力化が図られている。  Accordingly, when dynamically changing the operation clock frequency and the operation power supply voltage, the minimum operation frequency can be selected from among the operation frequencies that can process a predetermined number of block groups within a predetermined time. Lowering the operating frequency and operating power supply voltage, and controlling the voltage according to the required processing, lower power consumption.
[0010] ところで、ある一定の処理時間（例えば、ここでは 1フレームの処理時間 Tf)に完了 すべき処理（例えば、ここでは 1フレームの処理）に対しては、 1フレームの処理時間 を通してプロセッサを一定の動作電源電圧及び動作周波数で動作させて処理するこ とが好ましい。すなわち、 1フレームの処理時間を Tf (秒）とし、演算量 Kf (サイクル）と し、動作周波数 Ffとすると、動作周波数 Ff =KfZTf (サイクル Z秒）に設定し、 1フレ ームの処理時間 Τί¾®してプロセッサを一定の動作周波数 Ffで動作させることにより 、その処理時間 Tf内で動作周波数 Ffを何回も変動させる場合と比較して、より低消 費電力化が可能となる。 [0010] By the way, for processing that should be completed in a certain processing time (for example, processing time Tf of one frame here) (for example, processing of one frame here), the processor is passed through the processing time of one frame. It is preferable to operate and operate at a constant operating power supply voltage and operating frequency. That is, if the processing time for one frame is Tf (seconds), the amount of computation is Kf (cycle), and the operating frequency is Ff, the operating frequency is set to Ff = KfZTf (cycle Z seconds), and the processing time for one frame is set. By operating the processor at a constant operating frequency Ff Compared with the case where the operating frequency Ff is changed many times within the processing time Tf, the power consumption can be further reduced.
[0011] し力しながら、非特許文献 1では、処理時間 Tfの同期する単位が 1フレームである にもかかわらず、 1フレーム内で最大 N回の動作電源電圧及び動作周波数の変更が 行われており、低消費電力が十分に図られていな力つた。すなわち、本従来例のよう に多段階に動作電源電圧及び動作周波数を制御可能なプロセッサでの動画像符号 化又は復号化処理の低消費電力化は、 1フレームの処理中に何回も動作電源電圧 及び動作周波数を変更する必要があった。一方、上述のように、処理時間の制約の 単位がフレームであるため、 1フレームの処理中は処理を可能にする最低限の一定 の周波数で制御するのが好ましい。そのため、 1フレームの処理中に最大 N回動作 電源電圧及び動作周波数が変更される本従来例では十分な低消費電力化ができて いなかった。 However, in Non-Patent Document 1, the operating power supply voltage and the operating frequency are changed a maximum of N times within one frame even though the unit of synchronization of the processing time Tf is one frame. Therefore, low power consumption was not enough. In other words, the low power consumption of video encoding or decoding processing in a processor that can control the operating power supply voltage and operating frequency in multiple stages as in the conventional example is achieved by operating power supply many times during the processing of one frame. It was necessary to change the voltage and operating frequency. On the other hand, as described above, since the unit of processing time constraint is a frame, it is preferable to control at the minimum fixed frequency that enables processing during processing of one frame. Therefore, in this conventional example in which the power supply voltage and the operating frequency are changed at most N times during processing of one frame, the power consumption cannot be sufficiently reduced.
[0012] そこで、本願発明者等は下記特許文献 1の発明を完成させた。この発明は、一定の 動作電源電圧 ·動作周波数でプロセッサ 1を動作させながら符号ィ匕又は復号ィ匕処理 を行うことにより低消費電力化を図るものである。  Accordingly, the inventors of the present application completed the invention of Patent Document 1 below. According to the present invention, the power consumption is reduced by performing the encoding process or the decoding process while operating the processor 1 at a constant operating power supply voltage / operating frequency.
特許文献 1：特願 2003— 48535  Patent Document 1: Japanese Patent Application 2003—48535
[0013] また、プロセッサの低消費電力化を妨げる他の要因の一つとして、プロセッサを構 成する MOSトランジスタのサブスレツショルドリーク電流が挙げられる。サブスレツショ ルドリーク電流は、半導体基板に形成される MOSトランジスタのゲート電圧がしき ヽ 値電圧以下のとき流れる微少電流である。このサブスレツショルドリーク電流による消 費電力は、 MOSトランジスタの微細化が高まるにつれて支配的となる傾向にあり、半 導体基板に MOSトランジスタが集積されたプロセッサを使用して動画像の符号ィ匕又 は復号ィ匕を行う動画像符号ィ匕又は復号ィ匕システムにおいて、低消費電力化を妨げ る要因の一つとなって 、る。  [0013] Further, as one of the other factors hindering the reduction in power consumption of the processor, there is a sub-threshold leakage current of a MOS transistor constituting the processor. The subthreshold leakage current is a minute current that flows when the gate voltage of the MOS transistor formed on the semiconductor substrate is lower than the threshold voltage. The power consumption due to this subthreshold leakage current tends to dominate as MOS transistors become finer, and a video signal that uses a processor in which MOS transistors are integrated on a semiconductor substrate is used. This is one of the factors that hinder the reduction of power consumption in a video code key decoding system or decoding key system.
[0014] このサブスレツショルドリーク電流は、 1フレームの処理時間 Tf内でプロセッサの動 作周波数 Ffを何回も変動させる場合と比較して、処理時間 Τί¾®して一定の動作周 波数 Ffで動作させることにより低減され、プロセッサの低消費電力化が可能となる。 上記非特許文献 1の発明は、処理時間 Tfの同期する単位が 1フレームであるにも力 かわらず、 1フレーム内で最大 N回の動作周波数の変更が行われており、動作電源 電圧のみならずサブスレツショルドリーク電流の観点力もも好ましくな力つた。一方、 MOSトランジスタに関しては、 MOSトランジスタが形成される半導体領域の基板バイ ァス電圧を制御することにより、サブスレツショルドリーク電流を制御できることが知ら れている。 [0014] This sub-threshold leakage current is obtained at a constant operating frequency Ff at a constant operating frequency Ff as compared with the case where the operating frequency Ff of the processor is varied many times within the processing time Tf of one frame. It is reduced by operating, and the power consumption of the processor can be reduced. The invention of Non-Patent Document 1 described above is effective even when the unit for synchronizing the processing time Tf is one frame. Regardless, the operating frequency has been changed up to N times within one frame, and not only the operating power supply voltage but also the power of subthreshold leakage current is favorable. On the other hand, regarding MOS transistors, it is known that the subthreshold leakage current can be controlled by controlling the substrate bias voltage in the semiconductor region in which the MOS transistor is formed.
[0015] そこで、本願発明者等は下記特許文献 2の発明を完成させた。この発明は、フレー ムごとに符号化又は復号化に必要な演算量 (以下、必要演算量という）を計算し、一 定の基板バイアス電圧.動作周波数、又は、一定の基板バイアス電圧'動作電源電 圧 ·動作周波数でプロセッサ 1を動作させながら符号ィ匕又は復号ィ匕処理を行うことに より低消費電力化を図るものである。  Accordingly, the inventors of the present application have completed the invention of Patent Document 2 below. The present invention calculates the amount of computation required for encoding or decoding (hereinafter referred to as the necessary amount of computation) for each frame, and calculates a constant substrate bias voltage, an operating frequency, or a constant substrate bias voltage. The power consumption can be reduced by performing the encoding / decoding process while operating the processor 1 at the voltage / operating frequency.
特許文献 2 :特願 2003— 409641  Patent Document 2: Japanese Patent Application 2003—409641
発明の開示  Disclosure of the invention
発明が解決しょうとする課題  Problems to be solved by the invention
[0016] し力しながら、上記特許文献 1, 2においては、必要演算量計算手段で算出された 必要演算量 Kpが、符号化又は復号化処理に実際に必要となる演算量よりも小さい 場合、 1フレームの処理に割り当てられた時間 Te内に符号化処理又は復号化処理 が完了できないという破綻現象が生じる可能性があった。各特許文献 1, 2において も破綻現象を回避する破綻回避手段について開示したが、さらにいくつかの破綻回 避手段が考えられ、特に破綻回避手段に伴う画質の劣化を改善することが強く望ま れる。 However, in Patent Documents 1 and 2 described above, when the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the calculation amount actually required for the encoding or decoding process There was a possibility that the failure of the encoding process or the decoding process could not be completed within the time Te allocated for the processing of one frame. Each patent document 1 and 2 also disclosed failure avoidance means for avoiding the failure phenomenon. However, several failure avoidance means are conceivable, and it is strongly desired to improve the deterioration of image quality caused by the failure avoidance means. .
[0017] そこで本発明は、前記のような課題を解決するためのものであり、基板バイアス電圧 や動作電源電圧や動作周波数を制御することにより低消費電力化を実現しながらも 、破綻現象を回避可能な動画像符号化処理システム、動画像符号化又は復号化処 理システム、動画像符号化処理方法、及び、動画像符号化又は復号化処理方法を 提案することにある。 課題を解決するための手段  [0017] Therefore, the present invention is for solving the above-mentioned problems, and while realizing low power consumption by controlling the substrate bias voltage, the operating power supply voltage, and the operating frequency, the failure phenomenon is solved. The present invention proposes a video encoding processing system, a video encoding or decoding processing system, a video encoding processing method, and a video encoding or decoding processing method that can be avoided. Means for solving the problem
[0018] 本発明の動画像符号化処理システムは、連続する複数のフレームから構成される 動画像をフレーム単位で符号ィ匕する動画像符号ィ匕手段として機能するプロセッサを 備え、 [0018] The moving image encoding processing system of the present invention includes a processor functioning as a moving image encoding means for encoding a moving image composed of a plurality of continuous frames in units of frames. Prepared,
一のフレームの符号化に必要な必要演算量 Kpを計算する必要演算量計算手段と 、当該一のフレームの符号ィ匕処理に予め割り当てられている時間 Te内に当該必要 演算量 Kpを符号化処理可能な動作周波数を決定する動作決定手段とを備え、 当該プロセッサが、予め割り当てられて!/、る時間 Te内は当該動作決定手段により 決定された動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z又は 基板バイアス電圧で動作しながら当該動画像符号ィ匕手段により当該一のフレームの 符号化処理を行う動画像符号化処理システムであり、  Necessary calculation amount calculation means for calculating the necessary calculation amount Kp necessary for encoding one frame, and encoding the necessary calculation amount Kp within the time Te previously allocated to the encoding process of the one frame Operation determining means for determining a processing frequency that can be processed, and the processor is pre-allocated! /, The operation frequency determined by the operation determining means within the time period Te and an operation suitable for the operation frequency. A moving image encoding processing system that performs encoding processing of the one frame by the moving image encoding means while operating at a power supply voltage and a Z or substrate bias voltage;
所定のタイミングで符号ィヒ処理の実際の演算量を削減する第 1の破綻回避手段を 備えることを特徴とする。  It is characterized by comprising first failure avoiding means for reducing the actual calculation amount of the codeh processing at a predetermined timing.
また、本発明の動画像符号化処理方法は、プロセッサを使用して連続する複数の フレームから構成される動画像をフレーム単位で符号化する動画像符号化ステップ と、一のフレームの符号化に必要な必要演算量 Kpを計算する必要演算量計算ステ ップと、当該一のフレームの符号ィ匕処理に予め割り当てられている時間 Te内に当該 必要演算量 Kpを符号ィ匕処理可能な動作周波数を決定する動作決定ステップとを備 え、  Further, the moving image encoding processing method of the present invention includes a moving image encoding step for encoding a moving image composed of a plurality of consecutive frames using a processor in units of frames, and encoding of one frame. Necessary computation amount calculation step for calculating the necessary computation amount Kp, and operation that can signify the required computation amount Kp within the time Te allocated in advance for the signing processing of the one frame An operation determining step for determining the frequency,
当該プロセッサが、予め割り当てられて!/ヽる時間 Te内は当該動作決定ステップに おいて得られた動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z 又は基板バイアス電圧で動作しながら、動画像符号ィ匕又は復号化ステップにお ヽて 当該一のフレームの符号ィ匕処理を行う動画像符号ィ匕処理方法であり、  While the processor is operating at the operation frequency obtained in the operation determination step and the operation power supply voltage and Z or substrate bias voltage suitable for the operation frequency within the time period Te that is allocated in advance! A moving image code key processing method for performing the code key processing of the one frame in the moving image code key or decoding step;
所定のタイミングで符号ィ匕処理の実際の演算量を削減する第 1の破綻回避ステップ を備えることを特徴とする。  A first failure avoidance step is provided for reducing an actual calculation amount of the sign key processing at a predetermined timing.
第 1の破綻回避手段や第 1の破綻回避ステップは、符号ィ匕処理の一部を、それより も演算量の少ない処理に変更することなどにより、実際の演算量を削減する。  The first failure avoiding means and the first failure avoiding step reduce the actual amount of computation by changing a part of the sign process to a processing with a smaller amount of computation.
ここで、「符号化処理」とは、動作決定手段 ·動作決定ステップの処理の後に行われ る符号化処理手段 ·符号化処理ステップの処理を指し、「復号化処理」とは、動作決 定手段 ·動作決定ステップの処理の後に行われる復号化処理手段 ·復号化処理ステ ップの処理を指す。 [0019] 本発明の動画像符号化処理システム及び動画像符号化処理方法によれば、時間 Te内に完了すべき一のフレームの符号ィ匕処理に必要な必要演算量 Kpを算出し、プ 口セッサを時間 Te内に必要演算量 Κρを処理可能な動作周波数とし、一のフレーム を時間 Te内に符号ィ匕処理可能な最低限の (又は最低限に近 、)動作周波数、及び 、当該動作周波数に適する動作電源電圧及び Z又は基板バイアス電圧で符号化処 理を行うため、低消費電力化を図ることができる。たとえ、上記必要演算量計算手段 により算出された必要演算量 Kpが実際に必要な演算量よりも小さい場合であっても 、符号ィ匕処理の実際の演算量は第 1の破綻回避手段や第 1の破綻回避ステップによ り削減されるため、破綻現象を回避可能となる。 Here, the “encoding process” means the process of the operation determining means · the operation determining step and the encoding process means · the process of the encoding process step. The “decoding process” is the operation determining process. Means · Decoding processing means performed after the processing of the action determination step · Decoding processing step. [0019] According to the moving image coding processing system and the moving image coding processing method of the present invention, the necessary calculation amount Kp necessary for the coding process of one frame to be completed within the time Te is calculated, The operating frequency required for the mouth sensor in the time Te is set as an operating frequency that can be processed, and the minimum operating frequency (or close to the minimum) that allows one frame to be processed in the time Te, and Since the encoding process is performed with the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency, power consumption can be reduced. Even if the required calculation amount Kp calculated by the above-described required calculation amount calculation means is smaller than the actually required calculation amount, the actual calculation amount of the sign 匕 process is the first failure avoidance means or the first Since it is reduced by 1 failure avoidance step, the failure phenomenon can be avoided.
[0020] 前記第 1の破綻回避手段は、符号ィ匕が終了していないマクロブロックのうち、色差 信号の情報を持つブロックを無効ブロックとして処理を行うことを特徴とする。  [0020] The first failure avoiding means is characterized in that a block having color difference signal information is processed as an invalid block among macroblocks in which the sign is not finished.
[0021] この発明によれば、輝度信号の情報を持つブロック（以下、輝度ブロックとする）と色 差信号の情報を持つブロック（以下、色差ブロックとする）で構成されるマクロブロック に対し、色差ブロックのみを無効ブロック化処理し、輝度ブロックに関しては通常と同 様に符号化処理されることから、マクロブロックのうちすベてのブロックを無効ブロック 化処理する場合と比較して、破綻現象を回避しながらも画質を確保することができる 。ここで、無効ブロック化処理の対象を輝度ブロックではなく色差ブロックとした理由 は、色差ブロックの無効ブロック化処理のほうが画質劣化への影響が小さいためであ る。  [0021] According to the present invention, for a macro block composed of a block having luminance signal information (hereinafter referred to as luminance block) and a block having color difference signal information (hereinafter referred to as color difference block), Since only the chrominance block is processed as an invalid block and the luminance block is encoded in the same way as usual, the breakdown phenomenon is compared to the case where all the blocks in the macro block are processed as an invalid block. The image quality can be ensured while avoiding the problem. Here, the reason why the invalid block processing target is the chrominance block instead of the luminance block is that the chrominance block invalid block processing has less influence on image quality degradation.
[0022] 前記第 1の破綻回避手段は、符号ィ匕が終了していないマクロブロックに対し、フレ ーム内符号化処理を行うことを特徴とする。  [0022] The first failure avoiding means performs an intra-frame encoding process on a macroblock for which the encoding has not ended.
[0023] フレーム内符号ィ匕処理はフレーム間符号ィ匕処理よりも演算量が小さい。この発明に よれば、符号ィ匕が終了していないマクロブロックに対してフレーム内符号ィ匕処理を行 うことで、符号化処理に実際に必要な演算量を、必要演算量計算手段で算出された 必要演算量 Kpよりも小さく又は必要演算量 Kpに近い値とすることができる。これによ り、破綻現象を回避可能となる。  [0023] The intraframe code processing has a smaller amount of computation than the interframe code processing. According to the present invention, the necessary amount of calculation is calculated by the necessary amount of calculation means by performing the intra-frame code processing on the macroblock for which the code processing has not been completed. It can be set to a value smaller than or closer to the required calculation amount Kp. This makes it possible to avoid bankruptcy.
[0024] 前記第 1の破綻回避手段は、符号ィ匕が終了していないマクロブロックに対し、動き ベクトル検出を行うことなく動きベクトルを 0としてフレーム間符号ィ匕処理を行うことを 特徴とする。 [0024] The first failure avoiding means performs inter-frame code key processing with a motion vector set to 0 without performing motion vector detection for a macroblock for which the code key has not ended. Features.
[0025] フレーム間符号ィ匕処理のうち動きベクトル検出処理に必要な演算量は、符号化処 理の演算量の大部分を占める要素である。本発明によれば、動きベクトル検出処理 を行わずに動きベクトルを強制的に 0としてフレーム間符号ィ匕処理を行うため、動き ベクトル検出処理を行う通常のフレーム間符号ィ匕処理よりも演算量が小さくなり、実 際の符号ィ匕処理の演算量を、上記必要演算量 Kpよりも小さく又は必要演算量 Kpに 近い値とすることができる。これにより、破綻現象を回避可能となる。  [0025] Of the interframe code processing, the amount of computation required for the motion vector detection processing is an element that occupies most of the amount of computation in the encoding processing. According to the present invention, since the motion vector is forcibly set to 0 without performing the motion vector detection process, and the interframe code processing is performed, the calculation amount is larger than that of the normal interframe code processing that performs the motion vector detection processing. As a result, the amount of computation of the actual sign process is smaller than the required amount of computation Kp or close to the required amount of computation Kp. Thereby, it becomes possible to avoid the failure phenomenon.
[0026] 前記第 1の破綻回避手段は、符号ィ匕が終了していないマクロブロックに対し、量子 ィ匕ステップサイズを大きくして符号ィ匕処理を行うことを特徴とする。  [0026] The first failure avoidance means is characterized in that a code step is performed with a larger quantum step size for a macroblock that has not been finished.
[0027] この発明によれば、符号ィ匕が終了していないマクロブロックに対し量子ィ匕ステップサ ィズを大きくして符号化処理を行うことで、当該マクロブロックで発生する有効ブロック の数、有効係数の数を抑制し、有効ブロック、有効係数に対して行われる処理 (たと えば、動画像符号ィ匕方式として MPEG4を用いる場合、 IDCT処理、 IQ処理、 VLC 処理)の実行回数を削減し、符号化処理の実際の演算量を上記必要演算量 Kpより も小さく又は必要演算量 Kpに近い値とすることができる。これにより、破綻現象を回 避可能となる。  [0027] According to the present invention, the number of effective blocks generated in a macroblock is increased by performing the encoding process with the quantum step size being increased for a macroblock for which the code is not finished. The number of effective coefficients is suppressed, and the number of executions of processes performed on effective blocks and effective coefficients (for example, when MPEG4 is used as a moving image encoding method, IDCT processing, IQ processing, VLC processing) is reduced. The actual calculation amount of the encoding process can be made smaller than the required calculation amount Kp or close to the required calculation amount Kp. This makes it possible to avoid the bankruptcy phenomenon.
[0028] また、本発明の動画像符号化処理システムは、前記第 1の破綻回避手段により前 記一のフレームよりも後に符号ィ匕される後続フレームの符号ィ匕処理に与えられる影 響を緩和する緩和手段を備えることを特徴とする。  [0028] Further, the moving image encoding processing system of the present invention has an effect given to the encoding process of the subsequent frame encoded after the first frame by the first failure avoidance means. A mitigating means for mitigating is provided.
[0029] 上記の第 1の破綻回避手段は、通常行われるはずの符号化処理の処理内容を変 更して破綻回避を行っているため、一のフレームにおいて第 1の破綻回避手段が実 行された場合、後続フレームの画質は少なからず第 1の破綻回避手段による処理の 影響を受けることとなる。本発明によれば、一のフレームにおいて第 1の破綻回避手 段による処理が実行された場合に後続フレームの符号化処理への影響を緩和する 緩和手段により緩和処理を行うことで、後続フレームの符号化処理による画質の劣化 を抑制することができる。  [0029] Since the first failure avoidance means described above performs the failure avoidance by changing the processing content of the encoding process that should normally be performed, the first failure avoidance means is executed in one frame. If this happens, the image quality of the subsequent frame will be affected by the processing by the first failure avoidance method. According to the present invention, when the processing by the first failure avoidance means is executed in one frame, the mitigation processing is performed by the mitigation means that mitigates the influence on the encoding processing of the subsequent frame. Degradation of image quality due to encoding processing can be suppressed.
[0030] 前記緩和手段は、当該一のフレームにおいて第 1の破綻回避手段による処理が実 行されたマクロブロックに対応する後続フレームのマクロブロックに対し、量子化ステ ップサイズを小さくすることを特徴とする。 [0030] The mitigation means applies a quantization step to the macroblock of the subsequent frame corresponding to the macroblock for which the processing by the first failure avoidance means has been executed in the one frame. The feature is to reduce the size of the tape.
[0031] この発明によれば、量子化ステップサイズを小さくすることで、対応するマクロブロッ クに割り当てられる符号量を大きくして画質を改善することができ、第 1の破綻回避手 段による破綻回避処理が行われた場合でも、破綻回避処理が行われた一のフレーム よりも後に符号化される後続フレームに対する影響を緩和することができる。  [0031] According to the present invention, by reducing the quantization step size, it is possible to improve the image quality by increasing the code amount allocated to the corresponding macroblock, and the failure due to the first failure avoiding means. Even when the avoidance process is performed, the influence on the subsequent frame encoded after the one frame on which the failure avoidance process is performed can be reduced.
[0032] また、前記第 1の破綻回避手段により前記一のフレームの符号化処理の演算量が 削減された場合は、当該一のフレームよりも後に符号ィ匕処理される後続フレームの必 要演算量を増加させることを特徴とする。たとえば、前記後続フレームの必要演算量 、又は、必要演算量の算出に用いられる要素について、 m倍 (mは 1以上の実数）、 又は、 0より大きい実数 nを加算することにより、前記後続フレームの必要演算量を増 加させる。  [0032] Further, when the calculation amount of the encoding process of the one frame is reduced by the first failure avoidance means, the necessary calculation of the subsequent frame to be encoded after the one frame It is characterized by increasing the amount. For example, by adding m times (m is a real number greater than or equal to 1) or a real number n greater than 0, the subsequent frame is added to the element used for calculating the necessary calculation amount or the necessary calculation amount of the subsequent frame. Increase the required amount of computation.
[0033] 第 1の破綻回避手段や第 1の破綻回避ステップの処理が実行された場合、後続フ レームについて上記必要演算量 Kpが実際に必要な演算量よりも小さくなる確率が高 い。この発明によれば、第 1の破綻回避手段や第 1の破綻回避ステップの処理の後 に、後続フレームの必要演算量を増加させるため、後続フレームに関して上記必要 演算量が実際の演算量を満たす可能性が高くなり、後続フレームにおける破綻現象 を回避することができる。  [0033] When the first failure avoiding means or the first failure avoiding step is executed, the required calculation amount Kp is likely to be smaller than the actually required calculation amount for the subsequent frame. According to the present invention, after the processing of the first failure avoiding means and the first failure avoidance step, the necessary amount of computation for the subsequent frame satisfies the actual amount of computation in order to increase the necessary amount of computation for the subsequent frame. The possibility increases and the failure phenomenon in subsequent frames can be avoided.
[0034] 本発明の動画像符号化又は復号化処理システムは、連続する複数のフレームから 構成される動画像をフレーム単位で符号化または復号化する動画像符号化又は復 号ィ匕手段として機能するプロセッサを備え、  [0034] The moving image encoding or decoding processing system of the present invention functions as a moving image encoding or decoding means for encoding or decoding a moving image composed of a plurality of consecutive frames in units of frames. Processor,
一のフレームの符号化又は復号化に必要な必要演算量 Kpを計算する必要演算量 計算手段と、当該一のフレームの符号ィ匕又は復号ィ匕処理に予め割り当てられている 時間 Te内に前記必要演算量 Kpを符号化又は復号化処理可能な動作周波数を決 定する動作決定手段とを備え、  Necessary amount of computation required for encoding or decoding one frame. Necessary amount of computation for calculating Kp, and within the time Te previously assigned to the encoding / decoding processing of the one frame. Operation determining means for determining an operating frequency capable of encoding or decoding the necessary amount of computation Kp,
当該プロセッサが、予め割り当てられている時間 Te内は、当該動作決定手段により 決定された動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z又は 基板バイアス電圧で動作しながら当該動画像符号ィ匕又は復号ィ匕手段により当該一 のフレームの符号化又は復号化処理を行う動画像符号化又は復号化処理システム であり、 While the processor is operating at the operating frequency determined by the operation determining means and the operating power supply voltage and the Z or substrate bias voltage suitable for the operating frequency within the pre-allocated time Te, A moving image encoding or decoding processing system that performs encoding or decoding processing of the one frame by the 匕 or decoding means And
前記必要演算量算出手段で算出された必要演算量 Kpが実際に必要な演算量より も小さい場合、当該一のフレームよりも後に符号ィ匕又は復号ィ匕される後続フレームの 必要演算量を増加させる第 2の破綻回避手段を備えることを特徴とする。  When the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount, the required calculation amount of the subsequent frame that is encoded or decoded after the one frame is increased. The second failure avoiding means is provided.
また、本発明の動画像符号化又は復号化処理方法は、プロセッサを使用して連続 する複数のフレーム力 構成される動画像をフレーム単位で符号ィ匕又は復号ィ匕する 動画像符号化又は復号化ステップと、一のフレームの符号化又は復号化に必要な 必要演算量 Kpを計算する必要演算量計算ステップと、当該一のフレームの符号ィ匕 又は復号化処理に予め割り当てられている時間 Te内に前記必要演算量 Kpを符号 化又は復号ィ匕処理可能な動作周波数を決定する動作決定ステップとを備え、 当該プロセッサが、予め割り当てられて!/ヽる時間 Te内は当該動作決定ステップによ り得られた動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z又は 基板バイアス電圧で動作しながら、動画像符号ィ匕又は復号化ステップにお ヽて当該 一のフレームの符号化又は復号化処理を行う動画像符号化又は復号化処理方法で あり、  Also, the moving image encoding or decoding processing method of the present invention encodes or decodes a moving image composed of a plurality of consecutive frame forces using a processor in units of frames. Step, a necessary computation amount calculating step for calculating a necessary computation amount Kp required for encoding or decoding one frame, and a time Te previously allocated to the encoding or decoding processing of the one frame. An operation determining step for determining an operating frequency capable of encoding or decoding the necessary computation amount Kp, and the processor determines the operation determining step within the time period Te /! While operating with the obtained operating frequency and the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency, the video coding key or decoding step A moving picture encoding or decoding method for performing encoding or decoding of frames,
前記必要演算量算出手段で算出された必要演算量 Kpが実際に必要な演算量より も小さい場合、当該一のフレームよりも後に符号ィ匕又は復号ィ匕される後続フレームの 必要演算量を増加させる第 2の破綻回避ステップを備えることを特徴とする。  When the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount, the required calculation amount of the subsequent frame that is encoded or decoded after the one frame is increased. The second failure avoidance step is provided.
[0035] 一のフレームにつ 、て上記必要演算量 Kpが実際に必要な演算量よりも小さ 、場 合、後続フレームについても同様に上記必要演算量 Kpが実際に必要な演算量より も小さくなる確率が高い。この発明によれば、第 2の破綻回避手段や第 2の破綻回避 ステップが、後続フレームの必要演算量を増加させるため、後続フレームに関して上 記必要演算量が実際の演算量を満たす可能性が高くなり、後続フレームにおける破 綻現象を回避することができる。なお、第 2の破綻回避手段'ステップの処理は、第 1 の破綻回避手段'ステップの有無にかかわらず、所定フレームの必要演算量 Kpが実 際の演算量よりも小さ 、場合に行われる。  [0035] For one frame, the required calculation amount Kp is smaller than the actually required calculation amount. In this case, the necessary calculation amount Kp is also smaller than the actually required calculation amount for the subsequent frames. The probability of becoming high. According to the present invention, since the second failure avoiding means and the second failure avoiding step increase the necessary amount of computation for the subsequent frame, the necessary amount of computation for the subsequent frame may satisfy the actual amount of computation. It becomes higher and the failure phenomenon in the subsequent frame can be avoided. The processing of the second failure avoiding means 'step is performed when the required calculation amount Kp of the predetermined frame is smaller than the actual calculation amount regardless of the presence or absence of the first failure avoiding means' step.
[0036] 前記第 2の破綻回避手段や第 2の破綻回避ステップは、前記後続フレームの必要 演算量、又は、必要演算量の算出に用いられた要素について、 m倍 (mは 1以上の 実数)、又は、 0より大きい実数 nを加算することを特徴とする。 [0036] The second failure avoiding means and the second failure avoiding step may be performed by multiplying the necessary calculation amount of the subsequent frame or an element used for calculating the required calculation amount by m times (m is 1 or more). Real number) or a real number n greater than 0 is added.
[0037] この発明によれば、後続フレームの処理における必要演算量計算手段で算出され た必要演算量の値、又は、必要演算量の算出に用いられた要素の値について、 m倍 (mは 1以上の実数)又は 0より大きい実数 nを加算することで、後続フレームの必要 演算量 Kpを増加させ、後続フレームでの破綻現象の発生を回避することができる。 増加対象とする要素、及び、 mや ηの値としては、予め実験等により適切な要素 直を 導出してシステム内に設定するのが好ましい。 According to the present invention, the value of the required calculation amount calculated by the required calculation amount calculation means in the processing of the subsequent frame or the value of the element used for calculating the required calculation amount is m times (m is By adding a real number n greater than 1) or a real number n greater than 0, it is possible to increase the necessary amount of computation Kp of the subsequent frame and avoid the occurrence of a failure phenomenon in the subsequent frame. As elements to be increased, and values of m and η, it is preferable to derive appropriate elements from the experiment beforehand and set them in the system.
[0038] 本発明の動画像符号化又は復号化処理システムは、連続する複数のフレームから 構成される動画像をフレーム単位で符号化または復号化する動画像符号化又は復 号ィ匕手段として機能するプロセッサを備え、 [0038] The moving image encoding or decoding processing system of the present invention functions as moving image encoding or decoding means for encoding or decoding a moving image composed of a plurality of consecutive frames in units of frames. Processor,
一のフレームの符号化又は復号化に必要な必要演算量 Kpを計算する必要演算量 計算手段と、当該一のフレームの符号ィ匕又は復号ィ匕処理に予め割り当てられている 時間 Te内に前記必要演算量 Kpを符号化又は復号化処理可能な動作周波数を決 定する動作決定手段とを備え、  Necessary amount of computation required for encoding or decoding one frame. Necessary amount of computation for calculating Kp, and within the time Te previously assigned to the encoding / decoding processing of the one frame. Operation determining means for determining an operating frequency capable of encoding or decoding the necessary amount of computation Kp,
当該プロセッサが、予め割り当てられている時間 Te内は、当該動作決定手段により 決定された動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z又は 基板バイアス電圧で動作しながら動画像符号ィ匕又は復号ィ匕手段により当該一のフレ ームの符号化又は復号化処理を行う動画像符号化又は復号化処理システムであり、 所定のタイミングで前記時間 Teを延長する第 3の破綻回避手段を備えることを特徴 とする。  During the pre-allocated time Te, the moving image encoding code is operated while operating at the operating frequency determined by the operation determining means and the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency. Alternatively, it is a moving image encoding or decoding processing system that performs encoding or decoding processing of the one frame by decoding decoding means, and includes third failure avoiding means that extends the time Te at a predetermined timing. It is characterized by having.
また、本発明の動画像符号化又は復号化処理方法は、プロセッサを使用して連続 する複数のフレーム力 構成される動画像をフレーム単位で符号ィ匕又は復号ィ匕する 動画像符号化ステップと、一のフレームの符号化又は復号化に必要な必要演算量 K Pを計算する必要演算量計算ステップと、当該一のフレームの符号化又は復号ィ匕処 理に予め割り当てられている時間 Te内に前記必要演算量 Kpを符号化又は復号ィ匕 処理可能な動作周波数を決定する動作決定ステップとを備え、  Further, the moving image encoding or decoding processing method of the present invention includes a moving image encoding step of encoding or decoding a moving image composed of a plurality of continuous frame forces using a processor in units of frames. The necessary calculation amount calculation step for calculating the necessary calculation amount KP required for encoding or decoding one frame, and the time Te allocated in advance for the encoding or decoding processing of the one frame. An operation determining step for determining an operating frequency capable of encoding or decoding the necessary calculation amount Kp, and
当該プロセッサが、予め割り当てられて!/ヽる時間 Te内は当該動作決定ステップに おいて得られた動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z 又は基板バイアス電圧で動作しながら、動画像符号ィ匕又は復号化ステップにお ヽて 当該一のフレームの符号化又は復号化処理を行う動画像符号化又は復号化処理方 法であり、 Within the time period Te that the processor is pre-allocated! / Turns on, the operating frequency obtained in the operation determining step, the operating power supply voltage suitable for the operating frequency, and Z Alternatively, a moving image encoding or decoding processing method that performs encoding or decoding processing of the one frame in the moving image encoding or decoding step while operating at a substrate bias voltage,
所定のタイミングで当該時間 Teを延長する第 3の破綻回避ステップを備えることを 特徴とする。  A third failure avoidance step is provided which extends the time Te at a predetermined timing.
[0039] この発明によれば、一のフレームの符号ィ匕又は復号ィ匕処理が予め割り当てられた 時間 Teに完了できない場合は、次の後続フレームの処理に割り当てられている時間 を用いて前記一のフレームの処理時間 Teを延長し、当該一のフレームの符号ィ匕又 は復号化処理を完了する。これにより、符号化又は復号化の実際の処理量を削減す ることなく、かつ画質を劣悪にすることなぐ破綻現象を回避することができる。  [0039] According to the present invention, when the encoding / decoding processing of one frame cannot be completed at the time Te allocated in advance, the time allocated to the processing of the next subsequent frame is used. The processing time Te of one frame is extended, and the encoding / decoding processing of the one frame is completed. As a result, it is possible to avoid a failure phenomenon that does not deteriorate the image quality without reducing the actual processing amount of encoding or decoding.
[0040] さらに、前記第 3の破綻回避手段は、前記一のフレームの符号化処理に予め割り当 てられている時間 Teを延長した場合に、前記一のフレームよりも後に符号ィ匕又は復 号ィ匕される後続フレームにつ 、て、当該後続フレームの符号化処理又は復号化処 理に予め割り当てられて!/ヽる時間 Teを変更することを特徴とする。  [0040] Furthermore, the third failure avoiding means, when extending the time Te previously assigned to the encoding process of the one frame, is adapted to return the code after the one frame. For the subsequent frame to be signaled, the time Te that is assigned in advance to the encoding process or the decoding process of the subsequent frame is changed.
[0041] この発明によれば、一のフレームの時間 Teを延長することにより、後続フレームに 割り当てられる時間が短縮されることとなるが、次の後続フレームの符号ィ匕又は復号 化処理につ!、ては、後続フレームの符号ィ匕又は復号ィ匕処理に割り当てられて!/、る時 間から、上記一のフレームに割り当てた延長処理時間を差し引いた時間を、後続フレ ームの符号ィ匕又は復号ィ匕処理の時間 Teとするなどのように時間 Teを変更するため、 延長時間に応じて変更された時間内で後続フレームの符号ィヒ又は復号ィヒ処理可能 なプロセッサの動作周波数が決定され、その動作周波数及びそれに適する基板バイ ァス電圧や動作電源電圧でプロセッサが動作する。したがって、後続フレームの符号 化又は復号化処理においても低消費電力化が実現される。一のフレームに割り当て られた時間 Teの延長により後続フレームに割り当てられた時間 Teが短縮されても、 後続フレームの時間 Teを変更することにより、後続フレームの処理において破綻現 象が発生する可能性を低くすることができる。  [0041] According to the present invention, by extending the time Te of one frame, the time allocated to the subsequent frame is shortened. !, The time obtained by subtracting the extended processing time assigned to the above one frame from the time assigned to the encoding or decoding processing of the subsequent frame, The operation of the processor that can process the coding or decoding of the subsequent frame within the time changed in accordance with the extension time to change the time Te, such as the time Te of decoding or decoding processing The frequency is determined, and the processor operates at the operating frequency and the appropriate substrate bias voltage or operating power supply voltage. Therefore, low power consumption is realized in the encoding or decoding processing of subsequent frames. Even if the time Te assigned to the succeeding frame is shortened by extending the time Te assigned to one frame, changing the time Te of the succeeding frame may cause a failure in the processing of the succeeding frame. Can be lowered.
[0042] 前記第 3の破綻回避手段は、前記一のフレームの符号化処理に予め割り当てられ ている時間 Teを延長した場合に、次に符号ィ匕されるフレームを格納する入力フレー ムメモリに対して、フレームの書き込み先を変更することを特徴とする。 [0042] The third failure avoiding means stores an input frame for storing a frame to be encoded next when the time Te previously allocated to the encoding process of the one frame is extended. It is characterized in that the frame write destination is changed with respect to the memory.
[0043] この発明によれば、第 3の破綻回避手段により時間 Teが延長されると、順次入力さ れてくるフレームは、一のフレームを上書きすることなぐ書き込み先を変更して格納 されるため、入力フレームメモリに前記一のフレームを保持することができ、符号化処 理を続行して完了させることが可能となる。  [0043] According to this invention, when the time Te is extended by the third failure avoiding means, the sequentially input frames are stored by changing the write destination without overwriting one frame. Therefore, the one frame can be held in the input frame memory, and the encoding process can be continued and completed.
[0044] 前記第 3の破綻回避手段は、前記一のフレームの符号化処理に予め割り当てられ ている時間 Teを延長した場合に、次に符号ィ匕されるフレームを格納する入力フレー ムメモリに対して書き込み不可とすることを特徴とする。  [0044] The third failure avoiding means is provided for an input frame memory that stores a frame to be encoded next when the time Te previously allocated to the encoding process of the one frame is extended. The writing is impossible.
[0045] この発明によれば、前記第 3の破綻回避手段により時間 Teが延長されると、入カフ レームメモリはフレームの書き込みが不可となるため、後続のフレームに上書きされる ことなく一のフレームを保持することができ、符号ィ匕処理を続行して完了させることが 可能となる。さらに、複数フレームおきにフレームを符号ィ匕処理するに際しては、入力 フレームメモリに符号ィ匕が予定されて 、るフレームが入力される前に、入力フレームメ モリが書き込み可能となれば、時間 Teが延長されな力つた場合と同様のフレームを 対象に符号化処理を行うことができる。  [0045] According to the present invention, when the time Te is extended by the third failure avoiding means, the input frame memory cannot write the frame, so that the subsequent frame is not overwritten. The frame can be held, and the sign key process can be continued and completed. Further, when a frame is encoded every two or more frames, if an input frame memory can be written before the frame is input and the frame is scheduled to be input to the input frame memory, the time Te Encoding processing can be performed on the same frame as when the power is not extended.
[0046] 上記動画像符号化システムや動画像符号化又は復号化システムは、前記一のフレ ームより前に符号化処理されるフレームを前フレームとすると、動画像符号化処理を 行う場合において、前記必要演算量計算手段は、所定フレームと前フレームとの動き 量、所定フレームのアクティビティの量、前フレームのアクティビティの量、前フレーム の量子化ステップサイズの平均値、前フレームの量子化ステップサイズの平均値とそ の一つ前のフレームの量子化ステップサイズの平均値の差、前フレームのマクロブロ ックマッチング回数、前フレームの有効ブロック数、前フレームの有効係数の数、前フ レームの符号ィ匕に実際に要した演算量、前フレームの発生ビット数、所定フレームの 符号ィ匕ビットレート、所定フレームについてフレーム内符号ィ匕又はフレーム間符号ィ匕 のいずれであるかの種類、必要演算量計算手段により算出された前フレームの必要 演算量のうち、一つ以上の要素を使用して必要演算量を計算することが好ましい。  [0046] The moving image encoding system, the moving image encoding or decoding system, when performing a moving image encoding process, assuming that a frame to be encoded before the one frame is a previous frame. The necessary calculation amount calculating means includes a motion amount between the predetermined frame and the previous frame, an amount of activity of the predetermined frame, an amount of activity of the previous frame, an average value of a quantization step size of the previous frame, and a quantization step of the previous frame. The difference between the average value of the size and the average value of the quantization step size of the previous frame, the number of macroblock matching of the previous frame, the number of effective blocks of the previous frame, the number of effective coefficients of the previous frame, the sign of the previous frame The amount of computation actually required for the key, the number of bits generated in the previous frame, the sign key rate of the predetermined frame, and the predetermined frame The required amount of computation using one or more elements of the type of whether it is an intra-frame code 符号 or inter-frame code 匕, and the required amount of computation of the previous frame calculated by the required computation amount calculation means Is preferably calculated.
[0047] 上記動画像符号ィ匕又は復号ィ匕システムは、上記一のフレームより前に復号ィ匕処理 されるフレームを前フレームとすると、動画像復号化処理を行う場合において、前記 必要演算量計算手段は、所定フレームの符号化データのビット数、前記所定フレー ムがフレーム内符号ィ匕されたものであるか又はフレーム間符号ィ匕されたものであるか の種類、所定フレーム若しくは前フレームの動きベクトルの大きさの平均値、所定フレ ーム若しくは前フレームの動きベクトルの大きさの分散、所定フレーム若しくは前フレ ームの有効ブロック数、所定フレーム若しくは前フレームの有効係数の数、所定フレ ーム若しくは前フレームのビットレート、所定フレーム若しくは前フレームの符号量、 所定フレーム若しくは前フレームの量子化ステップサイズの平均値、量子化ステップ サイズの平均値の差 (所定フレームと 1つ前のフレームの量子化ステップサイズの差， もしくは 1つ前のフレームの量子化ステップサイズと 2つ前のフレームの量子化ステツ プサイズの差)、前フレームの復号化に実際に要した演算量、必要演算量計算手段 により算出された前フレームの必要演算量のうち一つ以上の要素を使用して必要演 算量を計算することが好まし 、。 [0047] In the case where the moving image decoding process is performed when the moving image decoding process is performed by assuming that a frame subjected to the decoding process before the one frame is a previous frame, The required calculation amount calculation means includes the number of bits of encoded data of a predetermined frame, the type of whether the predetermined frame is intra-frame encoded or inter-frame encoded, predetermined frame Or the average value of the motion vector size of the previous frame, the variance of the motion vector size of the predetermined frame or the previous frame, the number of effective blocks of the predetermined frame or the previous frame, the effective coefficient of the predetermined frame or the previous frame Number, the bit rate of the predetermined frame or the previous frame, the code amount of the predetermined frame or the previous frame, the average value of the quantization step size of the predetermined frame or the previous frame, the difference of the average value of the quantization step size (1 Difference in quantization step size of previous frame, or quantization step size of previous frame Difference between the quantization step size of the previous frame and the previous frame), the amount of calculation actually required for decoding the previous frame, and one or more elements of the required amount of calculation of the previous frame calculated by the required calculation amount calculation means It is preferable to calculate the required amount of computation using.
前記複数の要素はそれぞれ符号化又は復号化処理にお!ヽて必要演算量 Kpに影 響を与える要素である。本発明によれば、前記要素のうち一つ以上が必要演算量計 算手段の要素として使用されて必要演算量 Kpが計算されるため、必要演算量計算 手段により計算される必要演算量 Kpが現実に符号化又は復号化処理を行ったとき の演算量により近い値となる。したがって、算出された必要演算量 Kpが現実の演算 量よりも大き過ぎて低消費電力化が阻害される可能性が少なぐまた、必要演算量 K pが現実の演算量よりも小さくて符号ィ匕又は復号ィ匕処理が時間内に完了しないという 破綻現象が発生しにくい。  Each of the plurality of elements is an element that affects the required amount of computation Kp in the encoding or decoding process. According to the present invention, since one or more of the above elements are used as elements of the required amount of calculation means and the required amount of calculation Kp is calculated, the required amount of calculation Kp calculated by the required amount of calculation means is The value is closer to the amount of computation when encoding or decoding is actually performed. Therefore, it is less likely that the calculated required calculation amount Kp is too large compared to the actual calculation amount and the reduction in power consumption is hindered. Also, the required calculation amount Kp is smaller than the actual calculation amount and the sign It is difficult for the failure phenomenon that the defect or decryption process is not completed in time.
例えば 1フレームの符号化処理の必要演算量はマクロブロックマッチングなどの要 素処理の実行回数によって決定づけられる。要素処理はマクロブロックマッチング回 数だけ繰り返し実行される処理，有効ブロックの数だけ繰り返し実行される処理，有 効係数の数だけ繰り返し実行される処理に分類され、マクロブロックマッチング回数と 有効ブロック数と有効係数の数の全てが最大となるとき 1フレームの符号ィ匕処理に必 要な演算量は最大 (最悪）となる。実際の符号ィ匕処理においてマクロブロックマツチン グ回数，有効ブロック数，有効係数の数はフレーム毎に最大値以下で変動するため 、フレーム毎に符号ィ匕処理に必要な演算量は変動することとなる。本願発明では符 号ィ匕処理に必要な演算量がフレーム毎に異なるものの、時間的に近いフレーム間で はマクロブロックマッチング回数，有効ブロック数，有効係数の数，符号化処理の必 要演算量が近い値となる性質に着目し、符号化処理の必要演算量を予測するため に用いるパラメータとして、予測を行う時点ですでに符号ィ匕処理を完了している前フ レームのマクロブロックマッチング回数，有効ブロック数，有効係数の数，符号化処理 の必要演算量などを採用することで各フレームの必要演算量 Kを、実際に符号化処 理を行ったときの演算量に近似した値とすることができる。 For example, the amount of computation required for the encoding process of one frame is determined by the number of executions of element processing such as macroblock matching. Element processing is classified into processing that is repeatedly executed for the number of times of macroblock matching, processing that is repeatedly executed for the number of effective blocks, and processing that is repeatedly executed for the number of effective coefficients. When all of the effective coefficients are maximized, the amount of computation required for one frame of code processing is the maximum (worst). In actual code processing, the number of macroblock matching, the number of effective blocks, and the number of effective coefficients fluctuate below the maximum value for each frame, so the amount of computation required for code processing must vary from frame to frame. It becomes. In the present invention, Although the amount of computation required for signal processing varies from frame to frame, the number of macroblock matching, the number of effective blocks, the number of effective coefficients, and the amount of computation required for encoding processing are close to each other between frames that are close in time. As a parameter used to predict the required computational complexity of the encoding process, the number of macroblock matching and the number of effective blocks of the previous frame that has already completed the encoding process at the time of prediction By adopting the number of effective coefficients and the amount of computation required for the encoding process, the required amount of computation K for each frame can be approximated to the amount of computation when the encoding process is actually performed. .
同様に 1フレームの復号化処理の必要演算量も復号化処理の要素処理の実行回 数によって決定づけられる。要素処理は有効ブロックの数だけ繰り返し実行される処 理、有効係数の数だけ繰り返し実行される処理に分類され、有効ブロック数と有効係 数の数の全てが最大となるとき 1フレームの復号ィ匕処理に必要な演算量は最大 (最 悪）となる。実際の復号ィ匕処理において有効ブロック数，有効係数の数はフレーム毎 に最大値以下で変動するため、フレーム毎に復号化処理に必要な演算量は変動す ることとなる。本願発明では復号ィ匕処理に必要な演算量がフレーム毎に異なるものの 、時間的に近いフレーム間では有効ブロック数，有効係数の数，復号化処理の必要 演算量が近い値となる性質に着目し、復号化処理の必要演算量を予測するために 用いるパラメータとして、予測を行う時点ですでに復号ィ匕処理を完了している前フレ ームの有効ブロック数，有効係数の数，符号化処理の必要演算量などを採用するこ とで各フレームの必要演算量 Kを、実際に復号化処理を行ったときの演算量に近似 した値とすることができる。  Similarly, the amount of computation required for the decoding process for one frame is also determined by the number of executions of the element process of the decoding process. Element processing is classified into processing that is repeatedly executed by the number of effective blocks, and processing that is repeatedly executed by the number of effective coefficients. When all of the number of effective blocks and the number of effective coefficients are maximized, one-frame decoding is performed.演算 The amount of computation required for processing is the maximum (worst). In the actual decoding process, the number of effective blocks and the number of effective coefficients fluctuate below the maximum value for each frame, so the amount of computation required for the decoding process varies for each frame. In the present invention, although the amount of computation required for the decoding process varies from frame to frame, attention is paid to the property that the number of effective blocks, the number of effective coefficients, and the amount of computation required for the decoding process are close to each other between temporally close frames. As the parameters used to predict the required computational complexity of the decoding process, the number of effective blocks, the number of effective coefficients, and the number of effective coefficients of the previous frame that have already been decoded at the time of prediction By adopting the amount of computation required for processing, the required amount of computation K for each frame can be approximated to the amount of computation when decoding is actually performed.
発明の効果 The invention's effect
以上説明したように、本発明の動画像符号化処理システム、動画像符号化又は復 号化処理システム、及び、動画像符号化処理方法、動画像符号化又は復号化処理 方法によれば、これから符号ィ匕又は復号ィ匕する一のフレーム (未来に符号化又は復 号ィ匕するフレーム）に対して、符号ィ匕又は復号ィ匕に要する必要演算量 Kpを予測する 計算を行い、その所定フレームの処理に割り当てられた時間 Te内は、必要演算量 K pを処理するに必要な最小限又はそれに近い動作周波数とし、フレーム単位に動作 周波数と動作電源電圧、又は、動作周波数と基板バイアス電圧、又は、動作周波数 と基板バイアス電圧と動作電源電圧が動的に制御されるため、低消費電力を実現す ることがでさる。 As described above, according to the video encoding processing system, video encoding or decoding processing system, video encoding processing method, video encoding or decoding processing method of the present invention, For one frame to be encoded or decoded (frame to be encoded or decoded in the future), a calculation for predicting the necessary amount of computation Kp required for the code or decoding is performed, and the predetermined value is calculated. The time Te allocated for frame processing is the minimum or close operating frequency required to process the required amount of computation K p, and the operating frequency and operating power supply voltage or operating frequency and substrate bias voltage for each frame. Or operating frequency In addition, since the substrate bias voltage and the operating power supply voltage are dynamically controlled, low power consumption can be realized.
[0050] また、破綻回避手段を備えるため、必要演算量計算手段で算出された必要演算量 Kpが実際に必要な演算量よりも小さい場合に起きる破綻現象を回避することができ 、符号ィ匕又は復号ィ匕処理された動画像が劣悪になるのを防止することができる。さら に、破綻回避手段による処理が実行された場合に後続フレームの符号化処理への 影響を緩和する緩和手段により緩和処理を行うことで、後続フレームの符号化処理 による画質の劣化を抑制することができる。  [0050] Further, since the failure avoiding means is provided, it is possible to avoid the failure phenomenon that occurs when the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount. Alternatively, it is possible to prevent the moving image subjected to the decoding process from being deteriorated. In addition, when processing by the failure avoidance means is executed, mitigation processing is performed by mitigation means that mitigates the effect on subsequent frame encoding processing, thereby suppressing image quality deterioration due to subsequent frame encoding processing. Can do.
図面の簡単な説明  Brief Description of Drawings
[0051] [図 1]本発明の第 1の実施の形態の動画像符号ィ匕処理システムの動作を示した概略 ブロック図。  [0051] FIG. 1 is a schematic block diagram showing the operation of the moving image code processing system according to the first embodiment of the present invention.
[図 2]本発明の第 1の実施の形態の動画像符号ィ匕処理システムの実装例を示す図。  FIG. 2 is a diagram showing an implementation example of a moving image code key processing system according to the first embodiment of the present invention.
[図 3]上記実施の形態の動画像符号ィ匕処理システムとしてコンピュータを機能させる 動画像符号化処理プログラムの概略フローチャートを示す図。  FIG. 3 is a diagram showing a schematic flowchart of a moving image encoding processing program that causes a computer to function as the moving image encoding process system of the embodiment.
[図 4]トリプルゥヱル構造を示す断面図。  FIG. 4 is a cross-sectional view showing a triple tool structure.
[図 5]上記実施の形態の動画像符号ィ匕処理システムに使用されるプロセッサの動作 電源電圧 ·基板バイアス電圧 ·動作周波数を示す概念図。  FIG. 5 is a conceptual diagram showing the operation power supply voltage / substrate bias voltage / operation frequency of the processor used in the moving image code processing system of the embodiment.
[図 6]動作電源電圧及び動作周波数を一定とすることにより低消費電力化を図ること ができることを説明する説明図。  FIG. 6 is an explanatory diagram for explaining that power consumption can be reduced by keeping the operating power supply voltage and operating frequency constant.
[図 7]上記実施の形態における割り込みを行う際の時間と演算残量の関係を説明す る説明図。  FIG. 7 is an explanatory diagram for explaining the relationship between the time for performing an interrupt and the remaining calculation amount in the embodiment.
[図 8]本発明の第 2の実施の形態の動画像符号ィ匕処理システムの動作を示した概略 ブロック図。  FIG. 8 is a schematic block diagram showing the operation of the moving image code processing system of the second embodiment of the present invention.
[図 9]上記実施の形態の動画像符号ィ匕処理システムとしてコンピュータを機能させる 動画像符号化処理プログラムの概略フローチャートを示す図。  FIG. 9 is a diagram showing a schematic flowchart of a moving image encoding processing program that causes a computer to function as the moving image encoding process system of the embodiment.
[図 10]本発明の第 3の実施の形態の動画像復号ィ匕処理システムの動作を示した概 略ブロック図。  FIG. 10 is a schematic block diagram showing an operation of a video decoding process system according to a third embodiment of the present invention.
[図 11]上記実施の形態の動画像符号ィ匕処理システムとしてコンピュータを機能させる 動画像符号ィ匕処置プログラムの概略フローチャートを示す図。 FIG. 11 causes a computer to function as the moving image code processing system of the above embodiment. The figure which shows the schematic flowchart of a moving image code | symbol treatment program.
[図 12]通常の符号化処理の処理時間推移を示す図。  FIG. 12 is a graph showing a processing time transition of normal encoding processing.
[図 13]第 3の実施の形態における符号ィ匕処理の処理時間推移を示す図。  FIG. 13 is a diagram showing a transition of processing time of sign key processing in the third embodiment.
[図 14]第 3の実施の形態における符号ィ匕処理の処理時間推移を示す図。  FIG. 14 is a diagram showing a processing time transition of the sign key processing in the third embodiment.
[図 15]第 3の実施の形態における符号ィ匕処理の処理時間推移を示す図。  FIG. 15 is a diagram showing a transition of processing time of sign key processing in the third embodiment.
[図 16]本発明の第 4の実施の形態の動画像復号ィ匕処理システムの動作を示した概 略ブロック図。  FIG. 16 is a schematic block diagram showing the operation of the moving image decoding system according to the fourth embodiment of the present invention.
[図 17]本発明の第 5の実施の形態の動画像符号ィ匕処理システムの動作を示した概 略ブロック図。  FIG. 17 is a schematic block diagram showing the operation of the moving picture code processing system according to the fifth embodiment of the present invention.
[図 18]上記実施の形態の動画像符号ィ匕処理システムに使用されるプロセッサの動作 電源電圧'動作周波数の関係を説明する概念図。  FIG. 18 is a conceptual diagram illustrating the relationship between the operation power supply voltage and the operation frequency of the processor used in the moving image code processing system of the above embodiment.
[図 19]本発明の第 6の実施の形態の動画像符号ィ匕処理システムの動作を示した概 略ブロック図。  FIG. 19 is a schematic block diagram showing an operation of a moving image code processing system according to a sixth embodiment of the present invention.
[図 20]上記実施の形態の動画像符号ィ匕処理システムに使用されるプロセッサの基板 バイアス電圧 ·動作周波数の関係を説明する概念図。  FIG. 20 is a conceptual diagram for explaining a relationship between a substrate bias voltage and an operating frequency of a processor used in the moving image code processing system of the embodiment.
[図 21]実施例におけるプロセッサの動作周波数と動作電源電圧、基板バイアス電圧 の関係の例を示す図。  FIG. 21 is a diagram showing an example of the relationship between the operating frequency of the processor, the operating power supply voltage, and the substrate bias voltage in the example.
[図 22]動画像符号ィ匕処理システムについて従来の低消費電力化を行う手法を示した 図。  FIG. 22 is a diagram showing a conventional technique for reducing power consumption in a moving image code processing system.
[図 23]フレーム単位に動画像符号ィ匕又は復号ィ匕の演算量が異なる状態を示す概念 図。  FIG. 23 is a conceptual diagram showing a state in which the amount of calculation of a moving image code key or decoding key varies from frame to frame.
符号の説明 Explanation of symbols
S1(S11, S12, S13, S14), S2, S3, S5, S6 動画像符号ィ匕処理システム S4 動画像復号化処理システム S1 (S11, S12, S13, S14), S2, S3, S5, S6 Video code processing system S4 Video decoding system
1, 51, 61 プロセッサ  1, 51, 61 processor
2, 42 必要演算量計算手段  2, 42 Required computational complexity calculation means
3, 53, 63 動作決定手段  3, 53, 63 Action determining means
4, 54, 64 動作制御手段 5 動画像符号化手段 4, 54, 64 Operation control means 5 Video coding means
45 動画像復号化手段 45 Video decoding means
6 局部復号フレームメモリ  6 Local decoding frame memory
7 入力フレームメモリ  7 Input frame memory
8 要素メモリ  8-element memory
91, 92, 93, 94, 95 第 1の破綻回避手段  91, 92, 93, 94, 95
96 第 2の破綻回避手段  96 Second bankruptcy avoidance measure
97, 98 第 3の破綻回避手段  97, 98 Third method of avoiding bankruptcy
10 処理済マクロブロック数レジスタ  10 Processed macroblock count register
101 入力画像データ  101 Input image data
102 動作電源電圧 ·基板バイアス電圧 ·動作周波数指示  102 Operating supply voltage · Substrate bias voltage · Operating frequency indication
103 局部復号データ 103 Local decoding data
104 動作電源電圧 ·基板バイアス電圧 ·動作周波数指示  104 Operating power supply voltage / Substrate bias voltage / Operating frequency indication
105 動作電源電圧 ·基板バイアス電圧 ·動作周波数供給 105 Operating supply voltage · Substrate bias voltage · Operating frequency supply
106 符号化データ 106 Encoded data
107 前フレームの量子化ステップサイズの平均値、  107 Average value of quantization step size of previous frame,
108 各フレームについてフレーム内符号ィ匕であるカ レーム間符号ィ匕である力: の種類  108 The type of force that is the inter-frame code 力 that is the intra-frame code に つ い て for each frame:
109 動画像の符号ィ匕ビットレート  109 Video code rate
110 前フレーム（過去のフレーム）のアクティビティの量  110 Amount of activity in the previous frame (past frame)
111 前フレームのマクロブロックマッチング回数  111 Macroblock matching count for previous frame
112 前フレームの有効ブロック数  112 Number of valid blocks in previous frame
113 前フレームの有効係数の数  113 Number of effective coefficients of previous frame
114 前フレームの量子化ステップサイズの平均値とその一つ前のフレームの量 子ィ匕ステップサイズの平均値の差  114 Difference between the average value of the quantization step size of the previous frame and the amount of the previous frame.
115 前フレームの符号化に実際に要した処理量  115 The amount of processing actually required to encode the previous frame
116 必要演算量計算手段により算出された前フレームの必要演算量  116 Required calculation amount of the previous frame calculated by the required calculation amount calculation means
117 処理済マクロブロック数 45 動画像復号化手段 117 Number of processed macroblocks 45 Video decoding means
46 局部復号フレームメモリ  46 Local decoding frame memory
401 入力符号化データ  401 Input encoded data
406 復号化データ  406 Decrypted data
502 動作電源電圧 ·動作周波数指示  502 Operating power supply voltage and operating frequency indication
505 動作電源電圧 ·動作周波数供給  505 Operating power supply voltage and operating frequency supply
602 基板バイアス電圧 ·動作周波数指示  602 Substrate bias voltage and operating frequency indication
605 基板バイアス電圧 ·動作周波数供給  605 Substrate bias voltage and operating frequency supply
p— sub P型半導体基板 p—sub P-type semiconductor substrate
n— well n型ゥエル n—well n- type
p— well p型ゥエル p—well p-type uel
発明を実施するための最良の形態 BEST MODE FOR CARRYING OUT THE INVENTION
以下、本発明の動画像符号化又は復号化処理システム及び動画像符号化又は復 号化処理方法について説明する。本発明の動画像符号ィ匕又は復号ィ匕システムは、 後述するプロセッサ 1が動画像符号化処理及び動画像復号化処理を行うものであり 、動画像符号化を行う場合は動画像符号化処理システムとして機能し、動画像復号 化を行う場合を動画像復号化処理システムとして機能する。たとえば、本発明の動画 像符号ィ匕又は復号ィ匕処理システムとしては、フレーム単位若しくは時間単位で符号 化又は復号ィ匕を行うものでも良ぐまた、復号ィ匕処理のみ又は符号ィ匕処理のみを行う ものでも良い。また，プロセッサ 1は動画像符号化処理又は動画像復号化処理をより 少ないサイクル数，より小さい消費電力，より小さいプログラムコード量で効率的に実 現するための動画像処理専用の演算器 (命令）を備えて!/、てもよ 、.動画像処理専 用の演算器 (命令)の例として，離散コサイン変換処理などの行列演算で用いる積和 演算器 (積和演算命令），動きベクトル検出処理でのブロックマッチング演算などの演 算で用いる差分絶対値和演算器 (差分絶対値和命令)などがあげられる.以下、説 明の便宜上、符号化を行う場合を動画像符号化システムとし、復号化を行う場合を動 画像復号化システムとし、動画像符号化処理と動画像復号化処理に分けて詳述する [0054] (第 1の実施の形態） Hereinafter, the moving image encoding or decoding processing system and moving image encoding or decoding processing method of the present invention will be described. The moving image encoding / decoding system of the present invention is such that a processor 1 described later performs moving image encoding processing and moving image decoding processing. When moving image encoding is performed, moving image encoding processing is performed. It functions as a system and functions as a video decoding processing system when performing video decoding. For example, the moving image encoding / decoding processing system of the present invention may be one that performs encoding or decoding in units of frames or in units of time, or only decoding processing or only encoding processing. It may be something that performs. In addition, processor 1 is a computing unit dedicated to moving image processing (command) for efficiently performing moving image coding processing or moving image decoding processing with a smaller number of cycles, smaller power consumption, and smaller program code amount. As an example of an arithmetic unit (instruction) dedicated to moving image processing, a product-sum arithmetic unit (product-sum operation instruction) used in matrix operations such as discrete cosine transform processing, motion vectors The difference absolute value sum operator (difference absolute value sum command) used in operations such as block matching operations in the detection process is given below. The case where decoding is performed is a moving image decoding system, which will be described in detail separately for moving image encoding processing and moving image decoding processing. [0054] (First embodiment)
本発明の第 1の実施の形態の動画像符号化処理システム S1は、必要演算量計算 手段 2で算出された必要演算量 Kpが所定フレームの現実の演算量よりも小さい場合 に生ずる、所定フレームの処理に割り当てられた時間内に処理が完了できないという 破綻現象の問題を解決するためのものであり、所定フレームの符号化処理に予め割 り当てられて 、る時間内に所定フレームの符号ィ匕処理が終了しな 、と判断した場合 には破綻現象を回避する処理を行う第 1の破綻回避手段 9を備え、動作周波数と基 板バイアス電圧と動作電源電圧をフレーム単位に一定に制御する（すなわち、所定 フレームの符号ィ匕処理または復号ィ匕処理に割り当てられた時間内はプロセッサがー 定の動作周波数 ·一定の基板バイアス電圧 ·一定の動作電源電圧で動作し、所定フ レームの符号化処理または復号化処理中は動作周波数や基板バイアス電圧や動作 電源電圧を変更しない）ことにより、サブスレツショルドリーク電流と充放電電流とを適 度に抑制し、低消費電力化を図るものである。本システム S1は、例えばマイクロコン ピュータが内蔵された携帯電話やパーソナルコンピュータ等の情報端末機器である コンピュータにより実現され、特に、そのコンピュータ内においてマルチメディア信号 処理部などの一部として機能するシステムであり、連続する所定数のフレーム力 構 成される動画像をフレーム単位で順次符号ィ匕を行うシステムである。  The moving image coding processing system S1 according to the first embodiment of the present invention has a predetermined frame that occurs when the required calculation amount Kp calculated by the required calculation amount calculation means 2 is smaller than the actual calculation amount of the predetermined frame. In order to solve the problem of the failure phenomenon that the process cannot be completed within the time allotted to the process, the code of the predetermined frame is allocated in advance to the encoding process of the predetermined frame. When it is determined that the process has not been completed, the first failure avoiding means 9 is provided to perform processing to avoid the failure phenomenon, and the operating frequency, the base bias voltage, and the operating power supply voltage are controlled to be constant for each frame. (That is, the processor operates at a constant operating frequency, a constant substrate bias voltage, and a constant operating power supply voltage within the time allotted to the encoding process or decoding process of a given frame. By not changing the operating frequency, substrate bias voltage, or operating power supply voltage during the encoding or decoding process for a given frame, the subthreshold leakage current and charging / discharging current are appropriately suppressed, resulting in low power consumption. It aims to make it easier. This system S1 is realized by a computer which is an information terminal device such as a mobile phone or a personal computer with a built-in microcomputer, for example, and is particularly a system that functions as a part of a multimedia signal processing unit in the computer. There is a system that sequentially encodes a moving image composed of a predetermined number of frame forces in units of frames.
[0055] 図 1は、本実施の形態の動画像符号ィ匕処理システム S1の動作を示した概略ブロッ ク図であり、図 3は本システム S1により実現される動画像符号ィ匕処理方法である。動 画像符号化処理システム S1は、動作電源電圧 Vdd及び基板バイアス電圧 Vbn, Vb P及び動作周波数 F力 Sr段階 (rは 2以上の整数）に可変であり（すなわち、 r段階の動 作電源電圧 Vdd及び基板バイアス電圧 Vbn、 Vbp及び動作周波数 Fで動作可能で あり）且つプログラムにより動作電源電圧及び基板バイアス電圧及び動作周波数を変 更可能なプロセッサ 1と、プロセッサ 1の動作電源電圧及び基板バイアス電圧及び動 作周波数を制御する動作制御手段 4と、所定のデータを記憶する記憶領域である局 部復号フレームメモリ 6と入力フレームメモリ 7と要素メモリ 8とを少なくとも備えるコンビ ユータ (特にコンピュータ内のマルチメディア信号処理部）である。ただし、 Vbnは n— チャネル MOSトランジスタの基板バイアス電圧、 Vbpは p—チャネル MOSトランジス タの基板バイアス電圧である。局部復号メモリ 6および入力フレームメモリ 7等は、動 作制御手段 4によりプロセッサ 1と同様に動作電圧'動作周波数'基板バイアス電圧 が制御されてもよい。本実施の形態では、点線で示される制御領域 CAに含まれる要 素（プロセッサ 1、局部復号フレームメモリ 6、要素メモリ 8、処理済みマクロブロック数 レジスタ 10、入力フレームメモリ 7a, 7b、等）について、動作周波数と動作電圧と基 板バイアス電圧が制御される。 [0055] Fig. 1 is a schematic block diagram showing the operation of the video code key processing system S1 of the present embodiment, and Fig. 3 shows a video code key processing method realized by the system S1. is there. The moving image coding processing system S1 is variable in operating power supply voltage Vdd and substrate bias voltage Vbn, Vb P and operating frequency F force S r (r is an integer of 2 or more) (ie, r operating power supply) Voltage Vdd and substrate bias voltage Vbn, Vbp, and operating frequency F), and the operating power supply voltage and substrate bias voltage and operating frequency can be changed by the program, and the operating power supply voltage and substrate bias of processor 1 A computer (in particular, a computer having at least an operation control means 4 for controlling a voltage and an operating frequency, and a local decoding frame memory 6, an input frame memory 7, and an element memory 8 which are storage areas for storing predetermined data. Multimedia signal processing unit). Where Vbn is the substrate bias voltage of the n-channel MOS transistor and Vbp is the p-channel MOS transistor. Substrate bias voltage. In the local decoding memory 6 and the input frame memory 7 and the like, the operation voltage “operation frequency” and the substrate bias voltage may be controlled by the operation control means 4 in the same manner as the processor 1. In the present embodiment, the elements (processor 1, local decoding frame memory 6, element memory 8, processed macroblock number register 10, input frame memories 7a, 7b, etc.) included in control area CA indicated by dotted lines The operating frequency, operating voltage, and substrate bias voltage are controlled.
[0056] プロセッサ 1は、トリプルゥエル構造をとる半導体素子であり、 nMOSトランジスタ、 p MOSトランジスタともに基板バイアス電圧が制御可能となっている。局部復号メモリ 6 及び入力フレームメモリ 7は半導体記憶素子であり、動作制御手段 4により、プロセッ サ 1と同様に動作電源電圧 ·基板バイアス電圧 ·動作周波数が制御される。 The processor 1 is a semiconductor element having a triple-well structure, and the substrate bias voltage can be controlled for both the nMOS transistor and the pMOS transistor. The local decoding memory 6 and the input frame memory 7 are semiconductor memory elements, and the operation control means 4 controls the operation power supply voltage / substrate bias voltage / operation frequency in the same manner as the processor 1.
[0057] 図 4はトリプルゥエル構造のプロセッサ 1の部分断面図である。プロセッサ 1は、 P型 半導体基板 p— subに n型ゥエル n— wellを形成し、さらに、 n型ゥエル n— wellに p型 ゥエル p— wellを形成することによってトリプルゥエル構造としたものである。 p型ゥェ ル p— wellには、 n—チャネル MOSトランジスタと p型ゥエルコンタクト層 p— Contactと が形成されている。 n—チャネル MOSトランジスタは、 n型の不純物層からなるソース Zドレイン層 S, Dと、ゲート電極 Gとを有する。 n型ゥエル n— wellには、 p—チャネル MOSトランジスタと n型ゥエルコンタクト層 n— Contactとが形成されている。 n—チヤ ネル MOSトランジスタは、 p型の不純物層力 なるソース Zドレイン層 S, Dと、ゲート 電極 Gとを有する。 n—チャネル MOSトランジスタが形成される半導体領域である p 型ゥエル p— wellには p型ゥエルコンタクト層 p— Contactを介して基板バイアス電圧 V bnが印加される。 p—チャネル MOSトランジスタが形成される半導体領域である n型 ゥエル n— wellには n型ゥエルコンタクト層 n— Contactを介して基板バイアス電圧 Vb pが印加される。 FIG. 4 is a partial cross-sectional view of the processor 1 having a triple-well structure. The processor 1 has a triple-well structure by forming an n-type well n-well on a p-type semiconductor substrate p-sub and further forming a p-type well p-well on an n-type well n-well. is there. In the p-type well p-well, an n-channel MOS transistor and a p-type well contact layer p-Contact are formed. The n-channel MOS transistor has a source Z drain layer S, D composed of an n-type impurity layer, and a gate electrode G. In the n-type n-well, a p-channel MOS transistor and an n-type contact layer n-contact are formed. The n-channel MOS transistor has a source Z drain layer S, D having a p-type impurity layer force, and a gate electrode G. A substrate bias voltage V bn is applied to a p-type well p-well, which is a semiconductor region where an n-channel MOS transistor is formed, via a p-type contact contact layer p-Contact. A substrate bias voltage Vb p is applied to an n- type well n — well, which is a semiconductor region where a p-channel MOS transistor is formed, via an n- type well contact layer n — Contact.
[0058] 動作制御手段 4は、 DC— DCコンバータなどを備えた動作電源電圧制御手段 4c、 n-チャネル MOSトランジスタの基板バイアス電圧を制御するための基板バイアス電 圧 Vbn発生手段 4d、 p-チャネル MOSトランジスタの基板バイアス電圧を制御するた めの基板バイアス電圧 Vbp発生手段 4e、 PLLなどを備えた動作周波数制御手段 4b 力もなる。ただし、動作制御手段 4の各要素は動画像符号ィ匕処理システム SIの外に 存在し、動画像符号化処理システム S1の外から動作電源電圧または基板バイアス 電圧または動作周波数を制御してもよい。プロセッサ 1、各メモリ 6, 7、動作制御手段 4は互 、に配線を介して接続されて!ヽる。 [0058] The operation control means 4 includes an operation power supply voltage control means 4c having a DC-DC converter, etc., a substrate bias voltage Vbn generation means 4d for controlling the substrate bias voltage of the n-channel MOS transistor, p-channel Substrate bias voltage Vbp generation means 4e for controlling the substrate bias voltage of MOS transistors 4e, operating frequency control means 4b with PLL, etc. There will be power. However, each element of the operation control means 4 may exist outside the moving image encoding processing system SI, and may control the operating power supply voltage, the substrate bias voltage, or the operating frequency from outside the moving image encoding processing system S1. . The processor 1, the memories 6 and 7, and the operation control means 4 are connected to each other via wiring.
[0059] プロセッサ 1は、プロセッサ 1上で動作する手段として、必要演算量計算手段 2と、動 作決定手段 3と、動画像符号化手段 5と第 1の破綻回避手段 9を備える。なお、符号 1 01は入力画像データ、符号 102は動作電源電圧及び基板バイアス電圧及び動作周 波数指示、符号 103は前フレームの局部復号データ、符号 105は動作電源電圧'基 板バイアス電圧'動作周波数供給、符号 106はフレームの符号化データ、符号 107 は前フレームの量子化ステップサイズの平均値の情報、符号 108は各フレームにつ いてフレーム内符号ィ匕である力フレーム間符号ィ匕であるかの種類、符号 109は動画 像の符号ィ匕ビットレートの情報、符号 110は前フレームのアクティビティ量、符号 111 は前フレームのマクロブロックマッチング回数、符号 112は前フレームの有効ブロック 数、符号 113は前フレームの有効係数の数、符号 114は前フレームの量子化ステツ プサイズの平均値とその一つ前のフレームの量子化ステップサイズの平均値の差、 符号 115は前フレームの符号ィ匕に実際に要した処理量、符号 116は必要演算量計 算手段 2により算出された前フレームの必要演算量である。要素メモリ 8は、後述する 必要演算量計算手段 2において使用される複数の要素のうち一部の要素（フレーム 内符号ィ匕であるかフレーム間符号ィ匕であるかの種類 108や、符号ィ匕ビットレート 109 や、フレームのアクティビティの量 110や、必要演算量計算手段 2により算出された必 要演算量 116)が記憶される記憶領域である。処理済マクロブロック数レジスタ 10は 、符号ィ匕処理済みのマクロブロック数 117の情報を一時的に蓄積するレジスタである 。動画像符号ィ匕手段 5には符号ィ匕方式として MPEG— 4が使用されるが、 H. 26X や MPEG— 1、 MPEG 2などの他の符号化方式が使用されて!、ても良!、。  The processor 1 includes necessary calculation amount calculation means 2, operation determination means 3, moving picture encoding means 5, and first failure avoidance means 9 as means that operate on the processor 1. Reference numeral 101 is input image data, reference numeral 102 is an operation power supply voltage and substrate bias voltage and operation frequency instruction, reference numeral 103 is local decoded data of the previous frame, reference numeral 105 is operation power supply voltage 'base bias voltage' operation frequency Supply, code 106 is the encoded data of the frame, code 107 is information on the average value of the quantization step size of the previous frame, and code 108 is a force inter-frame code that is an intra-frame code for each frame. The code 109 is the information on the video code bit rate, the code 110 is the activity amount of the previous frame, the code 111 is the number of macroblock matching in the previous frame, the code 112 is the number of effective blocks in the previous frame, and the code 113 Is the number of effective coefficients of the previous frame, and reference numeral 114 is the average of the quantization step size of the previous frame and the average of the quantization step size of the previous frame. Difference between the values, symbols 115 are code processing amount actually required to I spoon, reference numeral 116 necessary calculation amount of the previous frame calculated by necessary calculation amount calculations means 2 of the previous frame. The element memory 8 is a part of a plurality of elements used in the required amount-of-computation calculation means 2 described later (intra-frame code 匕 type 108 or code ィ. This is a storage area for storing the bit rate 109, the amount of frame activity 110, and the required amount of calculation 116) calculated by the required amount of calculation calculation means 2. The processed macroblock number register 10 is a register that temporarily stores information on the number of macroblocks 117 that have been subjected to sign processing. MPEG-4 is used as the video encoding method for moving image encoding means 5, but other encoding schemes such as H.26X, MPEG-1 and MPEG 2 can be used! ,.
[0060] 図 2に動画像符号ィ匕処理システム S1の実装例を示す。システム S1は、主にプロセ ッサ 1と、周辺装置として各種メモリ MR, 7a, 7bや各種インタフェース CI, DI, BI、動 作制御回路 4a、 PLL4b、 DC— DCコンバータ 4c、基板バイアス電圧発生回路 4d, 4e等を備えたハードウェアにより実現される。上記各構成要素は、バス Bl, B2等を 介して互!、に通信可能となって 、る。 FIG. 2 shows an implementation example of the video code key processing system S1. System S1 mainly consists of processor 1, various memories MR, 7a, 7b as peripheral devices, various interfaces CI, DI, BI, operation control circuit 4a, PLL4b, DC-DC converter 4c, substrate bias voltage generation circuit Realized by hardware equipped with 4d, 4e, etc. Each of the above components includes buses Bl, B2, etc. It becomes possible to communicate with each other via!
[0061] プロセッサ 1は、プロセッサコア laと、命令キャッシュメモリ lbと、データキャッシュメ モリ lcと、バスコントローラ BCとを備える。必要演算量計算手段 2,動作決定手段 3, 動画像符号化手段 5,破綻回避手段 9, 11は、メモリ MRに格納されたプログラムが 必要に応じてプロセッサコァ 1 a上で実行されることにより実現される。命令キャッシュ メモリ lbおよびデータキャッシュメモリ lcは、プロセッサコア la上で実行されるプログ ラムの処理の高速ィ匕を図るために設けられたキャッシュメモリである。  [0061] The processor 1 includes a processor core la, an instruction cache memory lb, a data cache memory lc, and a bus controller BC. Necessary calculation amount calculation means 2, motion determination means 3, video encoding means 5, failure avoidance means 9, and 11 are executed by executing a program stored in the memory MR as necessary on the processor core 1a. Realized. The instruction cache memory lb and the data cache memory lc are cache memories provided for high-speed processing of programs executed on the processor core la.
[0062] 局部復号フレームメモリ 6,要素メモリ 8,処理済みマクロブロック数レジスタ 10は、 図 2のメモリ MRに集約されるとともに、前フレームの量子化ステップサイズの平均値 1 07,各フレームにつ 、てフレーム内符号化である力フレーム間符号化であるかの種 類 108,動画像符号化のビットレート 109,前フレーム（過去のフレーム）のァクテイビ ティの量 110,前フレームのマクロブロックマッチング回数 111,前フレームの有効ブ ロック数 112,前フレームの有効係数の数 113,前フレームの量子化ステップサイズ の平均値とその一つ前のフレームの量子化ステップサイズの平均値の差 114,前フ レームの符号ィ匕に実際に要した処理量 115,必要演算量計算手段により算出された 前フレームの必要演算量 116,処理済みマクロブロック数 117はメモリ MRにデータと して格納される。局部復号データ 103は、バスコントローラ BCを介してメモリ MRとプ 口セッサコア la間で信号 100j, 100k, 1001として送受信される。  [0062] The local decoding frame memory 6, the element memory 8, and the processed macroblock number register 10 are aggregated in the memory MR of Fig. 2 and the average value of the quantization step size of the previous frame 107, for each frame. Type of intra-frame coding or inter-frame coding 108, moving picture coding bit rate 109, previous frame (past frame) activity amount 110, previous frame macroblock matching 111, number of effective blocks of previous frame 112, number of effective coefficients of previous frame 113, difference between average value of quantization step size of previous frame and average value of quantization step size of previous frame 114, The actual amount of processing required for the sign of the previous frame 115, the required amount of previous frame 116 calculated by the required amount calculation means, and the number of processed macro blocks 117 are stored in the memory MR. It is stored in. Locally decoded data 103 is transmitted and received as signals 100j, 100k, and 1001 between memory MR and processor core la via bus controller BC.
[0063] 二つの入力フレームメモリ 7a, 7bは、図 1の入力フレームメモリ 7に相当する。カメラ インタフェース CIから入力されたビデオデータ（入力画像データ 101)は、バス B2を 介して入力フレームメモリ 7a (又は入力フレームメモリ 7b)に入力される。入力フレー ムメモリ（ # 0) 7aと入力フレームメモリ（ # 1) 7bは 1フレームの処理が終わるごとに用 途が入れ替わる。すなわち、 i番目のフレームの処理で、信号 100hにより入力フレー ムメモリ（ # 1) 7bに入力画像データが書き込まれ、動画像符号化処理手段による符 号ィ匕処理のために信号 100οにより入力フレームメモリ（# 0) 7aから入力画像データ が読み出されたとき、（i+ 1)番目のフレームの処理では、信号 100iにより入力フレー ムメモリ（ # 0) 7aに入力画像データが書き込まれ、動画像符号化処理手段による符 号化処理のために、信号 100pにより入力フレームメモリ（# 1) 7bから入力画像デー タが読み出される。したがって、信号 lOOhにより入力フレームメモリ（ # 1) 7bに入力 画像データが書き込まれているときは信号 ΙΟΟρが発生せず、逆に信号 ΙΟΟρにより 画像が読み出されているときは信号 100hが発生しない。同様に、信号 100iにより入 カフレームメモリ（# 0) 7aに入力画像データが書き込まれているときは信号 100οが 発生せず、信号 100οにより入力フレームメモリ（ # 0) 7aから入力画像データが読み 出されているときは信号 100iが発生しない。このとき、 i番目のフレームの処理におい ては入力フレームメモリ（ # 0) 7aが、（i+ 1)番目のフレームの処理にぉ 、ては入力 フレームメモリ（ # 1) 7bが動作周波数，基板バイアス電圧及び動作電圧の制御対象 となる。上記説明のように、入力フレームメモリを 2フレーム分用意し、それぞれの動 作周波数を独立に設定できるようにすることで、常に一定の動作周波数であるカメラ インタフェース CIからの入力画像データの書き込み動作と、必要演算量の算出値に 基づいて動作周波数が変動する入力画像データの読み出し動作を、互いに妨げる ことなく実行することがでさる。 The two input frame memories 7a and 7b correspond to the input frame memory 7 of FIG. Video data (input image data 101) input from the camera interface CI is input to the input frame memory 7a (or input frame memory 7b) via the bus B2. The input frame memory (# 0) 7a and the input frame memory (# 1) 7b are switched in use every time one frame is processed. In other words, in the processing of the i-th frame, the input image data is written to the input frame memory (# 1) 7b by the signal 100h, and the input frame memory is input by the signal 100ο for the encoding process by the moving image encoding processing means. When the input image data is read from (# 0) 7a, the input image data is written to the input frame memory (# 0) 7a by the signal 100i in the processing of the (i + 1) -th frame, and the moving image is encoded. For encoding processing by the processing means, input image data is input from the input frame memory (# 1) 7b by the signal 100p. Data is read. Therefore, the signal ΙΟΟρ does not occur when the input image data is written to the input frame memory (# 1) 7b by the signal lOOh, and conversely, the signal 100h does not occur when the image is read by the signal ΙΟΟρ. . Similarly, when input image data is written to the input frame memory (# 0) 7a by the signal 100i, the signal 100ο is not generated, and the input image data is read from the input frame memory (# 0) 7a by the signal 100ο. The signal 100i is not generated when it is output. At this time, the input frame memory (# 0) 7a is used for the processing of the i-th frame, and the input frame memory (# 1) 7b is used for the processing of the (i + 1) -th frame. Voltage and operating voltage are subject to control. As described above, the input frame memory is prepared for two frames, and each operation frequency can be set independently, so that the input image data is always written from the camera interface CI at a constant operation frequency. As a result, it is possible to execute the reading operation of the input image data whose operating frequency varies based on the calculated value of the necessary calculation amount without interfering with each other.
動作制御回路 4aは、 PLL4b, DC— DCコンバータ 4c,基板バイアス電圧発生回 路 4d, 4eと互いに信号を送受信可能となっており、これらは動作制御手段 4として機 能している。動作制御回路 4aは、プロセッサコア laからの信号 lOOeにより動作電源 電圧'基板バイアス電圧'動作周波数指示 102を受け、その指示 102に基づいて PL L4bに対して信号 lOOuを発信し、 DC— DCコンバータ 4cに対して信号 ΙΟΟνを発信 し，基板バイアス電圧発生回路 4dに対して信号 lOOwを発信し，基板バイアス電圧 発生回路 4eに対して信号 ΙΟΟχを発信する。 PLL4bは信号 lOOuに基づいて動作 周波数信号 100aを発信し、 DC— DCコンバータ 4cは信号 ΙΟΟνに基づいて動作電 源電圧 100bを供給し，基板バイアス電圧発生回路 4dは信号 lOOwに基づ 、て nM OS基板バイアス電圧 100cを供給し，基板バイアス電圧発生回路 4eは信号 ΙΟΟχに 基づいて pMOS基板バイアス電圧 100dを供給する。これにより、図 2において点線 で示される制御領域 CAに含まれる要素（プロセッサ 1、メモリ MR、入力フレームメモ リ 7a, 7b、バスコントローラ BC等）について、動作周波数と基板バイアス電圧と動作 電源電圧力 s制御される。信号 100e, lOOj, 100k, 1001, 100m, 100ο, lOOp, 10 Oq, lOOr, 100sは、 PLL4bが出力する動作周波数信号 100a, DC— DCコンパ一 タ 4cが出力する電源電圧供給 100b,基板バイアス電圧発生回路 4dが出力する nM OS基板バイアス電圧 100c,基板バイアス電圧発生回路 4eが出力する pMOS基板 バイアス電圧 100dの値に応じて周波数と信号レベルが変化する。 The operation control circuit 4a can transmit and receive signals to and from the PLL 4b, the DC-DC converter 4c, and the substrate bias voltage generation circuits 4d and 4e, and these function as the operation control means 4. The operation control circuit 4a receives the operation power supply voltage 'substrate bias voltage' operation frequency instruction 102 by the signal lOOe from the processor core la, and transmits the signal lOOu to the PL L4b based on the instruction 102, and the DC-DC converter Signal ΙΟΟν is transmitted to 4c, signal lOOw is transmitted to substrate bias voltage generation circuit 4d, and signal ΙΟΟχ is transmitted to substrate bias voltage generation circuit 4e. The PLL 4b transmits the operating frequency signal 100a based on the signal lOOu, the DC-DC converter 4c supplies the operating power voltage 100b based on the signal ΙΟΟν, and the substrate bias voltage generating circuit 4d generates nM based on the signal lOOw. The OS substrate bias voltage 100c is supplied, and the substrate bias voltage generation circuit 4e supplies the pMOS substrate bias voltage 100d based on the signal ΙΟΟχ. As a result, for the elements (processor 1, memory MR, input frame memories 7a and 7b, bus controller BC, etc.) included in the control area CA indicated by the dotted line in FIG. s controlled. Signals 100e, lOOj, 100k, 1001, 100m, 100ο, lOOp, 10 Oq, lOOr, 100s are the operating frequency signals output by PLL4b 100a, DC—DC comparator Power supply voltage output 100b output from the substrate 4c, nM OS substrate bias voltage 100c output from the substrate bias voltage generation circuit 4d, pMOS substrate output from the substrate bias voltage generation circuit 4e The frequency and signal level depend on the value of the bias voltage 100d Change.
[0065] プロセッサ 1上で動作する動画像符号ィ匕手段 5による符号ィ匕後の符号ィ匕データ 10 6は、バス B1を介してビットストリームインタフェース BIに信号 100mとして送信されて 信号 100ηとして出力され、符号ィ匕処理の過程で生成される局部復号データ 106は 局部復号フレームメモリ 6として機能するメモリ MRに信号 100jとして送信される。また 、画像のデータなどは、バス B1を介してメモリから信号 100qとして読み出され、ディ スプレイインタフェース DIに送信される。ディスプレイインタフェース DIに受信された 信号 100qは、信号 100tによるビデオデータとして出力される。ビデオデータは、ディ スプレイインタフェース DIと接続されるモニタを介して、動画像として出力'表示される [0065] Code key data 10 6 after being encoded by the moving image code key means 5 operating on the processor 1 is transmitted as a signal 100m to the bit stream interface BI via the bus B1 and output as a signal 100η. Then, the local decoded data 106 generated in the process of the code decoding process is transmitted as a signal 100j to the memory MR functioning as the local decoded frame memory 6. Also, image data and the like are read out from the memory as a signal 100q via the bus B1 and transmitted to the display interface DI. The signal 100q received by the display interface DI is output as video data by the signal 100t. Video data is output and displayed as moving images via a monitor connected to the display interface DI.
[0066] 動作制御回路 4a,ディスプレイインタフェース DI,ビットストリームインタフェース BI は常に一定の動作電源電圧で動作するが、これらの間で送受信される信号 100e, 1 00q, 100mは制御領域 CAに含まれる要素（プロセッサ 1やメモリ MRや入力フレー ムメモリ 7a, 7b等）の基板バイアス電圧と動作電源電圧の変更に応じて信号レベル が変動する。この影響を吸収するために、動作制御回路 4a,ディスプレイインタフエ ース DI,ビットストリームインタフェース BIは、受信した信号 100e, 100q, 100mの信 号レベルを補正するレベルコンバータを備えることが望ましい。 [0066] The operation control circuit 4a, the display interface DI, and the bit stream interface BI always operate at a constant operation power supply voltage, but signals 100e, 100q, and 100m transmitted and received between them are elements included in the control area CA. The signal level fluctuates according to changes in the substrate bias voltage and operating power supply voltage (processor 1, memory MR, input frame memory 7a, 7b, etc.). In order to absorb this influence, it is desirable that the operation control circuit 4a, the display interface DI, and the bit stream interface BI include a level converter that corrects the signal level of the received signals 100e, 100q, and 100m.
[0067] 次に、本実施の形態の動画像符号化処理システム S1の動作を説明する。動画像 符号ィ匕処理システム S1は、動画像符号ィ匕処理プログラム Prglによりコンピュータ (特 にコンピュータ内のマルチメディア信号処理部）を下記の所定の手段として機能させ ることにより実現される。また、本システム S1により、下記のステップ力 なる本発明の 動画像符号化方法が実現される。以下、順次符号化されるフレームのうちこれから符 号ィ匕される任意の一のフレームを所定フレーム（すなわち、あるフレームが符号ィ匕さ れた時点を基準とすると次に符号ィ匕されるフレームであり、換言すると、その時点に おいて未だに符号ィ匕処理されておらず未来に符号ィ匕処理が行われる予定であるフ レーム、現フレームともいう）、所定フレームより前に符号ィ匕された一のフレーム (過去 に符号ィ匕されたフレーム）を前フレームとし、所定フレームを符号化する処理にっ ヽ て説明するが、 V、ずれのフレームにつ 、ても同様の処理が行われる。 [0067] Next, the operation of the moving image coding processing system S1 of the present embodiment will be described. The moving image code processing system S1 is realized by causing a computer (in particular, a multimedia signal processing unit in the computer) to function as the following predetermined means by the moving image code processing program Prgl. Further, the system S1 realizes the moving picture coding method of the present invention having the following step force. Hereinafter, an arbitrary frame to be encoded from among the sequentially encoded frames is defined as a predetermined frame (that is, a frame to be encoded next when a certain frame is encoded). In other words, at this point in time, the code is not processed yet, and the frame that is scheduled to be processed in the future (also called the current frame) is encoded before the predetermined frame. Taichi no frame (past The process of encoding a predetermined frame with the previous frame as a previous frame) will be described, but the same process is performed for V and shifted frames.
[0068] 動画像符号化処理プログラム Prglは、後述するステップ 1からステップ 5においてコ ンピュータを下記のように機能させる。（ステップ 1)所定フレームの画像情報を入カフ レームメモリ 7に入力する。（ステップ 2)所定フレームの必要演算量 Kpを計算させる 必要演算量計算手段 2として機能させる。（ステップ 3)算出された必要演算量 Κρに 応じてプロセッサの動作周波数 F及び動作電源電圧 Vdd及び基板バイアス電圧 Vb n, Vbpを決定する動作決定手段 3として機能させる。（ステップ 4)算出された動作周 波数 F及び動作電源電圧 Vdd及び基板バイアス電圧 Vbn, Vbpでプロセッサ 1を動 作させる制御を行わせる動作制御手段 4として機能させる。（ステップ 5)所定フレーム の画像情報を符号化させる動画像符号ィ匕手段 5として機能させる。以上、ステップ 1 力 ステップ 5の処理を入力フレームメモリ 7に入力されるフレームの順番（すなわち、 符号化される順番）に、すべてのフレームに対して行うことで、動画像の符号化を行う 。以下、詳細に説明する。  [0068] The moving image encoding processing program Prgl causes the computer to function as follows in steps 1 to 5 described later. (Step 1) Input image information of a predetermined frame into the input frame memory 7. (Step 2) The required calculation amount Kp for a predetermined frame is calculated. (Step 3) It is made to function as the operation determining means 3 for determining the operating frequency F, the operating power supply voltage Vdd, and the substrate bias voltage Vb n, Vbp of the processor according to the calculated necessary calculation amount Κρ. (Step 4) It is made to function as the operation control means 4 for controlling the operation of the processor 1 with the calculated operation frequency F, operation power supply voltage Vdd, and substrate bias voltage Vbn, Vbp. (Step 5) It is made to function as the moving image encoding means 5 for encoding the image information of a predetermined frame. As described above, the processing of step 1 and step 5 is performed for all frames in the order of frames input to the input frame memory 7 (that is, the order of encoding), thereby encoding the moving image. Details will be described below.
[0069] (ステップ 1)入力された入力画像データは、フレームの同期をとるため、フレームを 一時的に記憶する記憶領域である入力フレームメモリ 7にー且格納される。  (Step 1) The inputted input image data is stored in the input frame memory 7 which is a storage area for temporarily storing the frames in order to synchronize the frames.
[0070] (ステップ 2：必要演算量計算ステップ)必要演算量計算手段 2は、入力フレームメモ リ 7にアクセスして所定フレームの入力画像データ 101を取得し、所定フレームの符 号化処理に必要な必要演算量 Kpを計算する。必要演算量 Κρの計算方法は様々な 方法が考えられるが、たとえば、所定フレームの符号ィ匕処理の演算量に影響を与え る要素を一つ以上使用して計算することが望ましい。要素としては、例えば、動画像 符号ィ匕処理において、処理内容が動き補償である場合は、動きの激しい映像では演 算量が多ぐ一方、動きの少ない映像では演算量が少ないことに注目して、所定フレ 一ムと前フレームとの動き量として差分絶対値和で計算される歪み値や、また、各々 のフレームのアクティビティ量として隣接画素差分絶対値和で計算される値や、マク ロブロックマッチング回数や、有効ブロック数や、有効係数の数や、符号化ビットレー トゃ、発生ビット数や、前フレームの符号ィ匕に実際に要した演算量や、必要演算量計 算手段 2により算出された前フレームの必要演算量が挙げられる。ここで、各要素そ れぞれについて、一つの要素の値のみ変化し、他の要素の値が変化しないと仮定し たときに、その一つの要素の値が大きい場合は小さい場合に比較して必要演算量が 相対的に大きくなるようにし、その一つの要素の値が小さ!/、場合は大き!、場合と比較 して必要演算量が相対的に小さくなるようにする。また、所定フレームがフレーム内符 号ィ匕である場合はフレーム間符号ィ匕である場合と比較して必要演算量 Kpが相対的 に小さぐフレーム間符号ィ匕である場合はフレーム内符号ィ匕である場合と比較して必 要演算量 Κρが相対的に大きくなるようにする。すなわち、これらの複数の要素は所定 フレームの符号ィ匕処理のために必要な必要演算量に影響を与える要素であるため、 必要演算量計算手段 2が、これらの要素に応じて必要演算量 Κρ (サイクル)を増減 するように計算を行うことにより、必要演算量計算手段 2により計算される必要演算量 Κρが現実に符号ィ匕処理を行ったときの演算量により近い値となる。 [0070] (Step 2: Necessary Computation Calculation Step) Necessary computation calculation means 2 accesses the input frame memory 7 to acquire the input image data 101 of the predetermined frame, and is necessary for the encoding processing of the predetermined frame. Calculate the necessary amount of computation Kp. There are various methods for calculating the required amount of computation Κρ. For example, it is desirable to calculate by using one or more elements that affect the amount of computation of the sign 匕 processing of a predetermined frame. As an element, for example, in the case of moving image code processing, when the processing content is motion compensation, the amount of computation is large for a video with a lot of motion, while the amount of computation is small for a video with little motion. Thus, the distortion value calculated by the sum of absolute differences as the amount of motion between the predetermined frame and the previous frame, the value calculated by the sum of absolute differences of adjacent pixels as the amount of activity of each frame, and the macro Depending on the number of block matching, the number of effective blocks, the number of effective coefficients, the encoding bit rate, the number of generated bits, the calculation amount actually required for the sign of the previous frame, and the required calculation amount calculation means 2 The required amount of calculation of the previous frame calculated can be mentioned. Here, each element Assuming that only the value of one element changes and the value of the other element does not change for each, the required amount of computation is relatively smaller when the value of that one element is large than when it is small. The value of one element is small! /, If it is large, the required amount of computation is relatively small compared to the case. In addition, when the predetermined frame is an intra-frame code 内, the intra-frame code は is required when the required calculation amount Kp is relatively small compared to the case of an inter-frame code 匕. Compared to the case of 匕, the required amount of computation Κρ is made relatively large. That is, since these multiple elements are elements that affect the required amount of calculation necessary for the sign processing of a predetermined frame, the required amount of calculation calculation means 2 determines the required amount of calculation Κρ according to these elements. By calculating so as to increase or decrease (cycle), the required calculation amount Κρ calculated by the required calculation amount calculation means 2 becomes a value closer to the calculation amount when the actual sign processing is performed.
[0071] たとえば、本実施の形態では、関数 Gを使用して計算し、入力フレームメモリ 7に記 憶されている所定フレームの入力画像データ 101と、局部復号フレームメモリ 6に蓄 積されている復号ィ匕された前フレームの局部復号データ 103とを比較して、入力画 像の動きの大きさの予測（計算）を行う。この前フレームの局部復号データ 103は、所 定フレームよりも前に符号ィ匕が行われる前フレームの符号ィ匕処理にぉ 、て、前フレ ームを符号化して形成した前フレームの符号化データ 106を、ローカルデコーダで 復号化することにより形成され、局部復号フレームメモリ 6に記憶されている。動きの 大きさの予測 (計算)の一例として、例えば差分絶対値和を用いる。以下に、差分絶 対値和∑と必要演算量 Κρの求め方を説明する。なお、前フレームの画像データとし ては、符号ィ匕後にローカルデコーダにより復号ィ匕された局部復号データ 106を使用 しても良いが、入力された前フレームの入力画像データをそのまま使用しても良い。  For example, in the present embodiment, calculation is performed using the function G, and the input image data 101 of a predetermined frame stored in the input frame memory 7 and the local decoding frame memory 6 are stored. By comparing the decoded local decoding data 103 of the previous frame, the motion magnitude of the input image is predicted (calculated). The local decoded data 103 of the previous frame is encoded by the previous frame formed by encoding the previous frame in accordance with the code key processing of the previous frame in which the code is performed before the predetermined frame. The data 106 is formed by decoding with a local decoder and stored in the local decoding frame memory 6. For example, the sum of absolute differences is used as an example of the prediction (calculation) of the magnitude of movement. The following explains how to find the difference absolute value sum ∑ and the required amount of computation Κρ. As the image data of the previous frame, the local decoded data 106 decoded by the local decoder after encoding may be used, or the input image data of the input previous frame may be used as it is. good.
[0072] 入力フレームメモリ 7に蓄積された所定フレームの入力画像データ 101を X(i,j) (iは 画像の水平方向の座標、 jは垂直方向の座標）、後述する局部復号フレームメモリ 6 に蓄積された前フレームの局部復号データ 103を Y(i,j) (iは画像の水平方向の座標 、 jは垂直方向の座標）とすると、所定フレームと前フレームとの動き量は、差分絶対 値和 Z=∑ I X(i,j)-Y(i,j) Iをすベての（またはサンプルした)画素に対して計算す る。この差分絶対値和の値を Zとする。一方、フレームのアクティビティ量においては 、 X(i,j)において隣接画素差分絶対値和 W、つまり、水平方向 Wh=∑ I x(i,j)-x(i- i,j) I、垂直方向 wv=∑ I x(u)—x(u-i) Iを計算することにより求められ、全ての[0072] Input image data 101 of a predetermined frame stored in the input frame memory 7 is represented by X (i, j) (i is a horizontal coordinate of the image, j is a vertical coordinate), and a local decoded frame memory 6 to be described later. If the local decoding data 103 of the previous frame stored in is Y (i, j) (where i is the horizontal coordinate of the image and j is the vertical coordinate), the amount of motion between the predetermined frame and the previous frame is the difference Calculate the sum of absolute values Z = ∑ IX (i, j) -Y (i, j) I for all (or sampled) pixels. Let Z be the value of the sum of absolute differences. On the other hand, in terms of frame activity, , X (i, j) adjacent pixel difference absolute value sum W, that is, horizontal direction Wh = ∑ I x (i, j) -x (i-i, j) I, vertical direction w v = ∑ I x ( u) —x (ui)
(又はサンプルした)入力画像に対して計算する。この隣接画素差分絶対値和の値（ すなわち各フレームのアクティビティ量)を wとする。 Calculate for (or sampled) input images. The value of the adjacent pixel difference absolute value sum (that is, the activity amount of each frame) is set to w.
[0073] 差分絶対値和を Z、所定フレームのアクティビティ量を Wa、前フレーム (過去のフレ ーム）のアクティビティ量を Wb、前フレームの平均量子化ステップサイズ（量子化ステ ップサイズの平均値）を Qprev、前フレームのマクロブロックマッチング回数を M、前フ レームの有効ブロック数を B、前フレームの有効係数の数を C、前フレームの符号ィ匕 に実際に要した処理量を S、所定フレームの符号ィ匕ビットレートを BR、前フレームの 量子化ステップサイズの平均値とその一つ前のフレームの量子化ステップサイズの 平均値の差を A Qprev、前フレームの実際の発生ビット数を D、必要演算量計算手 段により算出された前フレームの必要演算量を Kp，とおくと、これらの要素のうち一つ 以上の要素を使用して、必要演算量 Kpは、  [0073] The sum of absolute differences is Z, the amount of activity of the predetermined frame is Wa, the amount of activity of the previous frame (past frame) is Wb, and the average quantization step size of the previous frame (average value of quantization step size) Qprev, M for the number of macroblock matching in the previous frame, B for the number of effective blocks for the previous frame, C for the number of effective coefficients for the previous frame, S for the amount of processing actually required for the sign of the previous frame, BR is the frame code bit rate, A Qprev is the difference between the average quantization step size of the previous frame and the average quantization step size of the previous frame, and the actual number of bits generated in the previous frame. If the required amount of computation for the previous frame calculated by D, the required amount of computation calculator is Kp, then using one or more of these elements, the required amount of computation Kp is
Kp = G (Z, Wa, Wb, Qprev, M, B, C, S, BR, Δ Qprev, D, Kp，）  Kp = G (Z, Wa, Wb, Qprev, M, B, C, S, BR, Δ Qprev, D, Kp,)
で計算される。ただし、 Gは Z, Wa, Wb, Qprev, M, B, C, S, BR, Δ Qprev, D, Calculated by Where G is Z, Wa, Wb, Qprev, M, B, C, S, BR, Δ Qprev, D,
Kp'のうち、一以上の要素から導き出される関数である。その一例としては、 A function derived from one or more elements of Kp '. As an example,
Kp=j + a M+ j8 B+ y C+ δ Ζ+ ε AQprev  Kp = j + a M + j8 B + y C + δ Ζ + ε AQprev
が挙げられるが、これに限られるわけではない。また、必要演算量 Kpの計算に使用 される要素として、所定フレームがフレーム内符号ィ匕である力フレーム間符号ィ匕であ るかの種類 Iが使用される。所定フレームがフレーム内符号ィ匕である場合の必要演算 量 Kpは小さ 、値と、フレーム間符号ィ匕である場合の必要演算量 Kpは大き 、値とな る。すなわち、必要演算量計算手段 2は、差分絶対値和 Zを使用するときは差分絶対 値和 Z=∑ I Xij-Yij Iを計算した後に、必要演算量 Kp = G (Z, Wa, Wb, Qprev, M, B, C, S, BR, Δ Qprev, D, Kp，）を計算する。  However, it is not limited to this. In addition, as an element used for calculating the necessary amount of computation Kp, type I is used as to whether the predetermined frame is a force inter-frame code 匕 that is an intra-frame code 匕. The required amount of computation Kp when the predetermined frame is an intraframe code is small and the value, and the necessary amount of computation Kp when the predetermined frame is an interframe code is large and a value. That is, the necessary calculation amount calculation means 2 calculates the required calculation amount Kp = G (Z, Wa, Wb, after calculating the difference absolute value sum Z = ∑ I Xij-Yij I when using the difference absolute value sum Z. Qprev, M, B, C, S, BR, ΔQprev, D, Kp,).
[0074] 以下、上記関数 Gについて説明する。前フレームと所定フレームの間で画像の変化 が大きい (小さい)場合、すなわち差分絶対値和 Zが大きい (小さい)場合、所定フレ ームで実行されるマクロブロックマッチングの回数は大きく（小さく）なり、所定フレーム の動き検出処理に必要な演算量 (実行されるマクロブロックマッチング回数に依存す る）が大きく（小さく）なる。また、所定フレームのアクティビティ量 Waが大きい（小さい） 場合、所定フレームは画像の高周波成分を多く（少なく）含むことを意味し、この場合 、所定フレームの符号化処理で発生する有効ブロックの数、有効係数の数は大きく ( 小さく）なり、所定フレームの IDCT処理に必要な演算量 (発生する有効ブロックの数 に依存する）、 IQ処理に必要な演算量 (発生する有効係数の数に依存する）、 VLC 処理に必要な演算量 (発生する有効係数の数に依存する）は大きく（小さく）なる。し たがって、上記関数 Gは Z, Waなどのパラメータが大きい（小さい)場合、 Kpを大きく (小さく)設定するように構成する。 Hereinafter, the function G will be described. When the image change between the previous frame and the predetermined frame is large (small), that is, when the sum of absolute differences Z is large (small), the number of macroblock matching performed in the predetermined frame becomes large (small). The amount of computation required for motion detection processing for a given frame (depending on the number of macroblock matching operations to be performed Becomes larger (smaller). Further, when the activity amount Wa of the predetermined frame is large (small), it means that the predetermined frame includes a lot (small) of high frequency components of the image. In this case, the number of effective blocks generated in the encoding process of the predetermined frame, The number of effective coefficients increases (decreases), the amount of computation required for IDCT processing for a given frame (depends on the number of valid blocks generated), and the amount of computation required for IQ processing (depends on the number of generated effective coefficients) ), The amount of computation required for VLC processing (depending on the number of effective coefficients generated) becomes larger (smaller). Therefore, the function G is configured so that Kp is set large (small) when parameters such as Z and Wa are large (small).
[0075] 動画像は連続するフレーム間での相関が大きいため、符号化処理で実行されるマ クロブロックマッチング回数、符号化処理で発生する有効ブロック数、有効係数の数 、符号化処理で必要となる演算量、アクティビティ量は、時間的に連続するフレーム 間で非常に近い値となる。したがって、 Μ, Β, C, S, Wbが大きい（小さい)場合、所 定フレームにおいてもマクロブロックマッチング回数、有効ブロック数、有効係数の数 、符号ィ匕処理に必要となる演算量、アクティビティ量が大きく（小さく）なる確率が高い 。さらに、必要演算量計算手段で算出される予測演算量が実際の符号化処理に要し た演算量に近い値となる場合、 S Kp'となる。したがって，上記関数 Gは M, B, C, S, Wb, Kp'などのパラメータが大きい (小さい)場合、 Kpを大きく（小さく）設定する ように構成する。 [0075] Since a moving image has a large correlation between consecutive frames, the number of macroblock matching operations performed in the encoding process, the number of effective blocks generated in the encoding process, the number of effective coefficients, and the necessary in the encoding process The amount of computation and the amount of activity are very close to each other between consecutive frames. Therefore, when Μ, Β, C, S, Wb is large (small), the number of macroblock matching, the number of effective blocks, the number of effective coefficients, the amount of computation and the amount of activity required for sign key processing even in a given frame There is a high probability that will be larger (smaller). Further, when the predicted calculation amount calculated by the necessary calculation amount calculation means is close to the calculation amount required for the actual encoding process, S Kp ′ is obtained. Therefore, the above function G is configured to set Kp large (small) when parameters such as M, B, C, S, Wb, Kp 'are large (small).
[0076] ターゲットビットレートが大き ヽ（小さ 、）場合、量子化ステップサイズの値は小さく（ 大きく)設定され、その結果、符号化処理で発生する有効ブロックの数、有効係数の 数は大きく（小さく）なる。また、前フレームの発生ビット数力ターゲットビットレートと比 較して大き 、（小さ 、）場合、所定フレームの量子化ステップサイズの値は小さく（大き く）設定され、符号化処理で発生する有効ブロックの数、有効係数の数は小さく（大き く）なる。したがって、上記関数 Gは所定フレームの符号ィ匕ビットレート BRが大きい（ 小さい)場合、 Kpを大きく（小さく）設定するように、前フレームの実際の発生ビット数 Dが BRと比較して大き 、 (小さ ヽ)場合、 Kpを小さく（大きく）設定するように構成する 。さらに、前フレームの平均量子化ステップサイズ Qprevや前フレームの量子化ステ ップサイズの平均値とその一つ前のフレームの量子化ステップサイズの平均値の差 A Qprevを考慮することで、上記関数 Gが算出する Kpが実際に所定フレームを符号 化するために必要となる演算量に近 、値とすることができる。 [0076] When the target bit rate is large 小 さ (small), the value of the quantization step size is set small (large). As a result, the number of effective blocks and the number of effective coefficients generated in the encoding process are large ( Small). Also, if the number of generated bits in the previous frame is large (small) compared to the target bit rate, the quantization step size value for a given frame is set small (large), which is an effective value generated in the encoding process. The number of blocks and the number of effective coefficients become smaller (larger). Therefore, the above-mentioned function G has a larger number of generated bits D in the previous frame compared to BR so that Kp is set to be large (small) when the code key bit rate BR of the predetermined frame is large (small). If (small ヽ), configure Kp to be small (large). Furthermore, the difference between the average quantization step size Qprev of the previous frame and the average quantization step size of the previous frame and the average quantization step size of the previous frame. By considering A Qprev, Kp calculated by the above function G can be made close to the amount of computation required to actually encode a predetermined frame.
[0077] なお、動画像の符号化ビットレート 109や、所定フレーム及び前フレームについて フレーム内符号化であるかフレーム間符号化であるかの種類 108や、前フレームの アクティビティの量 110や、必要演算量計算手段により算出された前フレームの必要 演算量 116は要素が記憶される記憶領域である要素メモリ 8に予め記憶されており、 必要演算量 Kpの計算時に必要演算量計算手段 2に読み込まれて使用される。前フ レームの量子化ステップサイズの平均値 107、前フレームのマクロブロックマッチング 回数 111、前フレームの有効ブロック数 112、前フレームの有効係数の数 113、前フ レームの量子化ステップサイズの平均値とその一つ前のフレームの量子化ステップ サイズの平均値との差 114、及び前フレームの符号ィ匕に実際に要した処理量 115は 前フレームの符号ィ匕処理が行われたときに動画像符号ィ匕手段 5から必要演算量計 算手段 2にフィードバックされる。必要演算量計算手段 2においては、これらの要素の うち一つの要素のみを使用しても良 、し、複数の要素を組み合わせて使用しても良 い。 [0077] It should be noted that the encoding bit rate 109 of the moving image, the type 108 of intra-frame encoding or inter-frame encoding for the predetermined frame and the previous frame, the amount of activity 110 of the previous frame, and the necessary The required amount of calculation 116 of the previous frame calculated by the calculation amount calculation means is stored in advance in the element memory 8, which is a storage area for storing elements, and is read into the calculation amount calculation means 2 when calculating the required calculation amount Kp. Used. Average value of quantization step size of previous frame 107, number of macroblock matchings of previous frame 111, number of effective blocks of previous frame 112, number of effective coefficients of previous frame 113, average value of quantization step size of previous frame The difference between the average value of the quantization step size of the previous frame and the average value 114 of the previous frame and the amount of processing 115 actually required for the previous frame code It is fed back from the image code key means 5 to the necessary calculation amount calculation means 2. In the necessary calculation amount calculation means 2, only one of these elements may be used, or a plurality of elements may be used in combination.
[0078] (ステップ 3：動作決定ステップ)動作決定手段 3は、必要演算量 Kpの値をもとに、 所定フレームの処理に対する動作周波数 Fe (サイクル Z秒)を予測する計算を行う。 すなわち、符号ィ匕方式により処理時間が規定されている最小単位は 1フレームであり 、所定フレームの符号ィ匕処理に割り当てられた時間を Te (秒）とすると、所定フレーム に必要とされる動作周波数 Fe (サイクル Z秒）、すなわち時間 Te (秒）内に前記必要 演算量 Kpを符号化処理可能な動作周波数 Fe (サイクル Z秒）は Fe=KpZTeで表 されることから、動作決定手段 3は動作周波数 Fe=KpZTeを計算する。ただし、所 定フレームの符号ィ匕処理に割り当てられた時間 Teは、 1フレームの処理の制限時間 Tfから、所定フレームに対する演算量を予測する時間 Tp及びプロセッサの動作周 波数'動作電源電圧 ·基板バイアス電圧を変更する時間 Tsを引いた時間である。図 5 に示すように、プロセッサ 1および (又は）局部復号メモリ 6等を含めた周辺装置がサ ポートする動作電源電圧 ·基板バイアス電圧 ·動作周波数が r段階 (rは 2以上の整数 )で変更可能な場合、動作決定手段 3は、 F (n) >Fe且つ Fe >F (n— 1)となる動作 周波数 F (n)を所定フレームの符号化処理を行う動作周波数として選択する計算を 行い、その動作周波数 F (n)に適する動作電源電圧 Vdd (n)及び基板バイアス電圧 Vbn, Vbp (n)を選択する計算を行い、プロセッサ 1および (又は）局部復号メモリ 6等 を含めた周辺装置をその動作周波数 F (n)と動作電源電圧 Vdd (n)と基板バイアス 電圧 Vbn, Vbp (n)で動作させるように、動作電源電圧'基板バイアス電圧'動作周 波数を動作制御手段 4に指示する (符号 102)。なお、 nは 1以上 r以下の整数である (Step 3: Operation Determination Step) The operation determination means 3 performs calculation for predicting the operation frequency Fe (cycle Z seconds) for the processing of a predetermined frame based on the value of the necessary calculation amount Kp. In other words, the minimum unit for which the processing time is defined by the code key method is one frame. If the time allocated to the code key processing of a predetermined frame is Te (seconds), the operation required for the predetermined frame Since the operating frequency Fe (cycle Z seconds) capable of encoding the required amount of computation Kp within the time Fe (cycle Z seconds), ie, the time Te (seconds), is expressed by Fe = KpZTe. Calculates the operating frequency Fe = KpZTe. However, the time Te allocated to the code processing for a given frame is the time Tp for predicting the amount of computation for a given frame from the time limit Tf for processing one frame and the operating frequency of the processor 'operating power supply voltage · board The time to change the bias voltage is the time minus Ts. As shown in Fig. 5, the operating power supply voltage, substrate bias voltage, and operating frequency supported by peripheral devices including processor 1 and / or local decoding memory 6 are changed in r stages (r is an integer of 2 or more). When possible, the operation determining means 3 performs an operation in which F (n)> Fe and Fe> F (n— 1). Calculation is performed to select the frequency F (n) as the operating frequency for encoding the predetermined frame, and the operating power supply voltage Vdd (n) and the substrate bias voltage Vbn, Vbp (n) suitable for the operating frequency F (n) are calculated. Perform calculations to select and operate peripheral devices including processor 1 and / or local decoding memory 6 etc. at operating frequency F (n), operating power supply voltage Vdd (n), and substrate bias voltage Vbn, Vbp (n) The operation control means 4 is instructed to supply the operation power supply voltage 'substrate bias voltage' to the operation frequency (reference numeral 102). N is an integer between 1 and r
[0079] 図 5は、動作周波数'動作電源電圧'基板バイアス電圧の関係を示す図である。動 作決定手段 3において、各動作周波数に対し、プロセッサ 1、又は、プロセッサ 1及び 局部復号メモリ 6等を含めた周辺装置で消費される電流が所定値以下となるように動 作電源電圧'基板バイアス電圧の組合せが予め設定されている。たとえば、サブスレ ッショルドリーク電流 1stと充放電電流 ledとの関係から、各動作周波数ごとに消費電 力 Pが最小となる動作電源電圧 Vddと基板バイアス電圧 Vbn, Vbpを実験や計算等 により求め、この動作電源電圧 Vddと基板バイアス電圧 Vbn, Vbpの組み合わせとす ることが望ましい。ここで、電流の最小化を図る際、各電流要素を 1つ以上用いて合 計した電流を計算に用いる。なお、動作決定手段 3に内蔵するハードウェアおよび（ 又は)プログラムで、動作周波数に応じた動作電源電圧に対し、自動的に基板バイァ ス電圧が計算されてもよい。また、動作決定手段 3に内蔵するハードウェアおよび (又 は)プログラムで、動作周波数に対し、動作電源電圧と基板バイアス電圧が計算され てもよい。 FIG. 5 is a diagram showing the relationship between the operating frequency “operating power supply voltage” and the substrate bias voltage. In the operation determining means 3, the operation power supply voltage 'substrate is set so that the current consumed by the peripheral device including the processor 1 or the processor 1 and the local decoding memory 6 is equal to or less than a predetermined value for each operating frequency. A combination of bias voltages is preset. For example, based on the relationship between the subthreshold leakage current 1st and the charge / discharge current led, the operating power supply voltage Vdd and the substrate bias voltage Vbn, Vbp that minimize the power consumption P for each operating frequency are obtained by experiments and calculations. A combination of power supply voltage Vdd and substrate bias voltage Vbn, Vbp is desirable. Here, when minimizing the current, the total current using one or more current elements is used in the calculation. Note that the substrate bias voltage may be automatically calculated with respect to the operating power supply voltage corresponding to the operating frequency by hardware and / or a program built in the operation determining means 3. Further, the operating power supply voltage and the substrate bias voltage may be calculated with respect to the operating frequency by hardware and / or a program built in the operation determining means 3.
[0080] (ステップ 4)動作制御手段 4は、動作決定手段 3から指示を受けた動作電源電圧 V dd (n)及び基板バイアス電圧 Vbn (n) , Vbp (n)及び動作周波数 F (n)の値をプロセ ッサ 1および (又は）局部復号メモリ 6等を含めた周辺装置に供給し (符号 105)、その 動作電源電圧 Vdd (n)及び基板バイアス電圧 Vbn (n) , Vbp (n)及び動作周波数 F ( n)でプロセッサ 1と周辺装置を一定に動作させる制御を行う。これにより、プロセッサ 1 および (又は）局部復号メモリ 6等を含めた周辺装置は、一定の動作電源電圧 Vdd (n )及び基板バイアス電圧 Vbn (n) , Vbp (n)及び動作周波数 F (n)で動作することに なる。具体的には、動作制御手段 4に内蔵する動作電源電圧制御手段 4cにより一定 の動作電源電圧 Vdd (n)でプロセッサ 1を動作させる制御を行 ヽ、基板バイアス電圧 Vbn制御手段 4dにより n チャネル MOSトランジスタに対する一定の基板バイアス 電圧 Vbn (n)でプロセッサ 1を動作させる制御を行 ヽ、基板バイアス電圧 Vbp制御手 段 4bにより p—チャネル MOSトランジスタに対する一定の基板バイアス電圧 Vbp (n) でプロセッサ 1を動作させる制御を行い、動作周波数制御手段 (PLL) 4bにより一定 の動作周波数 F (n)でプロセッサ 1を一定に動作させる制御を行う。 (Step 4) The operation control means 4 receives the operation power supply voltage V dd (n), the substrate bias voltages Vbn (n) and Vbp (n) and the operation frequency F (n) received from the operation determination means 3 Is supplied to peripheral devices including processor 1 and / or local decoding memory 6 (symbol 105), and its operating power supply voltage Vdd (n) and substrate bias voltage Vbn (n), Vbp (n) The processor 1 and the peripheral device are controlled to operate at a constant frequency F (n). As a result, the peripheral device including the processor 1 and / or the local decoding memory 6 and the like has a constant operating power supply voltage Vdd (n), a substrate bias voltage Vbn (n), Vbp (n), and an operating frequency F (n). Will work. Specifically, the operation power supply voltage control means 4c built in the operation control means 4 is constant. The processor 1 is controlled to operate at the operating power supply voltage Vdd (n), and the processor 1 is controlled to operate at the constant substrate bias voltage Vbn (n) for the n-channel MOS transistor by the substrate bias voltage Vbn control means 4d.基板 The substrate bias voltage Vbp control means 4b controls the processor 1 to operate at a constant substrate bias voltage Vbp (n) for the p-channel MOS transistor, and the operating frequency control means (PLL) 4b controls the constant operating frequency F. In (n), the processor 1 is controlled to operate constantly.
[0081] 基板バイアス電圧制御は、基板バイアス電圧 Vbn (n) , Vbp (n)に対し、 n—チヤネ ル MOSトランジスタに対し適切な基板バイアス電圧 Vbn (n)を印加し、 p—チャネル MOSトランジスタに対し適切な基板バイアス電圧 Vbp (n)を印加することにより行う。 具体的には、 n—チャネル MOSトランジスタに対する基板バイアス電圧 Vbn (n)とグ ランド電位 Vssとの電位差を Vbbn (n)とし、 p チャネル MOSトランジスタに対する 基板バイアス電圧 Vbp (n)と動作電源電圧 Vdd (n)との電位差を Vbbp (n)とする。 すなわち、 [0081] Substrate bias voltage control applies p-channel MOS transistor by applying appropriate substrate bias voltage Vbn (n) to n-channel MOS transistor for substrate bias voltages Vbn (n) and Vbp (n). Is applied by applying an appropriate substrate bias voltage Vbp (n). Specifically, the potential difference between the substrate bias voltage Vbn (n) for the n-channel MOS transistor and the ground potential Vss is Vbbn (n), and the substrate bias voltage Vbp (n) and the operating power supply voltage Vdd for the p-channel MOS transistor. The potential difference from (n) is Vbbp (n). That is,
Vbn (n) =Vbbn (n) +Vss  Vbn (n) = Vbbn (n) + Vss
Vbp (n) = Vbbp (n) + Vdd (n)  Vbp (n) = Vbbp (n) + Vdd (n)
の関係が成り立つ。電圧 Vbbn (n)と Vbbp (n)と動作電源電圧 Vdd (n)は独立に設 定できる。ただし、 Vbbn(n)は、 n—チャネル MOSトランジスタのドレイン—ソース間 pn接合に印加された電圧であり、この電圧が拡散電位 ν φを超えないようにし、 Vbb p (n)は、 p チャネルトランジスタのドレイン一ソース間 pn接合に印加された電圧で あり、この電圧が拡散電位 V φを下回らないようにする。拡散電位 V φは通常 0. 6 Vである。  The relationship holds. The voltages Vbbn (n) and Vbbp (n) and the operating power supply voltage Vdd (n) can be set independently. However, Vbbn (n) is the voltage applied to the drain-source pn junction of the n-channel MOS transistor, and this voltage must not exceed the diffusion potential ν φ, and Vbb p (n) is the p-channel This is the voltage applied to the pn junction between the drain and source of the transistor, so that this voltage does not fall below the diffusion potential Vφ. The diffusion potential V φ is usually 0.6 V.
[0082] (ステップ 5：動画像符号化ステップ)動画像符号化手段 5は、動画像符号化処理プ ログラム Prglによりコンピュータのプロセッサ 1上で実現される手段であり、プロセッサ 1を使用して入力フレームメモリ 7に格納された入力画像データを動画像符号化を行 う単位でアクセスし、符号ィ匕処理を行う手段である。すなわち、動画像符号化手段 5 は、入力フレームメモリ 7から所定フレームの入力画像データ 101を取得し、符号化し て符号化データ 106を生成する。ステップ 4において、プロセッサ 1および (又は）局 部復号メモリ 6等を含めた周辺装置は動作制御手段 4から供給された一定の動作電 源電圧 Vdd (n)及び基板バイアス電圧 Vbn (n) , Vbp (n)及び動作周波数 F (n)で 動作している状態となっているため、ステップ 5では、動作制御手段 4がその一定の 動作周波数 F (n)及び動作電源電圧 Vdd (n)及び基板バイアス電圧 Vbn (n) , Vbp ( n)でプロセッサ 1および (又は）局部復号メモリ 6等を含めた周辺装置を動作させなが ら、そのプロセッサ 1を使用して符号ィ匕を行う動画像符号ィ匕手段 5が所定フレームの 符号ィ匕を行うこととなる。たとえば動きの激し 、画像 (所定フレームの入力画像データ 101)に対してはプロセッサ 1および (又は）局部復号メモリ 6等を含めた周辺装置を 高 ヽ周波数で一定に動作させ、動きの少な 、画像に対しては低 、周波数で一定に 動作させることにより低消費電力化を図ることが可能になる。さらに、動画像符号化手 段 5は、符号ィ匕データ 106を復号する機能を有するローカルデコーダを備えており、 所定フレームの符号ィ匕データ 106はローカルデコーダにより復号されて局部復号フ レームメモリ 6に局部復号データ 103として蓄積される。この所定フレームの局部復号 データ 103は所定フレームの次に符号ィ匕されるフレームについて必要演算量 Kpを 計算する際に使用される。所定フレームの符号化データ 106は伝送路を通じて送信 されたり、蓄積メディアに蓄積されたりする。 [0082] (Step 5: moving picture coding step) The moving picture coding means 5 is a means realized on the processor 1 of the computer by the moving picture coding processing program Prgl. This is a means for accessing the input image data stored in the frame memory 7 in units of moving image encoding and performing the encoding process. That is, the moving image encoding means 5 acquires the input image data 101 of a predetermined frame from the input frame memory 7 and encodes it to generate the encoded data 106. In step 4, the peripheral devices including the processor 1 and / or the local decoding memory 6 and the like operate at a constant operating voltage supplied from the operation control means 4. Since the operation is performed at the source voltage Vdd (n), the substrate bias voltage Vbn (n), Vbp (n) and the operating frequency F (n), in step 5, the operation control means 4 While operating peripheral devices including the processor 1 and / or the local decoding memory 6 at the operating frequency F (n), the operating power supply voltage Vdd (n), and the substrate bias voltage Vbn (n), Vbp (n) Therefore, the moving image code key means 5 that performs the code key using the processor 1 performs the key key code. For example, in the case of intense motion, peripheral devices including the processor 1 and / or the local decoding memory 6 etc. are constantly operated at a high frequency for an image (input image data 101 of a predetermined frame). Low power consumption can be achieved by operating the image at a constant frequency at a low frequency. Furthermore, the moving picture encoding unit 5 includes a local decoder having a function of decoding the code key data 106. The code key data 106 of a predetermined frame is decoded by the local decoder and is then decoded in the local decoding frame memory 6 Is stored as locally decoded data 103. The local decoded data 103 of the predetermined frame is used when calculating the necessary calculation amount Kp for the frame encoded next to the predetermined frame. The encoded data 106 of a predetermined frame is transmitted through a transmission path or stored in a storage medium.
[0083] なお、動作電源電圧及び基板バイアス電圧の制御は、動作周波数 Fに応じて動作 電源電圧 Vdd、 p—チャネル MOSトランジスタの基板バイアス電圧 Vbp, n—チヤネ ル MOSトランジスタの基板バイアス電圧 Vbnのうち少なくとも 1つの電圧だけ制御し てもよい。 [0083] Note that the operation power supply voltage and the substrate bias voltage are controlled according to the operation frequency F by the operation power supply voltage Vdd, the substrate bias voltage Vbp of the p-channel MOS transistor, and the n-channel MOS transistor substrate bias voltage Vbn. Only at least one voltage may be controlled.
[0084] また、 1フレームの処理に用いられる動作周波数'動作電源電圧'基板バイアス電 圧を一定に動作させるだけでなぐ動作決定手段 3により動作周波数'動作電源電圧 '基板バイアス電圧を N組 (Nは正の整数)選択してもよい。この場合、 N組の動作周 波数'動作電源電圧'基板バイアス電圧各々の動作時間を計算し、プロセッサ 1を動 作させることとなる。  [0084] Further, the operation frequency 'operating power supply voltage' used for the processing of one frame, and the operation determining means 3 that only requires the substrate bias voltage to operate at a constant level is set to N sets of operating frequency 'operating power supply voltage' substrate bias voltage ( N may be a positive integer). In this case, the operating time of each of N sets of operating frequencies “operating power supply voltage” and substrate bias voltage is calculated, and processor 1 is operated.
[0085] (第 1の破綻回避ステップ）  [0085] (First failure avoidance step)
符号ィ匕処理システム S1は破綻現象を回避するための第 1の破綻回避手段を備え、 第 1の破綻回避ステップは動画像符号化ステップに割り込みを行うことで実行される 。第 1の破綻回避手段は、所定フレームの符号ィ匕処理が割り当てられた時間 Te内に 終了できないと判断された場合、符号ィ匕処理が割り当てられた時間 Te内に終了でき るように、通常の符号ィ匕処理の演算量を削減できる処理に切り替えて、破綻現象を 回避する。このとき、処理を切り替えるタイミング Tiは、割り込み処理を行って符号ィ匕 処理を一時中断して算出される。また演算量を削減する処理を行うマクロブロックの 数を最小限に抑えるため、符号ィ匕された処理に応じてタイミング Tiの更新を行っても よい。 The code processing system S1 includes first failure avoiding means for avoiding the failure phenomenon, and the first failure avoiding step is executed by interrupting the moving image encoding step. The first bankruptcy avoiding means is within the time Te when the code processing of a predetermined frame is allocated. When it is determined that the process cannot be completed, the process is switched to a process capable of reducing the amount of calculation of the normal code process so that the code process can be completed within the allocated time Te to avoid the failure phenomenon. At this time, the timing Ti at which the process is switched is calculated by interrupting the sign process and temporarily interrupting the sign process. In addition, in order to minimize the number of macroblocks that perform processing that reduces the amount of computation, the timing Ti may be updated according to the encoded processing.
[0086] (切り替えタイミング Tiの算出 ·決定）  [0086] (Calculation / determination of switching timing Ti)
第 1の破綻回避手段を実行するタイミング Tiの決定について具体的に説明する。図 7は割り込みを行う際の時間と演算残量の関係を示している。 1フレームの符号化処 理に割り当てられている時間 Te、 1フレームのマクロブロック数を MBとし、 1つのマク ロブロックの最大演算量を Kw、 1つのマクロブロックで第 1の破綻回避手段による処 理に必要な演算量を Ksとする。プロセッサ 1は、処理済マクロブロック数レジスタ 10か ら符号ィ匕処理が終了したマクロブロックの数 MBi (符号 117)の読み出しを行 、、切り 替えタイミング Tiを以下に示す数式で算出する。  The determination of the timing Ti for executing the first failure avoidance measure will be described in detail. Figure 7 shows the relationship between the time for interrupting and the remaining calculation. The time allocated for the encoding process of one frame Te, the number of macroblocks in one frame as MB, the maximum computation amount of one macroblock as Kw, and the processing by the first failure avoidance method with one macroblock. Let Ks be the amount of computation required for processing. The processor 1 reads from the processed macroblock number register 10 the number of macroblocks MBi (reference numeral 117) for which the sign process has been completed, and calculates the switching timing Ti using the following equation.
Ti=Te- (Ks X (MB-MBi) ) /F  Ti = Te- (Ks X (MB-MBi)) / F
ただし、（ステップ 5)内において、符号化処理を開始する前に、 MBi = 0として Tiを 予め算出する。 Tiの更新は、予め算出された時刻 Tiで行われ、その際割り込み処理 により符号ィ匕処理を一時中断し、 Tiの更新を行うか否かを KwX (MB -MBi) >F X (Te—Ti)か、 KwX (MB -MBi)く F X (Te—Ti)で判断する。ここで、判断する前 提として MBi≠ MBである。 KwX (MB -MBi)く F X (Te—Ti)となれば、演算量に 余裕があると判断され、上記の数式により Tiの更新が行われる。 KwX (MB -MBi) >F X (Te— Ti)となれば、演算量に余裕がないと判断され、第 1の破綻回避手段 9 により演算量を削減する処理に切り替える。なお、 Tiの更新を行うかの判断をする際 に MBi=MBである場合は、処理を切り替えることなく符号化処理ルーチンを終了す る。第 1の破綻回避手段としては、下記の例 1から例 4が挙げられる。  However, within (Step 5), Ti is calculated in advance with MBi = 0 before starting the encoding process. Ti is updated at a pre-calculated time Ti. At that time, the sign key processing is temporarily interrupted by interrupt processing, and whether or not to update Ti is determined. KwX (MB -MBi)> FX (Te-Ti ) Or KwX (MB -MBi) and FX (Te-Ti). Here, MBi ≠ MB as a precondition for judgment. If KwX (MB -MBi) becomes F X (Te-Ti), it is judged that there is a sufficient amount of calculation, and Ti is updated by the above formula. If KwX (MB -MBi)> F X (Te-Ti), it is determined that there is no margin in the amount of computation, and the first failure avoiding means 9 switches to processing for reducing the amount of computation. If MBi = MB when deciding whether to update Ti, the encoding process routine is terminated without switching the process. Examples 1 to 4 are listed below as the first means of avoiding bankruptcy.
[0087] (第 1の破綻回避手段の例 1)例 1の第 1の破綻回避手段 91は、ステップ 5において 動画像符号ィ匕手段 5が所定フレームの入力画像データ 101の符号ィ匕処理ルーチン を実行している際に、上記で決定されたタイミング Tiで、符号ィ匕が終了していないマ クロブロックに対して色差ブロックのみを強制的に無効ブロック化する処理を行う。マ クロブロックのうち色差ブロックのみ強制的に無効ブロック化する処理を行い、輝度ブ ロックは通常と同様に符号ィ匕処理を行うことで、画質が劣悪になるのを防ぐことができ 、すべてのブロックを強制的に無効ブロック化して破綻を回避するよりも画質を改善 することができる。 (Example of First Failure Avoiding Means 1) The first failure avoiding means 91 of Example 1 is the same as the first failure avoiding means 91 in step 5, in which the moving picture code key means 5 is a code key processing routine for the input image data 101 of a predetermined frame. When executing the above, the sign 匕 is not finished at the timing Ti determined above. A process of forcibly making only the color difference block an invalid block is performed on the black block. Only the color difference block of the macro block is forcibly converted to an invalid block, and the luminance block is processed in the same manner as normal, thereby preventing the image quality from being deteriorated. The image quality can be improved rather than forcibly making the block invalid and avoiding the failure.
[0088] (第 1の破綻回避手段の例 2)例 2の第 1の破綻回避手段 92は、所定フレームの符 号ィ匕処理ルーチンを実行している際に、上記で決定されるタイミング Tiにおいて、符 号ィ匕が終了していないマクロブロックに対して強制的にフレーム内符号ィ匕処理を行う 。フレーム間符号ィ匕を行う場合に比べ、最も処理量の割合を占める動きベクトル検出 処理を行わないので、実際の演算量を削減することができる。また、最後のマクロプロ ックまで符号ィ匕処理を行うので、例 1と比較して該当するマクロブロックの数が増える 力 画質をさらに改善することができる。  (Example 1 of first failure avoiding means 2) The first failure avoiding means 92 of Example 2 is the timing Ti determined as described above when executing the code processing routine of a predetermined frame. In step (3), intra-frame code processing is forcibly performed on macroblocks that have not been completed. Compared with the case where inter-frame coding is performed, since the motion vector detection process which occupies the most processing amount is not performed, the actual calculation amount can be reduced. In addition, since the sign key processing is performed up to the last macro block, the number of corresponding macro blocks increases compared to Example 1, and the image quality can be further improved.
[0089] (第 1の破綻回避手段の例 3)例 3の第 1の破綻回避手段 93は、所定フレームの符 号ィ匕処理ルーチンを実行している際に、上記で決定されるタイミング Tiにおいて、符 号ィ匕が終了していないマクロブロックに対し、強制的に動きベクトルの大きさを 0として フレーム間符号化処理を行う。例 3は、動きベクトル検出処理を行わずにフレーム間 符号ィ匕処理を行うため、演算量の削減量は小さく該当するマクロブロックの数は増え る力 次のフレームの符号化処理に与える影響は例 2に比べ小さぐ画質が劣悪にな るのを、さらに改善することができる。  (Example 1 of first failure avoiding means 3) The first failure avoiding means 93 of Example 3 performs the timing Ti determined as described above when executing the code processing routine of a predetermined frame. In, the inter-frame encoding process is performed forcibly with the size of the motion vector set to 0 for macroblocks that have not been completed. In Example 3, since the interframe coding process is performed without performing the motion vector detection process, the amount of calculation reduction is small and the number of corresponding macroblocks increases. The effect on the coding process of the next frame is Compared to Example 2, the smaller image quality can be further improved.
[0090] (第 1の破綻回避手段の例 4)例 4の第 1の破綻回避手段 94は、上記で決定される タイミング Tiにおいて、符号ィ匕が終了していないマクロブロックに対して強制的に量 子化ステップサイズを大きくして符号ィ匕処理を行う。これ〖こより、当該マクロブロックに 対して有効ブロック数、有効係数の数の発生が小さくなり、符号化処理のうち、有効 ブロックに対して行われる処理、有効係数に対して行われる可変長符号化処理の回 数を削減することができる。  [0090] (Example 4 of first failure avoiding means) The first failure avoiding means 94 of Example 4 compulsorily applies to a macroblock whose code is not finished at the timing Ti determined above. In addition, the quantization step size is increased and the sign key processing is performed. As a result, the occurrence of the number of effective blocks and the number of effective coefficients for the macroblock is reduced, and among the encoding processes, the processes performed on the effective blocks and the variable length encoding performed on the effective coefficients. The number of processes can be reduced.
[0091] 上記第 1の破綻回避手段の各例は、通常の符号化処理の内容を省略し、強制的に 動きベクトルを 0としたり、有効ブロック数、有効係数の発生を抑えることで必要演算 量を削減している。このため、所定フレームで第 1の破綻回避手段による破綻回避処 理を行うことにより、次のフレームの処理において、所定フレームで破綻回避処理を 実行したマクロブロックに対応する、次のフレームのマクロブロックでフレーム間符号 化処理を行った場合、画質に影響を及ぼす可能性がある。このため、所定フレーム で破綻回避処理による画質劣化の防止を図ったにも関わらず、次のフレームに画質 が劣悪になる可能性がある。 [0091] Each example of the first failure avoidance means described above omits the contents of the normal encoding process, forces the motion vector to 0, and suppresses the number of effective blocks and generation of effective coefficients. The amount is reduced. For this reason, the failure avoidance process by the first failure avoiding means in a predetermined frame is performed. If the interframe coding processing is performed on the macroblock of the next frame corresponding to the macroblock that has been subjected to the failure avoidance processing on the predetermined frame in the processing of the next frame, the image quality is affected. there is a possibility. For this reason, there is a possibility that the image quality of the next frame is deteriorated even though the image quality deterioration is prevented by the failure avoidance process at the predetermined frame.
[0092] そこで、所定フレームにおいて第 1の破綻回避手段を実行したマクロブロックに対応 する、次のフレームのマクロブロックの符号ィ匕処理への影響を緩和するため、当該マ クロブロックに対し量子化ステップサイズを小さくして符号ィ匕処理を行う緩和手段 95 により緩和処理を行う。この処理により、対応するマクロブロックに割り当てられる発生 符号量が大きくなり、マクロブロック力 得られる情報を大きくして画質の改善を図るこ とがでさる。  [0092] Therefore, in order to mitigate the influence of the macroblock of the next frame corresponding to the macroblock for which the first failure avoidance means has been executed in the predetermined frame, the quantization is performed on the macroblock. Relaxation processing is performed by the relaxation means 95 that performs the sign key processing by reducing the step size. This process increases the amount of generated code assigned to the corresponding macroblock, and increases the information obtained from the macroblock power to improve image quality.
[0093] (証明 1)  [0093] (Proof 1)
以下に、プロセッサの動作周波数を複数回変更しながら一のフレームを符号ィ匕する 従来技術と比較して、本願発明がよりサブスレツショルドリーク電流による消費電力を 低減できることを証明する。たとえば、プロセッサ 1の基板バイアス電圧及び動作周波 数は図 5に示すように r段階に可変とし、任意の一のフレームの必要演算量を Ktとし、 そのフレームの処理に割り当てられる時間を Ttとする。図 6 (a)に示すように、動作周 波数を Ftと設定し、プロセッサ 1を動作周波数 Ftで動作させるときの基板バイアス電 圧を Vbp、 Vbnとし、基板バイアス電圧 Vbp、 Vbnに適するしきい値電圧を Vtとし、 時間 Ttで必要演算量 Ktの処理が終了する場合を Caselとし、図 6 (b)に示すように、 初期値の動作周波数を h * Ftと設定し、プロセッサを動作周波数 h * Ftで動作させる ときの基板バイアス電圧を Vbpl、 Vbnlとし、基板バイアス電圧 Vbpl、 Vbnlに適す るしき 、値電圧を Vtlとし、時間 T1が経過した時点でプロセッサの動作周波数を h * FtZ2に変更し、プロセッサ 1を動作周波数 h* FtZ2で動作させるときの基板バイァ ス電圧を Vbp2、 Vbn2とし、基板バイアス電圧 Vbp2、 Vbn2に適するしきい値電圧を Vt2とし、時間 T1 +T2で必要演算量 Ktの処理が終了する場合を Case2とし、各 Cas el, Case2について前記任意の一のフレームを符号ィ匕する場合を考えてみる。ただし 、しきい値電圧について Vtl >Vt>Vt2であり、サブスレツショルドリーク電流による 消費電力は、 In the following, it is proved that the present invention can further reduce the power consumption due to the subthreshold leakage current as compared with the prior art in which one frame is encoded while changing the operating frequency of the processor a plurality of times. For example, the substrate bias voltage and operating frequency of processor 1 are variable in r steps as shown in Fig. 5, the required amount of computation for any one frame is Kt, and the time allotted for processing of that frame is Tt. . As shown in Fig. 6 (a), when the operating frequency is set to Ft and the processor 1 is operated at the operating frequency Ft, the substrate bias voltages are Vbp and Vbn, and the thresholds are suitable for the substrate bias voltages Vbp and Vbn. When the value voltage is Vt, processing of the required amount of computation Kt is completed at time Tt, Casel is set, and as shown in Fig. 6 (b), the initial operating frequency is set to h * Ft, and the processor is set to operating frequency. The substrate bias voltage when operating at h * Ft is Vbpl, Vbnl, the threshold voltage suitable for the substrate bias voltage Vbpl, Vbnl is Vtl, and the processor operating frequency is set to h * FtZ2 when time T1 has elapsed. When the processor 1 is operated at the operating frequency h * FtZ2, the substrate bias voltage is Vbp2 and Vbn2, the threshold voltage suitable for the substrate bias voltage Vbp2 and Vbn2 is Vt2, and the required amount of computation at time T1 + T2 Case 2 is when the Kt processing ends. For each Cas el, Case2 consider the case of spoon codes I said arbitrary one frame. However, the threshold voltage is Vtl>Vt> Vt2, and it depends on the subthreshold leakage current. Power consumption is
Pst=VddXI X10"(-Vt/S)  Pst = VddXI X10 "(-Vt / S)
o  o
I：定数、 Vdd:動作電源電圧、 Vgs:ゲート一ソース間電圧、  I: Constant, Vdd: Operating power supply voltage, Vgs: Gate-source voltage,
0  0
Vt：しき!/ヽ値電圧、 S：サブスレツショルド swing  Vt: Shiki! / Variable voltage, S: Subthreshold swing
と表される。これを用いて Caselのサブスレツショルドリーク電流による消費電力 Pst It is expressed. Using this, the power consumption due to the subthreshold leakage current of Casel Pst
1と Case2のサブスレツショルドリーク電流による消費電力 Pst2を計算すると、 When calculating the power consumption Pst2 due to the subthreshold leakage current of 1 and Case2,
Pstl=VddXI X10"(-Vt/S) XTt  Pstl = VddXI X10 "(-Vt / S) XTt
o  o
Pst2=VddXI X10"(-Vtl/S) XT1+I X10"(-Vt2/S) XT2  Pst2 = VddXI X10 "(-Vtl / S) XT1 + I X10" (-Vt2 / S) XT2
o o  o o
となり、 And
Pstl:Pst2=10"(-Vt/S) XTt: (10"(-Vtl/S) XTl + 10" (-Vtl/S) XT2)  Pstl: Pst2 = 10 "(-Vt / S) XTt: (10" (-Vtl / S) XTl + 10 "(-Vtl / S) XT2)
となる。ここで、たとえば h= 1.5、 Ta= 1/3 XTt, Tb = 2/3XTt, Vtl = 3 X S, Vt2 = S, Vt=2XSとすると、 It becomes. For example, if h = 1.5, Ta = 1/3 XTt, Tb = 2 / 3XTt, Vtl = 3 X S, Vt2 = S, Vt = 2XS,
Pstl： Pst2=10"2: (10_3/3 + 10"1 X 2/3) Pstl: Pst2 = 10 " 2 : (10 _3 / 3 + 10" 1 X 2/3)
=0.01:0.07  = 0.01: 0.07
となり、 Pstl <Pst2となる。すなわち、決められた演算量を一定時間で処理する場 合、同一演算量 Ktにもかかわらず、 Caselの場合のように、その時間内で処理が終 了可能な最小の動作周波数により、その処理時間を通してプロセッサの基板バイァ ス電圧を一定に動作させるほうが、従来のように処理時間中に動作周波数を変更す る Case2の場合よりも低消費電力であることがわかる。したがって、一定の基板バイァ ス電圧及び動作周波数でプロセッサ 1を動作させながら一のフレームの符号ィ匕処理 を行う本発明によれば、ブロックごとに基板バイアス電圧及び動作周波数が決定され るため一のフレームの符号ィヒ中に何度も動作周波数が変更される従来技術と比較し て、低消費電力化が図られることがわかる。 And Pstl <Pst2. In other words, when processing a certain amount of computation in a certain amount of time, the processing is performed with the minimum operating frequency that can be completed within that time, as in Casel, despite the same computation amount Kt. It can be seen that operating the processor's board bias voltage constant over time consumes less power than in Case 2, where the operating frequency is changed during the processing time as in the past. Therefore, according to the present invention, in which the processor 1 is operated at a constant substrate bias voltage and operating frequency, and the code frame processing of one frame is performed, the substrate bias voltage and operating frequency are determined for each block. It can be seen that the power consumption can be reduced compared to the prior art in which the operating frequency is changed many times during the frame coding.
(証明 2)  (Proof 2)
以下に、プロセッサの動作電源電圧及び動作周波数を複数回変更しながら一のフ レームを符号ィ匕する従来技術と比較して、本願発明がより低消費電力化を図ることが できることを証明する。たとえば、ある特定の時間 Ttにある特定の演算量 Ktを行う場 合、その特定の時間の間は、同一周波数で制御を行い、周波数 Ftを Hereinafter, it will be proved that the present invention can further reduce the power consumption as compared with the prior art in which one frame is encoded while changing the operating power supply voltage and operating frequency of the processor a plurality of times. For example, when performing a certain amount of computation Kt at a certain time Tt Control at the same frequency during that particular time and the frequency Ft
Ft=Kt/Tt  Ft = Kt / Tt
に設定すると低消費電力を実現できる。たとえば、プロセッサ 1の動作電源電圧及 び動作周波数は図 5に示すように r段階に可変とし、任意の一のフレームの必要演算 量を Ktとし、そのフレームの処理に割り当てられる時間を Ttとする。図 6 (a)に示すよ うに、動作周波数を Ftと設定し、プロセッサ 1を動作周波数 Ftで動作させるときの動 作電源電圧を Vddとし、時間 Ttで必要演算量 Ktの処理が終了する場合 (すなわち、 動作周波数が一定の場合)を Caselとし、図 6(b)に示すように、初期値の動作周波 数を h * Ftと設定し、プロセッサを動作周波数 h * Ftで動作させるときの動作電源電 圧を VIとし、時間 T1が経過した時点でプロセッサの動作周波数を h*FtZ2に変更 し、プロセッサ 1を動作周波数 h*FtZ2で動作させるときの動作電源電圧を V2とし、 時間 Tl +T2で必要演算量 Ktの処理が終了する場合 (すなわち、動作周波数の切 り替えが 1回行われる場合）を Case2とし、各 Casel, Case2について前記任意の一のフ レームを符号化する場合を考えてみる。どちらも同一の演算量、すなわち Kt (サイク ル）となる。一方、消費電力は、  When set to, low power consumption can be realized. For example, the operating power supply voltage and operating frequency of processor 1 are variable in r steps as shown in Fig. 5, the required amount of computation for any one frame is Kt, and the time allotted for processing of that frame is Tt. . As shown in Fig. 6 (a), when the operating frequency is set to Ft, the operating power supply voltage when operating processor 1 at the operating frequency Ft is Vdd, and the processing of the required amount of computation Kt is completed at time Tt. (I.e., when the operating frequency is constant) is set to Casel, and as shown in Fig. 6 (b), the initial operating frequency is set to h * Ft, and the processor is operated at the operating frequency h * Ft. The operating power supply voltage is VI, the processor operating frequency is changed to h * FtZ2 when time T1 has elapsed, the operating power supply voltage when operating processor 1 at the operating frequency h * FtZ2 is V2, and the time Tl + Case 2 where the processing of the required amount of computation Kt ends at T2 (that is, when the operating frequency is switched once) is Case 2, and the case where any one of the frames is encoded for Case 1 and Case 2. I'll think about it. Both have the same amount of computation, that is, Kt (cycle). On the other hand, power consumption is
P= a XCXfXVdd2Xt P = a XCXfXVdd 2 Xt
a：係数、 C:プロセッサのトランジスタ数  a: Coefficient, C: Number of transistors in the processor
f:動作周波数、 Vdd:動作電源電圧、 t:動作時間  f: Operating frequency, Vdd: Operating power supply voltage, t: Operating time
で表される。これを用いて Caselの消費電力 Paと Case2の消費電力 Pbを計算する と、  It is represented by Using this, the power consumption Pa of Casel and the power consumption Pb of Case2 are calculated.
Pa= a XCXFtXV2XTt Pa = a XCXFtXV 2 XTt
Pb= a XCX (hXFt) XV12XT1+ a XCX (hXFt/2) XV22XT2 となり、 Pb = a XCX (hXFt) XV1 2 XT1 + a XCX (hXFt / 2) XV2 2 XT2
Pa : Pb = V2 X Tt : (h X VI2 X Tl + (h/2) X V22 X T2) Pa: Pb = V 2 X Tt: (h X VI 2 X Tl + (h / 2) X V2 2 X T2)
となる。ここでたとえば h=l. 5、Tl = lZ3XTt、Tb = 2Z3XTt、 V=l, Vl = l. 5, V2 = 0. 75とすると、 It becomes. For example, if h = l. 5, Tl = lZ3XTt, Tb = 2Z3XTt, V = l, Vl = l. 5, V2 = 0.75,
Pa:Pb = l2: (1. 5X1. 5ソ3+(1. 5/2) XO. 752X (2/3) Pa: Pb = l 2 : (1.5X1.5 So 3+ (1.5 / 2) XO. 75 2 X (2/3)
= 1:1.41 となり、 Pa< Pbとなる。すなわち、決められた演算量を一定時間で処理する場合、同 一演算量 Ktにもかかわらず、 Caselの場合のように、その時間内で処理が終了可能 な最小の動作周波数により、その処理時間を通してプロセッサを一定に動作させるほ うが、従来のように処理時間中に動作周波数を変更する Case2場合よりも低消費電 力であることがわかる。したがって、一定の動作電源電圧及び動作周波数でプロセッ サ 1を動作させながら一のフレームの符号ィ匕処理を行う本発明によれば、ブロックごと に動作電源電圧及び動作周波数が決定されるため一のフレームの符号ィ匕中に何度 も動作電源電圧及び動作周波数が変更される従来技術と比較して、低消費電力化 が図られることがわ力る。 = 1: 1.41 And Pa <Pb. In other words, when processing a fixed amount of computation in a certain amount of time, the processing time is reduced by the minimum operating frequency that can be completed within that time, as in Casel, despite the same amount of computation Kt. It can be seen that the constant operation of the processor through the power consumption is lower than in Case 2 where the operating frequency is changed during the processing time as in the conventional case. Therefore, according to the present invention in which the processor 1 is operated at a constant operating power supply voltage and operating frequency and the code processing of one frame is performed, the operating power supply voltage and operating frequency are determined for each block. Compared with the prior art in which the operating power supply voltage and the operating frequency are changed many times during the sign of the frame, the power consumption can be reduced.
[0095] なお、プロセッサに用意されている動作周波数において、予測された動作周波数 以上の動作周波数と予測された動作周波数より低い動作周波数を選択し、予め定め られたタイミングで割り込みを行い、 2段階に動作周波数等を制御してもよい。その割 り込み時点において、必要演算量計算手段で算出された所定フレームの必要演算 量の残量が、符号ィヒ又は復号ィヒ処理手段による所定フレームの符号ィヒ又は復号ィ匕 処理に実際に必要な演算量の残量よりも小さい場合は、切替タイミングを早めて、高 V、動作周波数で動作する時間を長くするようにしても良 、。  [0095] Of the operating frequencies provided for the processor, an operating frequency that is greater than or equal to the predicted operating frequency and an operating frequency that is lower than the predicted operating frequency are selected, an interrupt is performed at a predetermined timing, and two steps are performed. The operating frequency may be controlled. At the time of the interrupt, the remaining amount of the required calculation amount of the predetermined frame calculated by the required calculation amount calculation means is actually used for the sign or decoding process of the predetermined frame by the code or processing unit. If it is smaller than the amount of computation required for the operation, the switching timing may be advanced so that the time for operating at a high V and operating frequency can be lengthened.
[0096] さらに、動画像符号化処理システム S1は、上記第 1の破綻回避手段により所定フレ ームの符号ィ匕処理の演算量が削減された場合に、所定フレームよりも後に符号ィ匕処 理される後続フレームの必要演算量を増加させるようにしても良 、。この後続フレー ムの必要演算量を増加させる処理は、所定フレームの後に符号化処理が行われる後 続フレームの必要演算量計算手段 2の処理にお!、て、後続フレームの必要演算量 K Pを所定値だけ増加させる処理を行うものであり、後続フレームの必要演算量計算ス テツプ（Step2)内で実行される。具体的には、所定フレームの符号ィ匕処理の後に行 われる後続フレームの必要演算量計算ステップ内において、必要演算量 Kpを m倍（ mは 1以上の実数)する。たとえば m= l. 1とすると、算出した必要演算量 Kpに対し、 10%の余裕を持たせる（増カロさせる）ことができる。また、必要演算量 Kpに実数 n (n は 0以上の実数)を加算しても良ぐ算出された必要演算量の値に関わらず一定の値 で余裕を持たせる (増加させる)ことができる。上述の例を用いると、最終的に算出さ れる必要演算量 Kpは、 [0096] Furthermore, the moving image coding processing system S1 performs code coding processing after a predetermined frame when the calculation amount of code coding processing for a predetermined frame is reduced by the first failure avoidance means. It is also possible to increase the amount of calculation required for subsequent frames to be processed. This process of increasing the required amount of calculation for the subsequent frame is performed by the required calculation amount calculation means 2 for the subsequent frame in which the encoding process is performed after the predetermined frame. This is a process for increasing by a predetermined value, and is executed in the necessary calculation amount calculation step (Step 2) of the subsequent frame. Specifically, the required amount of computation Kp is multiplied by m (m is a real number greater than or equal to 1) in the necessary amount of computation calculation step of the subsequent frame that is performed after the sign key processing of the predetermined frame. For example, if m = l. 1, it is possible to give a 10% margin (increased calories) to the calculated required calculation amount Kp. In addition, even if a real number n (where n is a real number greater than or equal to 0) is added to the required amount of computation Kp, a margin can be given (increased) at a constant value regardless of the calculated amount of necessary computation. . Using the example above, the final calculation is Necessary calculation amount Kp is
Kp = G(Z) Xm ··· (数式 1)  Kp = G (Z) Xm (Formula 1)
Kp = G(Z)+n ··· (数式 2)  Kp = G (Z) + n (Formula 2)
により求められる。 2式を組み合わせて、  Is required. Combining the two formulas,
Kp = G(Z) Xm+n · · · (数式 3)  Kp = G (Z) Xm + n (Equation 3)
としてちよい。  As good as.
[0097] また、所定フレームの必要演算量が実際に必要な演算量よりも小さぐ第 1の破綻 回避手段を実行した場合は、後続フレームの必要演算量計算ステップ内において必 要演算量を算出する関数 G(Z)に用いたいずれかの要素の係数を、 m倍 (mは 1以 上の実数)する。たとえば、破綻回避処理を実行したためにマクロブロックマッチング 回数 Mが小さくなつてしまった場合、上述の例を用いると、次のフレームで算出される 必要演算量 Kpは、  [0097] Also, when the first failure avoiding means is executed in which the required amount of calculation for a given frame is smaller than the actually required amount of calculation, the required amount of calculation is calculated within the required amount of calculation calculation step for the subsequent frame. Multiply the coefficient of any element used in the function G (Z) by m times (m is a real number greater than 1). For example, if the macroblock matching count M has become smaller due to the execution of failure avoidance processing, using the above example, the required amount of computation Kp calculated in the next frame is
Kp=j+ Xm) XM+ j8B+ yC+ δΖ+ ε AQprev  Kp = j + Xm) XM + j8B + yC + δΖ + ε AQprev
Kp=j+(a +η) XM+ j8B+ yC+ δΖ+ ε AQprev  Kp = j + (a + η) XM + j8B + yC + δΖ + ε AQprev
により求められる。 2式を組み合わせて  Is required. Combine the two formulas
Kp=j+ (a Xm + n) XM+ j8B+ yC+ δ Z+ ε AQprev  Kp = j + (a Xm + n) XM + j8B + yC + δ Z + ε AQprev
としてもよい。また、要素一つだけに限らず、いくつかの要素を増加しても良い。関数 に用いるすべての要素を m倍 (mは 1以上の実数）、あるいはすべての要素に実数 n ( nは 0以上の実数)を加算しても良ぐ上記の (数式 1)、（数式 2)、（数式 3)のように算 出された必要演算量の値に関わらず一定の値で増加させても良い。  It is good. Moreover, not only one element but several elements may be increased. (Equation 1), (Equation 2), where all elements used in the function can be multiplied by m (m is a real number greater than or equal to 1), or real numbers n (n is a real number greater than or equal to 0) can be added to all elements. ), (Equation 3) may be increased by a fixed value regardless of the value of the required amount of computation.
[0098] (第 2の実施の形態） [0098] (Second Embodiment)
図 8は、本実施の形態の動画像符号ィ匕処理システム S2の概略ブロック図であり、図 9は、このシステム S2により実現される動画像符号ィ匕方法を説明する概略フローチヤ ートである。本実施の形態の動画像符号ィ匕処理システム S2は、第 2の破綻回避手段 96を備える。第 2の破綻回避手段 96は、所定フレームの符号ィ匕処理が終了した後 に、所定フレームにおける必要演算量算出手段で算出された必要演算量 Kpと所定 フレームの符号ィ匕処理に実際に要した演算量とを比較して、「必要演算量 Kp<実際 に要した演算量」である場合は、所定フレームの後に符号化処理が行われる後続フ レームの必要演算量計算手段の処理にぉ 、て、後続フレームの必要演算量 Kpを増 加させる機能を備える。その他の点については第 1の実施の形態と同様である。動画 像符号ィ匕処理システム S2は、第 1の破綻回避手段'ステップと第 2の破綻回避手段' ステップの両方を備えるようにしても良いし、第 2の破綻回避手段'ステップのみを備 えても良い。 FIG. 8 is a schematic block diagram of the video code key processing system S2 of the present embodiment, and FIG. 9 is a schematic flowchart for explaining the video code key method realized by the system S2. . The moving image code processing system S2 of the present embodiment includes second failure avoiding means 96. The second failure avoiding means 96 is actually required for the required calculation amount Kp calculated by the required calculation amount calculating means for the predetermined frame and the code amount processing for the predetermined frame after the sign key processing for the predetermined frame is completed. Compared with the calculated amount of computation, if “required amount of computation Kp <actual amount of computation required”, the subsequent frame in which encoding processing is performed after a predetermined frame is performed. In addition to the processing of the required calculation amount calculation means for the frame, a function for increasing the required calculation amount Kp of the subsequent frame is provided. The other points are the same as in the first embodiment. The video image code processing system S2 may include both the first failure avoiding means' step and the second failure avoiding means' step, or only the second failure avoiding means' step. Also good.
[0099] (第 2の破綻回避ステップ） [0099] (Second failure avoidance step)
第 2の破綻回避手段 96は、必要演算量計算手段 2の一部として機能する手段であ り、必要演算量計算ステップ (Step2)内で実行される。具体的には、第 2の破綻回避 手段 96により、所定フレームにおける必要演算量算出手段で算出された必要演算 量 Kpと所定フレームの符号ィ匕処理に実際に要した演算量とを比較して、「必要演算 量 Kp<実際に要した演算量」である場合は、所定フレームの後に符号ィ匕処理が行 われる後続フレームの必要演算量計算手段の処理にぉ 、て、後続フレームの必要 演算量 Kpを増加させる。後続フレームの必要演算量 Kpを増加させるには、たとえば 、所定フレームよりも後に符号ィヒ処理される後続フレームの必要演算量計算ステップ 内において、必要演算量 Kpを m倍 (mは 1以上の実数)する。 m= l. 1とすると、算 出した必要演算量 Kpに対し、 10%の余裕を持たせる（増加させる）ことができる。ま た、必要演算量 Kpに実数 n (nは 0以上の実数)を加算しても良ぐ算出された必要演 算量の値に関わらず一定の値で余裕を持たせる（増カロさせる）ことができる。上述の 例を用いると、最終的に算出される必要演算量 Kpは、  The second failure avoiding means 96 is a means that functions as a part of the required amount-of-computation calculation means 2, and is executed in the required amount-of-computation calculation step (Step 2). Specifically, the second failure avoiding means 96 compares the required calculation amount Kp calculated by the required calculation amount calculation means in the predetermined frame with the calculation amount actually required for the sign key processing of the predetermined frame. When “Required amount of computation Kp <actual amount of computation required”, the required computation amount of the succeeding frame is calculated according to the processing of the necessary computation amount calculating means for the subsequent frame in which the sign process is performed after the predetermined frame. Increase the amount Kp. In order to increase the required calculation amount Kp of the subsequent frame, for example, in the required calculation amount calculation step of the subsequent frame that is processed after the predetermined frame, the required calculation amount Kp is multiplied by m (m is 1 or more). Real number). If m = l. 1, it is possible to give (increase) a 10% margin to the calculated required calculation amount Kp. In addition, even if a real number n (n is a real number greater than or equal to 0) is added to the required amount of computation Kp, a certain amount of margin is given regardless of the value of the required amount of computation. be able to. Using the above example, the required calculation amount Kp finally calculated is
Kp = G (Z) X m · · · (数式 1)  Kp = G (Z) X m (1)
Kp = G (Z) +n · · · (数式 2)  Kp = G (Z) + n (2)
により求められる。 2式を組み合わせて、  Is required. Combining the two formulas,
Kp = G (Z) X m+n · · · (数式 3)  Kp = G (Z) X m + n (3)
としてちよい。  As good as.
[0100] また、所定フレームの必要演算量が実際に必要な演算量よりも小さぐ第 1の破綻 回避手段を実行した場合は、後続フレームの必要演算量計算ステップ内において必 要演算量を算出する関数 G (Z)に用いたいずれかの要素の係数を、 m倍 (mは 1以 上の実数)する。たとえば、破綻回避処理を実行したためにマクロブロックマッチング 回数 Mが小さくなつてしまった場合、上述の例を用いると、次のフレームで算出される 必要演算量 Kpは、 [0100] In addition, when the first failure avoidance method in which the required amount of computation for a given frame is smaller than the actually required amount of computation is executed, the required amount of computation is calculated within the required computation amount calculation step for the subsequent frame. Multiply the coefficient of any element used in the function G (Z) by m times (m is a real number greater than 1). For example, macro block matching due to execution of bankruptcy avoidance processing If the number of times M has become small, using the above example, the required amount of computation Kp calculated in the next frame is
Kp=j+ Xm) XM+ j8B+ yC+ δΖ+ ε AQprev  Kp = j + Xm) XM + j8B + yC + δΖ + ε AQprev
Kp=j+(a +η) XM+ j8B+ yC+ δΖ+ ε AQprev  Kp = j + (a + η) XM + j8B + yC + δΖ + ε AQprev
により求められる。 2式を組み合わせて  Is required. Combine the two formulas
Kp=j+ (a Xm + n) XM+ j8B+ yC+ δ Z+ ε AQprev  Kp = j + (a Xm + n) XM + j8B + yC + δ Z + ε AQprev
としてもよい。また、要素一つだけに限らず、いくつかの要素を増加しても良い。関数 に用いるすべての要素を m倍 (mは 1以上の実数）、あるいはすべての要素に実数 n ( nは 0以上の実数)を加算しても良ぐ上記の (数式 1)、（数式 2)、（数式 3)のように算 出された必要演算量の値に関わらず一定の値で増加させても良い。  It is good. Moreover, not only one element but several elements may be increased. (Equation 1), (Equation 2), where all elements used in the function can be multiplied by m (m is a real number greater than or equal to 1), or real numbers n (n is a real number greater than or equal to 0) can be added to all elements. ), (Equation 3) may be increased by a fixed value regardless of the value of the required amount of computation.
[0101] (第 3の実施の形態） [0101] (Third embodiment)
本発明の第 3の実施の形態の動画像符号ィ匕処理システム S3は、第 1の破綻回避 手段に換えて所定フレームの符号ィ匕処理に割り当てられた時間が経過した際に、符 号ィ匕処理が完了していない場合は破綻現象を回避する処理を行う第 3の破綻回避 手段を備える。その他の点については第 2の実施の形態と同様である。  The moving image code key processing system S3 according to the third embodiment of the present invention replaces the first failure avoiding means with the sign code when the time allotted to the code key processing for a predetermined frame has elapsed. (3) A third failure avoiding means is provided for performing processing to avoid the failure phenomenon when the processing is not completed. The other points are the same as in the second embodiment.
[0102] (第 3の破綻回避ステップ） [0102] (Third bankruptcy avoidance step)
図 10は、第 3の破綻回避手段 97を備える動画像符号ィ匕処理システム S3の概略ブ ロック図であり、図 11は、このシステム S3により実現される動画像符号化処理方法を 説明する概略フローチャートである。第 3の破綻回避手段 97は、ステップ 5において 動画像符号ィ匕手段 5が所定フレームの入力画像データ 101の符号ィ匕処理ルーチン を実行して、割り当てられた時間 Teが経過した際に符号ィ匕処理が終了しているか終 了していないかを判定し、符号ィ匕処理が終了していない場合は、次のフレームに割り 当てられている時間を用いて符号ィ匕処理が完了するまで所定フレームの処理時間を 延長することにより、符号化処理が完了できな!/、と 、う破綻現象を回避できるようにし ている。  FIG. 10 is a schematic block diagram of the video code processing system S3 including the third failure avoiding means 97, and FIG. 11 is a schematic diagram for explaining the video encoding processing method realized by the system S3. It is a flowchart. The third failure avoiding means 97 executes the encoding process when the allocated time Te elapses when the moving image code receiving means 5 executes the code processing routine of the input image data 101 of the predetermined frame in step 5. It is determined whether the 匕 process has been completed or not, and if the sign process has not been completed, the time allocated to the next frame is used until the code process is completed. By extending the processing time of a predetermined frame, the encoding process cannot be completed! / And the failure phenomenon can be avoided.
[0103] (入力フレームメモリ）  [0103] (Input frame memory)
以下に、第 3の破綻回避手段 97について具体的に説明する。まず、 1フレームの処 理時間が符号ィ匕方式 (MPEGなど)の規定などにより時間 Tfに制限されている理由の 1つとして、入力画像のデータがフレームレートの間隔で入力フレームメモリに 1フレ ームごと記憶され、入力画像のフレームレートの間隔で入力フレームメモリのデータ 力 S 1フレームずつ更新されると 、うことが挙げられる。入力フレームメモリのデータが更 新される間隔はフレームレートによって異なり、 1フレームの符号ィ匕処理時間は入力 画像のフレームレートにより制限された時間が割り当てられる。ここで、入力フレームメ モリ 7は複数フレーム分 (ここでは 2フレーム分）のデータを記憶できるメモリであり、デ ータの先頭アドレスはメモリのアドレスで指定されているものとし、データは指定された アドレス力 順次格納されて 、くものとする。 The third failure avoiding means 97 will be specifically described below. First, the reason why the processing time of one frame is limited to the time Tf by the definition of the encoding method (MPEG etc.) As one, the input image data is stored in the input frame memory for each frame at the frame rate interval, and the input frame memory data force S is updated frame by frame at the input image frame rate interval. Can be mentioned. The interval at which the data in the input frame memory is updated varies depending on the frame rate, and the time for processing the code for one frame is limited by the frame rate of the input image. Here, the input frame memory 7 is a memory that can store data for multiple frames (here, 2 frames), and the start address of the data is specified by the memory address, and the data is not specified. Address power shall be stored sequentially.
[0104] (処理時間制御）  [0104] (Processing time control)
図 12は通常の符号化処理の場合の処理時間推移を示す図であり、図 13,図 14, 図 15は第 3の破綻回避手段によって第 3の破綻回避ステップを実行した符号ィ匕処理 の場合の処理時間推移を示す図である。たとえば、実時間処理において、 1秒間に 3 0枚の画像データが入力されれば、外部メモリのデータは 1Z30秒毎に更新される。 動画像符号化処理システムで 1秒間に 10枚の画像を符号化する場合、 3枚に 1枚の 割合で入力画像が符号ィ匕されることになる。ここで、符号ィ匕される 1フレームに割り当 てられる時間は Tf= lZlO秒であり、入力画像のフレームの番号を # 1, # 2, # 3, # 4, · · ·とすると、通常符号ィ匕されるフレームの番号は，図 12で示すように # 1, # 4 , # 7, # 10,…となる。第 3の破綻回避手段は、所定フレームにおいて、必要演算 量計算手段により算出された必要演算量 Kpが実際に必要な演算量 Kmよりも小さい 場合、つまり、符号ィ匕処理に割り当てられた時間が経過したにもかかわらず、符号ィ匕 処理が終了されな 、場合に対し、次フレームに割り当てられて!/、る符号化処理時間 を用いて所定フレームの処理時間 Teを延長し、所定フレームの符号化処理を続行し て処理を完了させるために、入力フレームメモリの状態信号を制御する手段である。 第 3の破綻回避手段としては、以下の例 1、例 2が挙げられる。  Fig. 12 is a diagram showing the transition of processing time in the case of normal encoding processing. Fig. 13, Fig. 14 and Fig. 15 are diagrams of the code processing where the third failure avoiding means is executed by the third failure avoiding means. It is a figure which shows the processing time transition in a case. For example, in real-time processing, if 30 image data are input per second, the data in the external memory is updated every 1Z30 seconds. When encoding 10 images per second with the moving image encoding processing system, the input image is encoded at a rate of 1 out of 3 images. Here, the time allotted to one frame to be encoded is Tf = lZlO seconds, and if the frame number of the input image is # 1, # 2, # 3, # 4, The numbers of the frames to be signed are # 1, # 4, # 7, # 10, as shown in Fig. 12. The third failure avoiding means is for a case where the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount Km in the predetermined frame, that is, the time allocated to the sign key processing. In spite of the elapse of time, the encoding process is not completed.In contrast, the processing time Te of the predetermined frame is extended by using the encoding processing time assigned to the next frame! This is a means for controlling the status signal of the input frame memory in order to continue the encoding process and complete the process. Examples 3 and 2 below are examples of third means of avoiding bankruptcy.
[0105] (第 3の破綻回避手段の例 1)第 3の破綻回避手段 97は後述する手順 1から手順 4 を実行する機能を備える。（手順 1)所定フレームにおいて符号ィ匕処理に割り当てら れた時間 Teの延長を行うため、入力フレームメモリの状態制御を行い、所定フレーム の符号化処理を続行する。（手順 2)所定フレームの符号化処理が完了した際に、次 フレームの符号ィ匕処理可能な時間を算出する。（手順 3)入力フレームメモリ状態の 切り替え制御を行う。以下、手順 1から手順 3について詳述する。 (Example of Third Failure Avoiding Means 1) The third failure avoiding means 97 has a function of executing Step 1 to Step 4 described later. (Procedure 1) In order to extend the time Te allocated to the encoding process in a predetermined frame, the state of the input frame memory is controlled and the encoding process of the predetermined frame is continued. (Procedure 2) When the encoding process for a given frame is completed, The time for which the frame sign can be processed is calculated. (Procedure 3) Control switching of input frame memory status. Steps 1 to 3 are described in detail below.
[0106] (手順 1 :時間 Teの延長）図 13に、破綻回避手段 97による符号ィ匕処理時間の推移 を示す。上記の例を用いると、 #4のフレームデータは入力フレームメモリ 7bに記憶さ れ、 # 7のフレームデータは入力フレームメモリ 7bに記憶されている。ここで、 #4の 符号ィ匕が符号ィ匕処理に割り当てられた時間 Teにおいて終了できなかった場合、 #4 の符号化処理時間 Teを延長して処理を続行するためには、 # 4のフレームデータを 保持する必要がある。そこで、入力フレームメモリの状態制御を行い、入力フレームメ モリ 7bは書き込み不可の状態に制御し、入力フレームメモリ 7aを書き込み可能な状 態に維持し、 # 8のフレームデータの書き込み先を入力フレームメモリ 7aに変更する 。入力フレームメモリ 7bは #4のフレームデータを保持でき、 #4の符号化処理は続 行可能となるので、 # 4の符号ィ匕処理が完了するまで時間を延長することができる。  (Procedure 1: Extension of time Te) FIG. 13 shows the transition of the sign key processing time by the failure avoiding means 97. Using the above example, # 4 frame data is stored in the input frame memory 7b, and # 7 frame data is stored in the input frame memory 7b. Here, if the sign # 4 of # 4 could not be finished at the time Te allocated for the sign key processing, to continue the process by extending the # 4 encoding time Te, It is necessary to hold the frame data. Therefore, the state of the input frame memory is controlled, the input frame memory 7b is controlled to be in a writable state, the input frame memory 7a is maintained in a writable state, and the frame data write destination of # 8 is set to the input frame. Change to memory 7a. Since the input frame memory 7b can hold the frame data of # 4 and the encoding process of # 4 can be continued, the time can be extended until the encoding process of # 4 is completed.
[0107] (手順 2:次のフレーム処理時間の算出） #4の符号化処理が完了した際、割り当て られた時間 Te経過後、 #4の符号ィ匕処理に費やした延長時間 Toが求められる。次 の符号ィ匕対象となるフレームの符号ィ匕処理時間 T1は、 Tl=Te— Toにより算出される 。ここで、時間 T1は、次の符号化対象となるフレームに対し、必要とされる演算量が必 要演算量計算手段により算出され、動作決定手段により動作周波数 ·動作電源電圧 および基板バイアス電圧を決定する際に用いるので、時間 T1を算出しておく。なお、 時間 T1は次フレームの必要演算量計算手段を実行する際に算出してもよい。  (Step 2: Calculation of the next frame processing time) When the encoding process of # 4 is completed, after the allotted time Te, the extension time To spent for the encoding process of # 4 is obtained. . The code processing time T1 of the next frame to be encoded is calculated by Tl = Te-To. Here, for the time T1, the required amount of calculation is calculated by the required calculation amount calculation means for the next frame to be encoded, and the operation frequency / operation power supply voltage and substrate bias voltage are calculated by the operation determination means. Calculate the time T1 because it will be used when making the decision. The time T1 may be calculated when executing the necessary calculation amount calculation means for the next frame.
[0108] (手順 3 :メモリ状態の切り替え制御）手順 2により算出された延長時間 Toにおいて、 Toく TfZ3の場合は、次の符号化対象フレームは # 8となる。 # 8のフレームデータ の書き込みが終了する際に、入力フレームメモリ 7の状態の切り替え制御を行うことで 、入力フレームメモリ 7aは # 8のフレームデータの読み込み状態に、入力フレームメ モリ 7bは次のフレームである # 9の書き込み状態に切り替わり、 # 8の符号化ステツ プに移行する。また、 TfZ3く Toく 2TfZ3の場合は、図 14に示すように、次の符号 化対象フレームは # 9となり、 # 9のフレームデータの書き込みが終了する際に、入 カフレームメモリ 7の状態制御の切り替えが同様に行われ、 # 9の符号化ステップに 移行する。以上の手順をもって、入力フレームメモリの状態を制御することにより、所 定フレームの処理時間を変更して時間を延長することができ、所定フレームの符号ィ匕 処理を完了させることで、破綻を完全に回避できる。 (Procedure 3: Memory state switching control) In the extended time To calculated in Procedure 2, if To is TfZ3, the next frame to be encoded is # 8. When the writing of the frame data of # 8 is completed, the input frame memory 7a is in the reading state of the frame data of # 8 by controlling the switching of the state of the input frame memory 7, and the input frame memory 7b Switch to the # 9 writing state, which is a frame, and shift to the # 8 encoding step. In addition, in the case of TfZ3 to To 2TfZ3, as shown in Fig. 14, the next frame to be encoded is # 9, and when the writing of frame data of # 9 is completed, the status control of input frame memory 7 is performed. Is switched in the same way, and the process proceeds to the encoding step # 9. By controlling the state of the input frame memory using the above procedure, The processing time of the fixed frame can be changed to extend the time, and the failure of the predetermined frame can be completely avoided by completing the encoding process of the predetermined frame.
[0109] また、次フレームの符号ィ匕処理時間 T1を算出することで、次フレームの符号ィ匕に最 適な動作周波数'動作電源電圧および基板バイアス電圧を決定することができ、後 続フレームに対しても効率良く低消費電力化を図ることができる。なお、時間 T1の算 出は、次の符号化対象となるフレームの符号化ステップの必要演算量計算手段を実 行する際に算出しても良い。 [0109] Also, by calculating the sign key processing time T1 of the next frame, it is possible to determine the optimum operating frequency for the sign key of the next frame, that is, the operating power supply voltage and the substrate bias voltage. The power consumption can be reduced efficiently. Note that the calculation of the time T1 may be performed when the necessary calculation amount calculating means for the encoding step of the next encoding target frame is executed.
[0110] (第 3の破綻回避手段の例 2)第 3の破綻回避手段 98は、第 3の破綻回避手段 97に 対し、入力フレームメモリの状態制御方法のみが異なり、その他の点については、第 3の破綻回避手段 97と同様である。図 15は、第 3の破綻回避手段 98による符号ィ匕 処理の時間推移を示したものである。第 3の破綻回避手段 98は、入力フレームメモリ の状態制御にぉ 、て、 # 4に割り当てられた符号化処理時間 Teにお 、て処理が終 了できな力つた場合、少なくとも延長時間 Toの間は入力フレームメモリ 7a, 7bともに 書き込み不可の状態に制御し、時間 Teを時間 Toだけ延長する。入力フレームメモリ は、入力フレームメモリ 7bに格納されている # 4のフレームデータと、入力フレームメ モリ 7aに格納されている # 7のフレームデータが保持された状態となる。 # 4のフレー ムについては時間 Teの延長により破綻現象が回避される。延長時間 To中に入力さ れる # 8のフレームは、入力フレームメモリに書き込まれない。 # 4の符号化処理が完 了した時点で、入力フレームメモリ 7a, 7bの書き込み '読み出しの状態切り替えを行 う。書き込み状態に制御された入力フレームメモリ 7bは、入力されたフレームデータ が書き込み可能となる。入力フレームメモリ 7aは # 7のフレームデータの読み出しが 行われ、入力フレームメモリ 7bには # 9, # 10のフレームが順次書き込まれる。延長 時間 Toにかかわらず、 # 4のフレームの符号化処理の後は、 # 7, # 10のフレーム データの符号化処理が順次行われる。通常の符号化処理と等 ヽフレーム番号を符 号化させるため、通常の符号ィ匕処理と同等の画質を得ることができる。  (Example of third failure avoiding means 2) The third failure avoiding means 98 is different from the third failure avoiding means 97 only in the state control method of the input frame memory. This is the same as the third bankruptcy avoidance measure 97. FIG. 15 shows the time transition of the sign key processing by the third bankruptcy avoiding means 98. The third failure avoiding means 98, when controlling the state of the input frame memory, if the processing cannot be completed in the encoding processing time Te assigned to # 4, at least the extension time To. During this time, the input frame memories 7a and 7b are controlled so that they cannot be written, and the time Te is extended by the time To. The input frame memory holds the # 4 frame data stored in the input frame memory 7b and the # 7 frame data stored in the input frame memory 7a. For the # 4 frame, the failure phenomenon is avoided by extending the time Te. The # 8 frame input during the extension time To is not written to the input frame memory. When the encoding process of # 4 is completed, the state of writing 'reading' of the input frame memories 7a and 7b is switched. The input frame memory 7b controlled to the writing state can write the input frame data. The frame data # 7 is read out from the input frame memory 7a, and the frames # 9 and # 10 are sequentially written into the input frame memory 7b. Regardless of the extension time To, after the encoding process of the # 4 frame, the encoding process of the # 7 and # 10 frame data is sequentially performed. Since the same frame number as that of the normal encoding process is encoded, an image quality equivalent to that of the normal encoding process can be obtained.
[0111] また、入力画像データのフレームレートと動画像符号化処理システムのフレームレ ートが等しい場合は、符号化されるフレームは # L # 2, # 3, # 4, # 5, # 6, # 7, • · 'であり、符号ィ匕処理時間を延長できる範囲は最大 Tfとなる。 # 4の符号化処理の 延長時間が Tfを超える場合は # 6を次のフレームとして符号ィ匕しても良い。 [0111] Also, if the frame rate of the input image data is equal to the frame rate of the video encoding system, the encoded frames are # L # 2, # 3, # 4, # 5, # 6 , # 7, • · ', and the range in which the sign key processing time can be extended is the maximum Tf. # 4 encoding process If the extension time exceeds Tf, you may sign # 6 as the next frame.
[0112] (第 4の実施の形態）  [0112] (Fourth embodiment)
本発明の第 4の実施の形態の動画像復号ィ匕処理システム S4は、符号化された動 画像を復号化するシステムである。図 16は動画像復号化処理システム S4の動作を 示した概略ブロック図である。本実施の形態の動画像復号ィ匕処理システム S4は、動 作電源電圧及び基板バイアス電圧及び動作周波数が r段階 (rは 2以上の整数）に用 意され且つプログラムにより動作電源電圧及び基板バイアス電圧及び動作周波数を 変更可能なプロセッサ 1と、前記プロセッサ 1の動作電源電圧及び基板バイアス電圧 及び動作周波数を制御する動作制御手段 4と、前フレームの復号化データを記憶す る局部復号フレームメモリ 46とを備える。また、局部復号メモリ 46は動作制御手段 4 により，プロセッサ 1と同様に動作電源電圧 ·基板バイアス電圧 ·動作周波数が制御さ れてもよい。プロセッサ 1は、プロセッサ 1上で動作する手段として、必要演算量計算 手段 42と、動作決定手段 3と、動画像復号化手段 35と、第 3の破綻回避手段 97とを 備える。符号 401は入力符号化データ、符号 102は動作電源電圧'基板バイアス電 圧 ·動作周波数指示、符号 105は動作電源電圧 ·基板バイアス電圧 ·動作周波数供 給、符号 406は復号ィ匕データであり、第 1の実施の形態と同一符号は同一機能又は それ相当の機能を有する部分である。符号ィ匕ではなく復号ィ匕を行う点及び下記以外 の点は第 3の実施の形態と同様である。  The moving image decoding processing system S4 according to the fourth embodiment of the present invention is a system for decoding an encoded moving image. FIG. 16 is a schematic block diagram showing the operation of the video decoding processing system S4. In the moving image decoding processing system S4 of this embodiment, the operation power supply voltage, the substrate bias voltage, and the operation frequency are prepared in r stages (r is an integer of 2 or more), and the operation power supply voltage and the substrate bias are programmed. The processor 1 capable of changing the voltage and the operating frequency, the operation control means 4 for controlling the operating power supply voltage, the substrate bias voltage and the operating frequency of the processor 1, and the local decoding frame memory 46 for storing the decoding data of the previous frame 46 With. In the local decoding memory 46, the operation power supply voltage, the substrate bias voltage, and the operation frequency may be controlled by the operation control means 4 in the same manner as the processor 1. The processor 1 includes necessary calculation amount calculating means 42, operation determining means 3, moving picture decoding means 35, and third failure avoiding means 97 as means that operate on the processor 1. Reference numeral 401 is input encoded data, reference numeral 102 is operating power supply voltage'substrate bias voltage / operating frequency instruction, reference numeral 105 is operating power supply voltage / substrate bias voltage / operating frequency supply, reference numeral 406 is decoding key data, The same reference numerals as those in the first embodiment are parts having the same function or equivalent functions. The point of performing decoding key instead of code key is the same as that of the third embodiment except for the following points.
[0113] 図 16に従って、動画像復号化処理システム S4の動作を説明する。以下、順次復号 化されるフレームのうちこれ力 復号化される任意の一のフレーム（すなわち、あるフ レームが復号化された時点を基準とすると次に復号化されるフレームであり、換言す ると、その時点において未だに復号化処理されておらず未来に復号化処理が行わ れる予定であるフレーム）を所定フレーム、所定フレームより前に復号ィ匕された一のフ レーム (過去に復号ィ匕されたフレーム）を前フレームとし、所定フレームを復号化する 処理について説明する力 いずれのフレームについても同様の処理が行われる。コ ンピュータを動画像復号化処理システム S4として機能させる動画像復号化処理プロ グラム Prg4は、前記動画像符号ィ匕処理プログラム Prglとほぼ同様である力 ステツ プ 5において、所定フレームの符号ィ匕データを復号ィ匕させる動画像復号ィ匕手段 45と してコンピュータ (詳しくはコンピュータに内蔵されるプロセッサ 1)を機能させる。動画 像復号化処理システム S4に入力されてきた入力符号化データ 401は、必要演算量 計算手段 42に入力される。必要演算量計算手段 42は符号化データ 401の一フレー ム分 (すなわち、所定フレームの符号ィ匕データ 401)の発生情報量 (ビット数) FBを計 算し、必要計算量 Kpを予測する計算を行う (必要演算量計算ステップ)。必要演算 量 Κρは、 [0113] The operation of the video decoding processing system S4 will be described with reference to FIG. Hereinafter, one of the frames that is sequentially decoded is arbitrarily decoded (that is, the frame that is decoded next on the basis of the time when a certain frame is decoded, in other words, At that time, a frame that has not been decoded yet and is scheduled to be decoded in the future) is a predetermined frame, and one frame that has been decoded before the predetermined frame (decoded in the past). The power to explain the process of decoding a predetermined frame with the previous frame as the previous frame. The same process is performed for both frames. The moving picture decoding processing program Prg4 that causes the computer to function as the moving picture decoding processing system S4 is substantially the same as the moving picture code processing program Prgl. Video decoding means 45 for decoding The computer (specifically, the processor 1 built in the computer) is made to function. The input encoded data 401 input to the moving image decoding processing system S4 is input to the necessary calculation amount calculation means 42. The required calculation amount calculation means 42 calculates the amount of information (number of bits) FB generated for one frame of the encoded data 401 (that is, the encoded data 401 of the predetermined frame), and predicts the required calculation amount Kp. (Required calculation step). The required amount of computation Κρ is
Kp = G (FB, MVa, MVv, B, C, BR, Q, A Q, I, E, P)  Kp = G (FB, MVa, MVv, B, C, BR, Q, A Q, I, E, P)
で表される。ここで、 FBは所定フレームもしくは前フレームの符号ィ匕データのビット数 、 MVaは所定フレームもしくは前フレームの動きベクトルの大きさの平均値、 MVvは 所定フレームもしくは前フレームの動きベクトルの大きさの分散、 Bは所定フレームも しくは前フレームの有効ブロック数、 Cは所定フレームもしくは前フレームの有効係数 の数、 BRは所定フレームもしくは前フレームのビットレート、 Qは所定フレームもしくは 前フレームの量子化ステップサイズの平均値、 Δ Qは所定フレームと前フレームの量 子化ステップサイズの平均値の差もしくは前フレームと前々フレームの量子化ステツ プサイズの平均値の差、 Iは所定フレーム力 ピクチャである力， Pピクチャであるか B ピクチャであるかの種類、 Eは前フレームの復号ィヒに要した演算量、 Pは必要演算量 計算手段により算出された前フレームの必要演算量を表す。以下、上記関数 Gにつ いて説明する。  It is represented by Where FB is the number of bits of the code data of the predetermined frame or the previous frame, MVa is the average value of the motion vector of the predetermined frame or the previous frame, and MVv is the size of the motion vector of the predetermined frame or the previous frame Variance, B is the number of effective blocks of the specified frame or previous frame, C is the number of effective coefficients of the specified frame or previous frame, BR is the bit rate of the specified frame or previous frame, Q is the quantization of the specified frame or previous frame The average step size, Δ Q is the difference between the average quantization step sizes of the predetermined frame and the previous frame, or the average difference between the quantization step sizes of the previous frame and the previous frame, and I is the predetermined frame power picture. A certain force, the type of P picture or B picture, E is the performance required for decoding the previous frame. The amount, P is represents a necessary calculation amount of the previous frame calculated by the required calculation amount calculating means. The function G will be described below.
[0114] 所定フレームの復号化に必要な演算量は、所定フレームの復号ィ匕で実行される ID CT処理、 IQ処理、 VLD処理の実行回数に依存する。また、 IDCT処理の実行回数 は所定フレームに含まれる有効ブロックの数に、 IQ処理および VLD処理の実行回 数は所定フレームに含まれる有効係数の数に依存する。すなわち、所定フレームに 含まれる有効ブロックの数や有効係数の数が大き ヽ（小さ ヽ）場合は、復号化処理に 必要な演算量は大きく（小さく）なる。したがって、上記関数 Gは、 B, Cが大きい (小さ V、)場合、 Kpを大きく (小さく)設定するように構成する。  [0114] The amount of computation required for decoding a predetermined frame depends on the number of times ID CT processing, IQ processing, and VLD processing are executed in decoding the predetermined frame. The number of executions of IDCT processing depends on the number of effective blocks included in the predetermined frame, and the number of executions of IQ processing and VLD processing depends on the number of effective coefficients included in the predetermined frame. That is, when the number of effective blocks and the number of effective coefficients included in a predetermined frame is large (small), the amount of calculation required for the decoding process is large (small). Therefore, the above function G is configured so that Kp is set large (small) when B and C are large (small V).
[0115] 前フレームと所定フレームの間で画像の変化が大きい（小さい）場合、動きベクトル の大きさの平均値 MVaや動きベクトルの大きさの分散 MVvが大きく（小さく）なるが、 このとき所定フレームの有効ブロックの数や有効係数の数は大きく（小さく）なり、符号 化処理に必要な演算量は大きく（小さく）なる。したがって、上記関数 Gは， MVaや M Vvが大き ヽ (小さく）場合、 Kpを大きく（小さく)設定するように構成する。 [0115] When the change in the image between the previous frame and the predetermined frame is large (small), the average value MVa of the motion vector size and the variance MVv of the motion vector size become large (small). The number of effective blocks and the number of effective coefficients in the frame increases (decreases), and the sign The amount of computation required for the digitization process is large (small). Therefore, the above function G is configured so that Kp is set large (small) when MVa and MVv are large (small).
[0116] 所定フレームが Iピクチャの場合、復号化データを生成するときに予測画像と差分 画像の加算を行わなくてよいので、復号ィ匕処理に必要な演算量は小さくなる。したが つて、上記関数 Gは、所定フレームが Iピクチャの場合、 Kpを小さく設定するように構 成する。 [0116] When the predetermined frame is an I picture, it is not necessary to add the prediction image and the difference image when generating the decoded data, so the amount of calculation required for the decoding process is small. Therefore, the above function G is configured to set Kp small when the predetermined frame is an I picture.
[0117] 符号化データのビット数 FBやフレームレート BRが大きい（小さい）場合、有効ブロッ クの数や有効係数の数は大きく（小さく）なる。したがって、上記関数 Gは、 FBや BR が大きい (小さい)場合、 Kpを大きく（小さく)設定するように構成する。また、量子化ス テツプサイズはビットレートの制御に際して値が変更されるため、量子化ステップサイ ズの平均値 Qや量子化ステップサイズの平均値の差 Δ Qを考慮することで、上記関 数 Gが算出する Kpが実際に所定フレームを復号ィ匕するために必要な演算量に近い 値とすることができる。  [0117] The number of bits of encoded data When the FB and frame rate BR are large (small), the number of effective blocks and the number of effective coefficients are large (small). Therefore, the above function G is configured so that Kp is set large (small) when FB and BR are large (small). Also, since the quantization step size is changed when the bit rate is controlled, the above function G can be calculated by taking into consideration the average value Q of the quantization step size and the difference ΔQ of the average value of the quantization step size. The Kp calculated by can be a value close to the amount of computation required to actually decode the predetermined frame.
[0118] 動画像は連続するフレーム間での相関が大きいため、 MVa, MVv, B, C, BR, F B, Qは所定フレームと前フレームとで近い値となる。したがって、これらのパラメータ を上記関数 Gで使用する場合は、所定フレームでの値を用いても良いし、前フレーム での値を用いても良い。所定フレームでの値を用いる場合は、入力符号化データを 受信した後、このデータの一部を復号ィ匕し、値を取り出して用いる。このとき、所定フ レームでの値を用いることで予測演算量 Kpを実際の復号ィ匕処理に必要な演算量に より近い値にすることができるメリットがある。前フレームでの値を用いる場合、所定フ レームの入力符号ィ匕データを受信する前に予測演算量 Kpを算出することができるた め、入力符号ィ匕データを受信しながら、受信済みのデータ分について復号ィ匕処理を 同時に行うことができるメリットがある。  [0118] Since a moving image has a large correlation between consecutive frames, MVa, MVv, B, C, BR, FB, and Q are close to each other between a predetermined frame and a previous frame. Therefore, when these parameters are used in the above function G, values in a predetermined frame may be used, or values in the previous frame may be used. When using a value in a predetermined frame, after receiving input encoded data, a part of this data is decoded and a value is extracted and used. At this time, there is an advantage that the predicted calculation amount Kp can be made closer to the calculation amount necessary for the actual decoding process by using the value in the predetermined frame. When the value in the previous frame is used, the prediction calculation amount Kp can be calculated before receiving the input code key data of the predetermined frame. There is an advantage that the decryption process can be performed simultaneously for minutes.
[0119] また、動画像は連続するフレーム間での相関が大きいため、所定フレームの復号化 処理に必要演算量は前フレームの復号化処理で実際に必要であった演算量 Eと近 い値となる。さらに、必要演算量計算手段で算出される予測演算量が実際の復号ィ匕 処理に要した演算量に近い値となる場合， P^Eとなる。したがって、 Eや Pを考慮す ることで、上記関数 Gが算出する Kpが実際に所定フレームを復号ィ匕するために必要 な演算量に近 、値とすることができる。 FBは一フレーム分の発生情報量 (ビット数)で ある。ただし、関数 Gは要素 FB, MVa, MVv, B, C, BR, D, Q, Δ Qprev, I, E, P の一つ以上の要素を用いて導き出される関数である。必要演算量 Kpは、所定フレー ムに必要と予測される演算性能 (周波数，サイクル)であり、所定フレーム内のビット数 FBが大きければ高い値と、ビット数 FBが小さければ低い値となる。また、必要演算量 Kpを予測する計算である必要演算量計算手段 42の要素として、所定フレームがフ レーム内符号ィ匕である力フレーム間符号ィ匕であるかの種類も使用することが可能で あり、所定フレームがフレーム内符号ィ匕である場合の必要演算量 Kpは小さい値と、 フレーム間符号ィ匕である場合の必要演算量 Kpは大きい値となる。さらに、必要計算 量 Kpは、動きベクトルの大きさの平均値 (これから復号ィ匕するフレームの，もしくは前 フレームのもの) MVa、動きベクトルの大きさの分散 （これから復号化するフレームの ,もしくは前フレームのもの) MVv、有効ブロック数 (これから復号化するフレームの，も しくは前フレームのもの) B、有効係数の数 (これから復号化するフレームの，もしくは 前フレームのもの) C、ビットレート (これから復号化するフレームの，もしくは前フレーム のもの) BR、発生情報量 (これから復号ィ匕するフレームの，もしくは前フレームのもの） FB、量子化ステップサイズの平均値 (これ力 復号化するフレームの，もしくは前フレ ームのもの) Q、量子化ステップサイズの平均値の差 (これ力 復号化するフレームの Qと 1つ前のフレームの Qの差，もしくは 1つ前のフレームの Qと 2つ前のフレームの Q の差) A Q、 Iピクチャである力 Ρピクチャである力 Βピクチャであるかの種類 I、前フレー ムの復号化に実際に要した演算量 Ε、前フレームの復号化に必要な演算量の予測 値 (すなわち、必要演算量計算手段により算出された前フレームの必要演算量) Ρに も影響され、これらを必要演算量計算手段 42において要素として使用しても良い。 例えば、動きベクトルの大きさの平均値 (これから復号ィ匕するフレームの，もしくは前フ レームのもの) MVa、動きベクトルの大きさの分散 （これから復号化するフレームの， もしくは前フレームのもの) MVv、有効ブロック数 (これから復号化するフレームの，も しくは前フレームのもの) B、有効係数の数 (これから復号化するフレームの，もしくは 前フレームのもの) Cについては、各要素それぞれについて、他の要素の値が変化し な 、と仮定したときに、要素の値が大き 、場合は小さ 、場合と比較して必要演算量 K pが相対的に大きくなるようにし、要素の値が小さ!、場合は大き!、場合と比較して必 要演算量 Kpが相対的に小さくなるようにする。必要演算量計算手段 42においては、 これらの要素のうち一つの要素のみを使用しても良いし、複数組み合わせて使用し ても良い。すなわち、これらの複数の要素は所定フレームの復号ィ匕処理のために必 要な必要演算量に影響を与える要素であるため、必要演算量計算手段 42が、これら の要素に応じて必要演算量 Kp (サイクル)を増減させるように計算を行うことにより、 必要演算量計算手段 42により計算される必要演算量 Kpが現実に復号化処理を行 つたときの演算量により近い値となる。 [0119] In addition, since a moving image has a large correlation between consecutive frames, the amount of computation required for decoding the predetermined frame is close to the amount of computation E actually required for the decoding processing of the previous frame. It becomes. Furthermore, if the predicted amount of computation calculated by the necessary amount-of-computation calculation means is close to the amount of computation required for the actual decoding process, P ^ E. Therefore, considering E and P, the Kp calculated by the above function G is necessary to actually decode the predetermined frame. It can be a value close to the amount of computation. FB is the amount of generated information (number of bits) for one frame. However, the function G is a function derived by using one or more elements FB, MVa, MVv, B, C, BR, D, Q, ΔQprev, I, E, and P. The required amount of computation Kp is the computation performance (frequency, cycle) that is predicted to be necessary for a given frame. The value is high when the number of bits FB in a given frame is large, and is low when the number of bits FB is small. In addition, it is also possible to use the type of whether a given frame is a force interframe code that is an intra-frame code as an element of the required complexity calculation means that is a calculation that predicts the required complexity Kp Therefore, the required calculation amount Kp when the predetermined frame is the intraframe code 匕 is small, and the required calculation amount Kp when the predetermined frame is the interframe code 匕 is a large value. Furthermore, the required calculation amount Kp is the average value of the motion vector size (for the frame to be decoded or the previous frame) MVA, the variance of the size of the motion vector (for the frame to be decoded or the previous frame) MVv, number of effective blocks (for frame to be decoded or previous frame) B, number of effective coefficients (for frame to be decoded or previous frame) C, bit rate ( The frame to be decoded or the previous frame) BR, the amount of generated information (the frame to be decoded or the previous frame) FB, the average quantization step size (this power of the frame to be decoded) Q, the difference between the average quantization step sizes (this is the difference between the Q of the frame to be decoded and the Q of the previous frame) , Or the difference between the Q of the previous frame and the Q of the second previous frame) AQ, I picture power 力 Picture power Β Picture type I, actually decoding the previous frame The amount of computation required Ε and the predicted value of the amount of computation required for decoding the previous frame (that is, the amount of computation required for the previous frame calculated by the required computation amount calculation means) 影響 are also affected. It may be used as an element in means 42. For example, the average motion vector size (from the frame to be decoded or from the previous frame) MVa, the variance of the motion vector size (from the frame to be decoded or from the previous frame) MVv , The number of effective blocks (from the frame to be decoded or from the previous frame) B, the number of effective coefficients (from the frame to be decoded or from the previous frame) C Assuming that the value of the element does not change, the value of the element is large. p is relatively large, and the element value is small !, large in some cases, and the required amount of computation Kp is relatively small compared to the case. In the necessary calculation amount calculation means 42, only one of these elements may be used or a plurality of elements may be used in combination. In other words, these multiple elements are elements that affect the required amount of calculation necessary for the decoding process of a predetermined frame, and therefore the required calculation amount calculating means 42 needs to calculate the necessary amount of calculation according to these elements. By calculating so as to increase or decrease Kp (cycle), the required calculation amount Kp calculated by the required calculation amount calculating means 42 becomes closer to the calculation amount when the decoding process is actually performed.
[0120] 動作決定手段 3 (動作決定ステップ)及び動作制御手段 4は、前記第 1の実施の形 態と同様である。動画像復号ィ匕手段 45は、所定フレームの入力符号ィ匕データ 401を 復号化して復号化データ 406を生成する (動画像復号化ステップ)。動画像復号ィ匕 手段 45による復号化処理に際しては、動作制御手段 4により動作制御手段 4により 一定の動作電源電圧及び基板バイアス電圧及び動作周波数でプロセッサ 1を動作さ せながら復号化処理が行われる。フレームごとに、そのフレームの復号ィ匕処理の前に 必要な必要演算量が算出され、その必要演算量に応じた一定の動作周波数及び動 作電源電圧及び基板バイアス電圧でプロセッサを動作させながらそのフレームの復 号ィ匕が行われるため、低消費電力化を図ることができる。復号化データ 406は、携帯 電話やパソコンの画像表示部に動画像として表示されたり、ハードディスク等の記憶 媒体に記憶されたりする。なお、プロセッサに用意されている動作周波数において、 予測された動作周波数以上の動作周波数と予測された動作周波数より低い動作周 波数を選択し、予め定められたタイミングで割り込みを行い、 2段階に動作周波数等 を制御してもよい。 [0120] The operation determining means 3 (operation determining step) and the operation control means 4 are the same as in the first embodiment. The moving picture decoding means 45 decodes the input code key data 401 of a predetermined frame to generate decoded data 406 (moving picture decoding step). In the decoding process by the moving picture decoding means 45, the operation control means 4 performs the decoding process while the processor 1 is operated by the operation control means 4 at a constant operating power supply voltage, substrate bias voltage and operating frequency. . For each frame, the required amount of computation required before decoding the frame is calculated, and the processor is operated while operating at a constant operating frequency, operating power supply voltage, and substrate bias voltage according to the required amount of computation. Since frame decoding is performed, low power consumption can be achieved. The decrypted data 406 is displayed as a moving image on an image display unit of a mobile phone or a personal computer, or stored in a storage medium such as a hard disk. In the operating frequency prepared for the processor, select an operating frequency that is higher than the predicted operating frequency and lower than the predicted operating frequency, interrupt at a predetermined timing, and operate in two stages. The frequency may be controlled.
[0121] 動画像復号ィ匕処理システム S4においても、上記第 3の破綻回避手段 97, 98を備 えることが好ましい。各破綻回避手段は第 3の実施の形態とほぼ同様であるが、符号 化処理の演算量ではなく復号化処理の演算量につ!、て判断する点で異なる。第 3の 破綻回避手段 97, 98により、破綻現象を回避することができる。  [0121] It is preferable that the moving image decoding process system S4 also includes the third failure avoiding means 97, 98. Each failure avoiding means is almost the same as in the third embodiment, but differs in that it judges not only the amount of computation of the encoding process but the amount of computation of the decoding process. The third failure prevention means 97, 98 can avoid the failure phenomenon.
[0122] 本発明の動画像符号ィ匕処理システムは、第 1の破綻回避手段 91, 92, 93, 94、緩 和手段 95を伴う第 1の破綻回避手段、第 2の破綻回避手段 96、第 3の破綻回避手 段 97, 98をそれぞれ単独で備えても良ぐまた、各手段を適宜組み合わせて備えて も良い。復号ィ匕処理システムは、各第 2の破綻回避手段 96と、第 3の破綻回避手段 9 7, 98をそれぞれ単独で備えてもよぐまた、各手段を適宜組み合わせて備えても良 い。たとえば、符号ィ匕処理システムの場合は、第 1の破綻回避手段 91, 92, 93, 94 、緩和手段 95を伴う第 1の破綻回避手段、及び、第 2の破綻回避手段 96の何れかを 備えるようにし、第 2の破綻回避手段 96により必要演算量を増加させても破綻を回避 できない場合は、第 1の破綻回避手段 91により、色差ブロックのみ無効ブロック化処 理を行うか、あるいは第 1の破綻回避手段 92により、フレーム内符号ィ匕処理を行うか 、あるいは第 1の破綻回避手段 93により、動きベクトルの大きさを 0としてフレーム間 符号化処理を行うか、あるいは第 1の破綻回避手段 94により、量子化ステップサイズ を大きくして符号ィ匕処理を行うようにしても良い。また、上記動画像符号化又は復号 化処理プログラムは、 2段階に動作周波数及び基板バイアス電圧及び動作電圧を制 御して低消費電力化を実現しても良いし，プログラムと同様の機能を備えるハードウ エアで実現されても良い。 [0122] The moving image code processing system of the present invention includes first failure avoiding means 91, 92, 93, 94, first failure avoiding means with mitigation means 95, second failure avoiding means 96, Third bankruptcy avoider The steps 97 and 98 may be provided independently, or each means may be provided in appropriate combination. The decryption processing system may be provided with each second failure avoiding means 96 and third failure avoiding means 97 and 98, or may be provided with an appropriate combination of each means. For example, in the case of a sign key processing system, any one of the first failure avoidance means 91, 92, 93, 94, the first failure avoidance means with the mitigation means 95, and the second failure avoidance means 96 is used. If the failure cannot be avoided even if the required amount of computation is increased by the second failure avoiding means 96, the first failure avoiding means 91 performs invalid block processing only on the color difference block or The intra-frame code processing is performed by the first failure avoiding means 92, or the inter-frame coding process is performed by setting the motion vector size to 0 by the first failure avoiding means 93, or the first failure The avoidance means 94 may increase the quantization step size and perform the code processing. In addition, the moving image encoding or decoding processing program may realize low power consumption by controlling the operating frequency, the substrate bias voltage, and the operating voltage in two stages, and has the same function as the program. It may be realized by hardware.
(第 5の実施の形態）  (Fifth embodiment)
上記第 1の実施の形態乃至第 4の実施の形態は、動作電源電圧、基板バイアス電 圧及び動作周波数を制御するものである力 本実施の形態は、動作電源電圧及び 動作周波数を制御することにより、低消費電力化を図るものである。なお、本実施の 形態は半導体に集積されたシステムに限らない。図 17は、本実施の形態の動画像 符号化システム S5の動作を示した概略ブロック図であり、図 18はプロセッサ 51の動 作電源電圧 ·動作周波数の関係を示す概念図である。本実施の形態の動画像符号 化処理システム S5は、上記第 1の実施の形態のプロセッサ 1に代えて、動作電源電 圧 Vdd及び動作周波数力 Sr段階 (rは 2上の整数）に可変であり（すなわち、 r段階の動 作電源電圧 Vdd及び動作周波数で動作可能であり）且つプログラムにより動作電源 電圧及び動作周波数を変更可能なプロセッサ 51とする。また、前記動作制御手段 5 4は、プロセッサ 1の動作電源電圧及び動作周波数を制御する。プロセッサ 51、又は 、プロセッサ 1及び周辺装置（局部部復号メモリ 6や入力フレームメモリ 7等）は動作制 御手段 52により動作電源電圧 ·動作周波数が制御される。 [0124] 動作決定手段 53は、 F (n) >Feであり、且つ Fe>F (n— 1)となる動作周波数 F (n )を所定フレームの符号化処理を行う動作周波数として選択する計算を行 ヽ、動作周 波数 F (n)に適する動作電源電圧 Vdd(n)を選択する計算を行い、プロセッサ 1及び (又は)局部復号メモリ 6等を含めた周辺装置をその動作周波数 F (n)及び動作電源 電圧 Vdd(n)で一定に動作させるように、動作制御手段 54に指示する（符号 502)。 動作制御手段 54は、動作決定手段 53から指示を受けた動作周波数 F (n)及び動作 電源電圧 Vdd(n)の値をプロセッサ 1及び (又は）局部復号メモリ 6等を含めた周辺装 置に供給し (符号 505)、その動作周波数 F (n)及び動作電源電圧 Vdd (n)でプロセ ッサ 1を一定に動作させる制御を行う。これにより、プロセッサ 1及び (又は）局部復号 メモリ 6等を含めた周辺装置は、動作電源電圧 Vdd (n)及び動作周波数 F (n)で動 作することになる。その他の点については、第 1の実施の形態とほぼ同様である。 The first to fourth embodiments control the operating power supply voltage, the substrate bias voltage, and the operating frequency. The present embodiment controls the operating power supply voltage and the operating frequency. Thus, low power consumption is achieved. Note that this embodiment is not limited to a system integrated in a semiconductor. FIG. 17 is a schematic block diagram showing the operation of the video encoding system S5 of the present embodiment, and FIG. 18 is a conceptual diagram showing the relationship between the operating power supply voltage and the operating frequency of the processor 51. The moving image coding processing system S5 of this embodiment is variable to the operating power supply voltage Vdd and the operating frequency force Sr stage (r is an integer on 2) instead of the processor 1 of the first embodiment. It is assumed that the processor 51 can operate at the r-stage operating power supply voltage Vdd and the operating frequency and can change the operating power supply voltage and operating frequency by a program. The operation control means 54 controls the operation power supply voltage and the operation frequency of the processor 1. The operation power supply voltage and the operation frequency of the processor 51, or the processor 1 and peripheral devices (such as the local decoding memory 6 and the input frame memory 7) are controlled by the operation control means 52. [0124] The operation determining unit 53 calculates F (n)> Fe and selects an operation frequency F (n) satisfying Fe> F (n-1) as an operation frequency for performing encoding processing of a predetermined frame. To calculate the operating power supply voltage Vdd (n) suitable for the operating frequency F (n), and the peripheral devices including the processor 1 and / or the local decoding memory 6 and the like are operated at the operating frequency F (n ) And the operation power supply voltage Vdd (n) is instructed to the operation control means 54 (reference numeral 502). The operation control means 54 sends the values of the operation frequency F (n) and the operation power supply voltage Vdd (n) received from the operation determination means 53 to peripheral devices including the processor 1 and / or the local decoding memory 6 and the like. (Symbol 505), and the processor 1 is controlled at a constant operating frequency F (n) and operating power supply voltage Vdd (n). As a result, peripheral devices including the processor 1 and / or the local decoding memory 6 operate at the operating power supply voltage Vdd (n) and the operating frequency F (n). The other points are almost the same as in the first embodiment.
[0125] 本動画像符号ィ匕処理システム S 5においても、上記第 1の破綻回避手段 91, 92, 9 3, 94, 95や、上記第 2の破綻回避手段 96、又は第 3の破綻回避手段 97, 98を備 えることが好ましい。  [0125] Also in the moving image code processing system S5, the first failure avoiding means 91, 92, 93, 94, 95, the second failure avoiding means 96, or the third failure avoidance. Preferably means 97, 98 are provided.
[0126] (第 6の実施の形態）  [0126] (Sixth embodiment)
上記第 1の実施の形態乃至第 4の実施の形態は、動作電源電圧、基板バイアス電 圧及び動作周波数を制御するものであるが、本実施の形態は、基板バイアス電圧及 び動作周波数を制御することにより、低消費電力化を図るものである。図 19は、本実 施の形態の動画像符号ィ匕システム S6の動作を示した概略ブロック図であり、図 20は プロセッサ 61の基板バイアス電圧'動作周波数の関係を示す概念図である。本実施 の形態の動画像符号ィ匕処理システム S6は、上記第 1の実施の形態のプロセッサ 1に 代えて、基板バイアス電圧 Vbn、 Vbp及び動作周波数力 Sr段階 (rは 2以上の整数）に 可変であり（すなわち、 r段階の基板バイアス電圧 Vbn、 Vbp及び動作周波数で動作 可能であり）且つプログラムにより基板バイアス電圧及び動作周波数を変更可能なプ 口セッサ 61とする。また、前記動作制御手段 64は、プロセッサ 1の基板バイアス電圧 及び動作周波数を制御する。プロセッサ 61、又は、プロセッサ 61及び周辺装置（局 部部復号メモリ 6や入力フレームメモリ 7等）は動作制御手段 64により基板バイアス電 圧，動作周波数が制御される。 [0127] 動作決定手段 63は、 F (n) >Feであり、且つ Fe>F (n— 1)となる動作周波数 F (n )を所定フレームの符号化処理を行う動作周波数として選択する計算を行 ヽ、動作周 波数 F (n)に適する基板バイアス電圧 Vbn (n)、 Vbp (n)を選択する計算を行!ヽ、プ 口セッサ 1及び (又は）局部復号メモリ 6等を含めた周辺装置をその動作周波数 F (n) 及び基板バイアス電圧 Vbn (n)、 Vbp (n)で動作させるように、基板バイアス電圧 .動 作周波数を動作制御手段 64に指示する (符号 602)。動作制御手段 64は、動作決 定手段 63から指示を受けた動作周波数 F (n)及び基板バイアス電圧 Vbn (n)、 Vbp (n)の値をプロセッサ 61及び (又は）局部復号メモリ 6等を含めた周辺装置に供給し（ 符号 605)、その動作周波数 F (n)及び基板バイアス電圧 Vbn (n)、 Vbp (n)でプロ セッサ 61を一定に動作させる制御を行う。これにより、プロセッサ 61及び (又は）局部 復号メモリ 6等を含めた周辺装置は、一定の基板バイアス電圧 Vbn (n)、 Vbp (n)及 び動作周波数 F (n)で動作することになる。その他の点については、第 1の実施の形 態とほぼ同様である。 The first to fourth embodiments control the operating power supply voltage, the substrate bias voltage, and the operating frequency. In this embodiment, the substrate bias voltage and the operating frequency are controlled. By doing so, the power consumption can be reduced. FIG. 19 is a schematic block diagram showing the operation of the moving image encoding system S6 of the present embodiment, and FIG. 20 is a conceptual diagram showing the relationship between the substrate bias voltage and the operating frequency of the processor 61. Moving picture coding I spoon processing system S6 in the present embodiment, in place of the processor 1 of the first embodiment, the substrate bias voltage Vbn, Vbp and operating frequency power S r step (r is an integer of 2 or more) It is assumed that the plugging device 61 is variable (that is, can operate at r-stage substrate bias voltages Vbn, Vbp and operating frequency) and can change the substrate bias voltage and operating frequency by a program. The operation control means 64 controls the substrate bias voltage and the operating frequency of the processor 1. The processor 61 or the processor 61 and peripheral devices (such as the local decoding memory 6 and the input frame memory 7) have their substrate bias voltage and operating frequency controlled by the operation control means 64. [0127] The operation determination unit 63 calculates F (n)> Fe and selects an operation frequency F (n) satisfying Fe> F (n-1) as an operation frequency for performing encoding processing of a predetermined frame. Calculating the substrate bias voltage Vbn (n), Vbp (n) suitable for the operating frequency F (n)! ヽ, including the processor 1 and / or the local decoding memory 6 etc. The operation control means 64 is instructed to operate the peripheral device at the operating frequency F (n) and the substrate bias voltages Vbn (n) and Vbp (n) (reference numeral 602). The operation control means 64 receives the values of the operating frequency F (n) and the substrate bias voltages Vbn (n) and Vbp (n) received from the operation determining means 63 from the processor 61 and / or the local decoding memory 6 and the like. Supplied to the included peripheral device (reference numeral 605), and controls the processor 61 to operate constantly at its operating frequency F (n) and substrate bias voltages Vbn (n) and Vbp (n). As a result, peripheral devices including the processor 61 and / or the local decoding memory 6 and the like operate at a constant substrate bias voltage Vbn (n), Vbp (n) and an operating frequency F (n). The other points are almost the same as in the first embodiment.
[0128] 本動画像符号ィ匕処理システム S6においても、上記第 1の破綻回避手段 91, 92, 9 3, 94, 95又は第 3の破綻回避手段 97, 98や、上記第 2の破綻回避手段 96を備え ることが好ましい。  [0128] Also in the moving picture code processing system S6, the first failure avoiding means 91, 92, 93, 94, 95 or the third failure avoiding means 97, 98, or the second failure avoiding means. Preferably means 96 are provided.
[0129] また、上記第 4の実施の形態の動画像復号ィ匕システム S4についても、動作周波数 に適する動作電源電圧及び基板バイアス電圧を制御することに代えて、動作周波数 に適する動作電源電圧、又は基板バイアス電圧を制御することが可能である。  [0129] Also, in the moving picture decoding system S4 of the fourth embodiment, instead of controlling the operation power supply voltage and the substrate bias voltage suitable for the operation frequency, the operation power supply voltage suitable for the operation frequency, Alternatively, the substrate bias voltage can be controlled.
[0130] (実施例）  [0130] (Example)
実施例を説明する。本実施例は、第 2の実施の形態の動画像符号ィ匕システム 2に、 第 1の破綻回避手段 91と第 2の破綻回避手段が含まれたシステムについての実施 例である。符号ィ匕の対象として 75枚のフレーム力も成る動画像データを使用し、符号 化されるフレームとして 35番目のフレームを例に説明する。各フレームは 144行 176 列の画素配列で構成されている。符号化処理としては、 MPEG— 4を使用する。図 2 1は、動画像符号ィ匕システム S2のプロセッサ 1における動作周波数と動作電源電圧 、基板バイアス電圧の関係の例を表している。動画像符号ィ匕システム S 2のプロセッ サ 1は、動作周波数 50MHz〜250MHz、動作電源電圧 0. 5V〜1. OV、基板バイ ァス電圧一 1. 0V〜0. 5Vであり、 5段階に可変となっている。 Examples will be described. The present example is an example of a system in which the moving image coding system 2 of the second embodiment includes the first failure avoiding means 91 and the second failure avoiding means. Using moving image data with 75 frames as the object of the encoding, the 35th frame will be described as an example of the encoded frame. Each frame is composed of a 144 × 176 pixel array. MPEG-4 is used as the encoding process. FIG. 21 shows an example of the relationship between the operating frequency, the operating power supply voltage, and the substrate bias voltage in the processor 1 of the video encoding system S2. The processor 1 of the video code system S 2 has an operating frequency of 50 MHz to 250 MHz, an operating power supply voltage of 0.5 V to 1. OV, The first voltage is 1.0V to 0.5V, and is variable in 5 steps.
[0131] まず、動画像符号化システム S2は、入力フレームメモリ 7にアクセスして、 35番目の フレームを取得し、必要演算量計算手段 2により、そのフレームの必要演算量 Kpを 計算する。必要演算量 Kpは、具体的には、まず、前フレームとして 34番目のフレー ムを使用し下記の数式により差分絶対値和 Zを算出する。 First, the moving image coding system S2 accesses the input frame memory 7, obtains the 35th frame, and calculates the required calculation amount Kp of the frame by the required calculation amount calculation means 2. Specifically, the required amount of computation Kp first calculates the sum of absolute differences Z using the following formula using the 34th frame as the previous frame.
Z=∑ I Xij-Yij I =41541  Z = ∑ I Xij-Yij I = 41541
次に、前フレームのマクロブロックマッチング回数 M = 308、前フレームの量子化ス テツプサイズ（量子化ステップサイズの平均値） Qprev= 3、前フレームの有効ブロック 数 B= 109、前フレームの有効係数の数 C = 620、前フレームの符号ィ匕に実際に要 した処理量 S = 3561303、所定フレームの符号化ビットレート BR= 65536を得る。 また、前フレームの量子化ステップサイズの平均値とそのひとつ前の 33番目のフレー ムの量子化ステップサイズの平均値の差 A Qprev=0を算出する。また、前フレーム の実際の発生ビット数 D = 56797を得る。次に、各要素を使用して下記の数式により 必要演算量 Kpを算出する。  Next, the number of macroblock matching times for the previous frame M = 308, the quantization step size of the previous frame (average quantization step size) Qprev = 3, the number of effective blocks of the previous frame B = 109, the effective coefficient of the previous frame The number C = 620, the processing amount actually required for the sign of the previous frame S = 3561303, and the coding bit rate BR = 65536 of the predetermined frame are obtained. Also, the difference A Qprev = 0 is calculated between the average value of the quantization step size of the previous frame and the average value of the quantization step size of the previous 33rd frame. Also, the actual number of bits generated in the previous frame, D = 56797, is obtained. Next, the required amount of computation Kp is calculated using the following formula using each element.
Kp=j + a M+ j8 B+ y C+ δ Ζ+ ε AQprev  Kp = j + a M + j8 B + y C + δ Ζ + ε AQprev
以上より、本実施例 1では必要演算量 Κρ = 6958217が得られる。  As described above, the required amount of computation Κρ = 6958217 is obtained in the first embodiment.
[0132] 次に、下記の数式により動作周波数を計算する。 [0132] Next, the operating frequency is calculated by the following equation.
Fe=Kp/Te = 6958217/ (1/15) = 104MHz  Fe = Kp / Te = 6958217 / (1/15) = 104MHz
F (n) >Feであり且つ Fe>F (n—l)となる F (n)を計算し、プロセッサ 1の 5段階に 可変な動作周波数のうち、動作周波数 F (3) = 150MHzを選択し、動作周波数 F (3 ) = 150MHzに適する動作電源電圧 Vdd (3) =0. 8V及び基板バイアス電圧 Vbn ( 3) =OV、 Vbp (3) =0. 8Vを決定する。プロセッサ 1を動作周波数 F= 150MHz及 び動作電源電圧 Vdd=0. 8V及び基板バイアス電圧 Vbn=OV、 Vbp = 0. 8Vで動 作させるように、動作制御手段 4に指示する。動作制御手段 4は、少なくともプロセッ サ 1を動作電源電圧 F= 150MHz及び動作電源電圧 Vdd=0. 8 V及び基板バイァ ス電圧 Vbn=0V、 Vbp = 0. 8Vで一定に動作させる制御を行う。また、上記破綻回 避手段 91を備える場合は、下記の数式より切り替えタイミング時間を算出しておく。 Calculate F (n) where F (n)> Fe and Fe> F (n—l), and select the operating frequency F (3) = 150 MHz among the variable operating frequencies in the five stages of processor 1. The operating power supply voltage Vdd (3) = 0.8V and the substrate bias voltage Vbn (3) = OV, Vbp (3) = 0.8V, which are suitable for the operating frequency F (3) = 150MHz, are determined. The operation control means 4 is instructed to operate the processor 1 at the operating frequency F = 150 MHz, the operating power supply voltage Vdd = 0.8V, the substrate bias voltage Vbn = OV, and Vbp = 0.8V. The operation control means 4 performs control to operate at least the processor 1 at a constant operating power supply voltage F = 150 MHz, operating power supply voltage Vdd = 0.8 V, substrate bias voltage Vbn = 0 V, Vbp = 0.8 V. When the failure avoiding means 91 is provided, the switching timing time is calculated from the following formula.
Ti=Te- (Ks X (MB-MBi) ) /F =0. 06666- (4750 X (99-0) ) /(150000000) Ti = Te- (Ks X (MB-MBi)) / F = 0. 06666- (4750 X (99-0)) / (150000000)
=0. 06353  = 0. 06353
ここで、 Ksは 1つのマクロブロックのうち、色差ブロックのみを無効ブロックとして処理 するために必要なサイクル数である。動画像符号化手段 5は、上記動作周波数 F= l 50MHz及び動作電源電圧 Vdd=0. 9V及び基板バイアス電圧 Vbn=0V、 Vbp = Here, Ks is the number of cycles required to process only the color difference block as an invalid block in one macroblock. The moving image encoding means 5 has the above operating frequency F = l 50 MHz, operating power supply voltage Vdd = 0.9V, substrate bias voltage Vbn = 0V, Vbp =
0. 8Vで一定に動作させられた状態のプロセッサ 1を使用して符号ィ匕処理を行い、符 号化データを生成する。 0. Perform the encoding process using processor 1 in a state where it is constantly operated at 8 V to generate encoded data.
[0133] さらに、符号ィ匕処理ルーチンを実行している際に、上記第 1の破綻回避手段 91を 備える場合は、上記 Tiが経過したタイミングで KwX (MB-MBi) >F X (Te— Ti) であるか否かの判断をする。本実施例 1では、 Ti=0. 06353のタイミングで Kw X ( MB-MBi)く F X (Te— Ti)であり、演算量に余裕があると判断し、切り替えタイミン グの更新を行う。このとき MBi= 83であったので、下記の数式により Tiが更新される  Further, when the first failure avoiding means 91 is provided during execution of the sign key processing routine, KwX (MB-MBi)> FX (Te—Ti ) Or not. In the first embodiment, at timing of Ti = 0.06353, Kw X (MB-MBi) and F X (Te-Ti) are determined, and it is determined that there is a sufficient amount of calculation, and the switching timing is updated. At this time, since MBi = 83, Ti is updated by the following formula
Ti=Te- (Ks X (MB-MBi) ) /F Ti = Te- (Ks X (MB-MBi)) / F
=0. 06666- (4750 X (99 - 83) ) /(150000000)  = 0. 06666- (4750 X (99-83)) / (150000000)
=0. 06662  = 0. 06662
上記のタイミングにおいては、 MBi≠MBであり、 KwX (MB-MBi) >F X (Te— Ti)であることから、残りのマクロブロック全てを色差ブロックのみ無効ブロック化する 処理に切り替える。  At the above timing, since MBi ≠ MB and KwX (MB-MBi)> F X (Te—Ti), the process switches to the process of making all the remaining macroblocks invalid blocks only for the color difference blocks.
[0134] 35番目のフレームのステップが終了し、次の 36番目のフレームのステップに移行 する。上記第 2の破綻回避手段 96を備える場合、必要演算量計算手段において、前 フレームのマクロブロックマッチング回数 M = 327、前フレームの量子化ステップサイ ズ（量子化ステップサイズの平均値） Qprev= 3、前フレームの有効ブロック数 B = 12 [0134] The 35th frame step ends, and the next 36th frame step is entered. When the second failure avoiding means 96 is provided, in the required calculation amount calculating means, the number of times of macroblock matching for the previous frame M = 327, the quantization step size of the previous frame (average quantization step size) Qprev = 3 , Number of valid blocks in previous frame B = 12
1、前フレームの有効係数の数 C = 749、前フレームの符号ィ匕に実際に要した処理 量 S = 7013933、前フレームの実際の発生ビット数 D = 83942を得て、且つ 1つ前 のフレームで破綻回避処理が実行された力、されていないかをチェックする。破綻回 避処理が実行されているため、第 2の破綻回避手段 96により、必要演算量の予測値 を増加する処理を行う。本実施例では、マクロブロックマッチング回数 Mと有効ブロッ ク数 Bと有効係数の数 Cを 1. 1倍し、下記の数式により必要演算量 Kpを算出する。1, the number of effective coefficients of the previous frame C = 749, the processing amount actually required for the sign of the previous frame S = 7013933, the actual number of generated bits D = 83942 of the previous frame, and the previous one Check whether or not the failure avoidance process is executed on the frame. Since the failure avoidance process has been executed, the second failure avoiding means 96 is used to predict the required amount of computation. Increase the process. In this embodiment, the number M of macroblock matching, the number B of effective blocks, and the number C of effective coefficients are multiplied by 1.1, and the necessary calculation amount Kp is calculated by the following formula.
Kp=j+ XI. l)M+(j8 XI. 1)Β+(γ XI. l)C+ δΖ+ ε AQprev 以上により、必要演算量 Kp = 7101472が得られる。 Kp = j + XI. L) M + (j8 XI. 1) Β + (γ XI. L) C + δΖ + ε AQprev By the above, the necessary amount of computation Kp = 71010472 is obtained.
また、第 1の破綻回避手段 95により、前のフレームで破綻回避処理を実行したマク ロブロックに対応する所定フレームのマクロブロックにおいて、量子化ステップサイズ Also, the quantization step size is determined in the macroblock of the predetermined frame corresponding to the macroblock for which the failure avoidance processing has been executed in the previous frame by the first failure avoiding means 95.
Q=4から Q = 3とする。これにより、前フレームの画質の影響を緩和し、画質をより改 善することができる。 Q = 4 to Q = 3. As a result, the influence of the image quality of the previous frame can be reduced and the image quality can be further improved.

Claims

請求の範囲 The scope of the claims
[1] 連続する複数のフレームから構成される動画像をフレーム単位で符号化する動画 像符号ィ匕手段として機能するプロセッサを備え、  [1] A processor that functions as a moving image encoding means for encoding a moving image composed of a plurality of continuous frames in units of frames,
一のフレームの符号化に必要な必要演算量 Kpを計算する必要演算量計算手段と Necessary computation amount necessary for encoding one frame
、当該一のフレームの符号ィ匕処理に予め割り当てられている時間 Te内に当該必要 演算量 Kpを符号化処理可能な動作周波数を決定する動作決定手段とを備え、 当該プロセッサが、予め割り当てられて!/、る時間 Te内は当該動作決定手段により 決定された動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z又は 基板バイアス電圧で動作しながら当該動画像符号ィ匕手段により当該一のフレームの 符号化処理を行う動画像符号化処理システムであり、 And an operation determining means for determining an operating frequency capable of encoding the required amount of computation Kp within a time Te allocated in advance for the encoding process of the one frame, and the processor is allocated in advance. During the period Te, the moving image code key means operates while operating at the operating frequency determined by the operation determining means and the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency. Is a moving image encoding processing system for encoding the frame of
所定のタイミングで符号ィヒ処理の演算量を削減する第 1の破綻回避手段を備えるこ とを特徴とする動画像符号化処理システム。  A moving image coding processing system comprising first failure avoiding means for reducing the amount of calculation of coding performance processing at a predetermined timing.
[2] 前記第 1の破綻回避手段は、符号ィ匕が終了していないマクロブロックのうち、色差 信号の情報を持つブロックを無効ブロックとして処理を行うことを特徴とする請求項 1 記載の動画像符号化処理システム。 [2] The moving image according to claim 1, wherein the first failure avoiding means performs processing using, as an invalid block, a block having color difference signal information among macroblocks in which the sign is not terminated. Image coding processing system.
[3] 前記第 1の破綻回避手段は、符号ィ匕が終了していないマクロブロックに対し、フレ ーム内符号化処理を行うことを特徴とする請求項 1記載の動画像符号化処理システ ム。 3. The moving image coding processing system according to claim 1, wherein the first failure avoiding means performs an intra-frame coding process on a macroblock for which the code is not finished. Mu.
[4] 前記第 1の破綻回避手段は、符号ィ匕が終了していないマクロブロックに対し、動き ベクトル検出を行うことなく動きベクトルを 0としてフレーム間符号ィ匕処理を行うことを 特徴とする請求項 1記載の動画像符号化処理システム。  [4] The first failure avoiding means performs inter-frame code key processing with a motion vector set to 0 without performing motion vector detection for a macroblock for which the code key has not ended. The moving image encoding processing system according to claim 1.
[5] 前記第 1の破綻回避手段は、符号ィ匕が終了していないマクロブロックに対し、量子 ィ匕ステップサイズを大きくして符号ィ匕処理を行うことを特徴とする請求項 1記載の動画 像符号化処理システム。  5. The first failure avoiding means performs code key processing by increasing a quantum step size for a macroblock for which code key has not been completed. Video image encoding processing system.
[6] 前記第 1の破綻回避手段が前記一のフレームよりも後に符号化される後続フレーム の符号化処理に与える影響を緩和する緩和手段を備えることを特徴とする請求項 1 乃至請求項 5のいずれか 1項に記載の動画像符号ィ匕処理システム。  6. The mitigating means for mitigating the influence of the first failure avoiding means on the encoding process of a subsequent frame encoded after the one frame. The moving image code processing system according to any one of the above.
[7] 前記緩和手段は、当該一のフレームにおいて第 1の破綻回避手段による処理が実 行されたマクロブロックに対応する後続フレームのマクロブロックに対し、量子化ステ ップサイズを小さくすることを特徴とする請求項 6に記載の動画像符号ィ匕処理システ ム。 [7] The mitigation means executes processing by the first failure avoidance means in the one frame. 7. The moving picture code processing system according to claim 6, wherein a quantization step size is reduced for a macroblock of a subsequent frame corresponding to the executed macroblock.
[8] 前記第 1の破綻回避手段により前記一のフレームの符号化処理の演算量が削減さ れた場合、当該一のフレームよりも後に符号ィ匕処理される後続フレームの必要演算 量を増加させることを特徴とする請求項 1乃至請求項 7のいずれか 1項に記載の動画 像符号化処理システム。  [8] When the calculation amount of the encoding process of the one frame is reduced by the first failure avoidance means, the required calculation amount of the subsequent frame to be encoded after the one frame is increased. The moving image coding processing system according to claim 1, wherein the moving image coding processing system according to claim 1 is used.
[9] 前記後続フレームの必要演算量、又は、必要演算量の算出に用いられる要素につ いて、 m倍 (mは 1以上の実数）、又は、 0より大きい実数 nを加算することにより、前記 後続フレームの必要演算量を増カロさせることを特徴とする請求項 8記載の動画像符 号化又は復号化処理システム。  [9] By adding m times (m is a real number greater than or equal to 1) or a real number n greater than 0, the necessary calculation amount of the subsequent frame or an element used to calculate the required calculation amount 9. The moving image encoding or decoding processing system according to claim 8, wherein the necessary amount of calculation for the subsequent frame is increased.
[10] 連続する複数のフレーム力 構成される動画像をフレーム単位で符号ィ匕または復 号ィ匕する動画像符号ィ匕又は復号ィ匕手段として機能するプロセッサを備え、  [10] A processor that functions as a moving image code key or a decoding key means for encoding or decoding a moving image composed of a plurality of continuous frame forces in units of frames,
一のフレームの符号化又は復号化に必要な必要演算量 Kpを計算する必要演算量 計算手段と、当該一のフレームの符号ィ匕又は復号ィ匕処理に予め割り当てられている 時間 Te内に前記必要演算量 Kpを符号化又は復号化処理可能な動作周波数を決 定する動作決定手段とを備え、  Necessary amount of computation required for encoding or decoding one frame. Necessary amount of computation for calculating Kp, and within the time Te previously assigned to the encoding / decoding processing of the one frame. Operation determining means for determining an operating frequency capable of encoding or decoding the necessary amount of computation Kp,
当該プロセッサが、予め割り当てられている時間 Te内は、当該動作決定手段により 決定された動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z又は 基板バイアス電圧で動作しながら当該動画像符号ィ匕又は復号ィ匕手段により当該一 のフレームの符号化又は復号化処理を行う動画像符号化又は復号化処理システム であり、  While the processor is operating at the operating frequency determined by the operation determining means and the operating power supply voltage and the Z or substrate bias voltage suitable for the operating frequency within the pre-allocated time Te, A moving image encoding or decoding processing system that performs encoding or decoding processing of the one frame by the 匕 or decoding means;
前記必要演算量算出手段で算出された必要演算量 Kpが実際に必要な演算量より も小さい場合、当該一のフレームよりも後に符号ィ匕又は復号ィ匕される後続フレームの 必要演算量を増加させる第 2の破綻回避手段を備えることを特徴とする動画像符号 化又は復号化処理システム。  When the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount, the required calculation amount of the subsequent frame that is encoded or decoded after the one frame is increased. A moving image encoding or decoding processing system comprising: a second failure avoiding means for causing a failure.
[11] 前記第 2の破綻回避手段は、前記後続フレームの必要演算量、又は、必要演算量 の算出に用いられる要素について、 m倍 (mは 1以上の実数）、又は、 0より大きい実 数 nを加算することを特徴とする請求項 10記載の動画像符号化又は復号化処理シス テム。 [11] The second failure avoiding means may calculate the necessary calculation amount of the subsequent frame or an element used for calculation of the required calculation amount by m times (m is a real number of 1 or more) or an actual value greater than 0. 11. The moving image encoding or decoding processing system according to claim 10, wherein the number n is added.
[12] 連続する複数のフレーム力 構成される動画像をフレーム単位で符号ィ匕または復 号ィ匕する動画像符号ィ匕又は復号ィ匕手段として機能するプロセッサを備え、  [12] A processor that functions as a moving image code key or decoding key means for encoding or decoding a moving image composed of a plurality of continuous frame forces in units of frames,
一のフレームの符号化又は復号化に必要な必要演算量 Kpを計算する必要演算量 計算手段と、当該一のフレームの符号ィ匕又は復号ィ匕処理に予め割り当てられている 時間 Te内に前記必要演算量 Kpを符号化又は復号化処理可能な動作周波数を決 定する動作決定手段とを備え、  Necessary amount of computation required for encoding or decoding one frame. Necessary amount of computation for calculating Kp, and within the time Te previously assigned to the encoding / decoding processing of the one frame. Operation determining means for determining an operating frequency capable of encoding or decoding the necessary amount of computation Kp,
当該プロセッサが、予め割り当てられている時間 Te内は、当該動作決定手段により 決定された動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z又は 基板バイアス電圧で動作しながら動画像符号ィ匕又は復号ィ匕手段により当該一のフレ ームの符号化又は復号化処理を行う動画像符号化又は復号化処理システムであり、 所定のタイミングで前記時間 Teを延長する第 3の破綻回避手段を備えることを特徴 とする動画像符号化又は復号化処理システム。  During the pre-allocated time Te, the moving image encoding code is operated while operating at the operating frequency determined by the operation determining means and the operating power supply voltage and Z or substrate bias voltage suitable for the operating frequency. Alternatively, it is a moving image encoding or decoding processing system that performs encoding or decoding processing of the one frame by decoding decoding means, and includes third failure avoiding means that extends the time Te at a predetermined timing. A moving image encoding or decoding processing system comprising:
[13] 前記第 3の破綻回避手段は、前記一のフレームの符号ィヒ処理に予め割り当てられ ている時間 Teを延長した場合に、前記一のフレームよりも後に符号ィ匕又は復号ィ匕さ れる後続フレームにつ 、て、当該後続フレームの符号化処理又は復号化処理に予 め割り当てられて!/ヽる時間 Teを変更することを特徴とする請求項 12記載の動画像符 号化又は復号化処理システム。  [13] The third failure avoiding means, when extending a time Te previously assigned to the code frame processing of the one frame, increases the code key or decoding error after the one frame. 13. The moving picture encoding or the moving picture encoding or decoding according to claim 12, characterized in that a time Te that is pre-allocated to the encoding process or the decoding process of the subsequent frame is changed for each subsequent frame to be recorded. Decryption processing system.
[14] 前記第 3の破綻回避手段は、前記一のフレームの符号ィヒ処理に予め割り当てられ ている時間 Teを延長した場合に、次に符号ィ匕されるフレームを格納する入力フレー ムメモリに対して、フレームの書き込み先を変更することを特徴とする請求項 12又は 請求項 13記載の動画像符号化処理システム。  [14] The third failure avoiding means may be provided in an input frame memory for storing a frame to be encoded next when the time Te previously allocated to the encoding process of the one frame is extended. 14. The moving image encoding processing system according to claim 12, wherein the frame writing destination is changed.
[15] 前記第 3の破綻回避手段は、前記一のフレームの符号ィヒ処理に予め割り当てられ ている時間 Teを延長した場合に、次に符号ィ匕されるフレームを格納する入力フレー ムメモリに対して書き込み不可とすることを特徴とする請求項 12又は請求項 13記載 の動画像符号化処理システム。  [15] The third failure avoiding means is provided in an input frame memory for storing a frame to be encoded next when the time Te previously allocated to the encoding process of the one frame is extended. 14. The moving image encoding processing system according to claim 12, wherein writing is impossible.
[16] プロセッサを使用して連続する複数のフレーム力も構成される動画像をフレーム単 位で符号化する動画像符号化ステップと、一のフレームの符号ィヒに必要な必要演算 量 Kpを計算する必要演算量計算ステップと、当該一のフレームの符号化処理に予 め割り当てられている時間 Te内に当該必要演算量 Κρを符号ィ匕処理可能な動作周 波数を決定する動作決定ステップとを備え、 [16] Using a processor, a moving image consisting of multiple consecutive frame forces Is assigned in advance to a video encoding step for encoding at a position, a required calculation amount calculating step for calculating the necessary calculation amount Kp for one frame encoding, and an encoding process for the one frame. An operation determining step for determining an operation frequency capable of sign-processing the required amount of computation Κρ within a predetermined time Te,
当該プロセッサが、予め割り当てられて!/ヽる時間 Te内は当該動作決定ステップに おいて得られた動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z 又は基板バイアス電圧で動作しながら、動画像符号ィ匕又は復号化ステップにお ヽて 当該一のフレームの符号ィ匕処理を行う動画像符号ィ匕処理方法であり、  While the processor is operating at the operation frequency obtained in the operation determination step and the operation power supply voltage and Z or substrate bias voltage suitable for the operation frequency within the time period Te that is allocated in advance! A moving image code key processing method for performing the code key processing of the one frame in the moving image code key or decoding step;
所定のタイミングで符号ィ匕処理の実際の演算量を削減する第 1の破綻回避ステップ を備えることを特徴とする動画像符号化処理方法。  A moving picture coding processing method comprising: a first failure avoidance step for reducing an actual calculation amount of a coding process at a predetermined timing.
[17] プロセッサを使用して連続する複数のフレーム力も構成される動画像をフレーム単 位で符号化又は復号化する動画像符号化又は復号化ステップと、一のフレームの符 号化又は復号化に必要な必要演算量 Kpを計算する必要演算量計算ステップと、当 該一のフレームの符号ィ匕又は復号ィ匕処理に予め割り当てられている時間 Te内に前 記必要演算量 Kpを符号化又は復号化処理可能な動作周波数を決定する動作決定 ステップとを備え、 [17] A moving image encoding or decoding step for encoding or decoding a moving image that also includes a plurality of continuous frame forces using a processor in units of frames, and encoding or decoding of one frame The required amount of computation Kp for calculating the required amount of computation Kp and the required amount of computation Kp are encoded within the time Te previously allocated to the encoding or decoding processing of the frame. Or an operation determining step for determining an operation frequency that can be decoded,
当該プロセッサが、予め割り当てられて!/ヽる時間 Te内は当該動作決定ステップによ り得られた動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z又は 基板バイアス電圧で動作しながら、動画像符号ィ匕又は復号化ステップにお ヽて当該 一のフレームの符号化又は復号化処理を行う動画像符号化又は復号化処理方法で あり、  The processor operates at the operation frequency obtained by the operation determination step and the operation power supply voltage and the Z or substrate bias voltage suitable for the operation frequency during the pre-allocated time! / A moving image encoding or decoding processing method that performs encoding or decoding processing of the one frame in the moving image encoding or decoding step;
前記必要演算量算出手段で算出された必要演算量 Kpが実際に必要な演算量より も小さい場合、当該一のフレームよりも後に符号ィ匕又は復号ィ匕される後続フレームの 必要演算量を増加させる第 2の破綻回避ステップを備えることを特徴とする動画像符 号化又は復号化処理方法。  When the required calculation amount Kp calculated by the required calculation amount calculation means is smaller than the actually required calculation amount, the required calculation amount of the subsequent frame that is encoded or decoded after the one frame is increased. A moving picture encoding or decoding processing method comprising a second failure avoidance step.
[18] プロセッサを使用して連続する複数のフレーム力も構成される動画像をフレーム単 位で符号化又は復号化する動画像符号化ステップと、一のフレームの符号ィ匕又は復 号化に必要な必要演算量 Kpを計算する必要演算量計算ステップと、当該一のフレ ームの符号ィ匕又は復号ィ匕処理に予め割り当てられている時間 Te内に前記必要演算 量 Kpを符号ィ匕又は復号ィ匕処理可能な動作周波数を決定する動作決定ステップとを 備え、 [18] Necessary for encoding / decoding a single frame, and a video encoding step for encoding / decoding a video that also includes multiple consecutive frame forces using a processor. Necessary amount of computation required Necessary amount of computation to calculate Kp An operation determining step for determining an operating frequency at which the necessary amount of computation Kp can be encoded or decoded within a time Te previously assigned to the code
当該プロセッサが、予め割り当てられて!/ヽる時間 Te内は当該動作決定ステップに おいて得られた動作周波数、及び、当該動作周波数に適する動作電源電圧及び Z 又は基板バイアス電圧で動作しながら、動画像符号ィ匕又は復号化ステップにお ヽて 当該一のフレームの符号化又は復号化処理を行う動画像符号化又は復号化処理方 法であり、  While the processor is operating at the operation frequency obtained in the operation determination step and the operation power supply voltage and Z or substrate bias voltage suitable for the operation frequency within the time period Te that is allocated in advance! A moving image encoding or decoding processing method that performs encoding or decoding processing of the one frame in the moving image encoding or decoding step.
所定のタイミングで当該時間 Teを延長する第 3の破綻回避ステップを備えることを 特徴とする動画像符号化又は復号化処理方法。  A moving image coding or decoding processing method comprising a third failure avoidance step of extending the time Te at a predetermined timing.
PCT/JP2005/012250 2004-07-02 2005-07-01 Video image encoding system, video image encoding or decoding system, video image encoding method, and video image encoding or decoding method WO2006004065A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2006528865A JPWO2006004065A1 (en) 2004-07-02 2005-07-01 Moving picture coding processing system, moving picture coding or decoding processing system, moving picture coding processing method, and moving picture coding or decoding processing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2004197188 2004-07-02
JP2004-197188 2004-07-02

Publications (2)

Publication Number Publication Date
WO2006004065A1 true WO2006004065A1 (en) 2006-01-12
WO2006004065A8 WO2006004065A8 (en) 2006-02-23

Family

ID=35782869

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2005/012250 WO2006004065A1 (en) 2004-07-02 2005-07-01 Video image encoding system, video image encoding or decoding system, video image encoding method, and video image encoding or decoding method

Country Status (2)

Country Link
JP (1) JPWO2006004065A1 (en)
WO (1) WO2006004065A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008104153A (en) * 2006-08-29 2008-05-01 Nvidia Corp Method and system for dynamic frequency adjustment during video decoding
JP2008136177A (en) * 2006-10-27 2008-06-12 Matsushita Electric Ind Co Ltd Motion detection apparatus, mos (metal-oxide semiconductor) integrated circuit and video system
JP2008141311A (en) * 2006-11-30 2008-06-19 Sharp Corp Decoding device and decoding method
US8098898B2 (en) 2006-10-27 2012-01-17 Panasonic Corporation Motion detection device, MOS (metal-oxide semiconductor) integrated circuit, and video system
JP2014220020A (en) * 2013-04-30 2014-11-20 富士通株式会社 Electronic circuit
CN111342937A (en) * 2020-03-17 2020-06-26 北京百瑞互联技术有限公司 Method and device for dynamically adjusting voltage and/or frequency of coding and decoding processor

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002232881A (en) * 2001-01-31 2002-08-16 Toshiba Corp Moving picture decoding method and device

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002232881A (en) * 2001-01-31 2002-08-16 Toshiba Corp Moving picture decoding method and device

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
KAWAGUCHI H. ET AL: "An LSI for Vdd-Hopping and MPEG1 System Based on the Chip", IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEM 2001, BEIKOKU, IEEE 2001 NEN 5 GATSU, 2001, pages 918 - 921, XP010542007 *
KAWAKAMI K. ET AL: "Feedfoward-gata Doteki Den'atsu Seigyo ni yoru MPEG4 Teishohi Denryokuka Algorithm", THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS GIJUTSU KENKYU HOKOKU ICD2003-30, NIPPON, THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS, 2003 NEN 5 GATSU, 2003, pages 25 - 30, XP002991854 *
LEE S AND SAKURAI TAKAYASU.: "Run-time Voltage Hopping for Low-power Real-time Systems", PROC. DESIGN AUTOMATION CONFERENCE, BEIKOKU, IEEE 2000 NEN 6 GATSU, 2000, pages 806 - 809, XP002991852 *
MORITA Y. ET AL: "D-23 Feedfoward Seigyo o Mochiita MPEG4 Teishohi Denryokuka Algorithm (2) -2 Dankai Dosa Den'atsu/Shuhasu Seigyoteki Yoji no Shohi Denryoku Sakugen Koka no Mitsumori", HEISEI 15 NENDO RECORD OF JOINT CONFERENCE OF ELECTRICAL AND ELECTRONICS ENGINEERS IN HOKURIKU, NIPPON, HEISEI 15 NENDO INSTITUTES OF ELECTRICAL AND INFORMATION ENGINEERS, JAPAN HOKURIKU SHIBU RENGO TAIKAI JIKKO IINKAI JIMUKYOKU, 22 September 2003 (2003-09-22), pages 192, XP002991855 *
MORITA Y. ET AL: "Dagazo Fugoka Shgori ni Okeru Denryoku Saishoka no tameno Feedfoward-gata Doteki Shuhasu/Den'atsu Algorithm", THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS GIJUTSU KENKYU HOKOKU, ICD2004-26, NIPPON, THE INSTITUTE OF ELECTRONICS, 2004 NEN 5 GATSU, 2004, pages 19 - 24, XP002996675 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008104153A (en) * 2006-08-29 2008-05-01 Nvidia Corp Method and system for dynamic frequency adjustment during video decoding
JP2012075145A (en) * 2006-08-29 2012-04-12 Nvidia Corp Method and system for dynamic frequency adjustment during video decoding
TWI413418B (en) * 2006-08-29 2013-10-21 Nvidia Corp Method, system, and clock frequency control for dynamic frequency adjustment during video decoding
TWI448161B (en) * 2006-08-29 2014-08-01 Nvidia Corp Method, system, and clock frequency control for dynamic frequency adjustment during video decoding
JP2008136177A (en) * 2006-10-27 2008-06-12 Matsushita Electric Ind Co Ltd Motion detection apparatus, mos (metal-oxide semiconductor) integrated circuit and video system
US8098898B2 (en) 2006-10-27 2012-01-17 Panasonic Corporation Motion detection device, MOS (metal-oxide semiconductor) integrated circuit, and video system
JP2008141311A (en) * 2006-11-30 2008-06-19 Sharp Corp Decoding device and decoding method
JP2014220020A (en) * 2013-04-30 2014-11-20 富士通株式会社 Electronic circuit
CN111342937A (en) * 2020-03-17 2020-06-26 北京百瑞互联技术有限公司 Method and device for dynamically adjusting voltage and/or frequency of coding and decoding processor
CN111342937B (en) * 2020-03-17 2022-05-06 北京百瑞互联技术有限公司 Method and device for dynamically adjusting voltage and/or frequency of coding and decoding processor

Also Published As

Publication number Publication date
WO2006004065A8 (en) 2006-02-23
JPWO2006004065A1 (en) 2008-07-31

Similar Documents

Publication Publication Date Title
US11425371B2 (en) Intra/inter mode decision for predictive frame encoding
US8659607B2 (en) Efficient video decoding migration for multiple graphics processor systems
US8660182B2 (en) MPEG motion estimation based on dual start points
US20070030899A1 (en) Motion estimation apparatus
US20110216829A1 (en) Enabling delta compression and modification of motion estimation and metadata for rendering images to a remote display
WO2006004065A1 (en) Video image encoding system, video image encoding or decoding system, video image encoding method, and video image encoding or decoding method
US7222244B2 (en) Semiconductor device including a prediction circuit to control a power status control circuit which controls the power status of a function circuit
US6947488B2 (en) Scalable MPEG-2 video decoder
JP3772183B2 (en) Moving picture coding processing system, moving picture decoding processing system, moving picture coding processing program, and moving picture decoding processing program
JP2009055542A (en) Moving image encoder and moving image encoding method
JPWO2004093458A1 (en) Moving image encoding or decoding processing system and moving image encoding or decoding processing method
WO2005096634A1 (en) Multi-task processing system and multi-task processing method
TWI543627B (en) Method for adaptively performing video decoding, adaptive complexity video decoder and adaptive audio/video playback system
US20180139439A1 (en) Video processing device for encoding frames constituting hierarchical structure
US20030133504A1 (en) Image coding integrated circuit capable of reducing power consumption according to data to be processed
US20080291062A1 (en) Image coding apparatus and method thereof
WO2005062621A1 (en) Moving picture encoding or decoding system and moving picture encoding or decoding method
US20130315296A1 (en) Systems and methods for adaptive selection of video encoding resources
JP2003116136A (en) Moving picture decoder
US8170356B2 (en) Linear temporal reference scheme having non-reference predictive frames
JP3772184B2 (en) Moving image encoding processing system, moving image decoding processing system, moving image encoding processing method, and moving image decoding processing method
US10063873B2 (en) Method for adaptively performing video decoding, and associated adaptive complexity video decoder and adaptive audio/video playback system
Hsia et al. A real-time chip implementation for adaptive video coding control
Watanabe et al. H. 264/AVC HDTV Video Codec LSI
Hsieh et al. Bandwidth-constrained motion estimation for real-time mobile video application

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

CFP Corrected version of a pamphlet front page
CR1 Correction of entry in section i

Free format text: IN PCT GAZETTE 02/2006 DUE TO A TECHNICAL PROBLEM AT THE TIME OF INTERNATIONAL PUBLICATION, SOME INFORMATION WAS MISSING UNDER (57), THE MISSING INFORMATION NOW APPEARS IN THE CORRECTED VERSION

121 Ep: the epo has been informed by wipo that ep was designated in this application
DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2006528865

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

122 Ep: pct application non-entry in european phase