CN103634604A - Multi-core DSP (digital signal processor) motion estimation-oriented data prefetching method - Google Patents

Abstract

The invention discloses a multi-core DSP (digital signal processor) motion estimation-oriented data prefetching method. According to the method, data of a coding block and a reference block are prefetched to a local storage according to the space relevancy of data in motion estimation and the data of a prediction motion vector prefetching coding bock and the reference block, so the parallel performing between the motion estimation operation of the current coding block and the data loading operation of the next code block and the reference block are realized, the influence of the storage to the speed of a multi-core processor in the motion estimation is reduced, and tests prove that the data prefetching method enables the motion estimation processing speed in the multi-core DSP parallel video coding to be obviously improved.

Description

A kind of data prefetching method towards multi-core DSP estimation

Technical field

The invention belongs to multi-media decoding and encoding field, be specifically related to a kind of data prefetching method for estimation in embedded multi-core dsp processor parallel video coding, is a kind of by the method for data pre-fetching accelerating video motion estimation process process.

Background technology

Estimation is one of chief component of the Video coding based on hybrid encoding frame, estimation be take data block and is completed prediction as unit, motion search, motion compensation, the operation such as dct transform and quantification, in Video coding, estimation be take data block as processing unit, H.264/AVC in coding, the data block of estimation comprises macro block (MB), sub-macro block, piece etc., in HEVC coding, the data block of estimation comprises coding unit (CU), predicting unit (PU) and converter unit (TU) etc., the estimation of P frame needs the data of current encoded frame and a reference frame, the estimation of B frame needs current encoded frame and a forward reference frame, the data of a backward reference frame, the throughput of the data of processing is very large.In the video sequence of 1080i form, the resolution of every two field picture is 1920 * 1080, export 60 two field pictures each second, with YUV(4:2:0) form represents that the coded data of colour information generation per second reaches 0.746Gbps, more than data in estimation reach 1.5Gbps, along with the raising of video quality, the video data volume of generation is also sharply increasing.In embedded system, Video coding more and more adopts multi-core DSP processor to realize, and embedded multi-core dsp processor is multistage storage organization, and each core independently has local memory separately, and all core is shared MSM memory and jumbo chip external memory.Local memory capacity is little, fastest; MSM memory is larger, and speed is slower, and external memory storage capacity is large, and speed is slow.Because the local storage space of multi-core DSP processor is little, can not store the data of whole coded frame and reference frame, coded frame and reference frame need to be divided into little data block, store the data of the present encoding piece of coded frame and reference block into internal storage and coded frame and reference frame storing memory externally.As shown in Figure 1, in embedded multi-core video encoder, first by video acquisition, obtain video data and be temporarily stored in jumbo external memory storage, when coding, by external memory storage, read internal storage and do computing, yet processor performance is with annual 60% speed increment, and memory access performance is increased less than 10% every year, performance gap between processor and memory is also in continuous increase, memory becomes systematic function bottleneck, in polycaryon processor, store bottleneck problem even more serious, estimation is because the data volume of needs is large, memory bottleneck becomes the key factor that affects processing speed.

In order to reduce the impact of storage bottleneck, adopt the polycaryon processor memory property of multistage storage organization to lean on the hit rate of Cache to guarantee, yet in multistage storage organization, Cache does not hit and can cause outside accessing operation long delay, the long delay time can reach up to a hundred processor clock cycles, reduced the execution speed of processor, the delay of reading to lose efficacy under worst case as TMS320C6678 memory Cache is 287 clock cycle, adds up to 287ns(core to be operated in 1GHz).In estimation, deal with data amount is large, and it is more obvious that Cache does not hit the impact causing.Data pre-fetching technology read in advance data before data are used, by the stand-by period of calculating and the overlapping reduction processor of accessing operation.Existing application number is that 200410101465.2 patent " method that in video encoding-decoding process, macro block data reads " is by setting up the mode resolve buffer district hit rate Problem of Failure of " macroblock address mapping table ", but this method only provides the indexed mode of macro block data in a kind of frame of video, can only reduce the impact that Cache lost efficacy and brings, do not realize the data pre-fetching problem of reference frame in coding, and the data that often need to search for a plurality of reference blocks in estimation, data volume is larger, also larger on the impact of estimation.Application number is that mode that 200710046929.8 patent " data pre-fetching system in video processing " increases " data pre-fetching module " between processor and memory realizes data block and looks ahead, but this mode of looking ahead by the realization of increase hardware cell is not suitable for business-like DSP embedded processor, owing to lacking synchronization mechanism, this method is not suitable for multi-core DSP processor Parallel Implementation simultaneously.

The present invention is according to the look ahead data of coded frame and reference frame of the data space correlation of motion estimation process in parallel encoding and motion vectors, realize data and read parallel with motion estimation process, effectively reduce the impact of storage bottleneck on multi-core DSP processor processing speed, experiment shows that the method effectively raises the execution speed of embedded multi-core dsp processor estimation.

Summary of the invention

In order to overcome the time delay that reference to storage causes when being applied to estimation of multi-core DSP processor, the present invention has announced a kind of according to the look ahead technology of data of coded frame and reference frame of data space correlation and motion vectors in estimation, in upper data block coding, use the look ahead data of next encoding block and reference of DMA, realized parallel that motion estimation data reads and processes.Experiment showed, that the method effectively raises the processing speed of multi-core DSP processor motion estimation operation.

For achieving the above object, the present invention has adopted following technical proposals:

A kind of data prefetching method towards multi-core DSP estimation (as shown in Figure 4), step is as follows:

Step 1, prefetch data block size is set, divides encoding block, reference block size, the data of encoding block and reference block are stored in the local memory of core, and memory block is set to Ping-Pang structure;

If the encoding block of the current execution estimation of step 2 belongs to P frame, carry out prediction, motion search operations, the data of look ahead next encoding block and reference block;

If the encoding block of the current execution estimation of step 3 belongs to B frame, carry out prediction, motion search operations, the data of look ahead next encoding block and forward direction reference block, backward reference block.

Described step 1 specifically comprises following operation:

(1) coded frame, reference frame are divided into encoding block and reference block according to the data block size of processing in multi-core DSP local memory capacity and estimation, the data of encoded image frame and reference frame are stored in outside mass storage, and the data of present encoding piece and reference block are stored in local memory;

(2) coded frame is divided into encoding block and code line, as shown in Figure 2, each core of multi-core DSP is carried out motion estimation operation with coding behavior unit, after pronucleus finishes a code line motion estimation operation, obtains immediately below first estimated coding of not taking exercises is capable to proceed;

(3) system control table is set, system control table comprises the code line indicating device of first estimation operation of not taking exercises in present encoding frame counter, current encoded frame, current encoded frame encoding state table, and system control table is that multinuclear shares, synchronous to realize each core;

(4) in local memory for present encoding piece and reference data piece arrange Ping-Pang structure memory block, present encoding be Pang, data pre-fetching is to Ping.

Described step 2 specifically comprises the steps:

(1), if present encoding piece belongs to P frame, carry out prediction and motion search operations in encoding block estimation;

(2) start DMA and the data of the next encoding block of present encoding piece are prefetched to the internal storage of core by external memory storage, source address, destination address, data block size are pressed formula (1) and are calculated;

$\left\{\begin{array}{c}\mathrm{DMA}\_\mathrm{src}=\mathrm{fenc}\_\mathrm{base}+\mathrm{corei}\_\mathrm{line}\×\mathrm{line}\_\mathrm{block}\×\mathrm{block}\_\mathrm{size}\\ +(\mathrm{corei}\_\mathrm{block}+1)\×\mathrm{block}\_\mathrm{width}\\ \mathrm{DMA}\_\mathrm{dst}=\mathrm{corei}\_\mathrm{fenc}\_\mathrm{base}\\ \mathrm{data}\_\mathrm{size}=\mathrm{block}\_\mathrm{size}\end{array}\right.---\left(1\right)$

Wherein block_width is encoding block width, block_hight is encoding block height, block_size is coded block size, block_size=block_width * block_hight, DMA_src is the source address of i core DMA carrying data, DMA_dst is the destination address of i core DMA carrying data, data_size is carrying size of data, corei_fenc_base is that the encoding block data of i core are in the initial address of storage inside, fenc_base is the initial address of current encoded frame, corei_line is i the code line that core is current, line_block is code line size, corei_block is the present encoding piece of i core.

(3) start DMA and the data of the reference block of the next encoding block of present encoding piece are prefetched to the internal storage of core by external memory storage, be about to the data pre-fetching of a plurality of reference blocks centered by motion vectors in reference frame to the local memory of core, the quantity of the reference data of looking ahead piece determines by method for searching motion, and source address, destination address, data block size press formula (2) calculating;

$\left\{\begin{array}{c}\mathrm{DMA}\_\mathrm{src}=\mathrm{fref}\_\mathrm{base}\\ +(\mathrm{mvp}\_x\×\mathrm{line}\_\mathrm{pix}-\mathrm{block}\_\mathrm{width})+(\mathrm{mvp}\_y-\mathrm{block}\_\mathrm{hight})\\ \mathrm{DMA}\_\mathrm{dst}=\mathrm{corei}\_\mathrm{ref}\_\mathrm{base}\\ \mathrm{data}\_\mathrm{size}=\mathrm{block}\_\mathrm{size}\×\mathrm{bref}\_\mathrm{size}\end{array}\right.---\left(2\right)$

Wherein DMA_src is the source address of i core DMA carrying data, DMA_dst is the destination address of i core DMA carrying data, fref_base is reference frame first address, line_pix is the pixel quantity of a line in reference frame, mvp_x, mvp_y is motion vector predictor, after finishing, present encoding piece motion search can obtain motion vector predictor according to the Forecasting Methodology of motion vector, corei_ref_base is the data storage first address of reference block, bref_size is the data block size of reference, other parameter cotypes (1);

(4) carry out motion compensation (MC), dct transform, quantization operation;

(5) upgrade control table, position, present encoding piece coding maker position;

(6) repeating step (1), (2), (3), (4), (5), until each encoding block motion estimation operation of present encoding row finishes.

Described step 3 specifically comprises the steps:

(1), if present encoding piece belongs to B frame, carry out prediction and motion search operations in encoding block estimation;

(2) start DMA and the data of the next encoding block of present encoding piece are prefetched to the local memory of core by external memory storage, source address, destination address, data block size are pressed formula (1) and are calculated;

(3) start DMA by the data pre-fetching of the forward direction reference block of the next data block of present encoding piece the internal storage to core, be about to the data pre-fetching of a plurality of reference blocks centered by motion vectors in reference frame to the local memory of core, the quantity of reference block of looking ahead determines by method for searching motion, and source address, destination address, data block size press formula (2) calculating;

(4) start DMA by the data pre-fetching of the backward reference block of the next data block of present encoding piece the internal storage to core, be about to the data pre-fetching of a plurality of reference blocks centered by motion vectors in reference frame to the local memory of core, the quantity of reference block of looking ahead determines by method for searching motion, and source address, destination address, data block size press formula (2) calculating;

(5) carry out motion compensation (MC), dct transform, quantization operation;

(6) upgrade control table, position, present encoding piece coding maker position;

(7) repeating step (1), (2), (3), (4), (5), (6), until each encoding block motion estimation operation of present encoding row finishes.

The present invention's advantage is compared with prior art:

1, the present invention has realized in estimation data and has read with data processing parallelly, has accelerated the coding rate of multi-core DSP processor;

2, reading in advance of the motion estimation data that data prefetching method of the present invention is realized, takes full advantage of bandwidth of memory and has reduced memory bottleneck problem;

3, the present invention takes full advantage of the DMA high speed data transfer parts of embedded multi-core dsp processor, has accelerated data transmission procedure.

Accompanying drawing explanation

Multinuclear encoder stores structural representation in Fig. 1 video monitoring;

In Fig. 2 the present invention, code line is divided schematic diagram;

Data store organisation schematic diagram in Fig. 3 specific embodiment of the invention;

A kind of flow chart of the data prefetching method towards multi-core DSP parallel video coding in Fig. 4 the present invention.

Embodiment

Provide the embodiment of the TMS320C6678LE evaluation board of TI of the present invention company below.

The TMS320C6678LE evaluation board of TI company comprises 1 TMS320C6678 chip, and external memory space is DDR3 memory, 512MB.TMS320C6678 chip comprises 8 cores, core0 is to core7, the operating frequency of each core is 1.0GHz, each core includes 32KB one-level Data Buffer Memory L1D and 32KB one-level program buffer storage L1P, each core has 512KB second-level storage and 4MB shared storage MSM, one-level Data Buffer Memory is made as to cache, and the encoding block in estimation, the data of reference block are stored in second-level storage, and the data of encoded video, reference frame are stored in external memory storage DDR3.

Estimation adopts diamond search method, search window 48 * 48, the data of 9 reference blocks centered by motion vector predictor of therefore looking ahead in reference frame.Video format is YUV4:2:0, resolution is 704 * 576, coded block size is 16 * 16, therefore a coded frame has 36 code lines, every code line has 44 encoding blocks, video initial address is 0x82000000, forward reference frame initial address is 0x80000000, backward reference frame initial address is 0x80800000, the present encoding blocks of data of each core, the data first address of reference block is corei_fenc_base, corei_ref_base0, corei_ref_base1, each memory block is Ping-Pang structure, be that current estimation is used Pang data, prefetch data stores Ping into.

Parallel encoding control table is set, control table comprises enc_tab, cur_enc_frame, first_unenc_line, coding-control table, motion vector are that multinuclear is shared data, be stored in the MSM memory in sheet, as shown in Figure 3, multinuclear is accessed with exclusive mode, and semaphore is semphone0, and other coding parameters store in the L2 of each core.Each core of core0～core7 arranges present encoding column indicator core0_line～core7_line and present encoding piece indicating device core0_block～core0_block, corei_line is used to refer to the line number of the code line of processing when pronucleus, and corei_block is used to refer to the numbering when pronucleus encoding block.

Data pre-fetching between DDR3 and L2 is realized with QDMA, core0～core7 is used respectively the QDMA0～QDMA7 of TCC0 in EDMA3 as the data pre-fetching passage of each core, be set to AB-sync, LINK mode, the PaRAMSet of QDMA0～QDMA7 is 00～02,10～12,20～22,30～32,40～42,50～52,60～62,70～72.

Specific implementation process is:

(1) core0 reads in video sequence, and recorded video sequence totalframes (total_frames);

(2) core0 initialization global parameter, the cur_enc_frame of control table is initialized as-1, and first_unenc_line is-1;

(3) judged whether not coded frame, whether cur_enc_frame is greater than total_frames, if be greater than, and end-of-encode, otherwise proceed to (4);

(4) initialization codes state table is " 0 ", and first_unenc_line is 0;

(5) judged whether not code line, whether first_unenc_line is greater than 36, if be greater than present encoding frame end, turns (3), otherwise, turn (6);

(6) obtain code line, each core reads first_unenc_line to corei_line by exclusive mode, corei_block=0, and first_unenc_line adds 1;

(7) by cur_enc_frame, corei_line, corei_block calculation code piece first address, Y, U, tri-component address of V are respectively:

$\left\{\begin{array}{c}Y=0x82000000+\mathrm{cur}\_\mathrm{enc}\_\mathrm{frame}\×608256\\ +\mathrm{corei}\_\mathrm{line}\×11264+\mathrm{corei}\_\mathrm{block}\×16\\ U=0x82000000+\mathrm{cur}\_\mathrm{enc}\_\mathrm{frame}\×608256+0x63000\\ +\mathrm{corei}\_\mathrm{line}\×5632+\mathrm{corei}\_\mathrm{block}\×8\\ V=0x82000000+\mathrm{cur}\_\mathrm{enc}\_\mathrm{frame}\×608256+0\mathrm{xBC}00\\ +\mathrm{corei}\_\mathrm{line}\×5632+\mathrm{corei}\_\mathrm{blick}\×8\end{array}\right.$

Data_size is respectively: 256,64,64;

(8) judge whether a left side, ,Shang, upper right, upper left motion search that present encoding piece is adjacent complete, complete and continue, do not complete and wait for that adjacent block motion search finishes;

(9) whether the data that judge present encoding piece are transported to L2, are to continue, otherwise wait for that carrying finishes;

(10) carry out prediction, the motion search operations of encoding block;

(11) according to next encoding block and the reference macroblock data thereof of looking ahead shown in Fig. 4, the source address of the prefetch data of P frame encoding block, B frame encoding block, destination address, data block size are calculated by formula (1) and formula (2), block_width is 16, block_hight is 16, line_pix is 704, fenc_base=0x82000000+cur_enc_frame * 608256, bref_size is 9;

(12) carry out motion compensation (MC), dct transform, quantization operation;

(13) enc_tab in change control table, i.e. enc_tab[corei_line, corei_block]=1;

(14) judging whether present encoding row finishes, and whether corei_block is less than 44, is to turn (8) to continue the processing of present encoding row, otherwise turn (5), carries out next code line processing;

(15) repeat (5) to (14), until all frame of video processing of video finish.

Test result is as shown in the table: (QP=27)

When video image group is IBBP BBP BBP BBP BB structure, average behavior improves approximately 15%.

Claims (4)

1. towards a data prefetching method for multi-core DSP estimation, it is characterized in that: step is as follows:

Step 1, prefetch data block size is set, divides encoding block, reference block size, the data of encoding block and reference block are stored in the local memory of core, and memory block is set to Ping-Pang structure;

If the encoding block of the current execution estimation of step 2 belongs to P frame, carry out prediction, motion search operations, the data of look ahead next encoding block and reference block;

If the encoding block of the current execution estimation of step 3 belongs to B frame, carry out prediction, motion search operations, the data of look ahead next encoding block and forward direction reference block, backward reference block.

2. a kind of data prefetching method towards multi-core DSP estimation according to claim 1, is characterized in that: described step 1 specifically comprises following operation:

(1) coded frame, reference frame are divided into encoding block and reference block according to the data block size of processing in multi-core DSP local storage capacity and estimation, the data of encoded image frame and reference frame are stored in outside mass storage, and the data of present encoding piece and reference block are stored in local memory;

(2) coded frame is divided into encoding block and code line, each core of multi-core DSP is carried out motion estimation operation with coding behavior unit, after pronucleus finishes a code line motion estimation operation, obtains immediately below first estimated coding of not taking exercises is capable to proceed;

(3) system control table is set, system control table comprises the code line indicating device of first estimation operation of not taking exercises in present encoding frame counter, current encoded frame, current encoded frame encoding state table, and system control table is that multinuclear shares, and realizes each core synchronous;

(4) in local memory for present encoding piece and reference data piece arrange Ping-Pang structure memory block, present encoding be Pang, data pre-fetching is to Ping.

3. a kind of data prefetching method towards multi-core DSP estimation according to claim 1, is characterized in that: described step 2 specifically comprises the steps:

(2.1), if present encoding piece belongs to P frame, carry out prediction and motion search operations in encoding block estimation;

(2.2) start DMA and the data of the next encoding block of present encoding piece are prefetched to the internal storage of core by external memory storage, source address, destination address, data block size are pressed formula (1) and are calculated;

$\left\{\begin{array}{c}\mathrm{DMA}\_\mathrm{src}=\mathrm{fenc}\_\mathrm{base}+\mathrm{corei}\_\mathrm{line}\×\mathrm{line}\_\mathrm{block}\×\mathrm{block}\_\mathrm{size}\\ +(\mathrm{corei}\_\mathrm{block}+1)\×\mathrm{block}\_\mathrm{width}\\ \mathrm{DMA}\_\mathrm{dst}=\mathrm{corei}\_\mathrm{fenc}\_\mathrm{base}\\ \mathrm{data}\_\mathrm{size}=\mathrm{block}\_\mathrm{size}\end{array}\right.---\left(1\right)$

Wherein block_width is encoding block width, block_hight is encoding block height, block_size is coded block size, block_size=block_width * block_hight, DMA_src is the source address of i core DMA carrying data, DMA_dst is the destination address of i core DMA carrying data, data_size is carrying size of data, corei_fenc_base is that the encoding block data of i core are in the initial address of storage inside, fenc_base is the initial address of current encoded frame, corei_line is i the code line that core is current, line_block is code line size, corei_block is the present encoding piece of i core,

(2.3) start DMA and the data of the reference block of the next encoding block of present encoding piece are prefetched to the internal storage of core by external memory storage, be about to the data pre-fetching of a plurality of reference blocks centered by motion vectors in reference frame to the local memory of core, the quantity of the reference data of looking ahead piece determines by method for searching motion, and source address, destination address, data block size press formula (2) calculating;

$\left\{\begin{array}{c}\mathrm{DMA}\_\mathrm{src}=\mathrm{fref}\_\mathrm{base}\\ +(\mathrm{mvp}\_x\×\mathrm{line}\_\mathrm{pix}-\mathrm{block}\_\mathrm{width})+(\mathrm{mvp}\_y-\mathrm{block}\_\mathrm{hight})\\ \mathrm{DMA}\_\mathrm{dst}=\mathrm{corei}\_\mathrm{ref}\_\mathrm{base}\\ \mathrm{data}\_\mathrm{size}=\mathrm{block}\_\mathrm{size}\×\mathrm{bref}\_\mathrm{size}\end{array}\right.---\left(2\right)$

Wherein DMA_src is the source address of i core DMA carrying data, DMA_dst is the destination address of i core DMA carrying data, fref_base is reference frame first address, line_pix is the pixel quantity of a line in reference frame, mvp_x, mvp_y is motion vector predictor, after finishing, present encoding piece motion search can obtain motion vector predictor according to the Forecasting Methodology of motion vector, corei_ref_base is the data storage first address of reference block, bref_size is the data block quantity of reference, the same formula of other parameters (1);

(2.4) carry out motion compensation (MC), dct transform, quantization operation;

(2.5) upgrade control table, position, present encoding piece coding maker position;

(2.6) repeating step (2.1), (2.2), (2.3), (2.4), (2.5), until each encoding block motion estimation operation of present encoding row finishes.

4. a kind of data prefetching method towards multi-core DSP estimation according to claim 1, is characterized in that: described step 3 specifically comprises the steps:

(3.1), if present encoding piece belongs to B frame, carry out prediction and motion search operations in encoding block estimation;

(3.2) start DMA and the data of the next encoding block of present encoding piece are prefetched to the local memory of core by external memory storage, source address, destination address, data block size are pressed formula (1) and are calculated;

(3.3) start DMA by the data pre-fetching of the forward direction reference block of the next data block of present encoding piece the internal storage to core, be about to the data pre-fetching of a plurality of reference blocks centered by motion vectors in reference frame to the local memory of core, the quantity of reference block of looking ahead determines by method for searching motion, and source address, destination address, data block size press formula (2) calculating;

(3.4) start DMA by the data pre-fetching of the backward reference block of the next data block of present encoding piece the internal storage to core, be about to the data pre-fetching of a plurality of reference blocks centered by motion vectors in reference frame to the local memory of core, the quantity of reference block of looking ahead determines by method for searching motion, and source address, destination address, data block size press formula (2) calculating;

(3.5) carry out motion compensation (MC), dct transform, quantization operation;

(3.6) upgrade control table, position, present encoding piece coding maker position;

(3.7) repeating step (3.1), (3.2), (3.3), (3.4), (3.5), (3.6), until each encoding block motion estimation operation of present encoding row finishes.

Images