WO2023019910A1 - 视频处理方法及装置、电子设备、存储介质、计算机程序、计算机程序产品 - Google Patents

视频处理方法及装置、电子设备、存储介质、计算机程序、计算机程序产品 Download PDF

Info

Publication number
WO2023019910A1
WO2023019910A1 PCT/CN2022/078398 CN2022078398W WO2023019910A1 WO 2023019910 A1 WO2023019910 A1 WO 2023019910A1 CN 2022078398 W CN2022078398 W CN 2022078398W WO 2023019910 A1 WO2023019910 A1 WO 2023019910A1
Authority
WO
WIPO (PCT)
Prior art keywords
energy
target frame
macroblock
size
translation
Prior art date
Application number
PCT/CN2022/078398
Other languages
English (en)
French (fr)
Inventor
许通达
高宸健
王岩
袁涛
秦红伟
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2023019910A1 publication Critical patent/WO2023019910A1/zh
Priority to US18/444,824 priority Critical patent/US20240195968A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock

Definitions

  • the present disclosure relates to the field of video coding, and in particular, to a video processing method and device, electronic equipment, storage media, computer programs, and computer program products.
  • a macroblock-level adaptive quantization method can be used to allocate more bit rates to smooth areas of video frames.
  • the above methods usually have the disadvantage of being difficult to adapt to complex scenarios in encoding.
  • the present disclosure proposes a video processing method and device, electronic equipment, storage medium, computer program, and computer program product, aiming at adapting to scenes with large resolution changes and reducing block effects during video encoding.
  • the present disclosure proposes a video processing method and device, electronic equipment, storage medium, computer program, and computer program product, aiming at adapting to scenes with large resolution changes and reducing block effects during video encoding.
  • An embodiment of the present disclosure provides a video processing method, the method comprising:
  • each of the first energy maps represents AC energy of at least one first macroblock corresponding to a macroblock size, wherein, each of the first macroblocks is obtained by segmenting the target frame corresponding to the size of the macroblock;
  • An adaptive quantization parameter corresponding to the target frame is determined according to the first energy graph, and the target frame is encoded by using the adaptive quantization parameter.
  • An embodiment of the present disclosure provides a video processing device, the device comprising:
  • a target frame determination module configured to determine the target frame in the video to be processed
  • the first energy map determination module is configured to respectively determine a plurality of first energy maps corresponding to the target frame according to at least two preset macroblock sizes, and each of the first energy maps represents at least one macroblock size corresponding to AC energy of a first macroblock, wherein each of the first macroblocks is obtained by dividing the target frame according to the size of the macroblock;
  • An energy map determination module configured to determine a first energy map corresponding to the target frame according to each of the first energy maps, and the first energy map is used to characterize the energy distribution in the target frame;
  • the parameter determination module is configured to determine an adaptive quantization parameter corresponding to the target frame according to the first energy map, and encode the target frame by using the adaptive quantization parameter.
  • An embodiment of the present disclosure provides an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.
  • An embodiment of the present disclosure provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the foregoing method is implemented.
  • An embodiment of the present disclosure provides a computer program, the computer program includes computer readable code, and when the computer readable code is read and executed by a computer, a part or part of the method in any embodiment of the present disclosure is realized. All steps.
  • An embodiment of the present disclosure provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, any embodiment of the present disclosure is realized Some or all of the steps in the method.
  • the target frame is segmented by different macroblock sizes, the corresponding AC energy after the target frame is divided by each macroblock size is calculated, and the first energy map is determined based on each first energy map containing each corresponding AC energy, and Determine the adaptive quantization parameter corresponding to the target frame according to the first energy map, so that the adaptive quantization parameter is associated with the energy characteristics of the target frame after being divided by various macroblock sizes, and the adaptive quantization parameter is applied to the coding of the target frame
  • video encoding can be performed based on macroblocks of different sizes, adapting to complex scenes with large resolution changes during video encoding.
  • FIG. 1 is a flowchart of a video processing method provided by an embodiment of the present disclosure
  • FIG. 2 is a flowchart of a process of determining a first energy map provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of a process of determining a first energy map provided by an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of determining AC energy provided by an embodiment of the present disclosure.
  • FIG. 5 is a schematic diagram of determining a first energy spectrum provided by an embodiment of the present disclosure
  • FIG. 6 is a schematic diagram of a process of determining an adaptive quantization parameter provided by an embodiment of the present disclosure
  • FIG. 7 is a schematic diagram of a data transmission process provided by an embodiment of the present disclosure.
  • FIG. 8 is a flowchart of a video processing method provided by an embodiment of the present disclosure.
  • FIG. 9 is a flowchart of a process of determining a second energy map provided by an embodiment of the present disclosure.
  • FIG. 10 is a schematic diagram of a process of determining a second energy map provided by an embodiment of the present disclosure.
  • FIG. 11 is a schematic diagram of a process of determining a translation frame provided by an embodiment of the present disclosure.
  • FIG. 12 is a schematic diagram of determining a first energy spectrum provided by an embodiment of the present disclosure.
  • Fig. 13 is a schematic diagram of determining an energy pyramid provided by an embodiment of the present disclosure.
  • FIG. 14 is a schematic flowchart of a video encoding provided by an embodiment of the present disclosure.
  • FIG. 15 is a schematic diagram of a video processing device provided by an embodiment of the present disclosure.
  • FIG. 16 is a block diagram of an electronic device provided by an embodiment of the present disclosure.
  • Fig. 17 is a block diagram of an electronic device provided by an embodiment of the present disclosure.
  • FIG. 1 is a flowchart of a video processing method provided by an embodiment of the present disclosure.
  • the video processing method may be performed by a terminal device or other processing devices, wherein the terminal device may be user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant) , PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc.
  • the video processing method may be implemented by a processor invoking computer-readable instructions stored in a memory.
  • the video processing method of the embodiment of the present disclosure can be performed on the predetermined video to be processed and the preset multiple macroblock sizes, and the adaptive quantization parameter corresponding to each frame can be determined , perform video coding according to each frame in the video to be processed and the corresponding adaptive quantization parameters.
  • the video processing method and video encoding in the embodiments of the present disclosure can be completed by the same device, or the terminal device or other devices execute the video processing method first, and then transmit the method to a video encoder for video encoding.
  • the video processing method of the embodiment of the present disclosure includes the following steps:
  • Step S10 determining the target frame in the video to be processed.
  • the embodiment of the present disclosure may execute the video processing method in a manner of separately processing each frame of the video to be processed. That is to say, each frame in the video to be processed may be used as a target frame for image processing, so as to determine the corresponding adaptive quantization parameter. After completing the image processing of the current target frame, re-determine the unprocessed frame in the video to be processed as a new target frame until the image processing of all frames in the video to be processed is completed, and then complete the video processing process of the video to be processed.
  • the processing sequence of the target frames may be sequentially determined based on the sequence of the time axis.
  • T1, T2, T3, T4, T5, T6, T7, T8, T9, and T10 are sequentially determined as target frames according to the order of the time axis, so as to determine the corresponding Adaptive quantization parameters.
  • the current target frame is input into the video encoder as the input frame, and the corresponding adaptive quantization parameter is input into the adaptive quantization interface of the video encoder for video encoding .
  • the corresponding adaptive quantization parameter is input into the adaptive quantization interface of the video encoder for video encoding .
  • Step S20 respectively determine a plurality of first energy maps corresponding to the target frame according to at least two preset macroblock sizes.
  • each first energy map includes AC energy of multiple first macroblocks of the same size, wherein each first macroblock is obtained by cutting the target frame corresponding to the size of the macroblock. That is to say, each first energy map is used to represent AC energy of multiple first macroblocks obtained after dividing the target frame by corresponding macroblock size.
  • the AC energy corresponding to each first macroblock is used to characterize the image texture complexity in the first macroblock. For the first macroblock with a flat image, the calculated AC energy is small, and for the first macroblock with complex image texture, the calculated AC energy is relatively large.
  • the size of each macroblock can be set according to a fixed ratio, that is, the size of each macroblock is a preset geometric sequence.
  • the process of determining the size of each macroblock is to first set a target size, and then reduce and enlarge the target size at a fixed ratio for multiple times to obtain a plurality of corresponding reduced sizes and enlarged sizes. Both the size and the enlarged size are used as a preset macroblock size, and the macroblock size is applied to the video processing method of the embodiment of the present disclosure.
  • the preset target size is N (N is a positive number) and the fixed ratio is 2
  • the corresponding two reduced sizes N/2 and N/4 can be obtained by reducing and enlarging the target size twice respectively
  • the five macroblock sizes are determined to be N/4, N/2, N, 2N and 4N in sequence.
  • Fig. 2 is a flowchart of a process of determining a first energy map provided by an embodiment of the present disclosure. As shown in FIG. 2, in a possible implementation manner, for any macroblock size, the process of determining a plurality of first energy maps corresponding to the target frame according to preset multiple macroblock sizes includes the following steps:
  • Step S21 Segment the target frame according to the size of any macroblock to obtain a plurality of corresponding first macroblocks.
  • the target frame is respectively segmented according to multiple macroblock sizes set according to a fixed ratio, to obtain multiple first macroblocks corresponding to each macroblock size.
  • the format of the target frame may be RGB format or YUV format, which is not limited here.
  • when the target frame format is YUV format it may be 444 type, 422 type, 420 type or 411 type.
  • the process of segmenting the target frame can be determined according to the format. For example, when the target frame format is YUV of type 422, multiple pixel values of the three channels Y, U, and V are respectively acquired according to the ratio of 2:1:1, so as to determine the corresponding first macroblock.
  • a plurality of first macroblocks corresponding to the macroblock size can be obtained. Taking the size of the target frame as 32 ⁇ 32 and the sizes of the macroblocks as 2 ⁇ 2, 4 ⁇ 4, 8 ⁇ 8, 16 ⁇ 16 and 32 ⁇ 32 as an example for illustration. Segment the target frame according to the size of each macroblock to obtain 256 first macroblocks with a size of 2 ⁇ 2, 64 first macroblocks with a size of 4 ⁇ 4, and 16 first macroblocks with a size of 8 ⁇ 8 , 4 first macroblocks with a size of 16 ⁇ 16 and 1 first macroblock with a size of 32 ⁇ 32.
  • Fig. 3 is a schematic diagram of a process of determining a first energy map provided by an embodiment of the present disclosure.
  • the target frame 30 is respectively segmented according to a plurality of preset macroblock sizes N, 2N, ..., kN with a fixed ratio of 2, to obtain a plurality of first macroblocks of different sizes.
  • the macroblock 31 is used to determine the first energy map 32 corresponding to each macroblock size through multiple first macroblocks of the same size.
  • Step S22 Determine the AC energy of each of the first macroblocks.
  • each first macroblock includes first macroblocks of the same size or different sizes. That is to say, the AC energy of all the first macroblocks obtained after being divided according to the size of each macroblock can be determined. The AC energy is used to characterize the energy feature of each pixel in the corresponding first macroblock.
  • the embodiment of the present disclosure may determine the corresponding AC energy according to the variance of all pixel values in the first macroblock and the number of pixels. In some embodiments, the variance of all pixel values in the first macroblock is calculated first, and then the variance is divided by the number of pixels included in the first macroblock to obtain the corresponding AC energy.
  • Fig. 4 is a schematic diagram of determining AC energy provided by an embodiment of the present disclosure.
  • the first macroblock 40 obtained after division, calculate the variance of each pixel value Y1-Y8, U1-U4, and V1-V4, and then divide it by the total number of pixels 16 included therein to obtain the corresponding AC energy E1.
  • the AC energy E1 can be filled into each pixel position of the first macroblock 40 as a pixel value.
  • Step S23 Determine the corresponding first energy map according to the plurality of first macroblocks obtained by dividing the target frame with the same macroblock size.
  • the size of multiple first macroblocks obtained by dividing the target frame with the same macroblock size is the same, that is, the first macroblock is determined according to the AC energy corresponding to multiple first macroblocks of the same size Energy diagram.
  • the pixel value of each pixel position in the first energy map corresponds to the AC energy of the first macroblock. That is to say, the first energy map corresponding to the size of each macroblock is respectively determined, and the size of each first energy map is the same as the target frame size.
  • fill in the AC energy of each first macroblock in the corresponding pixel position in the corresponding first energy map, and the corresponding pixel position is before the pixel in each first macroblock is divided. position in the target frame.
  • the method of determining the first energy map may be to first fill the AC energy corresponding to each first macroblock as a pixel value into each pixel position in the first macroblock, and then fill the first macroblock with the same size Each pixel value in the block is filled into the corresponding pixel position in the corresponding first energy map, or each first macroblock filled with AC energy as pixel value is spliced according to the position before segmentation to obtain the first energy map.
  • each first energy map since the first macroblock size of each first energy map is determined to be different, multiple first energy maps corresponding to the target frame are superimposed together to form a set of energy pyramids for characterizing the energy distribution of the corresponding target frame .
  • Step S30 Determine the first energy spectrum corresponding to the target frame according to each of the first energy diagrams.
  • the first energy map is used to characterize the energy distribution in the target frame.
  • the process of determining the first energy spectrum corresponding to the target frame may include: determining a first mean value of each first energy map at the same pixel position, and determining the first energy spectrum according to the first mean value corresponding to each pixel position.
  • each first energy map is superimposed and fused, and the first mean value of the pixel values of each first energy map at the same pixel position is calculated as the first energy map at the pixel position pixel value. That is to say, each first energy map is regarded as a plurality of channels, and the first energy map is obtained through channel average fusion.
  • the target frame corresponds to two first energy maps, the first energy map 1 is used to represent the AC energy of four first macroblocks with a size of 2 ⁇ 2, and the first energy map 2 is used to represent 16 AC energies with a size of 1 ⁇ 1
  • the AC energy of the first macroblock is taken as an example for illustration.
  • the pixel values corresponding to each pixel position in the first energy map 1 are respectively:
  • E1 , E2 , E3 , and E4 respectively represent AC energy of the four first macroblocks.
  • the pixel values corresponding to each pixel position in the first energy map 2 are respectively:
  • each pixel value respectively represents the AC energy of the 16 first macroblocks.
  • the first energy map 1 and the second energy map 2 are used as two channels for channel averaging, that is, the average value of the pixel values at the same pixel position is calculated to obtain the first energy map.
  • the pixel values in the first energy spectrum are respectively:
  • FIG. 5 is a schematic diagram of determining a first energy spectrum provided by an embodiment of the present disclosure.
  • the embodiment of the present disclosure determines the first energy pyramid 50 composed of a plurality of first energy images corresponding to the target frame, and performs channel averaging on the first energy pyramid 50, that is, A first energy map 51 for characterizing the energy distribution of the target frame can be obtained.
  • the embodiments of the present disclosure can separately divide the target frame by different macroblock sizes to perform energy estimation, and determine the first energy map representing the energy distribution of the target frame through the multiple first energy maps obtained after estimation, which can be used in subsequent video coding Adaptive multi-resolution video quantization process.
  • Step S40 Determine an adaptive quantization parameter corresponding to the target frame according to the first energy map, and encode the target frame by using the adaptive quantization parameter.
  • the adaptive quantization parameter corresponding to the target frame can be determined according to the first energy map representing the distribution of the target frame, which is used to perform adaptive quantization on the target frame during the video encoding process and improve the efficiency of video encoding. Effect.
  • the process of determining the adaptive quantization parameter corresponding to the target frame may include: determining the second energy spectrum corresponding to the first energy spectrum by means of average pooling. An adaptive quantization parameter corresponding to the target frame is determined according to the second energy graph, and the target frame is encoded by using the adaptive quantization parameter.
  • the purpose of performing average pooling on the first energy map is to reduce the size of the first energy map to obtain a second energy map with a smaller resolution as a candidate adaptive quantization parameter.
  • the average pooling process includes determining a target macroblock size, using the target macroblock size as a window and a step size, and performing average pooling on the first energy map to obtain the second energy map.
  • the target macroblock size can be determined according to multiple preset macroblock sizes, that is, the target macroblock size can be one of the preset multiple macroblock sizes, or can be determined according to one of the preset multiple macroblock sizes.
  • One is obtained by zooming. That is to say, the second energy map scales a map of a target macroblock size for the first energy map.
  • the average pooling process reduces the resolution while retaining the energy features in the first energy map, and improves the subsequent video coding process. s efficiency.
  • the target macroblock size may be the median value among the macroblock sizes. For example, when the size of each macroblock is N/4, N/2, N, 2N and 4N, the target macroblock size is determined to be N. And after determining that the size of the target macroblock is N, average pooling is performed on the first energy map with the size of N ⁇ N as the window and N as the step size to obtain the second energy map.
  • the product of the resolution of the second energy atlas and N is the resolution of the first energy atlas, that is, the second energy atlas is an image with a smaller resolution used to reflect the energy characteristics of the target frame.
  • the adaptive quantization parameter corresponding to the target frame is determined by way of histogram mapping.
  • a histogram mapping table corresponding to the second energy spectrum is determined.
  • the second energy spectrum is mapped according to the histogram mapping table to obtain an adaptive quantization parameter corresponding to the target frame.
  • the adaptive quantization parameter and the target frame are input into a video encoder to video encode the target frame based on the corresponding adaptive quantization parameter.
  • the mapping process can be to initialize a blank image with the same size as the second energy map, for each pixel value in the second energy map, determine the corresponding value in the histogram mapping table, and put each value It is stored in the same position on the blank image as the position of the corresponding pixel value, and the corresponding adaptive quantization parameter is obtained.
  • the values corresponding to the pixel values in the second energy spectrum in the histogram mapping table are determined, and the corresponding pixel values in the second energy spectrum are replaced according to the values to obtain adaptive quantization parameters.
  • FIG. 6 is a schematic diagram of a process of determining an adaptive quantization parameter provided by an embodiment of the present disclosure.
  • the second energy map 61 corresponding to the target frame is obtained by means of average pooling.
  • the adaptive quantization parameter 62 is obtained by performing histogram mapping on the second energy spectrum 61.
  • the histogram mapping process includes performing histogram statistics on the second energy spectrum 61 to obtain a corresponding histogram mapping table, and then obtaining the adaptive quantization parameter 62 by mapping the second energy spectrum 61 through the histogram mapping table.
  • FIG. 7 is a schematic diagram of a data transmission process provided by an embodiment of the present disclosure.
  • the target frame 70 is input into the video encoder 72 as an input frame of the video encoder.
  • the adaptive quantization parameter 71 determined based on the first energy map is also input into the adaptive quantization interface of the video encoder 72 as a parameter for video encoding the target frame 70 .
  • the embodiments of the present disclosure may determine corresponding adaptive quantization parameters based on the energy distribution characteristics of the target frame, so as to perform adaptive quantization adjustment and improve the efficiency of the video encoding process.
  • the target frame can be segmented by different macroblock sizes, the AC energy corresponding to the target frame divided by each macroblock size can be calculated respectively, and the first energy map can be determined based on each first energy map containing each corresponding AC energy, And determine the adaptive quantization parameter corresponding to the target frame according to the first energy map, so that the adaptive quantization parameter is associated with the energy characteristics of the target frame after being divided by various macroblock sizes, and when the adaptive quantization parameter is applied to the target frame
  • video encoding can be performed based on macroblocks of different sizes, adapting to complex scenes with large resolution changes during video encoding.
  • FIG. 8 is a flowchart of a video processing method provided by an embodiment of the present disclosure, which is used to characterize the implementation process of another video processing method of the embodiment of the present disclosure. As shown in FIG. 8, in an exemplary embodiment, the video processing method of the embodiment of the present disclosure includes the following steps:
  • Step S10' determine the target frame in the video to be processed.
  • step S10 the process of determining the target frame in this step is similar to step S10.
  • Step S20' respectively determine a plurality of first energy maps corresponding to the target frame according to a plurality of preset macroblock sizes.
  • step S20 the process of determining the first energy map in this step is similar to step S20.
  • Step S30' respectively determine a plurality of second energy maps corresponding to the target frame according to the size of each macroblock.
  • the corresponding multiple second energy maps are also determined according to the size of each macroblock, and the first energy map and the second energy map are determined.
  • Multiple macroblocks of a picture are of the same size.
  • the second energy map includes AC energies of multiple second macroblocks of the same size, wherein the second macroblocks are obtained by shifting and segmenting the target frame corresponding to the size of the macroblocks.
  • Fig. 9 is a flowchart of a process of determining a second energy map provided by an embodiment of the present disclosure. As shown in FIG. 9 , the process of determining multiple second energy maps corresponding to the target frame in this embodiment of the present disclosure may include the following steps:
  • Step S31' Perform translation processing on the target frame according to the size of each macroblock to obtain a plurality of corresponding translation frames.
  • the process of determining the plurality of second energy maps corresponding to the target frame may include: respectively performing translation processing on the target frame according to the size of each macroblock to obtain a plurality of corresponding translation frames.
  • the second energy map corresponding to the translation frame of each macroblock size is respectively determined. That is to say, the target frame is shifted according to the size of each macroblock, and then the second energy map corresponding to the target frame shifted according to the size of the current macroblock is determined according to the size of each macroblock.
  • the second energy map corresponding to translation frame 1 is determined according to macroblock size 1 , determine the second energy map corresponding to the translation frame 2 according to the macroblock size 2, and determine the second energy map corresponding to the translation frame 3 according to the macroblock size 3.
  • the process of determining the translation frame corresponding to each macroblock size includes: scaling each macroblock size according to a predetermined scaling ratio to obtain a corresponding translation size.
  • the translation process is performed on the target frame according to each translation size to obtain a plurality of corresponding translation frames. That is to say, the size of each macroblock is scaled with a predetermined scaling ratio, and then the target frame is translated according to the scaled translation size to obtain a translation frame, that is, the corresponding translation frame is obtained by translating each target frame by the corresponding translation size.
  • the translation size is determined to be N/8, N/4, N/2 , N and 2N, to respectively translate the target frame by N/8, N/4, N/2, N and 2N lengths in the preset translation direction.
  • the translation directions of the target frames respectively translated by each translation size are the same.
  • the translation direction may be any diagonal direction, for example, translation to the upper left corner, lower right corner translation, lower left corner translation or upper right corner translation along the diagonal.
  • the method of translating the target frame according to the corresponding translation size includes: by copying two adjacent edges of the target frame, adding pixel rows and pixel columns corresponding to the translation size on the two adjacent edges, and The pixels at the intersecting positions of two adjacent edges are copied to the blank area between the added pixel row and pixel column to obtain the corresponding candidate translation frame. On both sides of the candidate translation frame that has not been copied, pixel rows and pixel columns corresponding to the translation size are cropped to obtain the corresponding translation frame.
  • the predetermined translation direction Take the predetermined translation direction as an example to translate along the diagonal to the lower right corner for illustration.
  • the translation size is N
  • first copy the pixels in the left column of the target frame to the left N times, and the top row of pixels up N times, and copy a pixel in the upper left corner to the upper left corner N ⁇ N times to obtain the pixel row and pixel Blank positions between columns increase candidate translation frames of length N.
  • the manner of translation is not limited to the above examples, for example, 2 columns (2 rows) or more than 2 columns (2 rows) may also be copied at the same time.
  • Step S32' respectively determine the second energy map of the translation frame corresponding to each of the macroblock sizes.
  • the second energy map of the translation frame corresponding to the current macroblock size is determined according to each macroblock size.
  • the process of determining the second energy map is similar to the process of determining the first energy map above, and may include: segmenting the corresponding translation frame according to the size of each macroblock to obtain a plurality of corresponding second macroblocks.
  • AC energy is determined for each second macroblock.
  • a corresponding second energy map is determined through multiple second macroblocks corresponding to the same translation frame, and each pixel value in the second energy map is AC energy corresponding to the second macroblock.
  • the process of segmenting and shifting the frame according to the size of the macroblock, determining the AC energy of the second macroblock, and determining the second energy map according to the AC energy is similar to the process of determining the first energy map.
  • each second energy map since it is determined that the second macroblock size of each second energy map is different, multiple second energy maps corresponding to the target frame can be superimposed together to form another set of energy pyramids for characterizing the energy distribution of the corresponding target frame .
  • two different sets of energy pyramids corresponding to the target frame can be obtained.
  • Fig. 10 is a schematic diagram of a process of determining a second energy map provided by an embodiment of the present disclosure.
  • multiple translation frames 101 are obtained by first translating the target frame with different macroblock sizes, and then the translation frames 101 obtained after the corresponding macroblock size are segmented and translated, to determine a plurality of second macroblocks 102 .
  • the corresponding second energy map 103 is determined according to the plurality of second macroblocks 102 obtained after each translation frame 101 is divided.
  • FIG. 11 is a schematic diagram of a process of determining a translation frame provided by an embodiment of the present disclosure. Taking the translation size as 2 and the size of the target frame 110 as 4 ⁇ 4 as an example for illustration. As shown in FIG. 11 , in a possible implementation, when the preset translation direction of the target frame 110 is to translate along the diagonal to the lower right corner, first copy the first row at the top of the target frame 110 up twice and copy the first column on the left twice to the left, and copy the first pixel in the upper left corner four times at the unconnected positions of each row and column after copying to obtain a candidate translation frame 111 with a size of 6 ⁇ 6. In some embodiments, the right two columns and the bottom two rows of the candidate translation frame 111 are respectively cropped to obtain a 4 ⁇ 4 translation frame with the same size as the target frame 110 .
  • the embodiment of the present disclosure uses a low-complexity sliding window to translate the target frame, and according to the exchange energy of each part of the translated target frame, obtains an energy pyramid representing the energy of the target frame after translation of different sizes, thereby improving the robustness of energy estimation. , to reduce the block effect caused by the code rate mutation between macroblocks in the subsequent video encoding process.
  • Step S40' according to each of the first energy maps, determine the first energy map corresponding to the target frame.
  • the first energy graph is jointly determined according to each first energy graph and the second energy graph. That is to say, determine the second mean value of each first energy map and each second energy map at the same pixel position, and determine the first energy map according to the second mean value corresponding to each pixel position. After it is determined that the target frame corresponds to multiple first energy maps and multiple second energy maps, each first energy map and second energy map are superimposed and fused, and each first energy map and second energy map are calculated at the same pixel position The second mean value of the pixel values of is used as the pixel value of the first energy spectrum at the pixel position. That is to say, each of the first energy map and the second energy map is used as a plurality of channels, and the first energy map is obtained through channel average fusion.
  • Fig. 12 is a schematic diagram of determining a first energy spectrum provided by an embodiment of the present disclosure.
  • the embodiment of the present disclosure determines the first energy pyramid 120 composed of multiple first energy images corresponding to the target frame, and the multiple second energy images corresponding to the target frame.
  • the second energy pyramid 121 of the first energy pyramid 120 and the energy maps in the second energy pyramid 121 are respectively used as channels for channel averaging to obtain the first energy map 122 used to characterize the energy distribution of the target frame.
  • the target frame can be divided into different macroblock sizes to perform energy estimation, and a first energy map representing the energy distribution of the target frame can be determined through multiple first energy maps obtained after estimation.
  • energy estimation is performed by shifting and dividing the target frame by different macroblock sizes, and determining a second energy map representing the energy distribution of the target frame through multiple second energy maps obtained after estimation.
  • the first energy map determined based on multiple first energy maps and second energy maps can adapt to the multi-resolution video quantization process in subsequent video coding, and at the same time improve the robustness of energy estimation, reducing the number of problems in the subsequent video coding process
  • the block effect is caused by the sudden change of code rate between macroblocks.
  • Step S50' Determine the adaptive quantization parameter corresponding to the target frame according to the first energy map, and encode the target frame through the adaptive quantization parameter.
  • step S10 the process of determining the adaptive quantization parameter in this step is similar to step S10.
  • the embodiment of the present disclosure can respectively determine the AC energy of each part of the original image in the target frame based on multiple macroblock sizes, and the AC energy of each part after translation of different sizes, and based on the energy pyramid representing the AC energy after dividing the original image with different macroblock sizes Determining the energy spectrum with the energy pyramid that characterizes the translation of different macroblock sizes and exchanging energy after segmenting the original image, and generating the corresponding adaptive quantization parameters, so that the adaptive quantization parameters are compatible with the energy characteristics of the target frame after being segmented by various macroblock sizes and The energy characteristics of the target frame after translation and segmentation are correlated.
  • video encoding can be performed based on macroblocks of different sizes, and it is suitable for complex scenes with large resolution changes during video encoding. At the same time, it also enhances the robustness of the algorithm and reduces the block effect caused by the sudden change of the code rate between macroblocks in the video coding process.
  • multiple first energy maps corresponding to the target frame may be respectively determined according to multiple preset macroblock sizes.
  • the input target frame can be copied K shares, and the first energy map corresponding to each target frame can be determined according to at least two macroblock sizes, wherein one target frame can correspond to a group of first energy maps, and one group includes multiple A first energy map, where K is a positive integer.
  • the K target frames may be segmented based on the size of each macroblock to obtain K sets of first energy maps.
  • a first energy pyramid multi-scale energy pyramid
  • K first energy pyramids are obtained.
  • the target frame can also be shifted according to the size of each macroblock to obtain a plurality of corresponding shifted frames, and the second energy map of the shifted frame corresponding to each macroblock size is respectively determined.
  • the second energy maps corresponding to each macroblock size corresponding to the translation frame may be determined respectively, wherein one translation frame corresponds to a group of second energy maps, and one group includes multiple second energy maps.
  • the M translation frames may be segmented based on the size of each macroblock to obtain M groups of second energy maps.
  • a second energy pyramid multi-scale sliding window energy pyramid
  • M second energy pyramids can be obtained.
  • Fig. 13 is a schematic diagram of determining an energy pyramid provided by an embodiment of the present disclosure. As shown in Fig. 13 , taking 5 copies of the input target frame 1301 as an example, in the process of implementation, it can be determined according to the macroblock size 1302 The first energy map corresponding to each target frame. When each macroblock size 1302 is N/4, N/2, N, 2N, and 4N, the five target frames can be divided based on each macroblock size 1302 to obtain five groups of first energy maps.
  • the first energy pyramid 1303 can be formed according to the multiple first energy diagrams in each group, and then 5 first energy pyramids 1303 can be obtained, and the 5 first energy pyramids 1303 can respectively represent For: V[1], V[2], V[3], V[4], V[5].
  • the size of the macroblock can be scaled based on the preset scaling ratio to obtain translation Size, taking the reduction of the macroblock size according to the ratio of 1/2 as an example, the obtained translation size can be: N/8, N/4, N/2, N, 2N.
  • N/8, N/4, N/2, N, 2N column pixels and row pixels can be filled in the form of copies on the left edge and upper edge of the target frame, and the right edge of the target frame N/8, N/4, N/2, N, 2N columns and rows of pixels are cropped from the lower edge, and then five translation frames 1304 that have the same size as the original target frame 1301 but have been translated are obtained.
  • the sizes of the macroblocks are N/4, N/2, N, 2N and 4N
  • the five translation frames 1304 can be divided based on the sizes of the macroblocks respectively to obtain five sets of second energy maps.
  • a second energy pyramid 1305 can be formed according to multiple second energy maps in each group, and then 5 second energy pyramids 1305 can be obtained, and the 5 second energy pyramids 1305 can respectively represent These are: VS[1], VS[2], VS[3], VS[4], VS[5].
  • Fig. 14 is a schematic flow diagram of a video coding provided by an embodiment of the present disclosure. As shown in Fig. 14, after obtaining the first energy pyramid 141 and the second energy pyramid 142, the first energy pyramid 141 and the second energy pyramid 141 can be The energy maps in the pyramid 142 are respectively used as channels for channel averaging to obtain the first energy map 143 used to characterize the energy distribution of the target frame.
  • the first energy map 143 can be determined by means of average pooling
  • the second energy map can also be referred to as a macroblock-level energy map, after obtaining the second energy map 144, it is possible to determine the self corresponding to the target frame by means of histogram mapping.
  • Adaptive quantization parameters after the adaptive quantization parameters are obtained, the adaptive quantization parameters can be input through the adaptive quantization interface of the video encoder, and at the same time, the target frame is input into the video encoder, and the adaptive quantization parameters are passed through the video encoder Encode the target frame.
  • the embodiment of the present disclosure uses a low-complexity sliding window to translate the target frame, and according to the exchange energy of each part of the translated target frame, obtains an energy pyramid representing the energy of the target frame after translation of different sizes, thereby improving the robustness of energy estimation. , to reduce the block effect caused by the code rate mutation between macroblocks in the subsequent video encoding process.
  • the present disclosure also provides video processing devices, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any video processing method provided in the present disclosure, corresponding technical solutions and descriptions, and corresponding records in the method section .
  • Fig. 15 is a schematic diagram of a video processing device provided by an embodiment of the present disclosure. As shown in Figure 15, the device includes:
  • the target frame determination module 130 is configured to determine the target frame in the video to be processed
  • the first energy map determination module 131 is configured to respectively determine a plurality of first energy maps corresponding to the target frame according to at least two preset macroblock sizes, each of the first energy maps represents a macroblock size corresponding to AC energy of at least one first macroblock, wherein each of the first macroblocks is obtained by dividing the target frame according to the size of the macroblock;
  • the energy map determination module 132 is configured to determine a first energy map corresponding to the target frame according to each of the first energy maps, and the first energy map is used to characterize the energy distribution in the target frame;
  • the parameter determination module 133 is configured to determine an adaptive quantization parameter corresponding to the target frame according to the first energy map, and encode the target frame by using the adaptive quantization parameter.
  • the first energy map determination module includes: a sub-slicing module configured to respectively segment the target frame according to a plurality of preset macroblock sizes, to obtain The corresponding multiple first macroblocks; the energy calculation submodule is configured to determine the AC energy of each of the first macroblocks; the first energy map determination submodule is configured to divide the target frame according to the same macroblock size. A corresponding first energy map is determined for each first macroblock, and each pixel value in the first energy map is AC energy corresponding to the first macroblock.
  • the first calculation submodule includes: a first calculation unit configured to determine the corresponding AC energy according to the variance of all pixel values in the first macroblock and the number of pixels.
  • the energy spectrum determination module includes: a first mean value calculation submodule configured to determine the first mean value of each of the first energy patterns at the same pixel position; the first energy spectrum determination submodule , configured to determine the first energy spectrum according to the first mean value corresponding to each pixel position.
  • the device further includes: a second energy map determination module configured to determine at least two second energy maps corresponding to the target frame according to the sizes of the macroblocks, each of the The second energy map respectively represents the AC energy of at least one second macroblock corresponding to a macroblock size, wherein each second macroblock is obtained by shifting and segmenting the target frame corresponding to the macroblock size; the energy map The determination module includes: a second mean value calculation submodule configured to determine the second mean value of each of the first energy maps and each of the second energy maps at the same pixel position; the second energy map determination submodule is configured to determine according to each The second mean value corresponding to the pixel position determines the first energy spectrum.
  • the second energy map determination module includes: a translation submodule configured to perform translation processing on the target frame according to the size of each macroblock to obtain a plurality of corresponding translation frames;
  • the second energy map determination sub-module is configured to respectively determine the second energy map of the translation frame corresponding to each macroblock size.
  • the translation submodule includes: a first size determination unit configured to scale each macroblock size according to a predetermined scaling ratio to obtain a corresponding translation size; a translation unit configured to scale each macroblock size according to each The translation size performs translation processing on the target frames respectively to obtain a plurality of corresponding translation frames.
  • the translation unit includes: a copy subunit configured to copy two adjacent edges of the target frame, and respectively increase pixel rows and Pixel column, and copy the pixel at the intersection position of two adjacent edges to the blank area between the increased pixel row and pixel column to obtain the corresponding candidate translation frame; the cropping subunit is configured to be configured when the candidate translation frame is not The pixel rows and pixel columns corresponding to the translation size are cut off on both sides of the copy to obtain the corresponding translation frame.
  • the second energy map determination submodule includes: a segmentation unit configured to segment the corresponding translation frame according to the size of each macroblock to obtain a plurality of corresponding second macroblocks;
  • the energy calculation unit is configured to determine the AC energy of each second macroblock;
  • the energy map determination unit is configured to determine a corresponding second energy map through multiple second macroblocks corresponding to the same translation frame, and the second energy Each pixel value in the figure corresponds to the AC energy of the second macroblock.
  • the parameter determination module includes: a pooling submodule configured to determine a second energy spectrum corresponding to the first energy spectrum by means of average pooling; a parameter determination submodule configured to An adaptive quantization parameter corresponding to the target frame is determined according to the second energy graph, and the target frame is encoded by using the adaptive quantization parameter.
  • the pooling submodule includes: a second size determination unit configured to determine a target macroblock size; a pooling unit configured to use the target macroblock size as a window and a step size, performing average pooling on the first energy map to obtain a second energy map.
  • the parameter determining submodule includes: a mapping table determining unit configured to determine a histogram mapping table corresponding to the second energy spectrum; a mapping unit configured to Mapping the second energy spectrum to obtain an adaptive quantization parameter corresponding to the target frame; a data transmission unit configured to input the adaptive quantization parameter and the target frame into a video encoder, based on the corresponding adaptive quantization parameter Perform video encoding on the target frame.
  • the size of each macroblock is set according to a fixed ratio.
  • the target frame determination module includes: a target frame determination submodule configured to sequentially determine target frames in the video to be processed according to a time axis sequence.
  • the functions or included modules of the apparatus provided in the embodiments of the present disclosure may be configured to execute the methods described in the above method embodiments, and the implementation may refer to the descriptions of the above method embodiments.
  • Embodiments of the present disclosure also provide a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor.
  • Computer readable storage media may be volatile or nonvolatile computer readable storage media.
  • An embodiment of the present disclosure also proposes an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.
  • An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.
  • Electronic devices may be provided as terminals, servers, or other forms of devices.
  • Fig. 16 is a block diagram of an electronic device 1400 provided by an embodiment of the present disclosure.
  • the electronic device 1400 may be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, or a personal digital assistant.
  • electronic device 1400 may include one or more of the following components: processing component 1402, memory 1404, power supply component 1406, multimedia component 1408, audio component 1410, input/output (I/O) interface 1412, sensor component 1414 , and the communication component 1416.
  • the processing component 1402 generally controls the overall operations of the electronic device 1400, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.
  • the processing component 1402 may include one or more processors 1420 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 1402 may include one or more modules that facilitate interaction between processing component 1402 and other components. For example, processing component 1402 may include a multimedia module to facilitate interaction between multimedia component 1408 and processing component 1402 .
  • the memory 1404 is configured to store various types of data to support operations at the electronic device 1400 . Examples of such data include instructions for any application or method operating on the electronic device 1400, contact data, phonebook data, messages, pictures, videos, and the like.
  • the memory 1404 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM erasable Programmable Read Only Memory
  • PROM Programmable Read Only Memory
  • ROM Read Only Memory
  • Magnetic Memory Flash Memory
  • Magnetic or Optical Disk Magnetic Disk
  • the power supply component 1406 provides power to various components of the electronic device 1400 .
  • Power components 1406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 1400 .
  • the multimedia component 1408 includes a screen providing an output interface between the electronic device 1400 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user.
  • the touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action.
  • the multimedia component 1408 includes at least one of the following: a front camera; a rear camera. When the electronic device 1400 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.
  • the audio component 1410 is configured to output and/or input audio signals.
  • the audio component 1410 includes a microphone (MIC), which is configured to receive an external audio signal when the electronic device 1400 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. Received audio signals may be further stored in memory 1404 or sent via communication component 1416 .
  • the audio component 1410 also includes a speaker for outputting audio signals.
  • the I/O interface 1412 provides an interface between the processing component 1402 and a peripheral interface module, which may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.
  • Sensor assembly 1414 includes one or more sensors for providing various aspects of status assessment for electronic device 1400 .
  • the sensor component 1414 can detect the open/closed state of the electronic device 1400, the relative positioning of components, such as the display and the keypad of the electronic device 1400, the sensor component 1414 can also detect the electronic device 1400 or one of the electronic device 1400 Changes in position of components, presence or absence of user contact with electronic device 1400 , electronic device 1400 orientation or acceleration/deceleration and temperature changes in electronic device 1400 .
  • Sensor assembly 1414 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact.
  • the sensor assembly 1414 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications.
  • CMOS complementary metal-oxide-semiconductor
  • CCD charge-coupled device
  • the sensor component 1414 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.
  • the communication component 1416 is configured to facilitate wired or wireless communication between the electronic device 1400 and other devices.
  • the electronic device 1400 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof.
  • the communication component 1416 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication component 1416 also includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID Radio Frequency Identification
  • IrDA Infrared Data Association
  • UWB Ultra Wide Band
  • Bluetooth Bluetooth
  • electronic device 1400 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation for performing the methods described above.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable A programmable gate array
  • controller microcontroller, microprocessor or other electronic component implementation for performing the methods described above.
  • a non-volatile computer-readable storage medium such as a memory 1404 including computer program instructions, which can be executed by the processor 1420 of the electronic device 1400 to implement the above method.
  • Fig. 17 is a block diagram of an electronic device 1500 provided by an embodiment of the present disclosure.
  • the electronic device 1500 may be provided as a server.
  • electronic device 1500 includes processing component 1522 , which further includes one or more processors, and a memory resource represented by memory 1532 for storing instructions executable by processing component 1522 , such as application programs.
  • the application programs stored in memory 1532 may include one or more modules each corresponding to a set of instructions.
  • the processing component 1522 is configured to execute instructions to perform the above method.
  • Electronic device 1500 may also include a power supply component 1526 configured to perform power management of electronic device 1500, a wired or wireless network interface 1550 configured to connect electronic device 1500 to a network, and an input-output (I/O) interface 1558 .
  • the electronic device 1500 can operate based on the operating system stored in the memory 1532, such as the Microsoft server operating system (Windows ServerTM), the graphical user interface-based operating system (Mac OS XTM) introduced by Apple Inc., and the multi-user and multi-process computer operating system (UnixTM). ), a free and open source Unix-like operating system (LinuxTM), an open source Unix-like operating system (FreeBSDTM), or similar.
  • a non-volatile computer-readable storage medium such as a memory 1532 including computer program instructions, which can be executed by the processing component 1522 of the electronic device 1500 to implement the above-mentioned method.
  • the present disclosure can be a system, method and/or computer program product.
  • a computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present disclosure.
  • a computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device.
  • a computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disc read only memory
  • DVD digital versatile disc
  • memory stick floppy disk
  • mechanically encoded device such as a printer with instructions stored thereon
  • a hole card or a raised structure in a groove and any suitable combination of the above.
  • computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.
  • Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • Computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or Source or object code written in any combination, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages.
  • Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer such as use an Internet service provider to connect via the Internet).
  • LAN Local Area Network
  • WAN Wide Area Network
  • an electronic circuit such as a programmable logic circuit, field programmable gate array (FPGA), or programmable logic array (PLA) can be customized by utilizing state information of computer-readable program instructions, which can Various aspects of the present disclosure are implemented by executing computer readable program instructions.
  • These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.
  • each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the computer program product can be specifically realized by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) etc. wait.
  • a software development kit Software Development Kit, SDK

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本公开涉及一种视频处理方法及装置、电子设备、存储介质、计算机程序、计算机程序产品，所述方法在待处理视频中确定目标帧，并根据预设的至少两个宏块尺寸分别确定目标帧对应的多个第一能量图，其中，各第一能量图包括多个尺寸相同的第一宏块的交流能量，各第一宏块通过对应宏块尺寸切分目标帧得到。在一些实施例中，根据各第一能量图确定用于表征目标帧中的能量分布的第一能量图谱，根据第一能量图谱确定目标帧的自适应量化参数，通过所述自适应量化参数对目标帧进行编码。本公开实施例基于多个宏块尺寸分别确定目标帧各部分的交流能量，并根据不同宏块尺寸切分目标帧得到的各交流能量确定能量图谱，生成对应的自适应量化参数，能够适应较大分辨率变化的视频编码场景。

Description

视频处理方法及装置、电子设备、存储介质、计算机程序、计算机程序产品
相关申请的交叉引用
本公开基于申请号为202110961276.6、申请日为2021年08月20日、申请名称为“视频处理方法及装置、电子设备和存储介质”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。
技术领域
本公开涉及视频编码领域，尤其涉及一种视频处理方法及装置、电子设备、存储介质、计算机程序、计算机程序产品。
背景技术
在视频编码领域中，为了提升视频视觉质量，可以通过宏块级自适应量化方法为视频帧的平滑区域分配更多的码率。但上述方法通常会存在难以适应编码中的复杂场景的弊端。
发明内容
本公开提出了一种视频处理方法及装置、电子设备、存储介质、计算机程序、计算机程序产品，旨在适应较大分辨率变化的场景，以及减少视频编码过程产生块效应。
本公开提出了一种视频处理方法及装置、电子设备、存储介质、计算机程序、计算机程序产品，旨在适应较大分辨率变化的场景，以及减少视频编码过程产生块效应。
本公开实施例提供了一种视频处理方法，所述方法包括：
在待处理视频中确定目标帧；
根据预设的至少两个宏块尺寸分别确定所述目标帧对应的多个第一能量图，各所述第一能量图分别表征一个宏块尺寸对应的至少一个第一宏块的交流能量，其中，各所述第一宏块通过对应宏块尺寸切分所述目标帧得到；
根据各所述第一能量图确定所述目标帧对应的第一能量图谱，所述第一能量图谱用于表征所述目标帧中的能量分布；
根据所述第一能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码。
本公开实施例提供了一种视频处理装置，所述装置包括：
目标帧确定模块，配置为在待处理视频中确定目标帧；
第一能量图确定模块，配置为根据预设的至少两个宏块尺寸分别确定所述目标帧对应的多个第一能量图，各所述第一能量图分别表征一个宏块尺寸对应的至少一个第一宏块的交流能量，其中， 各所述第一宏块通过对应宏块尺寸切分所述目标帧得到；
能量图谱确定模块，配置为根据各所述第一能量图确定所述目标帧对应的第一能量图谱，所述第一能量图谱用于表征所述目标帧中的能量分布；
参数确定模块，配置为根据所述第一能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码。
本公开实施例提供了一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为调用所述存储器存储的指令，以执行上述方法。
本公开实施例提供了一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。
本公开实施例提供一种计算机程序，所述计算机程序包括计算机可读代码，在所述计算机可读代码被计算机读取并执行的情况下，实现本公开任一实施例中的方法的部分或全部步骤。
本公开实施例提供一种计算机程序产品，所述计算机程序产品包括存储了计算机程序的非瞬时性计算机可读存储介质，所述计算机程序被计算机读取并执行时，实现本公开任一实施例中的方法的部分或全部步骤。
本公开实施例通过不同宏块尺寸切分目标帧，分别计算目标帧被各宏块尺寸分割后对应的交流能量，基于包含各对应的交流能量的各第一能量图确定第一能量图谱，并根据所述第一能量图谱确定目标帧对应的自适应量化参数，使得自适应量化参数与通过多种宏块尺寸分割后目标帧的能量特征相关联，在自适应量化参数应用于目标帧的编码时，能够基于不同尺寸的宏块进行视频编码，适应视频编码时存在较大分辨率变化的复杂场景。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。根据下面参考附图对示例性实施例的详细说明，本公开的其它特征及方面将变得清楚。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1为本公开实施例提供的一种视频处理方法的流程图；
图2为本公开实施例提供的一种确定第一能量图过程的流程图；
图3为本公开实施例提供的一种确定第一能量图过程的示意图；
图4为本公开实施例提供的一种确定交流能量的示意图；
图5为本公开实施例提供的一种确定第一能量图谱的示意图；
图6为本公开实施例提供的一种确定自适应量化参数过程的示意图；
图7为本公开实施例提供的一种数据传输过程的示意图；
图8为本公开实施例提供的一种视频处理方法的流程图；
图9为本公开实施例提供的一种确定第二能量图过程的流程图；
图10为本公开实施例提供的一种确定第二能量图过程的示意图；
图11为本公开实施例提供的一种确定平移帧过程的示意图；
图12为本公开实施例提供的一种确定第一能量图谱的示意图；
图13为本公开实施例提供的一种确定能量金字塔的示意图；
图14为本公开实施例提供的一种视频编码的流程示意图；
图15为本公开实施例提供的一种视频处理装置的示意图；
图16为本公开实施例提供的一种电子设备的框图；
图17为本公开实施例提供的一种电子设备的框图。
具体实施方式
以下将参考附图详细说明本公开的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
另外，为了更好地说明本公开，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本公开同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本公开的主旨。
图1为本公开实施例提供的一种视频处理方法的流程图。该视频处理方法可以由终端设备或其它处理设备执行，其中，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。在一些可能的实现方式中，该视频处理方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
在一个示例性的应用场景中，可以通过对预先确定的待处理视频和预设的多个宏块尺寸，执行本公开实施例的视频处理方法，确定其中的每一帧对应的自适应量化参数，根据待处理视频中各帧和对应的自适应量化参数进行视频编码。在一些实施例中，本公开实施例中的视频处理方法和视频编码可以通过同一设备完成，或先由终端设备或其他设备执行视频处理方法后，传输至视频编码器进行视频编码。
如图1所示，在一个示例性的实施例中，本公开实施例的视频处理方法包括以下步骤：
步骤S10、在待处理视频中确定目标帧。
在一种可能的实现方式中，本公开实施例可以通过对待处理视频中每一帧分别进行处理的方式执行视频处理方法。也就是说，可以将待处理视频中各帧分别作为目标帧进行图像处理，以确定对应的自适应量化参数。在完成当前目标帧的图像处理后，重新在待处理视频中确定未处理的帧作为新的目标帧，直到完成待处理视频中全部帧的图像处理，进而完成待处理视频的视频处理过程。在一些实施例中，为了提高视频处理效率，目标帧的处理顺序可以基于时间轴顺序依次确定。
例如，当待处理视频中包括T1-T10帧时，根据时间轴顺序依次确定T1、T2、T3、T4、T5、T6、T7、T8、T9和T10为目标帧，以确定各目标帧对应的自适应量化参数。在一些实施例中，在确定当前目标帧对应的自适应量化参数后，将当前目标帧作为输入帧输入视频编码器，将对应的自适应量化参数输入视频编码器的自适应量化接口进行视频编码。同时，重新确定时间轴顺序上的下一帧作为目标帧。
步骤S20、根据预设的至少两个宏块尺寸分别确定所述目标帧对应的多个第一能量图。
在一种可能的实现方式中，确定多个预设的宏块尺寸，再根据各宏块尺寸分别确定目标帧对应的多个第一能量图。各第一能量图包括多个相同尺寸的第一宏块的交流能量，其中，各第一宏块通过对应宏块尺寸切分所述目标帧得到。也就是说，各第一能量图用于表征通过对应宏块尺寸划分目标帧后得到多个第一宏块的交流能量。各第一宏块对应的交流能量用于表征该第一宏块内的图像纹理复杂度。对于图像平坦的第一宏块，计算得到的交流能量较小，而对于图像纹理复杂的第一宏块，计算得到的交流能量较大。
在一些实施例中，各宏块尺寸可以按固定比例设定，即各宏块尺寸大小为预设的等比数列。在一些实施例中，各宏块尺寸的确定过程为先设定一个目标尺寸，再以固定比例分别缩小和放大目标尺寸多次，得到对应的多个缩小尺寸和放大尺寸，将目标尺寸、缩小尺寸和放大尺寸均作为预设的宏块尺寸，并将该宏块尺寸应用于本公开实施例的视频处理方法。例如，当预先设定的目标尺寸为N(N为正数)，固定比例为2时，可以通过分别缩小和放大目标尺寸两次得到对应的两个缩小尺寸N/2和N/4，以及对应的两个放大尺寸2N和4N，确定五个宏块尺寸依次为N/4、N/2、N、2N和4N。
图2为本公开实施例提供的一种确定第一能量图过程的流程图。如图2所示，在一种可能的实现方式中，对于任一宏块尺寸，根据预设的多个宏块尺寸分别确定目标帧对应多个第一能量图的过程包括以下步骤：
步骤S21、根据所述任一宏块尺寸切分所述目标帧，得到对应的多个第一宏块。
在一种可能的实现方式中，根据多个按照固定比例设定的宏块尺寸分别切分目标帧，得到各宏块尺寸对应的多个第一宏块。在一些实施例中，目标帧的格式可以为RGB格式或YUV格式，在此不做限定。在一些实施例中，当目标帧格式为YUV格式时，可以为444类型、422类型、420类型或411类型。目标帧切分的过程可以根据格式确定。例如，当目标帧格式为422类型的YUV时，按照2:1:1的比例分别获取Y、U和V三个通道的多个像素值，以确定对应的第一宏块。
其中，按照每一种宏块尺寸切分所述目标帧时，可以得到对应于该宏块尺寸的多个第一宏块。以目标帧的尺寸为32×32，各宏块尺寸分别为2×2、4×4、8×8、16×16和32×32为例进行说明。根据各宏块尺寸分别切分目标帧，得到256个尺寸为2×2的第一宏块、64个尺寸为4×4的第一宏块、16个尺寸为8×8的第一宏块、4个尺寸为16×16的第一宏块以及1个尺寸为32×32的第一宏块。
图3为本公开实施例提供的一种确定第一能量图过程的示意图。如图3所示，在确定目标帧30后，根据预设的多个以2为固定比例的宏块尺寸N，2N，…，kN分别切分目标帧30，得到多个不同尺寸的第一宏块31，以通过相同尺寸的多个第一宏块确定各宏块尺寸对应的第一能量图32。
步骤S22、确定各所述第一宏块的交流能量。
在一种可能的实现方式中，分别计算各第一宏块的交流能量，各第一宏块中包括相同尺寸和不同尺寸的第一宏块。也就是说，可确定根据各宏块尺寸切分后得到的全部第一宏块的交流能量。该交流能量用于表征对应第一宏块内部各像素的能量特征。
在一些实施例中，本公开实施例可以根据第一宏块中全部像素值的方差和像素数量确定对应的交流能量。在一些实施例中，先计算第一宏块中全部像素值的方差，再用该方差除以第一宏块中包括的像素数量，得到对应的交流能量。
图4为本公开实施例提供的一种确定交流能量的示意图。如图4所示，对于划分后得到的第一宏块40，计算其中各像素值Y1-Y8、U1-U4以及V1-V4的方差，再除以其中包括的像素总数16得到对应的交流能量E1。在一些实施例中，在计算得到第一宏块40的交流能量E1后，可以 将该交流能量E1作为像素值填入该第一宏块40的各像素位置。
本领域技术人员应理解，交流能量的计算方式不限于以上示例，只要能够表征第一宏块内部各像素的能量特征即可。
步骤S23、根据通过同一宏块尺寸分割目标帧得到的多个第一宏块确定对应的第一能量图。
在一个可能的实现方式中，同一宏块尺寸分割目标帧得到的多个第一宏块尺寸相同，即根据多个相同尺寸的第一宏块以及各第一宏块对应的交流能量确定第一能量图。其中，第一能量图中各像素位置的像素值为对应第一宏块的交流能量。也就是说，分别确定与各宏块尺寸对应的第一能量图，各第一能量图的尺寸与目标帧尺寸相同。对于相同尺寸的多个第一宏块，在对应第一能量图中的对应像素位置填入各第一宏块的交流能量，对应的像素位置即为各第一宏块中像素被切分之前在目标帧中的位置。
在一些实施例中，确定第一能量图的方式可以为先将各第一宏块对应的交流能量作为像素值填入第一宏块中各像素位置，再将相同尺寸的多个第一宏块中各像素值填入对应第一能量图中的对应像素位置，或将填入交流能量作为像素值的各第一宏块按照切分前的位置拼接，得到第一能量图。
在一些实施例中，由于确定各第一能量图的第一宏块尺寸不同，目标帧对应的多个第一能量图叠加在一起，能够形成一组用于表征对应目标帧能量分布的能量金字塔。
步骤S30、根据各所述第一能量图确定所述目标帧对应的第一能量图谱。
在一个可能的实现方式中，第一能量图谱用于表征目标帧中的能量分布。确定目标帧对应第一能量图谱的过程可以包括：确定各第一能量图在同一像素位置的第一均值，根据各像素位置对应的第一均值确定第一能量图谱。当确定目标帧对应多个第一能量图后，对各第一能量图进行叠加融合，计算各第一能量图在同一像素位置的像素值的第一均值，作为第一能量图谱在该像素位置的像素值。也就是说，将各第一能量图分别作为多个通道，通过通道平均的融合方式得到第一能量图谱。
以目标帧对应两个第一能量图，第一能量图1用于表征四个尺寸为2×2的第一宏块的交流能量，第一能量图2用于表征16个尺寸为1×1的第一宏块的交流能量为例进行说明。例如，在第一能量图1中各像素位置对应的像素值分别为：
Figure PCTCN2022078398-appb-000001
其中，E1、E2、E3、E4分别表征四个第一宏块的交流能量。
同时，第一能量图2中各像素位置对应的像素值分别为：
Figure PCTCN2022078398-appb-000002
其中，各像素值分别表征16个第一宏块的交流能量的情况下。将第一能量图1和第二能量图2作为两个通道进行通道平均，即计算其中同一像素位置的像素值均值，得到第一能量图谱。该第一能量图谱中各像素值分别为：
Figure PCTCN2022078398-appb-000003
图5为本公开实施例提供的一种确定第一能量图谱的示意图。如图5所示，在一种可能的实现方式中，本公开实施例通过确定目标帧对应的多个第一能量图组成的第一能量金字塔50，对该第一能量金字塔50进行通道平均即可得到用于表征目标帧能量分布的第一能量图谱51。
本公开实施例能够通过不同的宏块尺寸分别划分目标帧进行能量估算，并通过估算后得到的多个第一能量图确定表征目标帧能量分布的第一能量图谱，在后续的视频编码中可以适应多分辨率的视频量化过程。
步骤S40、根据所述第一能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码。
在一种可能的实现方式中，根据表征目标帧分布的第一能量图谱能够确定目标帧对应的自适应量化参数，用于在视频编码过程中对该目标帧进行自适应量化，提高视频编码的效果。在一些实施例中，确定目标帧对应的自适应量化参数的过程可以包括：通过平均池化的方式确定第一能量图谱对应的第二能量图谱。根据第二能量图谱确定目标帧对应的自适应量化参数，通过所述自适应量化参数对目标帧进行编码。
在一些实施例中，对第一能量图谱进行平均池化的目的在于缩小第一能量图谱的尺寸，得到分辨率较小的第二能量图谱，以作为候选的自适应量化参数。在一种可能的实现方式中，平均池化的过程包括确定目标宏块尺寸，将目标宏块尺寸作为窗口和步长，对第一能量图谱进行平均池化得到第二能量图谱。其中，目标宏块尺寸可以根据预设的多个宏块尺寸确定，即目标宏块尺寸可以为预设多个宏块尺寸中的一个，或者可以根据预设到的多个宏块尺寸中的一个进行缩放处理得到。也就是说，第二能量图谱为第一能量图谱缩放一个目标宏块尺寸的图谱，该平均池化过程在保留第一能量图谱中各能量特征的同时缩小了分辨率，提高了后续视频编码过程的效率。
在一些实施例中，为提高视频编码过程的效率，目标宏块尺寸可以为各宏块尺寸中的中位数值。例如，当各宏块尺寸分别为N/4、N/2、N、2N和4N时，确定所述目标宏块尺寸为N。并在确定目标宏块尺寸为N后，以尺寸N×N为窗口、N为步长对第一能量图谱进行平均池化，得到第二能量图谱。其中，第二能量图谱的分辨率与N的乘积为第一能量图谱的分辨率，即第二能量图谱为一个分辨率较小的用于体现目标帧能量特征的图像。
在一种可能的实现方式中，在得到第二能量图谱后，通过直方图映射的方式确定目标帧对应的自适应量化参数。在一些实施例中，确定第二能量图谱对应的直方图映射表。根据直方图映射表映射第二能量图谱，得到目标帧对应的自适应量化参数。将自适应量化参数和目标帧输入视频编码器，以基于对应的自适应量化参数对目标帧进行视频编码。在一些实施例中，该映射过程可以为初始化一个与第二能量图谱尺寸相同的空白图像，对于第二能量图谱中的各像素值，在直方图映射表中确定对应的数值，并将各数值存入该空白图像上与对应像素值位置相同的位置，得到对应的自适应量化参数。或者，确定第二能量图谱中的各像素值在直方图映射表中对应的数值，根据各数值替换第二能量图谱中对应的像素值，得到自适应量化参数。
图6为本公开实施例提供的一种确定自适应量化参数过程的示意图。如图6所示，在视频编码的应用场景下，本公开实施例在确定第一能量图谱60后，通过平均池化的方式得到目标帧对应的第二能量图谱61。在一些实施例中，通过对第二能量图谱61进行直方图映射的方式得到自 适应量化参数62。其中，直方图映射的过程包括对第二能量图谱61进行直方图统计得到对应的直方图映射表，再通过直方图映射表映射第二能量图谱61的方式得到自适应量化参数62。
图7为本公开实施例提供的一种数据传输过程的示意图。如图7所示，在得到用于表征目标帧能量特征的第一能量图谱后，将该目标帧70作为视频编码器的输入帧输入视频编码器72。同时，还将基于第一能量图谱确定的自适应量化参数71作为用于对目标帧70进行视频编码的参数输入视频编码器72的自适应量化接口。
在视频编码场景中，本公开实施例可以基于目标帧的能量分布特征确定对应的自适应量化参数，以进行自适应量化调整，提高视频编码过程的效率。
本公开实施例可以通过不同宏块尺寸切分目标帧，分别计算目标帧被各宏块尺寸分割后对应的交流能量，基于包含各对应的交流能量的各第一能量图确定第一能量图谱，并根据所述第一能量图谱确定目标帧对应的自适应量化参数，使得自适应量化参数与通过多种宏块尺寸分割后目标帧的能量特征相关联，在自适应量化参数应用于目标帧的编码时，能够基于不同尺寸的宏块进行视频编码，适应视频编码时存在较大分辨率变化的复杂场景。
图8为本公开实施例提供的一种视频处理方法的流程图，用于表征本公开实施例另一种视频处理方法的实现过程。如图8所示，在一个示例性的实施例中，本公开实施例的视频处理方法包括以下步骤：
步骤S10’、在待处理视频中确定目标帧。
在一种可能的实现方式中，该步骤确定目标帧的过程与步骤S10类似。
步骤S20’、根据预设的多个宏块尺寸分别确定目标帧对应的多个第一能量图。
在一种可能的实现方式中，该步骤确定第一能量图的过程与步骤S20类似。
步骤S30’、根据各所述宏块尺寸分别确定所述目标帧对应的多个第二能量图。
在一种可能的实现方式中，在确定目标帧对应的多个第一能量图的同时，还根据各宏块尺寸分别确定对应的多个第二能量图，确定第一能量图和第二能量图的多个宏块尺寸相同。在一些实施例中，第二能量图包括多个相同尺寸的第二宏块的交流能量，其中，第二宏块通过对应宏块尺寸平移、切分所述目标帧得到。
图9为本公开实施例提供的一种确定第二能量图过程的流程图。如图9所示，本公开实施例确定目标帧对应的多个第二能量图的过程可以包括以下步骤：
步骤S31’、根据各所述宏块尺寸分别对所述目标帧进行平移处理，得到对应的多个平移帧。
在一些实施例中，确定目标帧对应的多个第二能量图的过程可以包括：根据各宏块尺寸分别对目标帧进行平移处理，得到对应的多个平移帧。分别确定各宏块尺寸对应平移帧的第二能量图。也就是说，先分别根据各宏块尺寸对目标帧进行平移处理，再根据各宏块尺寸确定根据当前宏块尺寸平移过的目标帧对应的第二能量图。例如，当分别通过宏块尺寸1、宏块尺寸2和宏块尺寸3平移目标帧得到平移帧1、平移帧2和平移帧3时，根据宏块尺寸1确定平移帧1对应的第二能量图，根据宏块尺寸2确定平移帧2对应的第二能量图，根据宏块尺寸3确定平移帧3对应的第二能量图。
在一种可能的实现方式中，确定各宏块尺寸对应平移帧的过程包括：根据预定的缩放比例缩放各宏块尺寸，得到对应的平移尺寸。根据各平移尺寸分别对目标帧进行平移处理，得到对应的多个平移帧。也就是说，以预定的缩放比例分别对各宏块尺寸均进行缩放，再根据缩放后的平移尺寸平移目标帧得到平移帧，即通过将各目标帧平移对应的平移尺寸得到对应的平移帧。例如，当各宏块尺寸分别为N/4、N/2、N、2N和4N，预设的缩放比例为1/2时，确定平移尺寸分别为N/8、N/4、N/2、N和2N，以向预设的平移方向将目标帧分别平移N/8、N/4、N/2、N和2N个 长度。
在一些实施例中，通过各平移尺寸分别平移目标帧的平移方向相同。该平移方向可以为任意对角线方向，例如沿对角线向左上角平移、右下角平移、左下角平移或右上角平移。
在一种可能的实现方式中，根据对应平移尺寸平移目标帧的方式包括：通过拷贝目标帧两个相邻边缘，在两个相邻边缘分别增加对应平移尺寸的像素行和像素列，并将两个相邻边缘相交位置的像素拷贝至增加的像素行和像素列之间的空白区域，得到对应的候选平移帧。在候选平移帧未被拷贝的两侧，裁剪掉对应平移尺寸的像素行和像素列，得到对应的平移帧。
以预定的平移方向为沿对角线向右下角平移为例进行说明。当平移尺寸为N时，先将目标帧左侧一列像素向左复制N次，以及顶部一行像素向上复制N次，并将左上角的一个像素向左上角复制N×N次得到像素行和像素列之间的空白位置增加长度N的候选平移帧。再将候选平移帧的底部裁剪掉N行像素，以及右侧裁剪掉N列像素得到平移帧。通过以上处理，相当于将目标帧像右下角平移N行、N列，裁减掉溢出的行和列，并在左上角进行了填充。
在一些实施例中，平移的方式不限于上述示例，例如，也可以同时复制2列(2行)，或2列(2行)以上。
步骤S32’、分别确定各所述宏块尺寸对应平移帧的第二能量图。
在一种可能的实现方式中，根据各宏块尺寸确定与当前宏块尺寸对应的平移帧的第二能量图。该确定第二能量图的过程与上文确定第一能量图的过程类似，可以包括：根据各宏块尺寸切分对应的平移帧，得到对应的多个第二宏块。确定各第二宏块的交流能量。通过同一平移帧对应的多个第二宏块确定对应的第二能量图，第二能量图中各像素值为对应第二宏块的交流能量。
在一些实施例中，根据宏块尺寸切分平移帧、确定第二宏块的交流能量、以及根据交流能量确定第二能量图的过程与第一能量图的确定过程类似。
在一些实施例中，由于确定各第二能量图的第二宏块尺寸不同，目标帧对应的多个第二能量图叠加在一起能够形成另一组用于表征对应目标帧能量分布的能量金字塔。由此，可以获得目标帧对应的两组不同的能量金字塔。
图10为本公开实施例提供的一种确定第二能量图过程的示意图。如图10所示，本公开实施例在确定目标帧100后，先通过不同的宏块尺寸平移目标帧得到多个平移帧101，再通过对应宏块尺寸切分平移后得到的平移帧101，以确定多个第二宏块102。在一些实施例中，再根据各平移帧101被分割后得到的多个第二宏块102确定对应的第二能量图103。
图11为本公开实施例提供的一种确定平移帧过程的示意图。以平移尺寸为2，目标帧110的尺寸为4×4为例进行说明。如图11所示，在一种可能的实现方式中，当目标帧110预设的平移方向为沿对角线向右下角平移时，先将目标帧110中的顶部第一行向上拷贝2次和左侧第一列向左拷贝2次，并在拷贝后各行和各列未连接位置拷贝左上角的第一个像素4次，得到尺寸为6×6的候选平移帧111。在一些实施例中，将候选平移帧111右侧两列和底部两行分别裁剪掉，得到尺寸与目标帧110相同的4×4的平移帧。
本公开实施例通过低复杂度的滑动窗口平移目标帧，并根据平移后目标帧各部分的交流能量，得到表征目标帧平移不同尺寸后能量情况的能量金字塔，从而能够提高能量估算的鲁棒性，减少在后续视频编码过程中因宏块间码率突变产生块效应。
步骤S40’、根据各所述第一能量图确定所述目标帧对应的第一能量图谱。
在一种可能的实现方式中，第一能量图谱根据各第一能量图和第二能量图共同确定。也就是说，确定各第一能量图和各第二能量图在同一像素位置的第二均值，根据各像素位置对应的第二均值确定第一能量图谱。当确定目标帧对应多个第一能量图和多个第二能量图后，对各第一能量 图和第二能量图进行叠加融合，计算各第一能量图和第二能量图在同一像素位置的像素值的第二均值，作为第一能量图谱在该像素位置的像素值。也就是说，将各第一能量图和第二能量图分别作为多个通道，通过通道平均的融合方式得到第一能量图谱。
图12为本公开实施例提供的一种确定第一能量图谱的示意图。如图12所示，在一种可能的实现方式中，本公开实施例确定目标帧对应的多个第一能量图组成的第一能量金字塔120，以及目标帧对应的多个第二能量图组成的第二能量金字塔121，将该第一能量金字塔120和第二能量金字塔121中各能量图分别作为通道进行通道平均，得到用于表征目标帧能量分布的第一能量图谱122。
本公开实施例能够通过不同的宏块尺寸分别划分目标帧进行能量估算，并通过估算后得到的多个第一能量图确定表征目标帧的能量分布的第一能量图谱。同时通过不同宏块尺寸平移并划分目标帧进行能量估算，并通过估算后得到的多个第二能量图确定表征目标帧能量分布的第二能量图。基于多个第一能量图和第二能量图确定的第一能量图谱在后续的视频编码中可以适应多分辨率的视频量化过程，同时提高了能量估算的鲁棒性，减少在后续视频编码过程中因宏块间码率突变产生块效应。
步骤S50’、根据所述第一能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码。
在一种可能的实现方式中，该步骤确定自适应量化参数的过程与步骤S10类似。
本公开实施例可以基于多个宏块尺寸分别确定目标帧中原始图像各部分的交流能量，以及平移不同尺寸后各部分的交流能量，基于表征不同宏块尺寸分割原始图像后交流能量的能量金字塔和表征不同宏块尺寸平移、分割原始图像后交流能量的能量金字塔确定能量图谱，生成对应的自适应量化参数，使得自适应量化参数与通过多种宏块尺寸分割后的目标帧的能量特征以及平移分割后的目标帧的能量特征相关联，在自适应量化参数应用于目标帧的编码时，能够基于不同尺寸的宏块进行视频编码，适应视频编码时存在较大分辨率变化的复杂场景。同时，还增强了算法的鲁棒性，减少在视频编码过程中因宏块间码率突变产生块效应的问题。
在实现的过程中，可以根据预设的多个宏块尺寸分别确定目标帧对应的多个第一能量图。如，可以将输入的目标帧拷贝K份，并根据至少两个宏块尺寸分别确定各个目标帧对应的第一能量图，其中，一个目标帧可以对应一组第一能量图，一组包括多个第一能量图，其中，K为正整数。如，可以基于各个宏块尺寸分别对该K个目标帧进行切分，得到K组第一能量图。在得到K组第一能量图之后，可以根据各组中的多个第一能量图组成第一能量金字塔(多尺度能量金字塔)，进而得到K个第一能量金字塔。
同时，还可以根据各宏块尺寸分别对目标帧进行平移处理，得到对应的多个平移帧，分别确定各宏块尺寸对应平移帧的第二能量图。如，可以将输入的目标帧拷贝M份，基于预设的缩放比例对宏块尺寸进行缩放，得到平移尺寸，并向预设的平移方向将各个目标帧按照该平移尺寸平移，得到M个平移帧，其中，M为正整数，且M可以与K相同，也可以不同，可以根据需要设置。在得到平移帧之后，可以分别确定各个宏块尺寸对应平移帧的第二能量图，其中，一个平移帧对应一组第二能量图，一组包括多个第二能量图。如，可以基于各个宏块尺寸分别对该M个平移帧进行切分，得到M组第二能量图。在得到M组第二能量图之后，可以根据各组中的多个第二能量图组成第二能量金字塔(多尺度滑窗能量金字塔)，进而得到M个第二能量金字塔。
图13为本公开实施例提供的一种确定能量金字塔的示意图，如图13所示，以将输入的目标帧1301拷贝5份为例，在实现的过程中，可以根据宏块尺寸1302分别确定各个目标帧对应的第 一能量图。当各宏块尺寸1302分别为N/4、N/2、N、2N和4N时，可以分别基于各个宏块尺寸1302分别对该5个目标帧进行切分，得到5组第一能量图。在得到5组第一能量图之后，可以根据各组中的多个第一能量图组成第一能量金字塔1303，进而得到5个第一能量金字塔1303，该5个第一能量金字塔1303可分别表示为：V[1]、V[2]、V[3]、V[4]、V[5]。
同时，还可以将目标帧1301拷贝5份，当各宏块尺寸分别为N/4、N/2、N、2N和4N时，可以基于预设的缩放比例对宏块尺寸进行缩放，得到平移尺寸，以按照1/2的比例对宏块尺寸进行缩小为例，得到的平移尺寸可以为：N/8、N/4、N/2、N、2N。在实现的过程中，可以在目标帧的左边缘与上边缘分别以拷贝的形式填充N/8、N/4、N/2、N、2N列像素和行像素，并在目标帧的右边缘与下边缘裁剪N/8、N/4、N/2、N、2N列像素和行像素，进而得到5份与原目标帧1301的尺寸相同但平移过的平移帧1304。当各宏块尺寸分别为N/4、N/2、N、2N和4N时，可以分别基于各个宏块尺寸分别对该5个平移帧1304进行切分，得到5组第二能量图。在得到5组第二能量图之后，可以根据各组中的多个第二能量图组成第二能量金字塔1305，进而得到5个第二能量金字塔1305，该5个第二能量金字塔1305可分别表示为：VS[1]、VS[2]、VS[3]、VS[4]、VS[5]。
图14为本公开实施例提供的一种视频编码的流程示意图，如图14所示，在得到第一能量金字塔141和第二能量金字塔142之后，可以将该第一能量金字塔141和第二能量金字塔142中各能量图分别作为通道进行通道平均，得到用于表征目标帧能量分布的第一能量图谱143，在得到第一能量图谱143之后，可以通过平均池化的方式确定第一能量图谱143对应的第二能量图谱144，在一些实施例中，第二能量图谱也可称为宏块级能量图谱，在得到第二能量图谱144之后，可以通过直方图映射的方式确定目标帧对应的自适应量化参数，在得到自适应量化参数之后，可以通过视频编码器的自适应量化接口输入该自适应量化参数，同时将目标帧输入视频编码器，并通过该视频编码器基于该自适应量化参数对该目标帧进行编码。
本公开实施例通过低复杂度的滑动窗口平移目标帧，并根据平移后目标帧各部分的交流能量，得到表征目标帧平移不同尺寸后能量情况的能量金字塔，从而能够提高能量估算的鲁棒性，减少在后续视频编码过程中因宏块间码率突变产生块效应。可以理解，本公开提及的上述各个方法实施例，在不违背原理逻辑的情况下，均可以彼此相互结合形成结合后的实施例，限于篇幅。本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的执行顺序应当以其功能和可能的内在逻辑确定。
此外，本公开还提供了视频处理装置、电子设备、计算机可读存储介质、程序，上述均可用来实现本公开提供的任一种视频处理方法，相应技术方案和描述和参见方法部分的相应记载。
图15为本公开实施例提供的一种视频处理装置的示意图。如图15所示，所述装置包括：
目标帧确定模块130，配置为在待处理视频中确定目标帧；
第一能量图确定模块131，配置为根据预设的至少两个宏块尺寸分别确定所述目标帧对应的多个第一能量图，各所述第一能量图分别表征一个宏块尺寸对应的至少一个第一宏块的交流能量，其中，各所述第一宏块通过对应宏块尺寸切分所述目标帧得到；
能量图谱确定模块132，配置为根据各所述第一能量图确定所述目标帧对应的第一能量图谱，所述第一能量图谱用于表征所述目标帧中的能量分布；
参数确定模块133，配置为根据所述第一能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码。
在一种可能的实现方式中，对于任一宏块尺寸，所述第一能量图确定模块包括：切分子模块，配置为根据预设的多个宏块尺寸分别切分所述目标帧，得到对应的多个第一宏块；能量计算子模 块，配置为确定各所述第一宏块的交流能量；第一能量图确定子模块，配置为根据通过同一宏块尺寸分割目标帧得到的多个第一宏块确定对应的第一能量图，所述第一能量图中各像素值为对应第一宏块的交流能量。
在一种可能的实现方式中，所述第一计算子模块包括：第一计算单元，配置为根据所述第一宏块中全部像素值的方差和像素数量确定对应的交流能量。
在一种可能的实现方式中，所述能量图谱确定模块包括：第一均值计算子模块，配置为确定各所述第一能量图在同一像素位置的第一均值；第一能量图谱确定子模块，配置为根据各像素位置对应的第一均值确定第一能量图谱。
在一种可能的实现方式中，所述装置还包括：第二能量图确定模块，配置为根据各所述宏块尺寸分别确定所述目标帧对应的至少两个第二能量图，各所述第二能量图分别表征一个宏块尺寸对应的至少一个第二宏块的交流能量，其中，各所述第二宏块通过对应宏块尺寸平移、切分所述目标帧得到；所述能量图谱确定模块包括：第二均值计算子模块，配置为确定各所述第一能量图和各所述第二能量图在同一像素位置的第二均值；第二能量图谱确定子模块，配置为根据各像素位置对应的第二均值确定第一能量图谱。
在一种可能的实现方式中，所述第二能量图确定模块包括：平移子模块，配置为根据各所述宏块尺寸分别对所述目标帧进行平移处理，得到对应的多个平移帧；第二能量图确定子模块，配置为分别确定各所述宏块尺寸对应平移帧的第二能量图。
在一种可能的实现方式中，所述平移子模块包括：第一尺寸确定单元，配置为根据预定的缩放比例缩放各所述宏块尺寸，得到对应的平移尺寸；平移单元，配置为根据各所述平移尺寸分别对所述目标帧进行平移处理，得到对应的多个平移帧。
在一种可能的实现方式中，所述平移单元包括：拷贝子单元，配置为通过拷贝所述目标帧两个相邻边缘，在所述两个相邻边缘分别增加对应平移尺寸的像素行和像素列，并将两个相邻边缘相交位置的像素拷贝至增加的像素行和像素列之间的空白区域，得到对应的候选平移帧；裁剪子单元，配置为在所述候选平移帧未被拷贝的两侧裁剪掉对应平移尺寸的像素行和像素列，得到对应的平移帧。
在一种可能的实现方式中，所述第二能量图确定子模块包括：切分单元，配置为根据各所述宏块尺寸切分对应的平移帧，得到对应的多个第二宏块；能量计算单元，配置为确定各所述第二宏块的交流能量；能量图确定单元，配置为通过同一平移帧对应的多个第二宏块确定对应的第二能量图，所述第二能量图中各像素值为对应第二宏块的交流能量。
在一种可能的实现方式中，所述参数确定模块包括：池化子模块，配置为通过平均池化的方式确定所述第一能量图谱对应的第二能量图谱；参数确定子模块，配置为根据所述第二能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码。
在一种可能的实现方式中，所述池化子模块包括：第二尺寸确定单元，配置为确定目标宏块尺寸；池化单元，配置为将所述目标宏块尺寸作为窗口和步长，对所述第一能量图谱进行平均池化得到第二能量图谱。
在一种可能的实现方式中，所述参数确定子模块包括：映射表确定单元，配置为确定所述第二能量图谱对应的直方图映射表；映射单元，配置为根据所述直方图映射表映射所述第二能量图谱，得到所述目标帧对应的自适应量化参数；数据传输单元，配置为将所述自适应量化参数和所述目标帧输入视频编码器，基于对应的自适应量化参数对所述目标帧进行视频编码。
在一种可能的实现方式中，各所述宏块尺寸按照固定比例设定。
在一种可能的实现方式中，所述目标帧确定模块包括：目标帧确定子模块，配置为按照时间 轴顺序依次在所述待处理视频中确定目标帧。
在一些实施例中，本公开实施例提供的装置具有的功能或包含的模块可以配置为执行上文方法实施例描述的方法，其实现可以参照上文方法实施例的描述。
本公开实施例还提出一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。计算机可读存储介质可以是易失性或非易失性计算机可读存储介质。
本公开实施例还提出一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为调用所述存储器存储的指令，以执行上述方法。
本公开实施例还提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器执行上述方法。
电子设备可以被提供为终端、服务器或其它形态的设备。
图16为本公开实施例提供的一种电子设备1400的框图。例如，电子设备1400可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等终端。
参照图16，电子设备1400可以包括以下一个或多个组件：处理组件1402，存储器1404，电源组件1406，多媒体组件1408，音频组件1410，输入/输出(I/O)的接口1412，传感器组件1414，以及通信组件1416。
处理组件1402通常控制电子设备1400的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件1402可以包括一个或多个处理器1420来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件1402可以包括一个或多个模块，便于处理组件1402和其他组件之间的交互。例如，处理组件1402可以包括多媒体模块，以方便多媒体组件1408和处理组件1402之间的交互。
存储器1404被配置为存储各种类型的数据以支持在电子设备1400的操作。这些数据的示例包括用于在电子设备1400上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器1404可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件1406为电子设备1400的各种组件提供电力。电源组件1406可以包括电源管理***，一个或多个电源，及其他与为电子设备1400生成、管理和分配电力相关联的组件。
多媒体组件1408包括在所述电子设备1400和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件1408包括以下至少之一：前置摄像头；后置摄像头。当电子设备1400处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜***或具有焦距和光学变焦能力。
音频组件1410被配置为输出和/或输入音频信号。例如，音频组件1410包括一个麦克风(MIC)，当电子设备1400处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被 配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器1404或经由通信组件1416发送。在一些实施例中，音频组件1410还包括一个扬声器，用于输出音频信号。
I/O接口1412为处理组件1402和***接口模块之间提供接口，上述***接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件1414包括一个或多个传感器，用于为电子设备1400提供各个方面的状态评估。例如，传感器组件1414可以检测到电子设备1400的打开/关闭状态，组件的相对定位，例如所述组件为电子设备1400的显示器和小键盘，传感器组件1414还可以检测电子设备1400或电子设备1400一个组件的位置改变，用户与电子设备1400接触的存在或不存在，电子设备1400方位或加速/减速和电子设备1400的温度变化。传感器组件1414可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件1414还可以包括光传感器，如互补金属氧化物半导体(CMOS)或电荷耦合装置(CCD)图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件1414还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件1416被配置为便于电子设备1400和其他设备之间有线或无线方式的通信。电子设备1400可以接入基于通信标准的无线网络，如无线网络(WiFi)，第二代移动通信技术(2G)或第三代移动通信技术(3G)，或它们的组合。在一个示例性实施例中，通信组件1416经由广播信道接收来自外部广播管理***的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件1416还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在示例性实施例中，电子设备1400可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。
在示例性实施例中，还提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器1404，上述计算机程序指令可由电子设备1400的处理器1420执行以完成上述方法。
图17为本公开实施例提供的一种电子设备1500的框图。例如，电子设备1500可以被提供为一服务器。参照图17，电子设备1500包括处理组件1522，其进一步包括一个或多个处理器，以及由存储器1532所代表的存储器资源，用于存储可由处理组件1522的执行的指令，例如应用程序。存储器1532中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件1522被配置为执行指令，以执行上述方法。
电子设备1500还可以包括一个电源组件1526被配置为执行电子设备1500的电源管理，一个有线或无线网络接口1550被配置为将电子设备1500连接到网络，和一个输入输出(I/O)接口1558。电子设备1500可以操作基于存储在存储器1532的操作***，例如微软服务器操作***(Windows ServerTM)，苹果公司推出的基于图形用户界面操作***(Mac OS XTM)，多用户多进程的计算机操作***(UnixTM)，自由和开放原代码的类Unix操作***(LinuxTM)，开放原代码的类Unix操作***(FreeBSDTM)或类似。
在示例性实施例中，还提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器1532，上述计算机程序指令可由电子设备1500的处理组件1522执行以完成上述方法。
本公开可以是***、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是(但不限于)电存储设备、磁存储设备、光存储设备、电磁存储设备、 半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(Local Area Network，LAN)或广域网(Wide Area Network，WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这里参照根据本公开实施例的方法、装置(***)和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实施例的***、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序 执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的***来实现，或者可以用专用硬件与计算机指令的组合来实现。
该计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (32)

  1. 一种视频处理方法，所述方法包括：
    在待处理视频中确定目标帧；
    根据预设的至少两个宏块尺寸分别确定所述目标帧对应的多个第一能量图，各所述第一能量图分别表征一个宏块尺寸对应的至少一个第一宏块的交流能量，其中，各所述第一宏块通过对应宏块尺寸切分所述目标帧得到；
    根据各所述第一能量图确定所述目标帧对应的第一能量图谱，所述第一能量图谱用于表征所述目标帧中的能量分布；
    根据所述第一能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码。
  2. 根据权利要求1所述的方法，其中，对于任一宏块尺寸，所述根据预设的至少两个宏块尺寸分别确定所述目标帧对应的多个第一能量图，包括：
    根据所述任一宏块尺寸切分所述目标帧，得到对应的多个第一宏块；
    确定各所述第一宏块的交流能量；
    根据通过同一宏块尺寸分割目标帧得到的多个第一宏块确定对应的第一能量图，所述第一能量图中各像素值为对应第一宏块的交流能量。
  3. 根据权利要求2所述的方法，其中，所述确定各所述第一宏块的交流能量，包括：
    根据所述第一宏块中全部像素值的方差和像素数量确定对应的交流能量。
  4. 根据权利要求1-3中任意一项所述的方法，其中，所述根据各所述第一能量图确定所述目标帧对应的第一能量图谱，包括：
    确定各所述第一能量图在同一像素位置的第一均值；
    根据各像素位置对应的第一均值确定第一能量图谱。
  5. 根据权利要求1-3中任意一项所述的方法，其中，所述方法还包括：
    根据各所述宏块尺寸分别确定所述目标帧对应的至少两个第二能量图，各所述第二能量图分别表征一个宏块尺寸对应的至少一个第二宏块的交流能量，其中，各所述第二宏块通过对应宏块尺寸平移、切分所述目标帧得到；
    所述根据各所述第一能量图确定所述目标帧对应的第一能量图谱，包括：
    确定各所述第一能量图和各所述第二能量图在同一像素位置的第二均值；
    根据各像素位置对应的第二均值确定第一能量图谱。
  6. 根据权利要求5所述的方法，其中，所述根据各所述宏块尺寸分别确定所述目标帧对应的多个第二能量图，包括：
    根据各所述宏块尺寸分别对所述目标帧进行平移处理，得到对应的多个平移帧；
    分别确定各所述宏块尺寸对应平移帧的第二能量图。
  7. 根据权利要求6所述的方法，其中，所述根据各所述宏块尺寸分别对所述目标帧进行平移处理，得到对应的多个平移帧，包括：
    根据预定的缩放比例缩放各所述宏块尺寸，得到对应的平移尺寸；
    根据各所述平移尺寸分别对所述目标帧进行平移处理，得到对应的多个平移帧。
  8. 根据权利要求7所述的方法，其中，对于任一平移尺寸，所述根据各所述平移尺寸分别对所述目标帧进行平移处理，得到对应的多个平移帧，包括：
    通过拷贝所述目标帧两个相邻边缘，在所述两个相邻边缘分别增加对应平移尺寸的像素行和 像素列，并将两个相邻边缘相交位置的像素拷贝至增加的像素行和像素列之间的空白区域，得到对应的候选平移帧；
    在所述候选平移帧未被拷贝的两侧，裁剪掉对应平移尺寸的像素行和像素列，得到对应的平移帧。
  9. 根据权利要求6-8中任意一项所述的方法，其中，所述分别确定各所述宏块尺寸对应平移帧的第二能量图，包括：
    根据各所述宏块尺寸切分对应的平移帧，得到对应的多个第二宏块；
    确定各所述第二宏块的交流能量；
    通过同一平移帧对应的多个第二宏块确定对应的第二能量图，所述第二能量图中各像素值为对应第二宏块的交流能量。
  10. 根据权利要求1-9中任意一项所述的方法，其中，所述根据所述第一能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码包括：
    通过平均池化的方式确定所述第一能量图谱对应的第二能量图谱；
    根据所述第二能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码。
  11. 根据权利要求10所述的方法，其中，所述通过平均池化的方式确定所述第一能量图谱对应的第二能量图谱，包括：
    确定目标宏块尺寸；
    将所述目标宏块尺寸作为窗口和步长，对所述第一能量图谱进行平均池化得到第二能量图谱。
  12. 根据权利要求10或11所述的方法，其中，所述根据所述第二能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码，包括：
    确定所述第二能量图谱对应的直方图映射表；
    根据所述直方图映射表映射所述第二能量图谱，得到所述目标帧对应的自适应量化参数；
    将所述自适应量化参数和所述目标帧输入视频编码器，基于对应的自适应量化参数对所述目标帧进行视频编码。
  13. 根据权利要求1-12中任意一项所述的方法，其中，各所述宏块尺寸按照固定比例设定。
  14. 根据权利要求1-13中任意一项所述的方法，其中，所述在待处理视频中确定目标帧，包括：
    按照时间轴顺序依次在所述待处理视频中确定目标帧。
  15. 一种视频处理装置，所述装置包括：
    目标帧确定模块，配置为在待处理视频中确定目标帧；
    第一能量图确定模块，配置为根据预设的至少两个宏块尺寸分别确定所述目标帧对应的多个第一能量图，各所述第一能量图分别表征一个宏块尺寸对应的至少一个第一宏块的交流能量，其中，各所述第一宏块通过对应宏块尺寸切分所述目标帧得到；
    能量图谱确定模块，配置为根据各所述第一能量图确定所述目标帧对应的第一能量图谱，所述第一能量图谱用于表征所述目标帧中的能量分布；
    参数确定模块，配置为根据所述第一能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码。
  16. 根据权利要求15所述的装置，其中，对于任一宏块尺寸，所述第一能量图确定模块，包括：
    切分子模块，配置为根据预设的多个宏块尺寸分别切分所述目标帧，得到对应的多个第一宏块；
    能量计算子模块，配置为确定各所述第一宏块的交流能量；
    第一能量图确定子模块，配置为根据通过同一宏块尺寸分割目标帧得到的多个第一宏块确定对应的第一能量图，所述第一能量图中各像素值为对应第一宏块的交流能量。
  17. 根据权利要求16所述的装置，其中，所述第一计算子模块包括：
    第一计算单元，配置为根据所述第一宏块中全部像素值的方差和像素数量确定对应的交流能量。
  18. 根据权利要求15-17中任意一项所述的装置，其中，所述能量图谱确定模块包括：
    第一均值计算子模块，配置为确定各所述第一能量图在同一像素位置的第一均值；
    第一能量图谱确定子模块，配置为根据各像素位置对应的第一均值确定第一能量图谱。
  19. 根据权利要求15-17中任意一项所述的装置，其中，所述装置还包括：
    第二能量图确定模块，配置为根据各所述宏块尺寸分别确定所述目标帧对应的至少两个第二能量图，各所述第二能量图分别表征一个宏块尺寸对应的至少一个第二宏块的交流能量，其中，各所述第二宏块通过对应宏块尺寸平移、切分所述目标帧得到；
    所述能量图谱确定模块包括：
    第二均值计算子模块，配置为确定各所述第一能量图和各所述第二能量图在同一像素位置的第二均值；
    第二能量图谱确定子模块，配置为根据各像素位置对应的第二均值确定第一能量图谱。
  20. 根据权利要求19所述的装置，其中，所述第二能量图确定模块包括：
    平移子模块，配置为根据各所述宏块尺寸分别对所述目标帧进行平移处理，得到对应的多个平移帧；
    第二能量图确定子模块，配置为分别确定各所述宏块尺寸对应平移帧的第二能量图。
  21. 根据权利要求20所述的装置，其中，所述平移子模块包括：
    第一尺寸确定单元，配置为根据预定的缩放比例缩放各所述宏块尺寸，得到对应的平移尺寸；
    平移单元，配置为根据各所述平移尺寸分别对所述目标帧进行平移处理，得到对应的多个平移帧。
  22. 根据权利要求21所述的装置，其中，所述平移单元包括：
    拷贝子单元，配置为通过拷贝所述目标帧两个相邻边缘，在所述两个相邻边缘分别增加对应平移尺寸的像素行和像素列，并将两个相邻边缘相交位置的像素拷贝至增加的像素行和像素列之间的空白区域，得到对应的候选平移帧；
    裁剪子单元，配置为在所述候选平移帧未被拷贝的两侧裁剪掉对应平移尺寸的像素行和像素列，得到对应的平移帧。
  23. 根据权利要求20-22中任意一项所述的装置，其中，所述第二能量图确定子模块包括：
    切分单元，配置为根据各所述宏块尺寸切分对应的平移帧，得到对应的多个第二宏块；
    能量计算单元，配置为确定各所述第二宏块的交流能量；
    能量图确定单元，配置为通过同一平移帧对应的多个第二宏块确定对应的第二能量图，所述第二能量图中各像素值为对应第二宏块的交流能量。
  24. 根据权利要求15-23中任意一项所述的装置，其中，所述参数确定模块包括：
    池化子模块，配置为通过平均池化的方式确定所述第一能量图谱对应的第二能量图谱；
    参数确定子模块，配置为根据所述第二能量图谱确定所述目标帧对应的自适应量化参数，通过所述自适应量化参数对所述目标帧进行编码。
  25. 根据权利要求24所述的装置，其中，所述池化子模块包括：
    第二尺寸确定单元，配置为确定目标宏块尺寸；
    池化单元，配置为将所述目标宏块尺寸作为窗口和步长，对所述第一能量图谱进行平均池化得到第二能量图谱。
  26. 根据权利要求24或25所述的装置，其中，所述参数确定子模块包括：
    映射表确定单元，配置为确定所述第二能量图谱对应的直方图映射表；
    映射单元，配置为根据所述直方图映射表映射所述第二能量图谱，得到所述目标帧对应的自适应量化参数；
    数据传输单元，用于将所述自适应量化参数和所述目标帧输入视频编码器，基于对应的自适应量化参数对所述目标帧进行视频编码。
  27. 根据权利要求15-26中任意一项所述的装置，其中，各所述宏块尺寸按照固定比例设定。
  28. 根据权利要求15-27中任意一项所述的装置，其中，所述目标帧确定模块包括：
    目标帧确定子模块，配置为按照时间轴顺序依次在所述待处理视频中确定目标帧。
  29. 一种电子设备，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为调用所述存储器存储的指令，以执行权利要求1至14中任意一项所述的方法。
  30. 一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现权利要求1至14中任意一项所述的方法。
  31. 一种计算机程序，包括计算机可读代码，在计算机可读代码在设备上运行的情况下，设备中的处理器执行用于实现权利要求1至14中任一所述的方法。
  32. 一种计算机程序产品，配置为存储计算机可读指令，所述计算机可读指令被执行时使得计算机执行权利要求1至14中任一所述的方法。
PCT/CN2022/078398 2021-08-20 2022-02-28 视频处理方法及装置、电子设备、存储介质、计算机程序、计算机程序产品 WO2023019910A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/444,824 US20240195968A1 (en) 2021-08-20 2024-02-19 Method for video processing, electronic device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110961276.6A CN113612999B (zh) 2021-08-20 2021-08-20 视频处理方法及装置、电子设备和存储介质
CN202110961276.6 2021-08-20

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/444,824 Continuation US20240195968A1 (en) 2021-08-20 2024-02-19 Method for video processing, electronic device, and storage medium

Publications (1)

Publication Number Publication Date
WO2023019910A1 true WO2023019910A1 (zh) 2023-02-23

Family

ID=78309036

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/078398 WO2023019910A1 (zh) 2021-08-20 2022-02-28 视频处理方法及装置、电子设备、存储介质、计算机程序、计算机程序产品

Country Status (3)

Country Link
US (1) US20240195968A1 (zh)
CN (1) CN113612999B (zh)
WO (1) WO2023019910A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113612999B (zh) * 2021-08-20 2024-03-22 北京市商汤科技开发有限公司 视频处理方法及装置、电子设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109076212A (zh) * 2017-12-27 2018-12-21 深圳市大疆创新科技有限公司 码率控制的方法与编码装置
CN111866504A (zh) * 2020-07-17 2020-10-30 Oppo广东移动通信有限公司 一种编码方法、编码器及计算机可读存储介质
US20210006802A1 (en) * 2018-04-04 2021-01-07 SZ DJI Technology Co., Ltd. Encoding method and apparatus, image processing system, and computer-readable storage medium
CN113612999A (zh) * 2021-08-20 2021-11-05 北京市商汤科技开发有限公司 视频处理方法及装置、电子设备和存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8064517B1 (en) * 2007-09-07 2011-11-22 Zenverge, Inc. Perceptually adaptive quantization parameter selection
US20160088298A1 (en) * 2014-09-22 2016-03-24 Ximin Zhang Video coding rate control including target bitrate and quality control
US9628803B2 (en) * 2014-11-25 2017-04-18 Blackberry Limited Perceptual image and video coding
US10924741B2 (en) * 2019-04-15 2021-02-16 Novatek Microelectronics Corp. Method of determining quantization parameters
CN110365983B (zh) * 2019-09-02 2019-12-13 珠海亿智电子科技有限公司 一种基于人眼视觉***的宏块级码率控制方法及装置
CN112073723B (zh) * 2020-11-16 2021-02-02 北京世纪好未来教育科技有限公司 视频信息处理方法、装置、电子设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109076212A (zh) * 2017-12-27 2018-12-21 深圳市大疆创新科技有限公司 码率控制的方法与编码装置
US20210006802A1 (en) * 2018-04-04 2021-01-07 SZ DJI Technology Co., Ltd. Encoding method and apparatus, image processing system, and computer-readable storage medium
CN111866504A (zh) * 2020-07-17 2020-10-30 Oppo广东移动通信有限公司 一种编码方法、编码器及计算机可读存储介质
CN113612999A (zh) * 2021-08-20 2021-11-05 北京市商汤科技开发有限公司 视频处理方法及装置、电子设备和存储介质

Also Published As

Publication number Publication date
CN113612999B (zh) 2024-03-22
US20240195968A1 (en) 2024-06-13
CN113612999A (zh) 2021-11-05

Similar Documents

Publication Publication Date Title
US12014275B2 (en) Method for text recognition, electronic device and storage medium
KR102654563B1 (ko) 비디오 인코딩 및 디코딩을 위한 적응형 전달 함수
US20210097715A1 (en) Image generation method and device, electronic device and storage medium
US11470294B2 (en) Method, device, and storage medium for converting image from raw format to RGB format
WO2018120238A1 (zh) 用于处理文档的设备、方法和图形用户界面
WO2020007241A1 (zh) 图像处理方法和装置、电子设备以及计算机可读存储介质
KR101620933B1 (ko) 제스쳐 인식 메커니즘을 제공하는 방법 및 장치
US20210279892A1 (en) Image processing method and device, and network training method and device
TW202032425A (zh) 圖像處理方法及裝置、電子設備和儲存介質
US20240195968A1 (en) Method for video processing, electronic device, and storage medium
CN112991381B (zh) 图像处理方法及装置、电子设备和存储介质
WO2023160617A9 (zh) 视频插帧处理方法、视频插帧处理装置和可读存储介质
US8995784B2 (en) Structure descriptors for image processing
CN112990197A (zh) 车牌识别方法及装置、电子设备和存储介质
CN110874809A (zh) 图像处理方法及装置、电子设备和存储介质
WO2021057359A1 (zh) 图像处理方法、电子设备及可读存储介质
WO2023019870A1 (zh) 视频处理方法及装置、电子设备、存储介质、计算机程序、计算机程序产品
US9665925B2 (en) Method and terminal device for retargeting images
CN110858388B (zh) 一种增强视频画质的方法和装置
CN107730443B (zh) 图像处理方法、装置及用户设备
WO2022095318A1 (zh) 字符检测方法、装置、电子设备、存储介质及程序
CN113538310A (zh) 图像处理方法及装置、电子设备和存储介质
CN112837237A (zh) 视频修复方法及装置、电子设备和存储介质
CN109816620B (zh) 图像处理方法及装置、电子设备和存储介质
WO2023024437A1 (zh) 节点排布方式确定方法及装置、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22857231

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE