CN114972087A - Video processing method, device, equipment and computer storage medium - Google Patents
Video processing method, device, equipment and computer storage medium Download PDFInfo
- Publication number
- CN114972087A CN114972087A CN202210528392.3A CN202210528392A CN114972087A CN 114972087 A CN114972087 A CN 114972087A CN 202210528392 A CN202210528392 A CN 202210528392A CN 114972087 A CN114972087 A CN 114972087A
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- frames
- discontinuous
- video frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 22
- 230000002708 enhancing effect Effects 0.000 claims abstract description 15
- 238000012545 processing Methods 0.000 claims description 70
- 238000000034 method Methods 0.000 claims description 41
- 238000004891 communication Methods 0.000 claims description 15
- 239000013589 supplement Substances 0.000 claims description 7
- 238000012216 screening Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 238000001914 filtration Methods 0.000 description 6
- 239000013598 vector Substances 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 239000003086 colorant Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 3
- 238000009499 grossing Methods 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000002146 bilateral effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/52—Scale-space analysis, e.g. wavelet analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention relates to the technical field of multimedia playing, and discloses a video processing method, which comprises the following steps: determining discontinuous regions within a continuous frame of video; the discontinuous region is a region with discontinuous image characteristics of adjacent frames; determining feature information of the continuous frames; generating a plurality of video frames according to the characteristic information; and enhancing the discontinuous area according to the video frame to obtain a processed video. Through the mode, the embodiment of the invention improves the watching experience of the video.
Description
Technical Field
The embodiment of the invention relates to the technical field of computer data processing, in particular to a video processing method, a video processing device, video processing equipment and a computer storage medium.
Background
Current video typically focuses on capturing and processing regions of medium luminosity.
The inventor of the present application finds, in the course of implementing the embodiments of the present invention: the existing video processing technology has the problems that the brightness changes severely, and the image texture details are easily covered by the brightness changes, thereby causing the visual discontinuity and influencing the watching experience of users.
Disclosure of Invention
In view of the foregoing problems, embodiments of the present invention provide a video processing method, apparatus, device and computer, so as to solve the problem in the prior art that the viewing experience of a video is not good.
According to an aspect of an embodiment of the present invention, there is provided a video processing method, including:
determining discontinuous regions within a continuous frame of video; the discontinuous region is a region with discontinuous image characteristics of adjacent frames;
determining feature information of the continuous frames;
generating a plurality of video frames according to the characteristic information;
and enhancing the discontinuous area according to the video frame to obtain a processed video.
In an alternative manner, the feature information includes frequency domain feature information; the video frame comprises an approximate video frame; the approximate video frame is a video frame in which the frequency domain characteristic information in the front and back directions of the continuous frames meets a preset characteristic value interval.
In an optional manner, the frequency domain feature information includes gray value gradient change information; the method further comprises the following steps:
screening at least two contrast reference frames in the front and back directions of the continuous frames according to the gray value gradient change information and the characteristic value interval to obtain frames to be approximated;
and carrying out approximation processing on the frame to be approximated to obtain the approximated video frame.
In an alternative approach, the video frames comprise overlay video frames; the superimposed video frame is a key frame containing texture detail information of the successive frames.
In an alternative manner, the feature information includes frequency domain feature information; the discontinuous region comprises a plurality of target region points; the method further comprises the following steps:
determining at least one key frame corresponding to the target area point from the adjacent area of the target area point according to the frequency domain characteristic information;
generating a transparent image corresponding to the target area point according to the frequency domain characteristic information;
and overlapping the transparent image to the key frame of the target area point to obtain the overlapped video frame.
In an optional manner, the frequency domain feature information includes gray value gradient change information; the method comprises the following steps:
dividing the continuous frames into highlight areas and low light areas according to the gray value gradient change information;
determining a detail component relationship between the highlight region and the highlight region;
and generating the transparent image according to the detail component relation and the endpoint output information of the highlight area and the low light area.
In an alternative mode, the discontinuous region includes a plurality of target region points; the characteristic information comprises frequency domain characteristic information; the video frames comprise approximate video frames and overlay video frames; the approximate video frame is a video frame of which the change of the frequency domain characteristic information in the front and back directions of the continuous frames meets a preset condition; the superimposed video frame is a key frame containing texture detail information of the continuous frames; the method further comprises the following steps:
determining an optimal frame from the approximate video frame and the overlay video frame;
and carrying out frame replacement or frame supplement processing on the target area point according to the optimal frame to obtain the processed video.
According to another aspect of the embodiments of the present invention, there is provided a video processing apparatus including:
a first determining module for determining discontinuous areas within successive frames of a video; the discontinuous region is a region with discontinuous image characteristics of adjacent frames;
a second determining module, configured to determine feature information of the consecutive frames;
the generating module is used for generating a plurality of video frames according to the characteristic information;
and the processing module is used for enhancing the discontinuous area according to the video frame to obtain a processed video.
According to another aspect of the embodiments of the present invention, there is provided a video processing apparatus including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is configured to store at least one executable instruction that causes the processor to perform the operations of any one of the embodiments of the video processing method.
According to a further aspect of embodiments of the present invention, there is provided a computer-readable storage medium having stored therein at least one executable instruction for causing a video processing apparatus to perform operations of any one of the video processing method embodiments.
The embodiment of the invention determines the discontinuous area in the continuous frame of the video; the discontinuous region is a region where the image features of adjacent frames are discontinuous, thereby locating a frame position where the viewing experience is not smooth. Then determining the characteristic information of the continuous frames; generating a plurality of video frames according to the characteristic information; and finally, enhancing the discontinuous area according to the video frame to obtain a processed video. Therefore, the method and the device are different from the problem that the video watching experience of the region with large luminosity change is not smooth due to the fact that only the central luminosity region is intensively enhanced in the prior art, and the embodiment of the invention can optimize the smoothness and the fluency of the video frame, ensure clear details and improve the watching experience of the video.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and in order that the technical solutions of the embodiments of the present invention can be clearly understood, the embodiments of the present invention can be implemented according to the content of the description, and the above and other objects, features, and advantages of the embodiments of the present invention can be more clearly understood, the detailed description of the present invention is provided below.
Drawings
The drawings are only for purposes of illustrating embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 is a schematic flow chart illustrating a video processing method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart illustrating a video processing method according to another embodiment of the present invention;
fig. 3 is a flow chart illustrating a video processing method according to another embodiment of the present invention;
fig. 4 is a schematic structural diagram of a video processing apparatus according to an embodiment of the present invention;
fig. 5 shows a schematic structural diagram of a video processing device provided by an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be embodied in various forms and should not be limited to the embodiments set forth herein.
Fig. 1 shows a flow diagram of a video processing method, as performed by a computer processing device, provided by an embodiment of the invention. The computer processing device may include a cell phone, a notebook computer, etc. As shown in fig. 1, the method comprises the steps of:
step 10: determining discontinuous regions within a continuous frame of video; the discontinuous areas are areas with discontinuous image characteristics of adjacent frames.
In one embodiment of the invention, the type of video may be movies, stop motion animations, and dynamic graphics, among others. Consecutive frames refer to a plurality of image frames in a video that are consecutive in time sequence.
The continuous frames may be collected according to a set length of a time line, for example, a 2-minute short video is read, continuous frames are selected from a video of a section to be processed, assuming that the length of the time line is set to be 4 seconds, and 120 continuous frames can be collected within the length of the time line of 4 seconds for a video set with a frame rate of 30 fps. The image features may be features such as luminosity and sharpness, and the discontinuity of the image features may cause discontinuity of the video viewing experience, such as flickering or temporal blurring and temporal blurring of a specific picture area.
For example, the determination of the discontinuous region may be a region with a drastic luminance change, which may obscure the texture details of the image due to the drastic luminance change, thereby causing a discontinuous experience such as blurring the picture.
Step 20: determining feature information of the successive frames.
In an embodiment of the present invention, it is considered that in the normal time domain and spatial domain states, the image/video frame hides much information, and in the frequency domain state, the high-frequency and low-frequency regions distributed in the video frame are more easily exposed, and the processing in analyzing the local details of the image is also facilitated.
Therefore, the characteristic information can be frequency domain characteristic information after frequency domain transformation is carried out on the continuous frames, so that image detail characteristics of the continuous frames are reflected better, corresponding enhancement processing is conveniently carried out according to the characteristic information subsequently, images and videos which are difficult to process, such as blurs, too high exposure, shadows and the like, are repaired, and visual experience is improved.
In yet another embodiment of the present invention, the frequency domain transform may be a fourier transform.
Fourier transformation from a space domain to a frequency domain is carried out on the continuous frame image to obtain two-dimensional coordinate information, and the transformation process is as follows:
performing a two-dimensional discrete fourier transform according to:
setting the image size of continuous frames to be M multiplied by N, namely setting the number of total pixel points to be M multiplied by N; u is a frequency domain variable in the M direction, and u is 0,1,2, …, M-1; v is a frequency domain variable in the N direction, v is 0,1,2, …, N-1; (x, y) is a space domain image variable; f (x, y) is the gray scale value of the (x, y) point.
The continuous frame images are discretized into two-dimensional signals through the formula, the two-dimensional signals are converted into gradient changes on two-dimensional coordinates through two-dimensional Fourier transform, the two-dimensional coordinates are decomposed into two independent gray value changes of one-dimensional coordinates, and the two gray value changes of the one-dimensional coordinates are superposed to obtain the gradient changes of the two-dimensional coordinates in a frequency domain state; after fourier transformation, the abscissa is the frequency of the separated sinusoidal signal, and the ordinate corresponds to the weighting density, i.e. amplitude value.
Step 30: and generating a plurality of video frames according to the characteristic information.
In an embodiment of the present invention, when processing the discontinuous area according to the feature information, on one hand, denoising processing may be performed on the discontinuous area to obtain a clearer image. On the other hand, the discontinuous areas can be subjected to detail supplement processing, so that more discontinuous features included in the image become continuous. When the detail supplement processing is performed, the video frame can be supplemented by an approximate video frame whose spectral feature changes little compared with the continuous frame.
Thus, in one embodiment of the invention, the video frame comprises an approximate video frame; the approximate video frame is a video frame in which the frequency domain characteristic information in the front and back directions of the continuous frames meets a preset characteristic value interval.
The preset characteristic value interval can be limited by a characteristic value upper limit and a characteristic value lower limit.
In one embodiment of the invention, the frequency domain feature information comprises gray value gradient change information; step 30 further comprises:
step 301: and screening at least two contrast reference frames in the front and back directions of the continuous frames according to the gray value gradient change information and the characteristic value interval to obtain a frame to be approximated.
In one embodiment of the present invention, an upper threshold and a lower threshold of the gray value gradient change information may be set, and a comparison reference frame of the gray value gradient change information between the upper threshold and the lower threshold is determined as a frame to be approximated.
Step 302: and carrying out approximation processing on the frame to be approximated to obtain the approximated video frame.
In one embodiment of the invention, the approximation process may include at least one of a similarity transformation, a change of RGB channel, and a smoothness process.
In an embodiment of the invention, each frame to be approximated is analyzed, the area and the position of a target image are determined, rectangular grids with a preset value and a specified size are segmented in the target image area, marking points are selected, the positions of the marking points in the target image area can be determined by a bilinear interpolation method, and each preset rectangular grid is deformed by using a similarity transformation matrix.
Wherein, the formula of the similarity transformation is as follows:
the similarity transformation matrix is:
wherein, (x ', y') is the coordinates of the original image points;
(x i ，y i ) Setting the coordinates of the mark points;
(x, y are set target image coordinates;
(w cx ，w cy ) Is the set target image center.
In one embodiment of the invention, the process of changing the RGB channels comprises at least the following:
in the RGB three channels, each channel (0-255) has 256 levels, the three channels can generate 16777216 colors, the contrast reference frame is subjected to approximate color matching processing, the target image area can be divided into rectangular grids with preset values and specified sizes, colors of the rectangular grid area are separated according to a set mode to form a color system combination with three channels separated, the colors of the color system combination are subjected to approximate color extraction to generate a plurality of approximate colors with three channels separated, effective approximate colors are combined and extracted, and a plurality of approximate color combinations can be obtained to optimize the approximate video frame.
In an embodiment of the present invention, the smoothing process for the frame to be approximated may adopt methods such as mean filtering, block filtering, gaussian filtering, median filtering, bilateral filtering, and 2D filtering.
Given that texture detail in discontinuous regions is masked by luminosity variations, noise, or other discontinuous image feature variations, resulting in visual blurring and front-to-back non-uniformity, it is desirable to supplement the texture detail of the images of successive frames.
Thus, in a further embodiment of the present invention, said video frames in step 30 comprise overlay video frames; the superimposed video frame is a key frame containing texture detail information of the successive frames.
In yet another embodiment of the present invention, considering that more image information is included in the key frame, it is necessary to capture the determined key frame first, generate a transparent image containing engraved texture details, and superimpose the transparent image on the key frame of the target region point, thereby reducing blurred regions and de-noising in consecutive frames.
Thus, in a further embodiment of the invention, the characteristic information comprises frequency domain characteristic information; the discontinuous region comprises a plurality of target region points; step 30 further comprises:
step 303: and determining at least one key frame corresponding to the target area point from the adjacent areas of the target area point according to the frequency domain characteristic information.
In an embodiment of the present invention, the determining process of the keyframe may perform gradient coding on the image of the neighboring area according to the gray value gradient change information to obtain a feature vector corresponding to the neighboring area; determining the keyframe from the neighboring region according to the feature vector. And selecting according to the gray value gradient variation and the gradient code.
Wherein the gradient coding is to take local square blocks (2) n ×2 n ) And/n is an integer of 2-10, including 2 and 10/, pixel points are arranged in a lifting sequence, the clockwise direction can be set as a specified direction, one-dimensional characteristic vectors are extracted from two-dimensional blocks, Z two-dimensional blocks correspond to Z one-dimensional characteristic vectors, and a proper key frame is determined by the Z characteristic vectors.
Step 304: and generating a transparent image corresponding to the target area point according to the frequency domain characteristic information.
In one embodiment of the invention, the frequency domain feature information comprises gray value gradient change information; step 304 further comprises:
step 3041: and dividing the continuous frames into a highlight area and a low light area according to the gray value gradient change information.
In an embodiment of the present invention, the continuous frame image is divided into a high frequency region and a low frequency region according to the gray value gradient variation and a preset variation threshold.
Step 3042: and determining the detail component relation between the highlight area and the highlight area.
In one embodiment of the invention, the detail component relationship is used to characterize a linear relationship between the frequency domain characteristics of the highlight and highlight regions. The detail component relationship includes at least one of a horizontal detail component relationship, a vertical detail component relationship, and a diagonal detail component relationship.
Step 3043: and generating the transparent image according to the detail component relation and the endpoint output information of the highlight area and the low light area.
In one embodiment of the invention, the linear relation between the highlight region and the highlight region in the distribution region and the output value at the end point of the highlight region are processed, the output value at the end point of the highlight region is output, and the transparent image is obtained through compiling.
Therefore, the transparent image is a high-frequency image obtained according to the linear relation and the high and low light areas, and details of the original key frame are described. And performing parameter compensation on the figure details between the highlight area and the low highlight area through the transparent image, so as to reflect the visual detail effect in the real environment.
Step 305: and overlapping the transparent image to the key frame of the target area point to obtain the overlapped video frame.
In one embodiment of the invention, the transparent image is superposed on the key frame, so that the details of the key frame image are supplemented, and the effect of enhancing the visual experience is achieved.
Step 40: and enhancing the discontinuous area according to the video frame to obtain a processed video.
In one embodiment of the invention, the enhancement processing may be either a complementary frame processing or a replacement frame processing. Considering that the approximate video frame can make the image clearer and less noisy, and the superimposed video frame can reflect more texture details of the image, the video frame with the best effect after processing can be selected from the approximate video frame and the superimposed video frame. Wherein the processed video effect can be determined according to the feature continuity between frames of the video.
Thus, in one embodiment of the invention, the discontinuity area comprises a plurality of target area points; the characteristic information comprises frequency domain characteristic information; the video frames comprise approximate video frames and overlay video frames; the approximate video frame is a video frame of which the change of the frequency domain characteristic information in the front and back directions of the continuous frames meets a preset condition; the superimposed video frame is a key frame containing texture detail information of the successive frames.
In order to further improve the smoothness of the processed video, the approximate video frame and the superimposed video frame can be generated simultaneously, and the video frame with the best effect (namely the best frame) after the enhancement processing is performed is further improved.
In one embodiment of the present invention, when the approximate video frame and the overlay video frame exist at the same time, the video frame with the best effect (i.e. the best frame) after the enhancement processing can be enhanced, and the effect of the video enhancement processing can be further improved.
Specifically, the approximate video frame, the overlay video frame, and the original video frame may be combined to obtain a corresponding processed video. And then screening the target frame from the approximate video frame and the superposed video frame according to the smoothness degree of the processed video.
The smoothness degree can be the smoothness degree of the image features of the processed video, and is used for reflecting whether the video watching experience is smooth and continuous.
In an embodiment of the present invention, the determination of the smoothing degree may also be performed by performing visual comparison on the combined video, and the determination is performed according to the motion vector information of the forward and backward reference fields.
Step 402: and carrying out frame replacement or frame supplement processing on the target area point according to the optimal frame to obtain the processed video.
In one embodiment of the invention, the optimal frame is added to the target area point for frame complementing, or the video frame of the target area point is replaced, and after processing, the processed video is marked by the processing area
Generating a sequence with a specified length through a distributed sorting algorithm to mark and record a processing area; the video frame processed by replacing the frame or supplementing the frame can be called by reading the mark record and is mapped and displayed to the user space, so that the video watching experience of the user is improved.
In yet another embodiment of the present invention, the process of video processing may refer to FIG. 2.
As shown in fig. 2, a first video set to be processed is read, continuous frames with a preset time line length are selected from the first video set, and an area with discontinuous image features is determined as a discontinuous area in the continuous frames.
And then, carrying out Fourier transform from a space domain to a frequency domain on the continuous frames to obtain the correlation characteristics of the continuous frames, then generating a plurality of video frames according to the correlation characteristics, and selecting the optimal frame to load on the corresponding region point of the discontinuous region.
Referring to fig. 3, the process of generating video frames and selecting the best frame may refer to fig. 3, and as shown in fig. 3, the gray value gradient change amount after fourier transform of consecutive frames is calculated and gradient encoding is performed. And then, at least two comparison reference frames in the front and back directions of corresponding region points in the discontinuous region are taken to generate an approximate video frame. And meanwhile, taking a transparent image generated by processing the corresponding region point under gradient coding, and overlapping the transparent image to the key frame of the corresponding region point to obtain an overlapped video frame.
And respectively combining the approximate video frame and the superposed video frame to the original continuous frames, performing smoothness evaluation, and determining the optimal frame from the approximate video frame and the superposed video frame according to smoothness.
And finally, loading the optimal frame to the corresponding region point of the discontinuous region, and carrying out frame replacement or frame supplement processing to obtain the continuous frame which completes continuous processing.
As shown in fig. 2, the continuous frames after the serialization processing are subjected to inverse fourier transform from the frequency domain to the spatial domain, and a second video set is obtained and output to the client for playing.
The video processing method provided by the embodiment of the invention determines the discontinuous area in the continuous frame of the video; the discontinuous region is a region where the image features of adjacent frames are discontinuous, thereby locating a frame position where the viewing experience is not smooth. Then determining the characteristic information of the continuous frames; generating a plurality of video frames according to the characteristic information; and finally, enhancing the discontinuous area according to the video frame to obtain a processed video. Therefore, the video processing method provided by the embodiment of the invention is different from the problem that the video watching experience of the region with large luminosity change is not smooth due to the fact that only the central luminosity region is intensively enhanced in the prior art, and the video processing method can optimize the smoothness and smoothness of the video frame, ensure clear details and improve the watching experience of the video.
Fig. 4 is a schematic structural diagram illustrating a video processing apparatus according to an embodiment of the present invention. As shown in fig. 4, the apparatus 50 includes: a first determining module 501, a second determining module 502, a generating module 503, and a processing module 504.
The first determining module 501 is configured to determine a discontinuous area in a continuous frame of a video; the discontinuous area is an area with discontinuous image characteristics of adjacent frames;
a second determining module 502, configured to determine feature information of the consecutive frames;
a generating module 503, configured to generate a plurality of video frames according to the feature information;
a processing module 504, configured to perform enhancement processing on the discontinuous area according to the video frame, so as to obtain a processed video.
The operation process of the video processing apparatus provided in the embodiment of the present invention is substantially the same as that of the foregoing method embodiment, and is not described again.
The video processing device provided by the embodiment of the invention determines the discontinuous area in the continuous frame of the video; the discontinuous region is a region where the image features of adjacent frames are discontinuous, thereby locating a frame position where the viewing experience is not smooth. Then determining the characteristic information of the continuous frames; generating a plurality of video frames according to the characteristic information; and finally, enhancing the discontinuous area according to the video frame to obtain a processed video. Therefore, the video processing device provided by the embodiment of the invention is different from the problem that the video watching experience of the region with large luminosity change is not smooth due to the fact that only the central luminosity region is intensively enhanced in the prior art, and the video processing device can optimize the smoothness and smoothness of the video frame, ensure clear details and improve the watching experience of the video.
Fig. 5 is a schematic structural diagram of a video processing device according to an embodiment of the present invention, and the specific embodiment of the present invention does not limit the specific implementation of the video processing device.
As shown in fig. 5, the video processing apparatus may include: a processor (processor)602, a communication Interface 604, a memory 406, and a communication bus 408.
Wherein: the processor 402, communication interface 404, and memory 406 communicate with each other via a communication bus 408. A communication interface 404 for communicating with network elements of other devices, such as clients or other servers. The processor 402 is configured to execute the program 410, and may specifically perform the relevant steps in the embodiment of the video processing method described above.
In particular, program 410 may include program code comprising computer-executable instructions.
The processor 402 may be a central processing unit CPU or an application Specific Integrated circuit asic or one or more Integrated circuits configured to implement embodiments of the present invention. The video processing device includes one or more processors, which may be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 406 for storing a program 410. Memory 406 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 410 may specifically be invoked by the processor 402 to cause the video processing device to perform the following operations:
determining discontinuous regions within a continuous frame of video; the discontinuous region is a region with discontinuous image characteristics of adjacent frames;
determining feature information of the continuous frames;
generating a plurality of video frames according to the characteristic information;
and enhancing the discontinuous area according to the video frame to obtain a processed video.
The operation process of the video processing device provided by the embodiment of the present invention is substantially the same as that of the foregoing method embodiment, and is not described again.
The video processing equipment provided by the embodiment of the invention determines the discontinuous area in the continuous frame of the video; the discontinuous region is a region where the image features of adjacent frames are discontinuous, thereby locating a frame position where the viewing experience is not smooth. Then determining the characteristic information of the continuous frames; generating a plurality of video frames according to the characteristic information; and finally, enhancing the discontinuous area according to the video frame to obtain a processed video. Therefore, the video processing device provided by the embodiment of the invention is different from the problem that the video watching experience of the region with large luminosity change is not smooth due to the fact that only the central luminosity region is intensively enhanced in the prior art, and the video processing device can optimize the smoothness and smoothness of the video frame, ensure clear details and improve the watching experience of the video.
An embodiment of the present invention provides a computer-readable storage medium, where the storage medium stores at least one executable instruction, and when the executable instruction is executed on a video processing device, the video processing device is caused to execute a video processing method in any method embodiment described above.
The executable instructions may be specifically configured to cause the video processing device to perform the following operations:
determining discontinuous regions within a continuous frame of video; the discontinuous region is a region with discontinuous image characteristics of adjacent frames;
determining feature information of the continuous frames;
generating a plurality of video frames according to the characteristic information;
and enhancing the discontinuous area according to the video frame to obtain a processed video.
The operation process of the executable instructions stored in the computer storage medium provided by the embodiment of the present invention is substantially the same as that of the method embodiment, and is not described again.
The computer storage medium provided by the embodiment of the invention stores executable instructions by determining discontinuous areas in continuous frames of a video; the discontinuous region is a region where the image features of adjacent frames are discontinuous, thereby locating a frame position where the viewing experience is not smooth. Then determining the characteristic information of the continuous frames; generating a plurality of video frames according to the characteristic information; and finally, enhancing the discontinuous area according to the video frame to obtain a processed video. Therefore, the method is different from the problem that the video watching experience is not smooth in the region with large luminosity change due to the fact that only the central luminosity region is intensively enhanced in the prior art, and the executable instruction stored in the computer storage medium provided by the embodiment of the invention can optimize the smoothness and fluency of the video frame, guarantee clear details and improve the watching experience of the video.
The embodiment of the invention provides a video processing device, which is used for executing the video processing method.
Embodiments of the present invention provide a computer program that can be invoked by a processor to cause a video processing device to perform a video processing method in any of the above method embodiments.
Embodiments of the present invention provide a computer program product comprising a computer program stored on a computer-readable storage medium, the computer program comprising program instructions that, when run on a computer, cause the computer to perform a video processing method in any of the above-mentioned method embodiments.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems may also be used with the teachings herein. The required structure for constructing such a system will be apparent from the description above. In addition, embodiments of the present invention are not directed to any particular programming language. It is appreciated that a variety of programming languages may be used to implement the teachings of the present invention as described herein, and any descriptions of specific languages are provided above to disclose the best mode of the invention.
In the description provided herein, numerous specific details are set forth. It is understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the invention and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that the invention as claimed requires more features than are expressly recited in each claim.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names. The steps in the above embodiments should not be construed as limited to the order of execution unless otherwise specified.
Claims (10)
1. A method of video processing, the method comprising:
determining discontinuous regions within a continuous frame of video; the discontinuous region is a region with discontinuous image characteristics of adjacent frames;
determining feature information of the continuous frames;
generating a plurality of video frames according to the characteristic information;
and enhancing the discontinuous area according to the video frame to obtain a processed video.
2. The method of claim 1, wherein the feature information comprises frequency domain feature information; the video frame comprises an approximate video frame; the approximate video frame is a video frame in which the frequency domain characteristic information in the front and back directions of the continuous frames meets a preset characteristic value interval.
3. The method of claim 2, wherein the frequency domain feature information comprises gray value gradient change information; the generating a plurality of video frames according to the feature information includes:
screening at least two contrast reference frames in the front and back directions of the continuous frames according to the gray value gradient change information and the characteristic value interval to obtain frames to be approximated;
and carrying out approximation processing on the frame to be approximated to obtain the approximated video frame.
4. The method of claim 1, wherein the video frame comprises an overlay video frame; the superimposed video frame is a key frame containing texture detail information of the successive frames.
5. The method of claim 4, wherein the feature information comprises frequency domain feature information; the discontinuous region comprises a plurality of target region points; the generating a plurality of video frames according to the feature information includes:
determining at least one key frame corresponding to the target area point from the adjacent area of the target area point according to the frequency domain characteristic information;
generating a transparent image corresponding to the target area point according to the frequency domain characteristic information;
and overlapping the transparent image to the key frame of the target area point to obtain the overlapped video frame.
6. The method of claim 5, wherein the frequency domain feature information comprises gray value gradient change information; the generating of the transparent image corresponding to the target area point according to the frequency domain feature information includes:
dividing the continuous frames into highlight areas and low light areas according to the gray value gradient change information;
determining a detail component relationship between the highlight region and the highlight region;
and generating the transparent image according to the detail component relation and the endpoint output information of the highlight area and the low light area.
7. The method of claim 1, wherein the discontinuity region comprises a plurality of target region points; the characteristic information comprises frequency domain characteristic information; the video frames comprise an approximate video frame and an overlay video frame; the approximate video frame is a video frame of which the change of the frequency domain characteristic information in the front and back directions of the continuous frames meets a preset condition; the superimposed video frame is a key frame containing texture detail information of the continuous frames; the enhancing the discontinuous region according to the video frame to obtain a processed video, including:
determining an optimal frame from the approximate video frame and the overlay video frame;
and carrying out frame replacement or frame supplement processing on the target area point according to the optimal frame to obtain the processed video.
8. A video processing apparatus, characterized in that the apparatus comprises:
a first determining module for determining discontinuous areas within successive frames of a video; the discontinuous region is a region with discontinuous image characteristics of adjacent frames;
a second determining module, configured to determine feature information of the consecutive frames;
the generating module is used for generating a plurality of video frames according to the characteristic information;
and the processing module is used for enhancing the discontinuous area according to the video frame to obtain a processed video.
9. A video processing apparatus, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is configured to store at least one executable instruction that causes the processor to perform the operations of the video processing method according to any one of claims 1 to 7.
10. A computer-readable storage medium having stored therein at least one executable instruction that, when executed on a video processing device, causes the video processing device to perform operations of the video processing method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210528392.3A CN114972087A (en) | 2022-05-16 | 2022-05-16 | Video processing method, device, equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210528392.3A CN114972087A (en) | 2022-05-16 | 2022-05-16 | Video processing method, device, equipment and computer storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114972087A true CN114972087A (en) | 2022-08-30 |
Family
ID=82982766
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210528392.3A Pending CN114972087A (en) | 2022-05-16 | 2022-05-16 | Video processing method, device, equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114972087A (en) |
-
2022
- 2022-05-16 CN CN202210528392.3A patent/CN114972087A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Rao et al. | A Survey of Video Enhancement Techniques. | |
| US7149355B2 (en) | Image processing apparatus, image processing method, image processing program, and computer-readable record medium storing image processing program | |
| US7321699B2 (en) | Signal intensity range transformation apparatus and method | |
| EP2164040B1 (en) | System and method for high quality image and video upscaling | |
| Su et al. | Spatially adaptive block-based super-resolution | |
| Wei et al. | Contrast-guided image interpolation | |
| US9262811B2 (en) | System and method for spatio temporal video image enhancement | |
| JPH06245113A (en) | Equipment for improving picture still more by removing noise and other artifact | |
| KR101377733B1 (en) | -up-scaling | |
| KR101987079B1 (en) | Method for removing noise of upscaled moving picture with dynamic parameter based on machine learning | |
| Park et al. | High dynamic range and super-resolution imaging from a single image | |
| Sandić-Stanković et al. | Fast blind quality assessment of DIBR-synthesized video based on high-high wavelet subband | |
| Damkat | Single image super-resolution using self-examples and texture synthesis | |
| CA2842439A1 (en) | Adaptive upsampling for spatially scalable video coding | |
| CN111353955A (en) | Image processing method, device, equipment and storage medium | |
| Furuta et al. | Fast volume seam carving with multipass dynamic programming | |
| Xu et al. | Fmnet: Frequency-aware modulation network for sdr-to-hdr translation | |
| WO2014008329A1 (en) | System and method to enhance and process a digital image | |
| CN114449181B (en) | Image and video processing method and system, data processing device and medium | |
| RU2583725C1 (en) | Method and system for image processing | |
| Luzardo et al. | Real-time false-contours removal for inverse tone mapped HDR content | |
| US9154671B2 (en) | Image processing apparatus, image processing method, and program | |
| CN114972087A (en) | Video processing method, device, equipment and computer storage medium | |
| Lee et al. | Real-time Monte Carlo Denoising with Adaptive Fusion Network | |
| CN112911262B (en) | Video sequence processing method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |