CN112954348A

CN112954348A - Video encoding method and apparatus, electronic device, and storage medium

Info

Publication number: CN112954348A
Application number: CN202110194279.1A
Authority: CN
Inventors: 周波; 吴佳飞; 李树析; 张广程; 闫俊杰
Original assignee: Zhejiang Shangtang Technology Development Co Ltd
Current assignee: Zhejiang Shangtang Technology Development Co Ltd
Priority date: 2021-02-20
Filing date: 2021-02-20
Publication date: 2021-06-11
Anticipated expiration: 2041-02-20
Also published as: CN112954348B

Abstract

The present disclosure relates to a video encoding method and apparatus, an electronic device, and a storage medium, the method including: analyzing the picture motion intensity of the current image group of the video stream to determine the motion intensity of the current image group; under the condition that the motion intensity does not exceed the motion intensity threshold, determining the encoding parameters of the image group to be encoded according to the encoded current image group, wherein the acquisition time of the image group to be encoded is behind the acquisition time of the current image group; and coding the image group to be coded according to the coding parameters to obtain a coded image group. The embodiment of the disclosure can effectively predict the encoding parameters of the image group to be encoded, so that the encoding parameters of the image group to be encoded tend to be reasonable, thereby being beneficial to relieving the generation of respiratory effect.

Description

Video encoding method and apparatus, electronic device, and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a video encoding method and apparatus, an electronic device, and a storage medium.

Background

Currently, with the development of technology, in order to store, transmit and communicate video files, video coding processing is required for videos. The video encoding process generally allocates an encoding rate (i.e., a number of encoding bits) to image frames in the video.

In the related technology, aiming at the video with the video picture in a relatively static scene, the coding rate distribution is not reasonable enough when the video is coded, and the video coded under the unreasonable rate distribution may generate the respiratory effect of the video picture.

Disclosure of Invention

The present disclosure proposes a technical solution of video coding.

According to an aspect of the present disclosure, there is provided a video encoding method including: analyzing the picture motion intensity of a current image group of a video stream, and determining the motion intensity of the current image group; under the condition that the motion intensity does not exceed a motion intensity threshold, determining encoding parameters of an image group to be encoded according to an encoded current image group, wherein the acquisition time of the image group to be encoded is after the acquisition time of the current image group; and coding the image group to be coded according to the coding parameters to obtain a coded image group. According to the embodiment of the disclosure, the encoding parameters of the image group to be encoded, which is to be encoded, can be effectively predicted, so that the encoding parameters of the image group to be encoded tend to be reasonable, and the generation of the respiratory effect is favorably alleviated.

In a possible implementation manner, in a case that the motion intensity does not exceed the motion intensity threshold, determining, according to the current image group that has been encoded, an encoding parameter of the image group to be encoded includes: decoding the encoded current image group to obtain each frame image in the decoded current image group and a first code rate allocation proportion of each frame image, wherein the encoded current image group is obtained by encoding the current image group according to a preset encoding code rate; and determining the coding parameters of the group of pictures to be coded according to the first code rate distribution proportion. By the method, whether the code rate distribution of the current image group is reasonable or not can be determined, and then the unreasonable code rate distribution proportion can be effectively adjusted, so that the encoding parameters of the image group to be encoded tend to be reasonable.

In a possible implementation manner, the determining, according to the first rate allocation ratio, the encoding parameters of the group of pictures to be encoded includes: analyzing the definition of each frame of image to determine the definition of each frame of image; and adjusting the first code rate distribution proportion according to the definition of each frame of image so as to determine the encoding parameters of the image group to be encoded. By the method, the image quality difference between the key frame and the non-key frame in the coded image group to be coded can be reduced, and the effect of relieving the respiratory effect is achieved.

In a possible implementation manner, the adjusting the first code rate allocation ratio according to the definition of each frame of image to determine the encoding parameters of the image group to be encoded includes: determining a first difference value between the definition of each frame of image; under the condition that a first difference value exceeding a definition threshold exists, adjusting the first code rate distribution proportion according to a preset adjustment step length to obtain an adjusted second code rate distribution proportion; and determining the encoding parameters of the group of pictures to be encoded according to the second code ratio distribution proportion. By the method, the encoding parameters of the image group to be encoded determined according to the second code ratio distribution proportion tend to be reasonable under the condition that the definition difference between the frame images is large, and therefore the generation of the respiratory effect is effectively relieved.

In a possible implementation manner, the determining, according to the second code ratio distribution ratio, the encoding parameter of the group of pictures to be encoded includes: judging whether the second code rate distribution proportion exceeds a code rate distribution proportion threshold value or not; and under the condition that the second code rate distribution proportion does not exceed a code rate distribution proportion threshold, determining a code rate value corresponding to each frame of image according to the second code rate distribution proportion and a preset coding code rate, wherein the coding parameters of the image group to be coded comprise the code rate value corresponding to each frame of image. By the method, the upper limit of the distribution proportion of the adjusted second code rate can be effectively controlled, and the situation that the coded image quality and the video coding efficiency are reduced due to overhigh code rate of I-frame distribution is further favorably reduced.

In one possible implementation, the method further includes: and under the condition that the second code rate distribution proportion exceeds a code rate distribution proportion threshold, determining the encoding parameters of the group of pictures to be encoded according to a second difference value between the definition of each frame of picture in each group of pictures in a group of pictures set and the encoding parameters of each group of pictures, wherein the group of pictures set comprises coded pictures of which the motion intensity does not exceed the motion intensity threshold. By the method, when the second code rate distribution proportion does not meet the requirement that the code rate distributed by the I frame cannot be too high, the coding parameters of the image group to be coded can be effectively determined according to the definition difference and the coding parameters of each image group coded before.

In a possible implementation manner, in the case that the second code rate allocation ratio exceeds a code rate allocation ratio threshold, determining, according to a second difference between definitions of frames of images in each group of images in a group of images and coding parameters of each group of images, coding parameters of the group of images to be coded, includes: determining the minimum value of third difference values corresponding to each image group, wherein the third difference value is the maximum value of the second difference values corresponding to each image group; and taking the encoding parameter of the image group corresponding to the minimum value in the image group set as the encoding parameter of the image group to be encoded. By the method, the coding parameters of the image group with the minimum definition difference can be applied to the image group to be coded to relieve the generation of respiratory effect under the condition that the requirement that the code rate of I frame distribution cannot be too high is met.

In one possible implementation, each frame of image includes a key frame and at least one non-key frame; the first difference comprises a difference between the sharpness of the key frame and the sharpness of the non-key frame.

In a possible implementation manner, the analyzing the picture motion strength of the current image group of the video stream and determining the motion strength of the current image group includes: and inputting the current image group into a pre-trained first neural network to obtain a motion intensity score of the current image group, wherein the motion intensity score is used for representing the motion intensity of the current image group. By the method, the motion intensity of the current image group can be effectively and accurately obtained.

In a possible implementation manner, the analyzing the sharpness of each frame of image to determine the sharpness of each frame of image includes: and inputting the frame images into a pre-trained second neural network to obtain the definition scores of the frame images, wherein the definition scores are used for representing the definition of the frame images. By the method, the definition of each frame of image can be effectively and accurately obtained, and the first code rate distribution proportion can be accurately adjusted according to the accurate definition.

According to an aspect of the present disclosure, there is provided a video encoding apparatus including: the motion intensity analysis module is used for analyzing the picture motion intensity of the current image group of the video stream and determining the motion intensity of the current image group; the first encoding parameter determining module is used for determining encoding parameters of an image group to be encoded according to an encoded current image group under the condition that the motion intensity does not exceed a motion intensity threshold, wherein the acquisition time of the image group to be encoded is behind the acquisition time of the current image group; and the coding module is used for coding the image group to be coded according to the coding parameters to obtain a coded image group.

In one possible implementation manner, the first encoding parameter determining module includes: the decoding submodule is used for decoding the coded current image group to obtain each frame image in the decoded current image group and a first code rate allocation proportion of each frame image, wherein the coded current image group is obtained by coding the current image group according to a preset coding rate; and the first coding parameter determining submodule is used for determining the coding parameters of the group of pictures to be coded according to the first code rate distribution proportion.

In one possible implementation manner, the first encoding parameter determining sub-module includes: the definition analyzing unit is used for analyzing the definition of each frame of image and determining the definition of each frame of image; and the adjusting unit is used for adjusting the first code rate distribution proportion according to the definition of each frame of image so as to determine the encoding parameters of the image group to be encoded.

In a possible implementation manner, the adjusting unit is specifically configured to: determining a first difference value between the definition of each frame of image; under the condition that a first difference value exceeding a definition threshold exists, adjusting the first code rate distribution proportion according to a preset adjustment step length to obtain an adjusted second code rate distribution proportion; and determining the encoding parameters of the group of pictures to be encoded according to the second code ratio distribution proportion.

In a possible implementation manner, the determining, according to the second code ratio distribution ratio, the encoding parameter of the group of pictures to be encoded includes: judging whether the second code rate distribution proportion exceeds a code rate distribution proportion threshold value or not; and under the condition that the second code rate distribution proportion does not exceed a code rate distribution proportion threshold, determining a code rate value corresponding to each frame of image according to the second code rate distribution proportion and a preset coding code rate, wherein the coding parameters of the image group to be coded comprise the code rate value corresponding to each frame of image.

In one possible implementation, the method further includes: and the second encoding parameter determining module is used for determining the encoding parameters of the image group to be encoded according to a second difference value between the definitions of the frames of images in each image group in the image group set and the encoding parameters of each image group under the condition that the second code ratio distribution ratio exceeds a code ratio distribution ratio threshold, wherein the image group set comprises an encoded image group of which the motion intensity does not exceed the motion intensity threshold.

In one possible implementation manner, the second encoding parameter determining module includes: the determining submodule is used for determining the minimum value in third difference values corresponding to each image group, and the third difference value is the maximum value in second difference values corresponding to each image group; and the second encoding parameter determining submodule is used for taking the encoding parameter of the image group corresponding to the minimum value in the image group set as the encoding parameter of the image group to be encoded.

In a possible implementation manner, the exercise intensity analysis module is specifically configured to input the current image group into a pre-trained first neural network, so as to obtain an exercise intensity score of the current image group, where the exercise intensity score is used to characterize the exercise intensity of the current image group.

In a possible implementation manner, the sharpness analysis unit is specifically configured to input each frame of image into a pre-trained second neural network to obtain a sharpness score of each frame of image, where the sharpness score is used to represent sharpness of each frame of image.

According to an aspect of the present disclosure, there is provided an electronic device including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the memory-stored instructions to perform the above-described method.

According to an aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.

In the embodiment of the present disclosure, whether to adjust the encoding parameters of the image group to be encoded can be determined according to the motion intensity of the current image group of the video stream, so that the encoding parameters can be adjusted for the image group with low motion intensity, that is, the image is in a relatively static scene; in addition, the encoding parameters of the image group to be encoded can be effectively predicted according to the encoded current image group, so that the encoding parameters of the image group to be encoded tend to be reasonable, and the generation of the respiratory effect is favorably alleviated.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

Fig. 1 shows a flow diagram of a video encoding method according to an embodiment of the present disclosure.

Fig. 2 shows a flow diagram of a video encoding method according to an embodiment of the present disclosure.

Fig. 3 shows a flow chart of an encoding parameter adjustment method according to an embodiment of the present disclosure.

Fig. 4 illustrates a block diagram of a video encoding apparatus according to an embodiment of the present disclosure.

Fig. 5 shows a block diagram of an electronic device in accordance with an embodiment of the disclosure.

Fig. 6 illustrates a block diagram of an electronic device in accordance with an embodiment of the disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, description, and drawings of the present disclosure are used to distinguish between different objects and are not used to describe a particular order. The terms "comprises" and "comprising," when used in the specification and claims of this disclosure, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present disclosure.

In the field of video coding, it is common to have both key frames (independent frames with all information themselves, called I-frames) and non-key frames (e.g., P-frames, recording differences from a previous frame; B-frames, recording differences from a previous frame and a subsequent frame) in order to save storage space. Rate allocation has a significant impact on the image quality of the encoded video, especially with respect to rate allocation for I-frames (intra-predicted frames, key frames). If the code rate allocated to the I frame is too low, the image quality of the I frame is reduced, and due to the inter-frame prediction encoding mode in video encoding, the image quality of non-I frames (a forward inter-frame prediction frame P frame and a bidirectional inter-frame prediction frame B frame) subsequent to the I frame is also affected by the image quality of the I frame, so that the overall image quality of the video is reduced, and an obvious respiratory effect may be generated. If the code rate of the I frame is too large, the video coding efficiency is reduced, the quality of the coded image is also influenced, and meanwhile, the I frame with too high code rate may cause network delay and packet loss to cause video blocking; therefore, reasonable rate allocation is important to video coding efficiency and image quality after coding.

In some scenarios, since video coding generally requires high real-time processing performance, rate allocation may be performed in a predictive manner. In the related art, when the coding rate allocation is predicted, the coding parameters of the current Group of Pictures are generally determined by the average coding parameters of the previous Group of Pictures (GOP). The method is simple, but cannot output reasonable coding parameters according to video characteristics (such as different picture motion intensities and different picture complexities), and cannot effectively deal with the generation of respiratory effect.

The respiratory effect in the video means that the image quality is better due to the insertion of the I frame, the image quality is gradually worse after the I frame is switched to the non-I frame, and the image quality is suddenly better after the I frame is inserted. The respiratory effect is mainly caused by the difference of the coding modes and the image quality of the I frame and the P frame, and the unreasonable code rate distribution, so that the image distortion degree is different, and the image is discontinuous visually. The breathing effect is easier to be observed when the picture is in a relatively static scene, and particularly the breathing effect is more obvious in a low-code-rate relatively static scene; in a moving scene, most of the contents in the picture are changed, so that the picture is not easy to be observed visually.

According to the embodiment of the disclosure, the image group with low picture motion intensity (i.e. relatively static picture) can be analyzed for the difference in definition between the I frame and the non-I frame in the current encoded image group, and then the rate allocation of the image group to be encoded is adjusted according to the difference in definition, so that the difference in definition between the I frame and the non-I frame in the subsequent image group tends to be minimum, thereby effectively relieving the respiratory effect in the video.

According to the embodiment of the disclosure, the respiratory effect can be effectively relieved, the method can be suitable for various video encoders, the encoding efficiency can meet the requirement of real-time performance, the image quality after encoding at the specified code rate is ensured, and the loss of the image quality caused by two rounds of encoding can be avoided.

Fig. 1 shows a flowchart of a video encoding method according to an embodiment of the present disclosure, as shown in fig. 1, the video encoding method includes:

in step S11, analyzing the picture motion strength of the current image group of the video stream to determine the motion strength of the current image group;

in step S12, in the case that the motion intensity does not exceed the motion intensity threshold, determining the encoding parameters of the image group to be encoded according to the encoded current image group, wherein the acquisition time of the image group to be encoded is after the acquisition time of the current image group;

in step S13, the group of pictures to be encoded is encoded according to the encoding parameters, so as to obtain an encoded group of pictures to be encoded.

In one possible implementation, the video encoding method may be performed by an electronic device such as a terminal device or a server, where the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like, and the method may be implemented by a processor calling computer-readable instructions stored in a memory. Alternatively, the method may be performed by a server.

In one possible implementation, the video stream may be acquired in real time by an image acquisition device (e.g., various cameras) connected to the electronic device; the information may also be transmitted to the electronic device by other devices, or may also be obtained by the electronic device invoking a local storage, which is not limited in this disclosure.

In one possible implementation, the image groups may be divided according to a frame rate, for example, based on a known frame rate of the video stream, each image group may include: a plurality of frame images within a preset time period of 1 second or 2 seconds. It will be appreciated that the group of pictures may comprise one or more frames of pictures in a video stream, and the frames of pictures may be arranged in a time sequence. The specific value of the preset time period may be determined according to the encoding performance, the encoding mode, and the like of the video encoder, and the embodiment of the present disclosure is not limited thereto.

In one possible implementation, the video stream may be captured in real-time or non-real-time. For a video stream acquired in real time, the current image group may include an image group acquired within a current preset time period in the video stream; for a non-real-time captured video stream, the current group of pictures may refer to a group of pictures in the video stream for any preset time period, and the embodiment of the present disclosure is not limited thereto.

It is understood that the video stream includes multiple frames of images captured in chronological order, that is, the multiple frames of images in the video stream are arranged in a time sequence. The acquisition time of the image group to be encoded is after the acquisition time of the current image group, which can be understood as that the acquisition time of the image in the image group to be encoded is after the current image group. The group of pictures to be encoded may be a group of pictures that is consecutive after the current group of pictures, or may be a group of pictures to be encoded obtained by performing interval sampling, which is not limited in the embodiment of the present disclosure.

In one possible implementation, for a video stream acquired in real time, the current image group may include an image group in the currently acquired video stream; the group of images to be encoded may include a group of images to be acquired immediately after the current group of images. It can be understood that, when an image group to be encoded in a video stream is acquired, the acquired image group to be encoded is also a current image group. In other words, each group of pictures transmitted in the video stream can be subjected to motion intensity analysis, so that the encoding parameters can be effectively adjusted for the pictures with low motion intensity.

In one possible implementation, the picture motion strength may refer to a degree of variation of content in the video picture. It can be understood that, in a relatively static scene, the degree of change of the content in the picture is not high, that is, the picture motion intensity is not high; in a moving scene, the content in the picture has a higher degree of change, i.e., the picture has a higher motion intensity.

It should be noted that the magnitude of the picture motion intensity is relative, and for example, the picture motion intensity in a conference scene is lower than the picture motion intensity in a running scene. The person skilled in the art will understand, or at least after reading the embodiments of the present disclosure, the meaning of the level of the motion intensity of the picture.

In one possible implementation manner, in step S11, analyzing the picture motion strength of the current image group of the video stream to determine the motion strength of the current image group may include: and inputting the current image group into a pre-trained first neural network to obtain the motion intensity score of the current image group, wherein the motion intensity score is used for representing the motion intensity of the current image group.

In one possible implementation, the first neural network may, for example, employ at least: learning the convolutional neural network FlowNet of the optical flow. The FlowNet can realize the purpose of utilizing the motion information in the optical flow to output the motion intensity score of the image group. The embodiment of the present disclosure is not limited to the training mode and the specific structure of the first neural network. Of course, other types of neural networks may be used, as long as the intensity of the motion of the image set can be analyzed.

In a possible implementation manner, for example, the similarity between each frame of image in the current image group may also be analyzed through the first neural network; and scoring the movement intensity of the picture according to the analysis result of the similarity to obtain a movement intensity score. It can be understood that the higher the similarity is, the lower the motion degree of the picture is represented, and the lower the motion intensity score is; conversely, the higher the exercise intensity score.

In a possible implementation manner, the exercise intensity score output by the first neural network may be trained to be positively correlated with the exercise intensity, that is, the lower the exercise intensity score is, the lower the representative picture exercise intensity is; conversely, the higher the motion intensity score, the higher the representative picture motion intensity. Of course, the correlation may also be negative, and may be specifically set according to the training mode of the neural network and the actual requirement, and the embodiment of the present disclosure is not limited.

In the embodiment of the disclosure, the image motion intensity of the current image group is analyzed through the pre-trained first neural network, and the motion intensity of the current image group can be effectively and accurately obtained.

In one possible implementation manner, in step S12, the exercise intensity threshold may be set according to actual needs, historical experience, and the like, and the embodiment of the present disclosure is not limited thereto.

As described above, the exercise intensity score may characterize the exercise intensity. In one possible implementation, the motion intensity not exceeding the motion intensity threshold may include: the exercise intensity score does not exceed the exercise intensity threshold.

As described above, the breathing effect is significant in a video picture in a relatively static scene. For the case that the motion intensity does not exceed the motion intensity threshold, the motion intensity of the picture in the current image group may be considered not to be high in this case, and the picture in the current image group may be a relatively static scene. Therefore, the encoding parameters of the image group to be encoded can be predicted according to the encoded current image group aiming at the image group with low picture motion intensity, so as to realize the adjustment and optimization of the encoding parameters of the image group to be encoded.

In one possible implementation manner, in step S12, the encoded current group of pictures may be obtained by encoding the current group of pictures with an existing video encoder according to a preset encoding rate. The embodiment of the present disclosure does not limit the encoding mode of the current group of pictures. The group of pictures to be encoded may specifically refer to a group of pictures that is not encoded after a current group of pictures in the video stream.

In a possible implementation manner, whether the code rate distribution of the coded current image group is reasonable can be known through the coded current image group, so that the coding parameters can be effectively adjusted according to the unreasonable code rate distribution, and the adjusted coding parameters are obtained; and determining the adjusted encoding parameters as the encoding parameters of the image group to be encoded, so that the code rate distribution of the image group to be encoded tends to be reasonable.

In a possible implementation manner, there may also be a case where the motion intensity of the current group of pictures exceeds the motion intensity threshold, in which case, it may be considered that the picture corresponding to the current group of pictures is in a motion scene, and at this time, the encoding parameters of the current group of pictures may not be adjusted, and the subsequent group of pictures to be encoded is encoded by the video encoder according to the current encoding manner. It will be appreciated that due to the continuity of the image frames in the video stream, the current group of images is in a moving scene or a relatively stationary scene, after which there is a high probability that the group of images will also be in a moving scene or a relatively stationary scene.

In one possible implementation, the encoding parameter may include a code rate value allocated to each frame of image in the group of images, for example, the preset encoding code rate is 800kbps, a code rate value allocated to an I frame in the group of images may be 400kbps, and a code rate value allocated to 2P frames may be 200kbps, respectively. In step S13, the group of pictures to be encoded is encoded according to the encoding parameters to obtain an encoded group of pictures to be encoded, which may be the group of pictures to be encoded obtained by encoding the group of pictures to be encoded according to the code rate value allocated to each frame of picture in the encoding parameters. For example, the existing video encoder may be adopted to encode the image group to be encoded, and the embodiment of the present disclosure is not limited thereto.

In a possible implementation manner, in step S12, in the case that the motion intensity does not exceed the motion intensity threshold, determining the encoding parameters of the image group to be encoded according to the encoded current image group includes:

decoding the encoded current image group to obtain each frame image in the decoded current image group and a first code rate allocation proportion of each frame image, wherein the encoded current image group is obtained by encoding the current image group according to a preset encoding code rate; and determining the coding parameters of the image group to be coded according to the first code rate distribution proportion.

In one possible implementation, each frame image in the current group of images may include a key frame and at least one non-key frame. Wherein, the key frame may comprise an I frame, and the non-key frame may comprise a P frame and/or a B frame.

In one possible implementation, as described above, the encoded current group of pictures may be obtained by encoding the current group of pictures via a video encoder according to a preset encoding rate.

In a possible implementation manner, the preset coding rate may be set according to an actual video coding requirement, which is not limited in the embodiment of the present disclosure, and the preset coding rate may be used to code the entire video stream.

In one possible implementation, the decoding of the encoded current group of pictures may be implemented by a video decoder corresponding to the video encoder described above. The embodiment of the present disclosure does not limit the decoding manner of the encoded current group of pictures.

In a possible implementation manner, after the encoded current image group is decoded, each frame of image and the code rate corresponding to each frame of image can be obtained, and further, the first code rate allocation proportion can be known according to the code rate corresponding to each frame of image.

In a possible implementation manner, the first code rate allocation ratio may be a ratio between the code rate of each frame image and a preset coding code rate. For example, the preset coding rate is 800kbps, the allocated coding rate for the I-frame is 400kbps, the allocated coding rates for the 2P-frames are 200kbps, respectively, and the first coding rate allocation ratio may include 50%, 25%, and 25%.

In a possible implementation manner, determining the encoding parameters of the image group to be encoded according to the first rate allocation ratio may include: adjusting the first code rate distribution proportion to obtain the adjusted code rate distribution proportion; and then distributing the code rate of each frame of image according to the adjusted code rate distribution proportion and the preset coding code rate, and determining the coding parameters of the image group to be coded.

In a possible implementation manner, when the rate allocation of the current group of pictures is adjusted to be reasonable, the current first rate allocation proportion may not be adjusted, so that the group of pictures to be encoded is encoded according to the encoding parameters of the current group of pictures.

In a possible implementation manner, when the encoded current group of pictures is decoded, the sequence number of each frame of picture may also be obtained, and each frame of picture may be distinguished by the sequence number and the arrangement sequence of each frame of picture is identified, so that the first code rate allocation proportion of each frame of picture and the encoding parameter of the second picture may be determined conveniently.

In the embodiment of the disclosure, by decoding the encoded current image group to obtain each frame of image in the decoded current image group and the first code rate allocation proportion of each frame of image, whether the code rate allocation of the current image group is reasonable can be determined, and then the unreasonable code rate allocation proportion can be effectively adjusted to make the encoding parameters of the image group to be encoded tend to be reasonable.

In a possible implementation manner, the determining, according to the first rate allocation ratio, the encoding parameters of the group of images to be encoded includes:

analyzing the definition of each frame of image to determine the definition of each frame of image;

and adjusting the first code rate distribution proportion according to the definition of each frame of image so as to determine the encoding parameters of the image group to be encoded.

As described above, the generation of the breathing effect is mainly due to the difference in image quality between the I frame and the non-I frame. In one possible implementation, the image quality may be reflected by sharpness, and the difference in sharpness (i.e., the difference in image quality) between I-frames and non-I-frames reflects whether the rate allocation is reasonable. And then the first code rate distribution proportion can be adjusted according to the definition difference of the I frame and the non-I frame images so as to determine the coding parameters of the image group to be coded.

In a possible implementation manner, analyzing the sharpness of each frame of image to determine a sharpness difference of each frame of image may include: and inputting each frame of image into a pre-trained second neural network to obtain the definition score of each frame of image, wherein the definition score is used for representing the definition of each frame of image.

In one possible implementation, the second neural network may comprise a convolutional neural network, for example, a convolutional neural network of the type *** lenet, AlexNet, VGG, or the like may be employed. The training mode and the specific structure of the second neural network are not limited in the embodiments of the present disclosure.

In a possible implementation manner, the definition score output by the second neural network can be trained to be positively correlated with the definition, that is, the higher the definition score is, the clearer the representative image is, and the better the image quality is; conversely, the less clear the representative image, the worse the image quality. Of course, the correlation may also be negative, and may be specifically set according to the training mode of the neural network and the actual requirement, and the embodiment of the present disclosure is not limited.

In the embodiment of the disclosure, the definition of each frame of image is analyzed through the pre-trained second neural network, so that the definition of each frame of image can be effectively and accurately obtained, and the first code rate allocation proportion can be accurately adjusted according to the accurate definition.

In a possible implementation manner, adjusting the first code rate allocation ratio according to the definition of each frame of image to determine the encoding parameters of the group of images to be encoded may include: correspondingly adjusting the first code rate distribution proportion according to the definition scores of the frames of images; and determining the coding parameters of the image group to be coded according to the adjusted code rate distribution proportion.

In the embodiment of the disclosure, the first code rate allocation proportion can be effectively adjusted according to the definition of each frame of image to determine the encoding parameters of the image group to be encoded, so that the image quality difference between the key frame and the non-key frame in the encoded image group to be encoded is reduced, and the effect of relieving the respiratory effect is achieved.

In a possible implementation manner, the adjusting the first code rate allocation ratio according to the definition of each frame of image to determine the encoding parameters of the image group to be encoded includes:

determining a first difference value between the definition of each frame of image;

under the condition that a first difference value exceeding a definition threshold exists, adjusting the first code rate distribution proportion according to a preset adjustment step length to obtain an adjusted second code rate distribution proportion;

and determining the encoding parameters of the image group to be encoded according to the second code ratio distribution proportion.

As described above, each frame image may include a key frame and at least one non-key frame. Wherein, the key frame may comprise an I frame, and the non-key frame may comprise a P frame and/or a B frame. In one possible implementation, the first difference may include a difference between a sharpness of the key frame and a sharpness of at least one frame of the non-key frame. For example, the images of each frame in the current image group a are arranged in time series as "I frame, P₁Frame, P₂Frame, P₃Frame, P₄Frame ", the first difference value may include: i frame definition and P₁Difference between frame sharpness, I-frame sharpness and P₂Difference between frame sharpness, I-frame sharpness and P₃Difference between frame sharpness, I-frame sharpness and P₄At least one of differences between frame definitions.

As described above, the sharpness score may be used to characterize the sharpness of each frame of the image. In a possible implementation manner, the first difference between the sharpness of each frame of image may include: a first difference between the sharpness scores of the respective frame images.

In a possible implementation manner, the sharpness threshold may be set according to actual needs, historical experience, and the like, and the embodiment of the present disclosure is not limited thereto.

In a possible implementation manner, for a case that the first difference exceeds the sharpness threshold, the sharpness difference between the key frame and the non-key frame may be considered to be large, and accordingly, the code rate allocation of the current group of pictures may be considered to be unreasonable, and in this case, the first code allocation ratio may be adjusted.

In a possible implementation manner, for a case that the first difference does not exceed the definition threshold, the definition difference between the key frame and the non-key frame may be considered to be small, and accordingly, the rate allocation of the current group of pictures may be considered to be reasonable.

In a possible implementation manner, determining whether there is a first difference value exceeding the sharpness threshold in the first difference values may include: and judging whether the difference values between the key frame and the continuous multiframe non-key frames exceed a definition threshold value or not. Wherein the specific number of non-key frames is not limited by the embodiments of the present disclosure. For example, it may be set that the difference between the sharpness of P frames and the sharpness of I frames of consecutive 3 frames each exceed a sharpness threshold, and it is determined that there is a first difference that exceeds the sharpness threshold.

For example, following the above example of the current image group a, if a multi-frame is set to be 3 frames, there is a first difference value exceeding the definition threshold, which may be: i frame definition and P₁Difference between frame sharpness, I-frame sharpness and P₂Difference between frame sharpness, I-frame sharpness and P₃The difference between the frame definitions exceeds a definition threshold; it can also be: i frame definition and P₂Difference between frame sharpness, I-frame sharpness and P₃Difference between frame sharpness, I-frame sharpness and P₄The difference between the frame sharpness values exceeds the sharpness threshold.

In a possible implementation manner, the preset adjustment step size may be set according to an actual requirement, a specific form of the code rate allocation ratio, and the like, for example, the adjustment step size may be set to be 1%, which is not limited in this embodiment of the disclosure.

It can be known that, due to the coding mode of inter-frame prediction in video coding, a P frame is usually encoded and decoded with an I frame or a previous P frame as a reference frame, so the image quality of a non-I frame is affected by the image quality of the I frame, and the higher the image quality of the I frame is, the higher the image quality of the opposite non-I frame is. In a possible implementation manner, adjusting the first code rate allocation ratio according to a preset adjustment step to obtain a second code rate allocation ratio may include: and according to a preset adjusting step length, increasing the code rate distribution proportion of the key frame (I frame), and correspondingly reducing the code rate distribution proportion of the non-key frame (P frame and B frame) to obtain a second code rate distribution proportion.

For example, following the example that the first code rate allocation ratio includes 50%, 25%, and 25%, setting the adjustment step size to be 1%, and increasing the code rate allocation ratio of the key frame (I frame) according to the preset adjustment step size, the code rate allocation ratio of the I frame may be 51%, and accordingly, the code rate allocation ratio of the first P frame may still be 25%, the code rate allocation ratio of the second P frame may be reduced to 24%, and the second code rate allocation ratio may include 51%, 25%, and 24%.

In a possible implementation manner, the second code rate allocation ratio may be a code rate allocation ratio obtained after increasing the adjustment step size on the basis of the first code rate classification ratio, that is, the second code rate allocation ratio may be greater than the first code rate classification ratio.

It should be noted that the above manner of increasing the first code rate allocation ratio is a manner of adjusting the code rate allocation ratio provided in the embodiment of the present disclosure, and those skilled in the art can understand that the present disclosure should not be limited thereto. In fact, those skilled in the art can set the method for adjusting the code rate allocation ratio according to actual requirements, as long as the difference in definition between the key frame and the non-key frame in the reduced group of pictures can be achieved.

In a possible implementation manner, after the code rate allocation proportion of the key frame is increased according to the adjustment step size, the code rate allocation proportion of the multi-frame non-I frame may be determined based on a preset adjustment strategy. For example, it may be preset to reduce the code rate allocation ratio from the non-key frame of the last frame, that is, the current reduced code rate allocation ratio is the non-key code rate allocation ratio of the last frame, the next time the code rate allocation ratio of the non-key frame of the second frame to the last is reduced, and so on; it is also possible to preset the code rate allocation reduction ratio starting from the first frame non-key frame. Specifically, the adjustment strategy of the non-I frame may be set according to actual requirements, an encoding mode, a preset encoding rate, and the like, and the embodiment of the present disclosure is not limited.

In a possible implementation manner, determining the encoding parameters of the image group to be encoded according to the second code ratio allocation ratio may include: and allocating the preset coding rate according to the second code rate allocation proportion to obtain the code rate value allocated to each frame image, that is, determining the coding parameter of the image group to be coded, for example, the preset coding rate is 800kbps, allocating according to 51%, 25% and 24%, and obtaining that the code rate allocated to the I frame is 408kbps and the code rates allocated to the two P frames are 200kbps and 192kbps, respectively.

In the embodiment of the disclosure, the first code rate distribution proportion can be adjusted according to the preset adjustment step length under the condition that the definition difference between the frame images is large, so as to obtain the second code rate distribution proportion, so that the encoding parameters of the image group to be encoded determined according to the second code rate distribution proportion tend to be reasonable, and the generation of the respiratory effect is effectively relieved.

In a possible implementation manner, the determining, according to the second code ratio distribution ratio, an encoding parameter of an image group to be encoded includes:

judging whether the second code rate distribution proportion exceeds a code rate distribution proportion threshold value or not;

and under the condition that the second code rate distribution proportion does not exceed the code rate distribution proportion threshold, determining a code rate value corresponding to each frame of image according to the second code rate distribution proportion and a preset coding code rate, wherein the coding parameters of the image group to be coded comprise the code rate value corresponding to each frame of image.

As mentioned above, the code rate of the I-frame allocation cannot be too high. The above method may be to increase the code rate allocation ratio of the I frame when the first code rate allocation ratio is adjusted according to the adjustment step size. In this way, the code rate allocation ratio of the adjusted I frame may be too high, and the code rate allocated to the I frame may also be too high.

In a possible implementation manner, by setting a code rate allocation proportion threshold, whether the second code rate allocation proportion exceeds the code rate allocation proportion threshold is judged, so that the upper limit of the adjusted second code rate allocation proportion can be effectively controlled, and the situation that the code rate of the I-frame allocation is too high is further favorably reduced.

In a possible implementation manner, the rate allocation ratio threshold may be set according to actual needs, historical experience, and the like, for example, 80% or 85% may be set, and the embodiments of the present disclosure are not limited thereto.

In a possible implementation manner, the second code rate allocation proportion does not exceed the code rate allocation proportion threshold, and the adjusted second code rate allocation proportion is considered to be reasonable and effective; and then the code rate value corresponding to each frame of image can be determined according to the reasonable and effective second code rate distribution proportion and the preset coding parameters.

In a possible implementation manner, as described above, the preset coding rate may be allocated according to the second code rate allocation proportion, so as to obtain a code rate value allocated to each frame of image, that is, determine the coding parameters of the image group to be coded.

In the embodiment of the disclosure, the upper limit of the adjusted second code rate distribution proportion can be effectively controlled, thereby being beneficial to reducing the situation that the coded image quality and the video coding efficiency are reduced due to overhigh code rate of I frame distribution.

In one possible implementation, the method further includes:

and under the condition that the second code rate distribution proportion exceeds a code rate distribution proportion threshold, determining the encoding parameters of the image group to be encoded according to a second difference value between the definition of each frame image in each image group in the image group set and the encoding parameters of each image group, wherein the image group set comprises the encoded image group of which the motion intensity does not exceed the motion intensity threshold.

As described above, by setting the code rate allocation ratio threshold value, the adjustment upper limit of the second code rate allocation ratio can be controlled. For the case that the second code rate allocation ratio exceeds the threshold, it may be considered that, although the first code rate allocation ratio, that is, the coding parameter of the current group of pictures, may still be unreasonable in this case, the adjusted second code rate allocation ratio does not meet the requirement that the code rate allocated by the I frame cannot be too high. At this time, the encoding parameters of the group of images to be encoded may not be determined in accordance with the second encoding distribution ratio.

In one possible implementation, the set of image groups includes encoded image groups for which the motion strength does not exceed the motion strength threshold. That is, each group of pictures in the group of pictures may be an encoded group of pictures obtained according to the video encoding method in the embodiment of the present disclosure described above.

When the second code rate allocation proportion does not meet the requirement that the code rate allocated by the I frame cannot be too high, the encoding parameters can be selected according to a certain rule from the encoding parameters of the previously encoded image group, and the selected encoding parameters are determined as the encoding parameters of the image group to be encoded.

In a possible implementation manner, in the process of obtaining each encoded group of pictures according to the video encoding method in the embodiment of the present disclosure, a second difference between the degrees of sharpness of each frame of picture in each group of pictures and an encoding parameter of each group of pictures may be recorded, so that when the second code rate allocation ratio exceeds the code rate allocation ratio threshold, the encoding parameter of the group of pictures to be encoded is determined according to the second difference of each group of pictures and the encoding parameter of each group of pictures.

In the embodiment of the disclosure, when the second code rate allocation proportion does not meet the requirement that the code rate allocated by the I frame cannot be too high, the encoding parameter of the image group to be encoded can be effectively determined according to the definition difference and the encoding parameter of each image group encoded before.

In a possible implementation manner, in the case that the second code rate allocation ratio exceeds the code rate allocation ratio threshold, determining the encoding parameter of the image group to be encoded according to a second difference between the definitions of the frames of images in each image group in the image group set and the encoding parameter of each image group, includes:

determining the minimum value of the third difference values corresponding to each image group, wherein the third difference value is the maximum value of the second difference values corresponding to each image group;

and taking the encoding parameter of the image group corresponding to the minimum value in the image group set as the encoding parameter of the image group to be encoded.

As described above, the encoded group of pictures includes a key frame and at least one non-key frame, and the second difference between the sharpness of the respective frame pictures may include a difference between the sharpness of the key frame and the sharpness of the at least one non-key frame. That is, the second difference value may include more than one.

In a possible implementation manner, when the image group set includes a plurality of image groups, a maximum value of the second difference values may be determined for each image group, and then a minimum value of the maximum values is determined, so that the image group with the smallest difference in sharpness of each frame image in the image group set is determined. The sharpness difference, i.e. the image quality difference, is minimal and the coding parameters are relatively reasonable.

In a possible implementation manner, the encoding parameter of the image group corresponding to the minimum value in the image group set is used as the encoding parameter of the image group to be encoded, so that relatively reasonable encoding parameters can be applied to the image group to be encoded under the condition that the requirement that the code rate allocated by the I frame cannot be too high is met.

In a possible implementation manner, a group of pictures may also be included in the group of pictures set, that is, only the encoded current group of pictures is included in the group of pictures set, in which case, the encoding parameter of the encoded current group of pictures may be directly used as the encoding parameter of the group of pictures to be encoded.

In the embodiment of the disclosure, the encoding parameters of the image group with the minimum definition difference can be applied to the image group to be encoded to alleviate the generation of the respiratory effect under the condition that the requirement that the code rate allocated to the I frame cannot be too high is met.

Fig. 2 shows a flow diagram of a video encoding method according to an embodiment of the present disclosure. As shown in fig. 2, the method includes:

acquiring a video stream; analyzing the motion intensity of the current image group in the video stream, and encoding the image; judging whether the motion intensity of the current image group exceeds a motion intensity threshold value or not; under the condition that the exercise intensity exceeds the exercise intensity threshold value, the encoding parameters are not adjusted;

under the condition that the motion intensity does not exceed the motion intensity threshold value, carrying out image decoding on the coded current image group; performing definition analysis on a current image group after the image is decoded to obtain definition scores of an I frame and a P frame;

judging whether the definition difference value between the I frame and the P frame in the current image group exceeds a definition threshold value according to the definition scores of the I frame and the P frame; under the condition that the definition difference value does not exceed the definition threshold value, the coding parameters are not adjusted;

adjusting the encoding parameters when the sharpness difference exceeds a sharpness threshold; and feeding the adjusted coding parameters back to the image coding module so as to code the subsequent image group in the video stream according to the adjusted coding parameters.

In a possible implementation manner, the images in the image group may be YUV (a color coding mode) images, but may also be other types of images, for example, a grayscale image, and the embodiment of the present disclosure is not limited thereto.

In a possible implementation manner, the group of GOP pictures can be input into a motion intensity evaluation algorithm, so that the motion intensity of the pictures can be given according to the frame rate and the actual motion situation, and the motion intensity scores of the pictures represent the motion intensity. The motor strength assessment algorithm may use a CNN network (e.g., FlowNet),

in a possible implementation manner, the definition analysis function is started when the motion intensity reaches a set condition value (motion intensity threshold), a normally encoded image group is decoded, whether a video frame in the current image group is an I frame or a P frame and a frame number are recorded, the decoded video frame is sent to a definition analysis algorithm, the definition analysis algorithm gives a current definition score, and the definition score represents the definition. The sharpness detection algorithm may use a CNN network (e.g., GoogleNet) to input the decoded group of images and output a sharpness score for the images.

Fig. 3 shows a flow chart of an encoding parameter adjustment method according to an embodiment of the present disclosure. As shown in fig. 3, the method includes:

determining a definition difference value between the I frame and the P frame through definition analysis; judging whether the definition difference value exceeds a definition threshold value;

under the condition that the definition difference value exceeds the definition threshold value, the code rate allocation proportion of the I frame and the P frame is adjusted according to the preset 1 percent of adjustment step length; judging whether the adjusted code rate distribution proportion exceeds a code rate distribution proportion threshold value or not;

under the condition that the code rate distribution proportion threshold value is not exceeded, determining code rate values distributed by the I frame and the P frame according to the adjusted code rate distribution proportion, wherein the coding parameters comprise the code rate values;

and under the condition that the code rate distribution proportion threshold is exceeded, outputting the coding parameters corresponding to the image group with the minimum definition difference in the coding parameter adjusting process.

In a possible implementation mode, the code rate allocation is calculated according to the current preset coding code rate, the I frame type, the P frame type and the frame sequence number, the allocated code rate value is set to the image coding module, and then the coding parameter adjusting method is repeated for the subsequent image group until the definition difference value of each frame in the image group is adjusted to be not more than the definition threshold value, so that the closed-loop coding parameter adjusting process can be realized.

In a possible implementation manner, the video coding method can be applied to scenes such as intelligent video analysis, intelligent commerce and security monitoring.

According to the embodiment of the disclosure, the problem of respiratory effect caused by unreasonable code rate distribution when a video encoder in the related art encodes a video in a low-code-rate and relatively static scene can be solved. Namely, the method can solve the problem of optimization of the quality of the coded image of a hardware accelerated coder and the problem of image distortion caused by the breathing effect at a low code rate.

According to the embodiment of the disclosure, the definition difference of the currently coded image is analyzed based on deep learning, and the code rate allocation of the I frame and the P frame is dynamically adjusted according to the difference value so as to achieve the minimum definition difference, thereby effectively slowing down the respiratory effect and improving the image quality under the condition of low code rate.

In the field of video monitoring, in a scene with a poor network environment, the encoding code rate of a video can only be reduced in the related technology to improve fluency, and after the video encoding method disclosed by the embodiment of the disclosure is used, the respiratory effect of a relatively static environment image can be greatly improved. According to the video coding method disclosed by the embodiment of the disclosure, optimal code rate distribution can be obtained under the appointed coding code rate.

According to the embodiment of the disclosure, the video can be scored according to the definition and the intensity of the movement (movement strength) by using deep learning, and whether the code rate allocation of dynamic adjustment needs to be started or the code rate allocation proportion of I and P frames needs to be dynamically adjusted is determined according to the scoring result.

It is understood that the above-mentioned method embodiments of the present disclosure can be combined with each other to form a combined embodiment without departing from the logic of the principle, which is limited by the space, and the detailed description of the present disclosure is omitted. Those skilled in the art will appreciate that in the above methods of the specific embodiments, the specific order of execution of the steps should be determined by their function and possibly their inherent logic.

In addition, the present disclosure also provides a video encoding apparatus, an electronic device, a computer-readable storage medium, and a program, which can be used to implement any video encoding method provided by the present disclosure, and the corresponding technical solutions and descriptions and corresponding descriptions in the methods section are not repeated.

Fig. 4 shows a block diagram of a video encoding apparatus according to an embodiment of the present disclosure, as shown in fig. 4, the apparatus including:

a motion intensity analysis module 101, configured to analyze picture motion intensity of a current image group of a video stream, and determine motion intensity of the current image group;

a first encoding parameter determining module 102, configured to determine, according to an encoded current image group, an encoding parameter of an image group to be encoded when the motion intensity does not exceed a motion intensity threshold, where an acquisition time of the image group to be encoded is later than an acquisition time of the current image group;

and the encoding module 103 is configured to encode the image group to be encoded according to the encoding parameter to obtain an encoded image group.

In a possible implementation manner, the first encoding parameter determining module 102 includes: the decoding submodule is used for decoding the coded current image group to obtain each frame image in the decoded current image group and a first code rate allocation proportion of each frame image, wherein the coded current image group is obtained by coding the current image group according to a preset coding rate; and the first coding parameter determining submodule is used for determining the coding parameters of the group of pictures to be coded according to the first code rate distribution proportion.

In a possible implementation manner, the exercise intensity analysis module 101 is specifically configured to input the current image group into a pre-trained first neural network, so as to obtain an exercise intensity score of the current image group, where the exercise intensity score is used to represent an exercise intensity of the current image group.

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.

Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above-mentioned method. The computer readable storage medium may be a non-volatile computer readable storage medium.

An embodiment of the present disclosure further provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the memory-stored instructions to perform the above-described method.

The disclosed embodiments also provide a computer program product comprising computer readable code, which when run on a device, a processor in the device executes instructions for implementing a video encoding method as provided in any of the above embodiments.

The embodiments of the present disclosure also provide another computer program product for storing computer readable instructions, which when executed cause a computer to perform the operations of the video encoding method provided in any of the above embodiments.

The electronic device may be provided as a terminal, server, or other form of device.

Fig. 5 illustrates a block diagram of an electronic device 800 in accordance with an embodiment of the disclosure. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like terminal device.

Referring to fig. 5, electronic device 800 may include one or more of the following components: processing component 802, memory 804, power component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814, and communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing components 802 may include one or more processors 820 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front facing camera and/or a rear facing camera. The front camera and/or the rear camera may receive external multimedia data when the electronic device 800 is in an operation mode, such as a shooting mode or a video mode. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

The sensor assembly 814 includes one or more sensors for providing various aspects of state assessment for the electronic device 800. For example, the sensor assembly 814 may detect an open/closed state of the electronic device 800, the relative positioning of components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, orientation or acceleration/deceleration of the electronic device 800, and a change in the temperature of the electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include a light sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 may access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components for performing the above-described methods.

In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804, is also provided that includes computer program instructions executable by the processor 820 of the electronic device 800 to perform the above-described methods.

Fig. 6 illustrates a block diagram of an electronic device 1900 in accordance with an embodiment of the disclosure. For example, the electronic device 1900 may be provided as a server. Referring to fig. 6, electronic device 1900 includes a processing component 1922 further including one or more processors and memory resources, represented by memory 1932, for storing instructions, e.g., applications, executable by processing component 1922. The application programs stored in memory 1932 may include one or more modules that each correspond to a set of instructions. Further, the processing component 1922 is configured to execute instructions to perform the above-described method.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system, such as the Microsoft Server operating system (Windows Server), stored in the memory 1932^TM) Apple Inc. of the present application based on the graphic user interface operating System (Mac OS X)^TM) Multi-user, multi-process computer operating system (Unix)^TM) Free and open native code Unix-like operating System (Linux)^TM) Open native code Unix-like operating System (FreeBSD)^TM) Or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932, is also provided that includes computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above-described methods.

The present disclosure may be systems, methods, and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied thereon for causing a processor to implement various aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic memory device, a magnetic memory device, an optical memory device, an electromagnetic memory device, a semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present disclosure may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, the electronic circuitry that can execute the computer-readable program instructions implements aspects of the present disclosure by utilizing the state information of the computer-readable program instructions to personalize the electronic circuitry, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA).

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

Having described embodiments of the present disclosure, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A video encoding method, comprising:

analyzing the picture motion intensity of a current image group of a video stream, and determining the motion intensity of the current image group;

under the condition that the motion intensity does not exceed a motion intensity threshold, determining encoding parameters of an image group to be encoded according to an encoded current image group, wherein the acquisition time of the image group to be encoded is after the acquisition time of the current image group;

and coding the image group to be coded according to the coding parameters to obtain a coded image group.

2. The method according to claim 1, wherein determining the encoding parameters of the group of images to be encoded according to the encoded current group of images in the case that the motion intensity does not exceed the motion intensity threshold comprises:

decoding the encoded current image group to obtain each frame image in the decoded current image group and a first code rate allocation proportion of each frame image, wherein the encoded current image group is obtained by encoding the current image group according to a preset encoding code rate;

and determining the coding parameters of the group of pictures to be coded according to the first code rate distribution proportion.

3. The method according to claim 2, wherein said determining the encoding parameters of the group of pictures to be encoded according to the first rate allocation ratio comprises:

4. The method according to claim 3, wherein the adjusting the first code rate allocation ratio according to the sharpness of each frame of image to determine the encoding parameters of the group of images to be encoded comprises:

and determining the encoding parameters of the group of pictures to be encoded according to the second code ratio distribution proportion.

5. The method according to claim 4, wherein said determining the encoding parameters of the group of pictures to be encoded according to the second code rate allocation ratio comprises:

and under the condition that the second code rate distribution proportion does not exceed a code rate distribution proportion threshold, determining a code rate value corresponding to each frame of image according to the second code rate distribution proportion and a preset coding code rate, wherein the coding parameters of the image group to be coded comprise the code rate value corresponding to each frame of image.

6. The method according to claim 4 or 5, characterized in that the method further comprises:

determining the encoding parameters of the group of pictures to be encoded according to a second difference between the sharpness of each frame of picture in each group of pictures in the group of pictures set and the encoding parameters of each group of pictures, if the second code rate allocation ratio exceeds a code rate allocation ratio threshold,

wherein the set of image groups comprises encoded image groups for which the motion intensity does not exceed the motion intensity threshold.

7. The method according to claim 6, wherein in the case that the second code rate allocation ratio exceeds the code rate allocation ratio threshold, determining the encoding parameters of the group of pictures to be encoded according to a second difference between the sharpness of each frame picture in each group of pictures in the group of pictures and the encoding parameters of each group of pictures comprises:

determining the minimum value of third difference values corresponding to each image group, wherein the third difference value is the maximum value of the second difference values corresponding to each image group;

8. The method of claim 4, wherein each frame image comprises a key frame and at least one non-key frame; the first difference comprises a difference between the sharpness of the key frame and the sharpness of the non-key frame.

9. The method of claim 1, wherein analyzing the picture motion strength of the current group of pictures of the video stream to determine the motion strength of the current group of pictures comprises:

and inputting the current image group into a pre-trained first neural network to obtain a motion intensity score of the current image group, wherein the motion intensity score is used for representing the motion intensity of the current image group.

10. The method according to claim 4, wherein the analyzing the sharpness of the frame images to determine the sharpness of the frame images comprises:

and inputting the frame images into a pre-trained second neural network to obtain the definition scores of the frame images, wherein the definition scores are used for representing the definition of the frame images.

11. A video encoding apparatus, comprising:

the motion intensity analysis module is used for analyzing the picture motion intensity of the current image group of the video stream and determining the motion intensity of the current image group;

the encoding parameter determining module is used for determining encoding parameters of an image group to be encoded according to an encoded current image group under the condition that the motion intensity does not exceed a motion intensity threshold, wherein the acquisition time of the image group to be encoded is behind the acquisition time of the current image group;

and the coding module is used for coding the image group to be coded according to the coding parameters to obtain a coded image group.

12. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to invoke the memory-stored instructions to perform the method of any one of claims 1 to 10.

13. A computer readable storage medium having computer program instructions stored thereon, which when executed by a processor implement the method of any one of claims 1 to 10.