CN112954348B

CN112954348B - Video encoding method and device, electronic equipment and storage medium

Info

Publication number: CN112954348B
Application number: CN202110194279.1A
Authority: CN
Inventors: 周波; 吴佳飞; 李树析; 张广程; 闫俊杰
Original assignee: Zhejiang Shangtang Technology Development Co Ltd
Current assignee: Zhejiang Shangtang Technology Development Co Ltd
Priority date: 2021-02-20
Filing date: 2021-02-20
Publication date: 2024-04-02
Anticipated expiration: 2041-02-20
Also published as: CN112954348A

Abstract

The present disclosure relates to a video encoding method and apparatus, an electronic device, and a storage medium, the method comprising: analyzing the picture motion intensity of the current image group of the video stream, and determining the motion intensity of the current image group; under the condition that the motion intensity does not exceed the motion intensity threshold value, determining coding parameters of an image group to be coded according to the coded current image group, wherein the acquisition time of the image group to be coded is after the acquisition time of the current image group; and encoding the image group to be encoded according to the encoding parameters to obtain an encoded image group. The embodiment of the disclosure can realize effective prediction of the coding parameters of the image group to be coded, so that the coding parameters of the image group to be coded tend to be reasonable, thereby being beneficial to alleviating the generation of respiratory effect.

Description

Video encoding method and device, electronic equipment and storage medium

Technical Field

The disclosure relates to the field of computer technology, and in particular, to a video encoding method and apparatus, an electronic device, and a storage medium.

Background

Currently, with the development of technology, in order to facilitate storage, transmission and communication of video files, video encoding processing is required for video. Video coding processes typically allocate coding rates (i.e., the number of coded bits) to image frames in video.

The related art aims at the video with the video pictures in the relatively static scene, the coding code rate distribution is not reasonable enough when the video coding is carried out, and the breathing effect of the video pictures can be generated when the video is coded under the unreasonable code rate distribution.

Disclosure of Invention

The present disclosure proposes a technical solution for video coding.

According to an aspect of the present disclosure, there is provided a video encoding method including: analyzing the picture motion intensity of a current image group of a video stream, and determining the motion intensity of the current image group; under the condition that the motion intensity does not exceed a motion intensity threshold value, determining coding parameters of an image group to be coded according to a coded current image group, wherein the acquisition time of the image group to be coded is after the acquisition time of the current image group; and coding the image group to be coded according to the coding parameters to obtain a coded image group. According to the embodiment of the disclosure, the coding parameters of the image group to be coded can be effectively predicted, so that the coding parameters of the image group to be coded tend to be reasonable, and the breathing effect is favorably relieved.

In a possible implementation manner, the determining, according to the encoded current image group, the encoding parameter of the image group to be encoded if the motion intensity does not exceed the motion intensity threshold value includes: decoding the encoded current image group to obtain each frame image in the decoded current image group and a first code rate allocation proportion of each frame image, wherein the encoded current image group is obtained by encoding the current image group according to a preset encoding code rate; and determining the coding parameters of the image group to be coded according to the first code rate distribution proportion. By the method, whether the code rate distribution of the current image group is reasonable or not can be determined, and further, the unreasonable code rate distribution proportion can be effectively adjusted, so that the coding parameters of the image group to be coded tend to be reasonable.

In a possible implementation manner, the determining, according to the first code rate allocation ratio, the coding parameter of the to-be-coded image group includes: analyzing the definition of each frame of image to determine the definition of each frame of image; and adjusting the first code rate distribution proportion according to the definition of each frame of image so as to determine the coding parameters of the image group to be coded. By the method, the image quality difference between the key frames and the non-key frames in the coded image group to be coded can be reduced, and the effect of relieving respiratory effect is achieved.

In a possible implementation manner, the adjusting the first code rate allocation proportion according to the sharpness of each frame image to determine the coding parameters of the image group to be coded includes: determining a first difference between the sharpness of the frames of images; under the condition that a first difference value exceeding a definition threshold exists, adjusting the first code rate allocation proportion according to a preset adjusting step length to obtain an adjusted second code rate allocation proportion; and determining the coding parameters of the image group to be coded according to the second code rate distribution proportion. By the method, under the condition that the definition difference between the frames of images is large, the coding parameters of the image group to be coded, which are determined according to the second code rate distribution proportion, tend to be reasonable, and therefore the occurrence of respiratory effect is effectively relieved.

In a possible implementation manner, the determining, according to the second code rate allocation ratio, the coding parameter of the to-be-coded image group includes: judging whether the second code rate allocation proportion exceeds a code rate allocation proportion threshold value or not; and under the condition that the second code rate allocation proportion does not exceed a code rate allocation proportion threshold value, determining a code rate value corresponding to each frame of image according to the second code rate allocation proportion and a preset code rate, wherein the code parameters of the image group to be coded comprise the code rate value corresponding to each frame of image. By the mode, the upper limit of the adjusted second code rate allocation proportion can be effectively controlled, and the situations that the coded image quality and video coding efficiency are reduced due to the fact that the code rate allocated by an I frame is too high are reduced.

In one possible implementation, the method further includes: and under the condition that the second code rate distribution proportion exceeds a code rate distribution proportion threshold value, determining the coding parameters of the image group to be coded according to a second difference value between the definition of each frame image in each image group in an image group set and the coding parameters of each image group, wherein the image group set comprises the coded image group with the motion intensity not exceeding the motion intensity threshold value. By the method, when the second code rate allocation proportion does not meet the requirement that the code rate allocated by the I frame cannot be too high, the coding parameters of the image groups to be coded can be effectively determined according to the definition difference and the coding parameters of the previously coded image groups.

In one possible implementation manner, the determining, when the second bitrate allocation ratio exceeds a bitrate allocation ratio threshold, the coding parameters of the to-be-coded image group according to the second difference between the resolutions of the frame images in each image group in the image group set and the coding parameters of each image group includes: determining the minimum value of third difference values corresponding to the image groups, wherein the third difference value is the maximum value of the second difference values corresponding to the image groups; and taking the coding parameters of the image group corresponding to the minimum value in the image group set as the coding parameters of the image group to be coded. By the method, the coding parameters of the image group with the minimum definition difference can be applied to the image group to be coded under the condition that the requirement that the code rate of I frame allocation cannot be too high is met, so that the breathing effect is relieved.

In one possible implementation, each frame of image includes a key frame and at least one frame of non-key frame; the first difference comprises a difference between the sharpness of the key frame and the sharpness of the non-key frame.

In one possible implementation manner, the analyzing the picture motion intensity of the current image group of the video stream to determine the motion intensity of the current image group includes: and inputting the current image group into a pre-trained first neural network to obtain a motion intensity score of the current image group, wherein the motion intensity score is used for representing the motion intensity of the current image group. By the method, the motion intensity of the current image group can be obtained effectively and accurately.

In one possible implementation manner, the analyzing the sharpness of each frame of image to determine the sharpness of each frame of image includes: and inputting each frame of image into a pre-trained second neural network to obtain a definition score of each frame of image, wherein the definition score is used for representing the definition of each frame of image. By the mode, the definition of each frame of image can be effectively and accurately obtained, and the first code rate distribution proportion can be accurately adjusted according to the accurate definition.

According to an aspect of the present disclosure, there is provided a video encoding apparatus including: the motion intensity analysis module is used for analyzing the picture motion intensity of the current image group of the video stream and determining the motion intensity of the current image group; the first coding parameter determining module is used for determining coding parameters of an image group to be coded according to the coded current image group under the condition that the motion intensity does not exceed a motion intensity threshold value, and the acquisition time of the image group to be coded is after the acquisition time of the current image group; and the coding module is used for coding the image group to be coded according to the coding parameters to obtain a coded image group.

In one possible implementation manner, the first coding parameter determining module includes: the decoding submodule is used for decoding the coded current image group to obtain each frame image in the decoded current image group and a first code rate allocation proportion of each frame image, wherein the coded current image group is obtained by coding the current image group according to a preset coding code rate; and the first coding parameter determining submodule is used for determining the coding parameters of the image group to be coded according to the first code rate distribution proportion.

In one possible implementation, the first coding parameter determining submodule includes: the definition analysis unit is used for analyzing the definition of each frame of image and determining the definition of each frame of image; and the adjusting unit is used for adjusting the first code rate distribution proportion according to the definition of each frame of image so as to determine the coding parameters of the image group to be coded.

In one possible implementation, the adjusting unit is specifically configured to: determining a first difference between the sharpness of the frames of images; under the condition that a first difference value exceeding a definition threshold exists, adjusting the first code rate allocation proportion according to a preset adjusting step length to obtain an adjusted second code rate allocation proportion; and determining the coding parameters of the image group to be coded according to the second code rate distribution proportion.

In a possible implementation manner, the determining, according to the second code rate allocation ratio, the coding parameter of the to-be-coded image group includes: judging whether the second code rate allocation proportion exceeds a code rate allocation proportion threshold value or not; and under the condition that the second code rate allocation proportion does not exceed a code rate allocation proportion threshold value, determining a code rate value corresponding to each frame of image according to the second code rate allocation proportion and a preset code rate, wherein the code parameters of the image group to be coded comprise the code rate value corresponding to each frame of image.

In one possible implementation, the method further includes: and the second coding parameter determining module is used for determining the coding parameters of the image groups to be coded according to the second difference value between the definition of each frame image in each image group in the image group set and the coding parameters of each image group under the condition that the second code rate distribution proportion exceeds a code rate distribution proportion threshold value, wherein the image group set comprises the coded image groups with the motion intensity not exceeding the motion intensity threshold value.

In one possible implementation manner, the second coding parameter determining module includes: the determining submodule is used for determining the minimum value in third difference values corresponding to the image groups, wherein the third difference value is the maximum value in the second difference values corresponding to the image groups; and the second coding parameter determining submodule is used for taking the coding parameters of the image group corresponding to the minimum value in the image group set as the coding parameters of the image group to be coded.

In a possible implementation manner, the motion intensity analysis module is specifically configured to input the current image set into a pre-trained first neural network, and obtain a motion intensity score of the current image set, where the motion intensity score is used to characterize a motion intensity of the current image set.

In a possible implementation manner, the sharpness analysis unit is specifically configured to input the respective frame of images into a pre-trained second neural network, and obtain sharpness scores of the respective frame of images, where the sharpness scores are used to characterize sharpness of the respective frame of images.

According to an aspect of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the above method.

According to an aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method.

In the embodiment of the disclosure, whether to adjust the coding parameters of the image group to be coded can be judged according to the motion intensity of the current image group of the video stream, so that the coding parameters can be adjusted for the image group with low picture motion intensity, i.e. the picture in a relatively static scene; and the coding parameters of the image group to be coded can be effectively predicted according to the coded current image group, so that the coding parameters of the image group to be coded tend to be reasonable, thereby being beneficial to relieving the generation of respiratory effect.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the technical aspects of the disclosure.

Fig. 1 shows a flowchart of a video encoding method according to an embodiment of the present disclosure.

Fig. 2 shows a flowchart of a video encoding method according to an embodiment of the present disclosure.

Fig. 3 shows a flowchart of an encoding parameter adjustment method according to an embodiment of the present disclosure.

Fig. 4 shows a block diagram of a video encoding apparatus according to an embodiment of the present disclosure.

Fig. 5 shows a block diagram of an electronic device, according to an embodiment of the disclosure.

Fig. 6 shows a block diagram of an electronic device, according to an embodiment of the disclosure.

Detailed Description

Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, specification, and drawings of this disclosure are used for distinguishing between different objects and not for describing a particular sequential order. The terms "comprises" and "comprising" when used in the specification and claims of this disclosure are taken to specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Furthermore, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.

In the field of video coding, key frames (independent frames with all information, called I frames) and non-key frames (e.g. P frames, recording the difference from the previous frame; B frames, recording the difference from the previous and subsequent frames) are usually provided in order to save storage space. The code rate allocation has an important effect on the image quality of the encoded video, and in particular relates to the code rate allocation of I frames (intra-prediction frames, key frames). If the allocated code rate of the I frame is too low, the image quality of the I frame is reduced, and the image quality of the non-I frame (forward inter-frame predicted frame P frame, bi-directional inter-frame predicted frame B frame) following the I frame is also affected by the image quality of the I frame due to the inter-frame predicted coding mode in video coding, thereby reducing the overall image quality of the video and possibly generating obvious respiratory effects. If the I frame code rate is too large, the video coding efficiency is reduced, the coded image quality is also affected, and meanwhile, the I frame with the too high code rate can cause network delay and packet loss to cause video blocking; reasonable code rate allocation is therefore important for video coding efficiency and coded image quality.

In some scenarios, since video coding generally requires high real-time processing performance, code rate allocation can be performed by prediction. While the related art generally determines the coding parameters of the current group of pictures from the average coding parameters of the last group of pictures (GOP, group of Pictures) at the time of prediction code rate allocation. The method is simple, but cannot output reasonable coding parameters according to video characteristics (such as different picture motion intensities and different picture complexity), and cannot effectively cope with the generation of respiratory effects.

The respiratory effect in the video means that the image quality is improved by inserting the I frame, the image quality is gradually reduced after switching to the non-I frame, and then the image quality is suddenly improved by inserting the I frame. The respiratory effect is mainly caused by the difference of coding modes and image quality of I frames and P frames, and unreasonable code rate distribution is added, so that the image distortion degree is different, and visual image discontinuity is caused. The respiratory effect is easier to observe when the picture is in a relatively static scene, and particularly, the respiratory effect is more obvious under the conditions of low code rate and relatively static scene; in a sports scene, most of the contents in the picture are changed, so that the visual observation is not easy.

According to the embodiment of the disclosure, for the image group with low picture motion strength (i.e., the picture is relatively still), the sharpness difference of the I frame and the non-I frame images in the current encoded image group can be analyzed, and then the code rate distribution of the image group to be encoded is adjusted according to the sharpness difference, so that the sharpness difference trend of the I frame and the non-I frame images in the subsequent image group is minimized, and the respiratory effect in the video is effectively relieved.

According to the embodiment of the disclosure, not only can the respiratory effect be effectively relieved, but also the method can be suitable for various video encoders, the encoding efficiency can meet the requirement of instantaneity, the quality of the encoded image is ensured under the specified code rate, and the loss of the two-round encoding on the quality of the image can be avoided.

Fig. 1 shows a flowchart of a video encoding method according to an embodiment of the present disclosure, as shown in fig. 1, including:

in step S11, the picture motion intensity of the current image group of the video stream is analyzed to determine the motion intensity of the current image group;

in step S12, under the condition that the motion intensity does not exceed the motion intensity threshold, determining the encoding parameters of the image group to be encoded according to the encoded current image group, wherein the acquisition time of the image group to be encoded is after the acquisition time of the current image group;

In step S13, the image group to be encoded is encoded according to the encoding parameter, so as to obtain an encoded image group to be encoded.

In a possible implementation manner, the video encoding method may be performed by an electronic device such as a terminal device or a server, where the terminal device may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc., and the method may be implemented by a processor invoking computer readable instructions stored in a memory. Alternatively, the method may be performed by a server.

In one possible implementation, the video stream may be acquired in real time by an image acquisition device (e.g., various cameras) connected to the electronic device; the method may also be transmitted to the electronic device by other devices, or obtained by the electronic device invoking local storage, which is not limited in the embodiments of the present disclosure.

In one possible implementation, the image groups may be partitioned by frame rate, e.g., based on a known frame rate of the video stream, each image group may include: and multi-frame images within a preset time period of 1 second or 2 seconds and the like. It will be appreciated that the group of images may comprise one or more images in a video stream, and that the plurality of images may be arranged in a time series. The specific value of the preset time period may be determined according to the coding performance, the coding mode, etc. of the video encoder, which is not limited in the embodiments of the present disclosure.

In one possible implementation, the video stream may be acquired in real-time or non-real-time. For a video stream acquired in real time, the current image group may include an image group acquired within a current preset time period in the video stream; for a video stream that is not acquired in real time, the current image group may refer to an image group in the video stream for any preset period of time, which is not limited in the embodiments of the present disclosure.

It will be appreciated that the video stream comprises a plurality of frames of images acquired in chronological order of the acquisition time, i.e. the plurality of frames of images in the video stream are arranged in time series. The acquisition time of the group of images to be encoded is after the acquisition time of the current group of images, it is understood that the acquisition time of the images in the group of images to be encoded is after the current group of images. The image group to be encoded may be a continuous image group after the current image group, or may be an image group to be encoded obtained by performing interval sampling, which is not limited in the embodiment of the present disclosure.

In one possible implementation, for a video stream acquired in real-time, the current group of images may include a group of images in the currently acquired video stream; the group of pictures to be encoded may comprise a group of pictures to be acquired immediately after the current group of pictures. It will be appreciated that when a group of images to be encoded in a video stream is acquired, the acquired group of images to be encoded is also the current group of images. In other words, each group of images transmitted in the video stream can be subjected to motion intensity analysis, so that the encoding parameters can be effectively adjusted for the group of images with low picture motion intensity.

In one possible implementation, the picture motion strength may refer to the degree of variation in content in a video picture. It can be understood that in a relatively static scene, the content in the picture does not change to a high degree, that is, the picture motion intensity is not high; in a moving scene, the content in the picture has a higher degree of change, i.e., the picture has a higher moving intensity.

The picture motion intensity is relatively low in, for example, a conference scene, compared with a running scene. Those skilled in the art will understand, or at least will understand, the meaning of the intensity of motion of a picture upon reading the embodiments of the present disclosure.

In one possible implementation manner, in step S11, analyzing the picture motion intensity of the current image group of the video stream to determine the motion intensity of the current image group may include: and inputting the current image group into a pre-trained first neural network to obtain a motion intensity score of the current image group, wherein the motion intensity score is used for representing the motion intensity of the current image group.

In one possible implementation, the first neural network may, for example, at least employ: convolutional neural network FlowNet for learning optical flow. The use of motion information in the optical flow can be achieved by FlowNet, outputting the motion intensity score for the image group. The training method and specific structure of the first neural network are not limited in the embodiments of the present disclosure. Of course, other types of neural networks may be employed as long as the intensity of motion of the image set can be analyzed.

In one possible implementation, for example, the similarity between each frame of image in the current image group may also be analyzed by the first neural network; and scoring the picture motion intensity according to the analysis result of the similarity to obtain a motion intensity score. It can be understood that the higher the similarity, the lower the motion intensity score, which can represent that the motion degree of the picture is not high; conversely, the higher the exercise intensity score.

In one possible implementation, the motion intensity score output by the first neural network may be trained to correlate positively with the motion intensity, i.e., the lower the motion intensity score, the lower the representative picture motion intensity; conversely, the higher the motion intensity score, the higher the representative picture motion intensity. Of course, the correlation may be negative, and may be specifically set according to the training mode and the actual requirement of the neural network, which is not limited in the embodiments of the disclosure.

In the embodiment of the disclosure, the pre-trained first neural network is used for analyzing the picture motion intensity of the current image group, so that the motion intensity of the current image group can be effectively and accurately obtained.

In one possible implementation, in step S12, the exercise intensity threshold may be set according to actual requirements, historical experience, and the like, which is not limited by the embodiments of the present disclosure.

As described above, the athletic strength score may characterize athletic strength. In one possible implementation, the exercise intensity does not exceed the exercise intensity threshold, and may include: the exercise intensity score does not exceed the exercise intensity threshold.

As described above, the respiratory effect is more remarkable in the video picture in the relatively stationary scene. For the case where the motion intensity does not exceed the motion intensity threshold, it can be considered that in this case the picture motion intensity of the current image group is not high, and a relatively still scene may be in the picture of the current image group. Therefore, aiming at the image group with low picture motion intensity, the coding parameters of the image group to be coded can be predicted according to the coded current image group, so as to realize adjustment and optimization of the coding parameters of the image group to be coded.

In a possible implementation manner, in step S12, the encoded current image group may be obtained by encoding the current image group through an existing video encoder according to a preset encoding rate. The embodiment of the present disclosure is not limited to the encoding mode of the current image group. The group of pictures to be encoded may particularly refer to the group of pictures in the video stream that is not encoded after the current group of pictures.

In one possible implementation manner, through the encoded current image group, whether the code rate allocation of the encoded current image group is reasonable or not can be known, so that effective adjustment of the coding parameters can be performed aiming at unreasonable code rate allocation, and the adjusted coding parameters are obtained; and then the adjusted coding parameters are determined as the coding parameters of the image group to be coded, so that the code rate distribution trend of the image group to be coded is reasonable.

In one possible implementation manner, there may be a case where the motion intensity of the current image group exceeds the motion intensity threshold, in which case, the picture corresponding to the current image group may be considered to be in a motion scene, where the encoding parameters of the current image group may not be adjusted, and still the subsequent image group to be encoded is encoded by the video encoder in the current encoding manner. It will be appreciated that due to the continuity of image frames in the video stream, the current group of images is in a moving or relatively still scene, after which the group of images also corresponds with a high probability to be in a moving or relatively still scene.

In one possible implementation, the coding parameters may include a code rate value allocated for each frame of the image group, for example, the preset coding rate is 800kbps, the code rate value allocated for the I frame in the image group may be 400kbps, and the code rate values allocated for the 2P frames may be 200kbps. In step S13, the image group to be encoded is encoded according to the encoding parameters, so as to obtain an encoded image group to be encoded, which may be encoded according to the code rate value allocated to each frame of image in the encoding parameters, so as to obtain an encoded image group to be encoded. The image group to be encoded may be encoded by using the existing video encoder, for example, and the embodiment of the disclosure is not limited thereto.

In a possible implementation manner, in step S12, the determining, according to the encoded current image group, the encoding parameters of the image group to be encoded in the case that the motion intensity does not exceed the motion intensity threshold value includes:

decoding the encoded current image group to obtain each frame image in the decoded current image group and a first code rate allocation proportion of each frame image, wherein the encoded current image group is obtained by encoding the current image group according to a preset encoding code rate; and determining the coding parameters of the image group to be coded according to the first code rate distribution proportion.

In one possible implementation, each frame image in the current image group may include a key frame and at least one frame of non-key frames. Wherein the key frames may include I frames and the non-key frames may include P frames and/or B frames.

In one possible implementation, as described above, the encoded current image group may be obtained by encoding the current image group through a video encoder according to a preset encoding rate.

In one possible implementation, the preset coding rate may be set according to an actual video coding requirement, and the embodiment of the present disclosure is not limited, and the preset coding rate may be used to code the entire video stream.

In one possible implementation, decoding the encoded current group of pictures may be implemented by a video decoder corresponding to the video encoder described above. The embodiment of the disclosure does not limit the decoding mode of the encoded current image group.

In one possible implementation manner, after decoding the encoded current image group, each frame image and the code rate corresponding to each frame image can be obtained, and then the first code rate allocation proportion can be known according to the code rate corresponding to each frame image.

In one possible implementation, the first rate allocation ratio may be a ratio between the rates of the respective frame images and a preset encoding rate. For example, the preset coding rate is 800kbps, the I frame allocation rate is 400kbps, the 2P frames allocation rates are 200kbps, and the first rate allocation ratio may include 50%, 25% and 25%.

In one possible implementation manner, determining the coding parameters of the image group to be coded according to the first code rate allocation proportion may include: adjusting the first code rate allocation proportion to obtain an adjusted code rate allocation proportion; and then, according to the adjusted code rate allocation proportion and the preset code rate, the code rate of each frame of image is allocated, and the coding parameters of the image group to be coded are determined.

In one possible implementation manner, when the code rate allocation of the current image group is adjusted to be reasonable, the current first code rate allocation proportion may not be adjusted, so that the image group to be encoded is encoded according to the encoding parameters of the current image group.

In a possible implementation manner, when the encoded current image group is decoded, the sequence number of each frame image may be obtained, each frame image may be distinguished by the sequence number, and the arrangement sequence of each frame image is identified, so that the first code rate allocation proportion of each frame image and the encoding parameter of the second image may be determined conveniently.

In the embodiment of the disclosure, by decoding the encoded current image group to obtain each frame image in the decoded current image group and the first code rate allocation proportion of each frame image, whether the code rate allocation of the current image group is reasonable or not can be determined, and further, effective adjustment can be performed on the unreasonable code rate allocation proportion, so that the coding parameters of the image group to be encoded tend to be reasonable.

In one possible implementation manner, the determining the coding parameters of the image group to be coded according to the first code rate allocation proportion includes:

analyzing the definition of each frame of image to determine the definition of each frame of image;

and adjusting the first code rate distribution proportion according to the definition of each frame of image so as to determine the coding parameters of the image group to be coded.

As described above, the breathing effect occurs mainly due to the difference in image quality between I-frames and non-I-frames. In one possible implementation, the picture quality may be reflected by sharpness, and whether the code rate allocation is reasonable is reflected by the sharpness difference (i.e., picture quality difference) between the I-frames and the non-I-frames. And the first code rate distribution proportion can be adjusted according to the definition difference of the I frame image and the non-I frame image so as to determine the coding parameters of the image group to be coded.

In one possible implementation, the analyzing the sharpness of each frame of image to determine the sharpness difference of each frame of image may include: and inputting each frame of image into a pre-trained second neural network to obtain a definition score of each frame of image, wherein the definition score is used for representing the definition of each frame of image.

In one possible implementation, the second neural network may comprise a convolutional neural network, e.g., a type of convolutional neural network such as GoogLeNet, alexNet, VGG may be employed. The embodiments of the present disclosure are not limited with respect to the training manner and specific structure of the second neural network.

In one possible implementation, the sharpness score output by the second neural network may be trained to correlate positively with sharpness, i.e., the higher the sharpness score, the sharper the representative image and the better the image quality; conversely, the less sharp the representative image, the poorer the image quality. Of course, the correlation may be negative, and may be specifically set according to the training mode and the actual requirement of the neural network, which is not limited in the embodiments of the disclosure.

In the embodiment of the disclosure, the definition of each frame of image is analyzed through the pre-trained second neural network, so that the definition of each frame of image can be effectively and accurately obtained, and the first code rate allocation proportion can be accurately adjusted according to the accurate definition.

In one possible implementation manner, the adjusting the first code rate allocation proportion according to the definition of each frame image to determine the coding parameters of the image group to be coded may include: correspondingly adjusting a first code rate allocation proportion according to the definition scores of the frames of images; and determining the coding parameters of the image group to be coded according to the adjusted code rate distribution proportion.

In the embodiment of the disclosure, the first code rate allocation proportion can be effectively adjusted according to the definition of each frame of image so as to determine the coding parameters of the image group to be coded, so that the image quality difference between the key frames and the non-key frames in the coded image group to be coded is reduced, and the effect of relieving the respiratory effect is achieved.

In a possible implementation manner, the adjusting the first code rate allocation proportion according to the sharpness of each frame image to determine the coding parameters of the image group to be coded includes:

determining a first difference between the sharpness of each frame of image;

under the condition that a first difference value exceeding a definition threshold exists, adjusting the first code rate allocation proportion according to a preset adjusting step length to obtain an adjusted second code rate allocation proportion;

and determining the coding parameters of the image group to be coded according to the second code rate distribution proportion.

As described above, each frame of image may include a key frame and at least one frame of non-key frames. Wherein the key frames may include I frames and the non-key frames may include P frames and/or B frames. In one possible implementation, the first difference may include a difference between a sharpness of the key frame and a sharpness of the at least one non-key frame. For example, the frame images in the current image group a are arranged in time series as "I frame, P ₁ Frame, P ₂ Frame, P ₃ Frame, P ₄ Frame ", the first difference value may include: i frame sharpness and P ₁ Difference between frame definitions, I-frame definition and P ₂ Difference between frame definitions, I-frame definition and P ₃ Difference between frame definitions, I-frame definition and P ₄ At least one of differences between frame resolutions.

As described above, the sharpness of each frame image may be characterized by a sharpness score. In one possible implementation, the first difference between the sharpness of each frame of image may include: a first difference between sharpness scores of the frames of images.

In one possible implementation, the sharpness threshold may be set according to actual requirements, historical experience, etc., and the embodiments of the present disclosure are not limited in this regard.

In one possible implementation, for the case where the first difference exceeds the sharpness threshold, the sharpness gap between the key frame and the non-key frame may be considered to be large, and correspondingly, the code rate allocation of the current image group may be considered to be unreasonable, in which case the first code allocation ratio may be adjusted.

In one possible implementation manner, for the case that the first difference value does not exceed the sharpness threshold, the sharpness gap between the key frame and the non-key frame may be considered to be small, and correspondingly, the code rate allocation of the current image group may be considered to be reasonable, where the first code rate allocation proportion may not be adjusted, so that the image group to be encoded may be encoded according to the encoding parameter of the current image group.

In one possible implementation, determining whether the first difference value exceeds the sharpness threshold value includes: and judging whether the difference value between the key frame and the continuous multi-frame non-key frame exceeds a definition threshold. Wherein the embodiments of the present disclosure are not limited to a particular number of non-key frames. For example, it may be set that the differences between the sharpness of the P frames and the sharpness of the I frames of consecutive 3 frames both exceed the sharpness threshold, determining that there is a first difference that exceeds the sharpness threshold.

For example, following the example of the current image group a, setting a plurality of frames to 3 frames, there is a first difference value exceeding the sharpness threshold, which may be: i frame sharpness and P ₁ Difference between frame definitions, I-frame definition and P ₂ Difference between frame definitions, I-frame definition and P ₃ The differences between the frame definitions exceed the definition threshold; it may also be: i frame sharpness and P ₂ Difference between frame definitions, I-frame definition and P ₃ Difference between frame definitions, I-frame definition and P ₄ The differences between the frame definitions exceed the definition threshold.

In one possible implementation manner, the preset adjustment step may be set according to actual requirements, a specific form of the code rate allocation proportion, and the like, for example, the adjustment step may be set to 1%, which is not limited in this embodiment of the disclosure.

It is known that, because of the inter-prediction coding mode in video coding, P frames are usually coded and decoded with I frames or P frames of previous frames as reference frames, the image quality of non-I frames is affected by the image quality of I frames, and the higher the image quality of I frames, the higher the image quality of the relative non-I frames. In one possible implementation manner, adjusting the first code rate allocation proportion according to a preset adjustment step length to obtain a second code rate allocation proportion may include: and increasing the code rate allocation proportion of the key frames (I frames) according to a preset adjustment step length, and correspondingly reducing the code rate allocation proportion of the non-key frames (P frames and B frames) to obtain a second code rate allocation proportion.

For example, taking the example that the first rate allocation ratio includes 50%, 25% and 25%, the adjustment step is set to 1%, and the rate allocation ratio of the key frame (I frame) is increased according to the preset adjustment step, the rate allocation ratio of the I frame may be obtained to be 51%, and correspondingly, the rate allocation ratio of the first P frame may still be 25%, the rate allocation ratio of the second P frame may be reduced to 24%, and the second rate allocation ratio may include 51%, 25% and 24%.

In one possible implementation, the second code rate allocation proportion may be a code rate allocation proportion obtained after increasing the adjustment step size based on the first code rate classification proportion, that is, the second code rate allocation proportion may be greater than the first code rate classification proportion.

It should be noted that, the above manner of increasing the first code rate allocation ratio is a manner of adjusting the code rate allocation ratio provided by the embodiments of the present disclosure, and those skilled in the art will understand that the present disclosure should not be limited thereto. In practice, a person skilled in the art may set a mode of adjusting the code rate allocation ratio according to actual requirements, so long as the sharpness difference between the key frame and the non-key frame in the image group can be reduced.

In one possible implementation, after increasing the rate allocation proportion of the key frame according to the adjustment step, the rate allocation proportion of the multi-frame non-I frame may be determined based on a preset adjustment policy. For example, it may be preset to decrease the rate allocation proportion from the non-key frame of the last frame, that is, the rate allocation proportion of the non-key frame of the last frame is currently decreased, the rate allocation proportion of the non-key frame of the next-to-last frame is decreased next time, and so on; the reduction of the rate allocation ratio from the first frame non-key frame may also be preset. The adjustment strategy of the non-I frame can be specifically set according to the actual requirements, the coding mode, the preset coding rate and the like, and the embodiment of the disclosure is not limited.

In one possible implementation manner, determining the coding parameters of the image group to be coded according to the second code rate allocation proportion may include: and distributing the preset coding code rate according to the second code rate distribution proportion to obtain the code rate value of each frame of image distribution, namely determining the coding parameters of the image group to be coded, for example, the preset coding code rate is 800kbps, and distributing according to 51%, 25% and 24%, so that the I frame distribution code rate is 408kbps, and the two P frame distribution code rates are 200kbps and 192kbps respectively.

In the embodiment of the disclosure, under the condition that the difference of definition between each frame of image is large, the first code rate distribution ratio is adjusted according to the preset adjustment step length to obtain the second code rate distribution ratio, so that the coding parameters of the image group to be coded, which are determined according to the second code rate distribution ratio, tend to be reasonable, and the respiratory effect is effectively relieved.

In one possible implementation manner, the determining the coding parameters of the image group to be coded according to the second code rate allocation proportion includes:

judging whether the second code rate allocation proportion exceeds a code rate allocation proportion threshold value or not;

and under the condition that the second code rate allocation proportion does not exceed the code rate allocation proportion threshold value, determining the code rate value corresponding to each frame of image according to the second code rate allocation proportion and a preset code rate, wherein the code parameters of the image group to be coded comprise the code rate value corresponding to each frame of image.

As described above, the code rate of I-frame allocation cannot be too high. Since the above-mentioned method can be adopted to increase the rate allocation ratio of the I frame when the first rate allocation ratio is adjusted according to the adjustment step size. In this way, the rate allocation proportion of the I frame after adjustment may be too high, so that the rate allocated by the I frame is too high.

In one possible implementation manner, by setting the code rate allocation proportion threshold and judging whether the second code rate allocation proportion exceeds the code rate allocation proportion threshold, the upper limit of the adjusted second code rate allocation proportion can be effectively controlled, and the situation that the code rate of I frame allocation is too high is reduced.

In one possible implementation, the rate allocation proportion threshold may be set according to actual requirements, historical experience, and the like, for example, 80% or 85% may be set, which is not limited by the embodiments of the present disclosure.

In one possible implementation, the second rate allocation ratio does not exceed the rate allocation ratio threshold, and it can be considered that the adjusted second rate allocation ratio is reasonable and effective; and then the code rate value corresponding to each frame of image can be determined according to the reasonable and effective second code rate distribution proportion and the preset coding parameters.

In one possible implementation manner, as described above, the preset coding rate may be allocated according to the second rate allocation ratio, so as to obtain the allocated code rate value of each frame of image, that is, determine the coding parameters of the image group to be coded.

In the embodiment of the disclosure, the upper limit of the adjusted second code rate allocation proportion can be effectively controlled, so that the condition that the coded image quality and the video coding efficiency are reduced due to the fact that the code rate allocated by the I frame is too high is reduced.

In one possible implementation, the method further includes:

and under the condition that the second code rate allocation proportion exceeds the code rate allocation proportion threshold value, determining the coding parameters of the image group to be coded according to the second difference value between the definition of each frame image in each image group in the image group set and the coding parameters of each image group, wherein the image group set comprises the coded image group with the motion intensity not exceeding the motion intensity threshold value.

As described above, by setting the code rate allocation ratio threshold, the adjustment upper limit of the second code rate allocation ratio can be controlled. For the case where the second rate allocation ratio exceeds the rate allocation ratio threshold, it is considered that, although in this case, the first rate allocation ratio, that is, the coding parameters of the current image group, may still be unreasonable, in this case, the adjusted second rate allocation ratio has not satisfied the requirement that the rate of I frame allocation cannot be excessively high. At this time, the encoding parameters of the group of images to be encoded may be determined not in accordance with the second encoding allocation ratio.

In one possible implementation, the set of image groups includes encoded image groups whose motion strength does not exceed a motion strength threshold. That is, each image group in the image group set may be an encoded image group obtained according to the video encoding method in the embodiment of the present disclosure described above.

When the second code rate allocation proportion does not meet the requirement that the code rate allocated by the I frame cannot be too high, the coding parameters can be selected from the coding parameters of the previously coded image group according to a certain rule, and the selected coding parameters are determined as the coding parameters of the image group to be coded.

In a possible implementation manner, in the process of obtaining each encoded image group according to the video encoding method in the embodiment of the present disclosure, a second difference value between the sharpness of each frame image in each image group and an encoding parameter of each image group may be recorded, so that, when the second code rate allocation ratio exceeds the code rate allocation ratio threshold, the encoding parameter of the image group to be encoded is determined according to the second difference value of each image group and the encoding parameter of each image group.

In the embodiment of the disclosure, when the second code rate allocation proportion does not meet the requirement that the code rate allocated by the I frame cannot be too high, the coding parameters of the image groups to be coded can be effectively determined according to the definition difference and the coding parameters of the previously coded image groups.

In one possible implementation manner, in the case that the second rate allocation ratio exceeds the rate allocation ratio threshold, determining the coding parameters of the image group to be coded according to the second difference between the resolutions of the frame images in each image group in the image group set and the coding parameters of each image group includes:

Determining the minimum value in the third difference values corresponding to the image groups, wherein the third difference value is the maximum value in the second difference values corresponding to the image groups;

and taking the coding parameters of the image group corresponding to the minimum value in the image group set as the coding parameters of the image group to be coded.

As described above, the encoded group of images includes a key frame and at least one non-key frame, and the second difference between the sharpness of each frame of images may include a difference between the sharpness of the key frame and the sharpness of the at least one non-key frame. That is, the second difference may include more than one.

In one possible implementation manner, in the case that the image group set includes a plurality of image groups, a maximum value in the second difference value may be determined for each image group first, and then a minimum value in each maximum value is determined, so that the image group with the minimum sharpness difference of each frame image in the image group set is determined. The coding parameters are relatively reasonable with minimal sharpness differences, i.e. minimal image quality differences.

In one possible implementation manner, the coding parameters of the image group corresponding to the minimum value in the image group set are used as the coding parameters of the image group to be coded, so that the relatively reasonable coding parameters can be applied to the image group to be coded which is to be coded under the condition that the requirement that the code rate of I frame allocation cannot be too high is met.

In one possible implementation, it is also possible to include one group of pictures in the group of pictures set, i.e. only the current group of pictures that is encoded is included in the group of pictures set, in which case the encoding parameters of the current group of pictures that is encoded may be directly used as the encoding parameters of the group of pictures to be encoded.

In the embodiment of the disclosure, the coding parameters of the image group with the minimum definition difference can be applied to the image group to be coded under the condition that the requirement that the code rate of I frame allocation cannot be too high is met, so that the occurrence of respiratory effect is relieved.

Fig. 2 shows a flowchart of a video encoding method according to an embodiment of the present disclosure. As shown in fig. 2, the method includes:

acquiring a video stream; performing motion intensity analysis on a current image group in a video stream and performing image coding; judging whether the motion intensity of the current image group exceeds a motion intensity threshold value or not; under the condition that the motion intensity exceeds the motion intensity threshold value, the coding parameters are not adjusted;

in the case that the motion intensity does not exceed the motion intensity threshold, performing image decoding on the encoded current image group; performing definition analysis on the current image group after image decoding to obtain definition scores of the I frames and the P frames;

Judging whether the definition difference value between the I frame and the P frame in the current image group exceeds a definition threshold value according to the definition scores of the I frame and the P frame; under the condition that the definition difference value does not exceed the definition threshold value, the coding parameters are not adjusted;

adjusting the coding parameters under the condition that the definition difference exceeds a definition threshold; and feeding back the adjusted coding parameters to the image coding module so as to code the later image group in the video stream according to the adjusted coding parameters.

In one possible implementation, the images in the image group may be YUV (a color coding manner) images, and of course, may be other types of images, such as gray scale images, which is not limited to the embodiments of the present disclosure.

In one possible implementation, GOP image groups may be input into the motion intensity assessment algorithm, so that a score for the motion intensity of a picture, which characterizes the motion intensity, can be given according to the frame rate and the actual motion situation. The motion strength assessment algorithm may use a CNN network (e.g., flowNet),

in one possible implementation manner, the sharpness analysis function is started when the motion strength reaches a set condition value (motion strength threshold value), the normally encoded image group is firstly required to be decoded, whether the video frame in the current image group is an I frame or a P frame or a frame sequence number is recorded, the decoded video frame is sent to the sharpness analysis algorithm, the current sharpness score is given by the sharpness analysis algorithm, and the sharpness score characterizes the sharpness. The sharpness detection algorithm may use a CNN network (e.g., *** net) to input the decoded group of images and output a sharpness score for the images.

Fig. 3 shows a flowchart of an encoding parameter adjustment method according to an embodiment of the present disclosure. As shown in fig. 3, the method includes:

determining a definition difference value between the I frame and the P frame through definition analysis; judging whether the definition difference exceeds a definition threshold;

under the condition that the definition difference exceeds a definition threshold, the code rate allocation proportion of the I frame and the P frame is adjusted according to a preset adjustment step length of 1%; judging whether the adjusted code rate allocation proportion exceeds a code rate allocation proportion threshold value or not;

under the condition that the code rate allocation proportion threshold value is not exceeded, determining the code rate value allocated by the I frame and the P frame according to the adjusted code rate allocation proportion, wherein the code parameters comprise the code rate value;

and under the condition that the code rate distribution proportion threshold value is exceeded, outputting the coding parameters corresponding to the image group with the smallest definition difference value in the coding parameter adjusting process.

In one possible implementation manner, according to the current preset coding rate, I-frame and P-frame types and frame numbers, calculating the allocation of the coding rate, setting the allocated coding rate value to the image coding module, and repeating the above coding parameter adjustment method for the subsequent image group until the sharpness difference of each frame in the image group does not exceed the sharpness threshold value, so that the closed-loop coding parameter adjustment process can be realized.

In one possible implementation manner, the video coding method can be applied to intelligent video analysis, intelligent business, security monitoring and other scenes.

According to the embodiment of the disclosure, the problem of respiratory effect caused by unreasonable code rate distribution when a video encoder of related technology encodes video under a low code rate and relatively static scene can be solved. That is, the method can solve the problem of optimizing the quality of the coded image of the hardware-accelerated encoder and the problem of image distortion caused by the respiratory effect at a low code rate.

According to the embodiment of the disclosure, the definition difference of the currently encoded image is analyzed based on the deep learning, and the code rate distribution of the I frame and the P frame is dynamically adjusted according to the difference value so as to achieve the minimum definition difference, thereby effectively reducing the respiratory effect and improving the image quality under the condition of low code rate.

In the field of video monitoring, in a scene with a poor network environment, the video coding rate can only be reduced in the related art to improve the fluency, and after the video coding method of the embodiment of the disclosure is used, the respiratory effect of relatively static environment images can be greatly improved. According to the video coding method, the optimal code rate distribution can be obtained under the appointed code rate.

According to the embodiment of the disclosure, the video can be scored by utilizing deep learning to score the definition and the movement intensity (movement intensity), and whether the dynamic adjustment of code rate allocation or the dynamic adjustment of the code rate allocation proportion of I and P frames is required to be started or not is confirmed according to the scoring result.

It will be appreciated that the above-mentioned method embodiments of the present disclosure may be combined with each other to form a combined embodiment without departing from the principle logic, and are limited to the description of the present disclosure. It will be appreciated by those skilled in the art that in the above-described methods of the embodiments, the particular order of execution of the steps should be determined by their function and possible inherent logic.

In addition, the disclosure further provides a video encoding device, an electronic device, a computer readable storage medium, and a program, where the foregoing may be used to implement any one of the video encoding methods provided in the disclosure, and corresponding technical schemes and descriptions and corresponding descriptions referring to method parts are not repeated.

Fig. 4 shows a block diagram of a video encoding apparatus according to an embodiment of the present disclosure, as shown in fig. 4, the apparatus including:

a motion intensity analysis module 101, configured to analyze a motion intensity of a picture of a current image group of a video stream, and determine the motion intensity of the current image group;

A first coding parameter determining module 102, configured to determine, according to a current image group that has been coded, coding parameters of an image group to be coded, where the motion intensity does not exceed a motion intensity threshold, and an acquisition time of the image group to be coded is after an acquisition time of the current image group;

and the encoding module 103 is configured to encode the image group to be encoded according to the encoding parameter, so as to obtain an encoded image group.

In one possible implementation manner, the first coding parameter determining module 102 includes: the decoding submodule is used for decoding the coded current image group to obtain each frame image in the decoded current image group and a first code rate allocation proportion of each frame image, wherein the coded current image group is obtained by coding the current image group according to a preset coding code rate; and the first coding parameter determining submodule is used for determining the coding parameters of the image group to be coded according to the first code rate distribution proportion.

In a possible implementation manner, the motion intensity analysis module 101 is specifically configured to input the current image set into a pre-trained first neural network, and obtain a motion intensity score of the current image set, where the motion intensity score is used to characterize a motion intensity of the current image set.

In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.

The disclosed embodiments also provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the above-described method. The computer readable storage medium may be a non-volatile computer readable storage medium.

The embodiment of the disclosure also provides an electronic device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the above method.

Embodiments of the present disclosure also provide a computer program product comprising computer readable code which, when run on a device, causes a processor in the device to execute instructions for implementing the video encoding method provided in any of the embodiments above.

The disclosed embodiments also provide another computer program product for storing computer readable instructions that, when executed, cause a computer to perform the operations of the video encoding method provided in any of the above embodiments.

The electronic device may be provided as a terminal, server or other form of device.

Fig. 5 illustrates a block diagram of an electronic device 800, according to an embodiment of the disclosure. For example, electronic device 800 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or like terminal device.

Referring to fig. 5, an electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816.

The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interactions between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power supply component 806 provides power to the various components of the electronic device 800. The power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 800.

The multimedia component 808 includes a screen between the electronic device 800 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operational mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a Microphone (MIC) configured to receive external audio signals when the electronic device 800 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, audio component 810 further includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be a keyboard, click wheel, buttons, etc. These buttons may include, but are not limited to: homepage button, volume button, start button, and lock button.

The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of the electronic device 800. For example, the sensor assembly 814 may detect an on/off state of the electronic device 800, a relative positioning of the components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in position of the electronic device 800 or a component of the electronic device 800, the presence or absence of a user's contact with the electronic device 800, an orientation or acceleration/deceleration of the electronic device 800, and a change in temperature of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a photosensor, such as a Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices, either wired or wireless. The electronic device 800 may access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the electronic device 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for executing the methods described above.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 804 including computer program instructions executable by processor 820 of electronic device 800 to perform the above-described methods.

Fig. 6 illustrates a block diagram of an electronic device 1900 according to an embodiment of the disclosure. For example, electronic device 1900 may be provided as a server. Referring to FIG. 6, electronic device 1900 includes a processing component 1922 that further includes one or more processors and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by processing component 1922. The application programs stored in memory 1932 may include one or more modules each corresponding to a set of instructions. Further, processing component 1922 is configured to execute instructions to perform the methods described above.

The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. Electronic device 1900 may operate an operating system based on memory 1932, such as the Microsoft Server operating system (Windows Server) ^TM ) Apple Inc. developed graphical user interface based operating System (Mac OS X ^TM ) Multi-user multi-process computer operating system (Unix) ^TM ) Unix-like operating system (Linux) of free and open source code ^TM ) Unix-like operating system (FreeBSD) with open source code ^TM ) Or the like.

In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 1932, including computer program instructions executable by processing component 1922 of electronic device 1900 to perform the methods described above.

The present disclosure may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present disclosure.

The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.

Computer program instructions for performing the operations of the present disclosure can be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present disclosure are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information of computer readable program instructions, which can execute the computer readable program instructions.

Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The computer program product may be realized in particular by means of hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied as a computer storage medium, and in another alternative embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), or the like.

The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A video encoding method, comprising:

analyzing the picture motion intensity of a current image group of a video stream, and determining the motion intensity of the current image group;

under the condition that the motion intensity does not exceed a motion intensity threshold value, determining coding parameters of an image group to be coded according to a coded current image group, wherein the acquisition time of the image group to be coded is after the acquisition time of the current image group;

Coding the image group to be coded according to the coding parameters to obtain a coded image group;

wherein, under the condition that the motion intensity does not exceed the motion intensity threshold, determining the coding parameters of the image group to be coded according to the coded current image group, including:

decoding the encoded current image group to obtain each frame image in the decoded current image group and a first code rate allocation proportion of each frame image, wherein the encoded current image group is obtained by encoding the current image group according to a preset encoding code rate;

according to the definition of each frame of image, the first code rate distribution proportion is adjusted to determine the coding parameters of the image group to be coded;

wherein, according to the definition of each frame image, the first code rate allocation proportion is adjusted to determine the coding parameters of the image group to be coded, including:

determining a first difference between the sharpness of the frames of images;

2. The method according to claim 1, wherein said determining the coding parameters of the group of pictures to be coded according to the second code rate allocation ratio comprises:

and under the condition that the second code rate allocation proportion does not exceed a code rate allocation proportion threshold value, determining a code rate value corresponding to each frame of image according to the second code rate allocation proportion and a preset code rate, wherein the code parameters of the image group to be coded comprise the code rate value corresponding to each frame of image.

3. The method according to claim 1 or 2, characterized in that the method further comprises:

under the condition that the second code rate allocation proportion exceeds a code rate allocation proportion threshold value, determining the coding parameters of the image groups to be coded according to second difference values between the definition of each frame image in each image group in the image group set and the coding parameters of each image group,

wherein the set of image groups comprises encoded image groups whose motion strength does not exceed the motion strength threshold.

4. The method according to claim 3, wherein said determining the coding parameters of the group of pictures to be coded based on the second difference between the sharpness of each frame picture in each group of pictures in the group of pictures set and the coding parameters of each group of pictures in the case where the second rate allocation ratio exceeds the rate allocation ratio threshold comprises:

determining the minimum value of third difference values corresponding to the image groups, wherein the third difference value is the maximum value of the second difference values corresponding to the image groups;

5. The method of claim 1, wherein each frame of image comprises a key frame and at least one frame of non-key frame; the first difference comprises a difference between the sharpness of the key frame and the sharpness of the non-key frame.

6. The method of claim 1, wherein analyzing the picture motion intensity of a current group of images of the video stream to determine the motion intensity of the current group of images comprises:

And inputting the current image group into a pre-trained first neural network to obtain a motion intensity score of the current image group, wherein the motion intensity score is used for representing the motion intensity of the current image group.

7. The method of claim 1, wherein analyzing the sharpness of each of the frame images to determine the sharpness of each of the frame images comprises:

and inputting each frame of image into a pre-trained second neural network to obtain a definition score of each frame of image, wherein the definition score is used for representing the definition of each frame of image.

8. A video encoding apparatus, comprising:

the motion intensity analysis module is used for analyzing the picture motion intensity of the current image group of the video stream and determining the motion intensity of the current image group;

the coding parameter determining module is used for determining coding parameters of an image group to be coded according to the coded current image group under the condition that the motion intensity does not exceed a motion intensity threshold value, and the acquisition time of the image group to be coded is after the acquisition time of the current image group;

the coding module is used for coding the image group to be coded according to the coding parameters to obtain a coded image group;

determining a first difference between the sharpness of the frames of images;

9. An electronic device, comprising:

a processor;

a memory for storing processor-executable instructions;

wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any of claims 1 to 7.

10. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 7.