CN110288554B - Video beautifying method and device and electronic equipment - Google Patents
Video beautifying method and device and electronic equipment Download PDFInfo
- Publication number
- CN110288554B CN110288554B CN201910580723.6A CN201910580723A CN110288554B CN 110288554 B CN110288554 B CN 110288554B CN 201910580723 A CN201910580723 A CN 201910580723A CN 110288554 B CN110288554 B CN 110288554B
- Authority
- CN
- China
- Prior art keywords
- video frame
- leg
- video
- point
- key point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000012545 processing Methods 0.000 claims abstract description 39
- 238000001514 detection method Methods 0.000 claims abstract description 29
- 210000002414 leg Anatomy 0.000 claims description 125
- 210000001624 hip Anatomy 0.000 claims description 41
- 230000008569 process Effects 0.000 claims description 18
- 210000003423 ankle Anatomy 0.000 claims description 13
- 210000003127 knee Anatomy 0.000 claims description 13
- 238000009499 grossing Methods 0.000 claims description 8
- 238000003708 edge detection Methods 0.000 claims description 6
- 239000000126 substance Substances 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 4
- 238000004590 computer program Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 241001465754 Metazoa Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the disclosure provides a video beautifying method, a video beautifying device and electronic equipment, belonging to the technical field of image processing, wherein the method comprises the following steps: acquiring a first video frame and a second video frame which are adjacent in a target video, wherein the first video frame is earlier than the second video frame in the time dimension of the target video; respectively executing key point detection aiming at the hip and the leg of the target object on the first video frame and the second video frame to form a hip key point and a leg key point; determining a representative region and a leg center point of a leg of the target object on the first video frame and the second video frame based on the hip key point and the leg key point; and carrying out smooth beautification processing on the representation areas, the leg central points and the deformation parameters on the first video frame and the second video frame. Through the processing scheme disclosed by the invention, the effect of beautifying the video is improved.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a video beautification method and apparatus, and an electronic device.
Background
Image processing (image processing), also called video processing, is a technique for achieving a desired result in an image by a computer. Since the 20 th century, digital image processing was common. The main contents of the image processing technology include image compression, enhancement restoration, and matching description identification 3 parts, and common processes include image digitization, image coding, image enhancement, image restoration, image segmentation, image analysis, and the like. The image processing is to process the image information by using a computer to meet the visual psychology of people or the behavior of application requirements, has wide application, and is mainly used for mapping, atmospheric science, astronomy, beautifying, image identification improvement and the like.
With the popularization of smart phones, the use of mobile phones for photographing is increasing, and in the process of photographing by using a mobile phone, due to various reasons such as a photographing angle, a problem that a body proportion of a video formed by photographing by a person cannot meet a user expectation value exists.
Disclosure of Invention
In view of the above, embodiments of the present disclosure provide a method, an apparatus, and an electronic device for beautifying a video, which at least partially solve the problems in the prior art.
In a first aspect, an embodiment of the present disclosure provides a video beautification method, including:
acquiring a first video frame and a second video frame which are adjacent in a target video, wherein the first video frame is earlier than the second video frame in the time dimension of the target video;
respectively executing key point detection aiming at the hip and the leg of the target object on the first video frame and the second video frame to form a hip key point and a leg key point;
determining a representative region and a leg center point of a leg of the target object on the first video frame and the second video frame based on the hip key point and the leg key point;
and carrying out smooth beautification processing on the representation areas, the leg central points and the deformation parameters on the first video frame and the second video frame.
According to a specific implementation manner of the embodiment of the present disclosure, the performing smooth beautification processing on the representation area, the leg center point, and the deformation parameter on the first video frame and the second video frame includes:
respectively calculating jitter values of the representation area, the leg central point and the deformation parameter on the first video frame and the second video frame;
determining coordinates of the representation area and the leg center point and a final value of the deformation parameter based on the jitter value;
performing a stretching process on the deformation region based on the determined coordinates of the representation region and the leg center point, and the final value of the deformation parameter.
According to a specific implementation manner of the embodiment of the present disclosure, the performing smooth beautification processing on the representation area, the leg center point, and the deformation parameter on the first video frame and the second video frame includes:
when the extension direction of the leg is inconsistent with the vertical direction of the video frame of the target video, establishing a corrected vertical direction of the target object in the target video by taking the central point of the shoulder key point and the central point of the hip key point as continuity;
performing a stretching operation on the representative region of the leg portion based on the corrected vertical direction.
According to a specific implementation manner of the embodiment of the present disclosure, after performing smooth beautification processing on the representation area, the leg center point, and the deformation parameter on the first video frame and the second video frame, the method further includes:
acquiring a third video frame adjacent to the second video frame;
performing key point detection on the third video frame to obtain a hip key point and a leg key point;
performing image beautifying operation on a third video frame based on the hip key point and the leg key point on a second video frame and the third video frame to form a third beautified image;
and taking the third beautified image as a video frame of the beautified video.
According to a specific implementation manner of the embodiment of the present disclosure, the performing smooth beautification processing on the representation area, the leg center point, and the deformation parameter on the first video frame and the second video frame includes:
obtaining coordinates (X1, Y1) of a first keypoint on the first video frame and coordinates (X2, Y2) of a second keypoint on a second video frame, the second keypoint corresponding to the first keypoint;
calculating the coordinates (X2 ', Y2') after the smoothing of the second key point based on a preset smoothing coefficient L, wherein
X2’＝X1*L+X2*(1-L)
Y2’＝Y1*L+Y2*(1-L)。
According to a specific implementation manner of the embodiment of the present disclosure, the performing key point detection on the hip and the leg of the target object on the first video frame and the second video frame respectively to form a hip key point and a leg key point includes:
performing key detection on the leg of the target object to obtain a knee key point and an ankle key point;
taking the knee keypoints and the ankle keypoints as the leg keypoints.
According to a specific implementation manner of the embodiment of the present disclosure, the performing of the keypoint detection for the hip and the leg of the target object on the first video frame and the second video frame respectively includes:
performing edge detection on the target object to obtain contour points of the target object;
selecting any contour point as a starting scanning point, establishing a row pointer and a column pointer pointing to the starting scanning point, and a total pointer pointing to the right of the row pointer and pointing to the down of the column pointer;
when scanning other scanning points except the initial scanning point based on the contour point, correspondingly establishing a row pointer, a column pointer and a total pointer of other scanning points;
determining hip, knee, and ankle keypoints of the target object based on the row, column, and total pointers.
According to a specific implementation manner of the embodiment of the present disclosure, before performing the keypoint detection for the hip and the leg of the target object on the first video frame and the second video frame respectively, the method further includes:
transforming the target image into a grayscale image;
and carrying out edge detection on the gray level image to obtain an edge contour of the target object for key point detection.
In a second aspect, an embodiment of the present disclosure provides a video beautification device, including:
the device comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a first video frame and a second video frame which are adjacent in a target video, and the first video frame is earlier than the second video frame in the time dimension of the target video;
the detection module is used for respectively carrying out key point detection aiming at the hip and the leg of the target object on the first video frame and the second video frame to form a hip key point and a leg key point;
the determining module is used for determining the representing areas and the leg center points of the legs of the target object on the first video frame and the second video frame based on the hip key points and the leg key points;
and the beautifying module is used for carrying out smooth beautifying processing on the representation areas, the leg central points and the deformation parameters on the first video frame and the second video frame.
In a third aspect, an embodiment of the present disclosure further provides an electronic device, where the electronic device includes:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the video beautification method of any of the preceding first aspects or any implementation manner of the first aspect.
In a fourth aspect, the disclosed embodiments also provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the video beautification method of the first aspect or any implementation manner of the first aspect.
In a fifth aspect, the disclosed embodiments also provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the video beautification method of the first aspect or any of the implementations of the first aspect.
The video beautification scheme in the embodiment of the disclosure comprises the steps of obtaining a first video frame and a second video frame which are adjacent in a target video, wherein the first video frame is earlier than the second video frame in the time dimension of the target video;
respectively executing key point detection aiming at the hip and the leg of the target object on the first video frame and the second video frame to form a hip key point and a leg key point; determining a representative region and a leg center point of a leg of the target object on the first video frame and the second video frame based on the hip key point and the leg key point; and carrying out smooth beautification processing on the representation areas, the leg central points and the deformation parameters on the first video frame and the second video frame. Through the scheme disclosed by the invention, the effect of beautifying the video is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic diagram illustrating a video beautification process according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of key points based on a human body according to an embodiment of the present disclosure;
FIG. 3 is a schematic view of another video beautification process provided by the embodiment of the present disclosure;
FIG. 4 is a schematic view of another video beautification process provided by the embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of a video beautification apparatus according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of an electronic device provided in an embodiment of the present disclosure.
Detailed Description
The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
The embodiments of the present disclosure are described below with specific examples, and other advantages and effects of the present disclosure will be readily apparent to those skilled in the art from the disclosure in the specification. It is to be understood that the described embodiments are merely illustrative of some, and not restrictive, of the embodiments of the disclosure. The disclosure may be embodied or carried out in various other specific embodiments, and various modifications and changes may be made in the details within the description without departing from the spirit of the disclosure. It should be noted that the features in the following embodiments and examples may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the disclosure, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number of the aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present disclosure, and the drawings only show the components related to the present disclosure rather than the number, shape and size of the components in actual implementation, and the type, amount and ratio of the components in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated.
In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details.
The embodiment of the disclosure provides a video beautification method. The video beautification method provided by the embodiment can be executed by a computing device, the computing device can be implemented as software, or implemented as a combination of software and hardware, and the computing device can be integrally arranged in a server, a terminal device and the like.
Referring to fig. 1, a video beautification method provided by an embodiment of the present disclosure includes:
s101, a first video frame and a second video frame which are adjacent in a target video are obtained, and the first video frame is earlier than the second video frame in the time dimension of the target video.
The target video is a video which needs to be beautified, the target video can be a video obtained by shooting through equipment such as a smart phone and the like, and can also be a video obtained through other modes, and the target video is composed of a plurality of video frames.
The target video includes a target object, and generally, the target object may be a person, for example, a user takes a video including a person through a mobile phone or the like, and the person in the video constitutes the target object. The target object may be a human, an animal, or other types of objects.
And S102, respectively executing key point detection aiming at the hip and the leg of the target object on the first video frame and the second video frame to form a hip key point and a leg key point.
The target object is displayed with a hip region and a leg region on the target video. After the target video is formed, the key point detection can be performed on the hip region and the leg region, so that a plurality of hip key points, knee key points and ankle key points are obtained. Referring to fig. 2, fig. 2 shows the human body key points including P1 to P19. Where the hip and leg keypoints may contain a set of keypoints of multiple keypoints, for example, the hip keypoints may include keypoints P12, P3, and P16, and the leg keypoints may include knee keypoints P13, P17, and ankle keypoints P14, P18. The key points may include a left key point and a right key point according to the position of the key point distribution.
The detection of the key points in the hip region and the leg region can be performed by methods such as CPM (Convolutional Pose detector), PAF (Part Affinity Fields), and the like. The method of detecting the key point is not limited herein.
S103, determining the representing areas of the legs and the leg center points of the target object on the first video frame and the second video frame based on the hip key points and the leg key points.
After the hip key points and the leg key points are obtained, a representative region of the leg can be determined based on the hip key points and the leg key points, and a beautification operation can be performed on the leg of the target object based on the representative region of the leg.
In one approach, a first center point of a set of keypoints consisting of knee keypoints and ankle keypoints may be obtained, and a second center point of a set of hip keypoints may be obtained. The leg length can be obtained by calculating the distance d between the first center point and the second center point and multiplying the calculated distance by a preset coefficient gamma.
After the vertical direction of the leg and the leg length are obtained, a first rectangular region which covers a key point of the leg and is established with the second center point as a starting point and has a length of γ x d, where γ is a preset stretching ratio and γ >1, may be obtained, and the first rectangular region may be determined as a representation region of the leg.
The leg center point of the target object can be obtained by performing an average calculation of the coordinates of all the hip and leg key points.
In addition to acquiring the representation area of the leg, in order to ensure that the background area corresponding to the representation area of the leg performs similar beautification transformation with the representation area of the leg in the process of performing beautification on the representation area of the leg, it is also necessary to simultaneously acquire the background area of the target object corresponding to the representation area of the leg.
Correspondingly, a second rectangular area which takes the second central point as a starting point, has the length of gamma d and has the width the same as the horizontal width of the target image is established, and the second rectangular area is determined as the background area.
And S104, performing smooth beautification processing on the representation areas, the leg central points and the deformation parameters on the first video frame and the second video frame.
In the process of forming images on the first video frame and the second video frame by the leg region, the area of the leg region is slightly larger than that of other parts of the body due to shooting angles and the like, or the area of the leg region is expected to be smaller than that of the actual leg region by a user, so that the aesthetic degree of the leg of the target object is improved, and therefore the display of the target object on the first video frame and the second video frame can be adjusted in a manner of deforming the leg region, and the beautifying effect is achieved.
Before the deformation is performed, a deformation parameter of the leg region may be further obtained, where the deformation parameter indicates a scaling ratio of the leg region, for example, the deformation parameter may be 0.9, which indicates that a scaling process of 0.9 times is performed on the leg region.
The deformation parameter may be obtained in various ways, and as one way, the deformation parameter may be determined according to a value input by a user on a corresponding interactive interface. Alternatively, the deformation parameters may be automatically calculated by automatically calculating the proportions of the target object to the respective body parts on the target graph.
In the process of detecting key points of video frames, the key points of target objects on different video frames have a shaking phenomenon, so that the video formed after leg deformation has a discontinuous phenomenon, and therefore, smoothing processing needs to be performed on the first video frame and the second video frame after deformation. Before the smoothing process is performed, different deformation parameters can be set on different video frames in advance. Specifically, the determining the deformation parameter may include the following steps:
first, the jitter parameters of the leg representing region and the leg center point in the first video frame and the second video frame, and the preset default deformation parameter L1 are obtained.
By calculating the offset distances c1, c2, and c3 of the contour, the leg representation region, and the leg center point of the target object on the first video frame and the second video frame, respectively, the jitter parameter d can be determined, for example, d ═ c2+ c3)/2 × c 1. The jitter parameter d indicates the case where the key point is shifted over different video frames. The leg center point may be obtained by averaging a set of coordinates consisting of the hip key point, the knee key point, and the ankle key point.
The default morphing parameter L1 is a morphing parameter set in advance, and as a mode, the default morphing parameter L1 may be determined according to a value input by a user on the corresponding interactive interface. Alternatively, the default deformation parameter L1 may be calculated automatically by automatically calculating the proportions of the target object in the first video frame.
Next, the default morphing parameter L1 is used as the morphing parameter of the first video frame.
Since the first video frame is the starting video frame, the default deformation parameter L1 may be kept unchanged on the first video frame, and the deformation operation may be performed on the leg representation area of the first video frame based on the default deformation parameter L1.
Finally, based on the default deformation parameter L1 and the jitter parameter, a deformation parameter L2 at the second video frame is determined.
Specifically, the jitter parameter d may be normalized to obtain a normalized jitter parameter between 0 and 1, where the normalization is a common processing manner, and the normalization manner is not limited herein. Then, taking the product of the normalized jitter parameter and the default deformation parameter as the deformation parameter L2 of the second video frame, and performing a deformation operation on the leg portion representing region of the second video frame based on the deformation parameter L2 of the second video.
Through the scheme in this application, can guarantee to carry out effectual deformation to target object's shank region to target object's whole pleasing to the eye degree on the video has been improved.
As another optional implementation manner of the embodiment of the present disclosure, referring to fig. 3, performing smooth beautification processing on the representation area, the leg center point, and the deformation parameter on the first video frame and the second video frame may include:
s301, jitter values of the areas, the leg center points and the deformation parameters on the first video frame and the second video frame are calculated respectively.
By calculating the offset distances c1, c2, and c3 of the contour, the leg-representing region, and the leg center point of the target object on the first video frame and the second video frame, respectively, the shake value f can be determined, for example, f ═ (c1+ c2+ c 3)/2. The jitter value f represents the case where the keypoints are shifted over different video frames.
S302, determining the coordinates of the representing area and the central point of the leg part and the final value of the deformation parameter based on the jitter value.
In order to reduce jitter on the second video frame relative to the first video frame, the coordinates representing the region and the leg center point may be subtracted by the corresponding jitter value f as the final coordinates representing the region and the leg center point on the second video frame. Meanwhile, the distortion parameter L1 of the first video frame is multiplied by the jitter parameter d as the distortion parameter of the second video frame.
And S303, performing stretching processing on the deformation region based on the determined coordinates of the representation region and the central point of the leg part and the final value of the deformation parameter.
In the process of carrying out smooth beautification processing on the representation areas, the leg center points and the deformation parameters on the first video frame and the second video frame, when the extension direction of the legs is inconsistent with the vertical direction of the video frame of the target video, the center point of the shoulder key point and the center point of the hip key point are taken as the continuity, the corrected vertical direction of the target object in the target video is established, and based on the corrected vertical direction, stretching operation is carried out on the representation areas of the legs.
After smooth beautification processing is carried out on the representation areas, the leg central points and the deformation parameters on the first video frame and the second video frame, a third video frame adjacent to the second video frame can be obtained; performing key point detection on the third video frame to obtain a hip key point and a leg key point; performing image beautification operation on a third video frame based on the hip key point and the leg key point on a second video frame and the third video frame to form a third beautified image; and taking the third beautified image as a video frame of the beautified video.
Performing smooth beautification processing on the representation area, the leg center point and the deformation parameter on the first video frame and the second video frame, and may further include: obtaining coordinates (X1, Y1) of a first keypoint on the first video frame and coordinates (X2, Y2) of a second keypoint on a second video frame, the second keypoint corresponding to the first keypoint; and calculating coordinates (X2 ', Y2') after the smoothing of the second key point based on a preset smoothing coefficient L, wherein X2 is X1L + X2 (1-L), and Y2 is Y1L + Y2 (1-L).
In the process of detecting key points of the leg, key detection can be performed on the leg of the target object to obtain knee key points and ankle key points; taking the knee keypoints and the ankle keypoints as the leg keypoints.
Referring to fig. 4, as an alternative implementation manner of the embodiment of the present disclosure, performing keypoint detection for the hip and the leg of the target object on the first video frame and the second video frame respectively may include:
s401, performing edge detection on the target object to obtain contour points of the target object;
s402, selecting any contour point as an initial scanning point, establishing a row pointer and a column pointer pointing to the initial scanning point, and a total pointer pointing to the row pointer rightwards and the column pointer downwards;
s403, when scanning points other than the initial scanning point based on the contour point, correspondingly establishing a row pointer, a column pointer and a total pointer of other scanning points;
s404, determining the hip key point, the knee key point and the ankle key point of the target object based on the row pointer, the column pointer and the total pointer.
As an optional implementation manner of the embodiment of the present disclosure, before performing the keypoint detection for the hip and the leg of the target object on the first video frame and the second video frame respectively, the method further includes: transforming the target image into a grayscale image; and carrying out edge detection on the gray level image to obtain an edge contour of the target object for key point detection.
Corresponding to the above method embodiment, referring to fig. 5, the present disclosure also provides a video beautification apparatus, including:
an obtaining module 501, configured to obtain a first video frame and a second video frame that are adjacent to each other in a target video, where the first video frame is earlier than the second video frame in a time dimension of the target video;
a detecting module 502, configured to perform keypoint detection on the hip and the leg of the target object on the first video frame and the second video frame, respectively, to form a hip keypoint and a leg keypoint;
a determining module 503, configured to determine, based on the hip key point and the leg key point, a leg representing area and a leg center point of the target object on the first video frame and the second video frame;
and a beautification module 504, configured to perform smooth beautification processing on the representation area, the leg center point, and the deformation parameter on the first video frame and the second video frame.
The apparatus shown in fig. 5 may correspondingly execute the content in the above method embodiment, and details of the part not described in detail in this embodiment refer to the content described in the above method embodiment, which is not described again here.
Referring to fig. 6, an embodiment of the present disclosure also provides an electronic device 60, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the video beautification method of the method embodiments described above.
The disclosed embodiments also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the foregoing method embodiments.
The disclosed embodiments also provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the video beautification method of the aforementioned method embodiments.
Referring now to FIG. 6, a schematic diagram of an electronic device 60 suitable for use in implementing embodiments of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., car navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 6, electronic device 60 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 60 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 60 to communicate with other devices wirelessly or by wire to exchange data. While the figures illustrate an electronic device 60 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or installed from the storage means 608, or installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: acquiring at least two internet protocol addresses; sending a node evaluation request comprising the at least two internet protocol addresses to node evaluation equipment, wherein the node evaluation equipment selects the internet protocol addresses from the at least two internet protocol addresses and returns the internet protocol addresses; receiving an internet protocol address returned by the node evaluation equipment; wherein the obtained internet protocol address indicates an edge node in the content distribution network.
Alternatively, the computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving a node evaluation request comprising at least two internet protocol addresses; selecting an internet protocol address from the at least two internet protocol addresses; returning the selected internet protocol address; wherein the received internet protocol address indicates an edge node in the content distribution network.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first obtaining unit may also be described as a "unit obtaining at least two internet protocol addresses".
It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.
The above description is only for the specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present disclosure should be covered within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (9)
1. A method for beautifying video, comprising:
acquiring a first video frame and a second video frame which are adjacent in a target video, wherein the first video frame is earlier than the second video frame in the time dimension of the target video;
respectively executing key point detection aiming at the hip and the leg of the target object on the first video frame and the second video frame to form a hip key point and a leg key point;
determining a representative region and a leg center point of a leg of the target object on the first video frame and the second video frame based on the hip key point and the leg key point;
carrying out smooth beautification processing on the representation area, the leg central point and the deformation parameter on the first video frame and the second video frame, wherein the smooth beautification processing comprises the following steps: when the extension direction of the leg is not consistent with the vertical direction of the video frame of the target video, establishing a corrected vertical direction of the target object in the target video by taking the central point of the shoulder key point and the central point of the hip key point on the first video frame and the second video frame as a connecting line; performing a stretching operation on a representation area of the leg portion based on the corrected vertical direction;
the smooth beautification processing is performed on the representation area, the leg center point and the deformation parameter on the first video frame and the second video frame, and the smooth beautification processing comprises the following steps: respectively calculating jitter values of the representing area of the leg, the center point of the leg and the deformation parameter on the first video frame and the second video frame; determining coordinates of the representing region of the leg and the center point of the leg, and a final value of the deformation parameter, based on the jitter value; performing a stretching process on the representative region of the leg based on the determined coordinates of the representative region of the leg and the leg center point, and the final value of the deformation parameter;
wherein the jitter value and the jitter parameter are determined based on the contour of the target object, the leg representation area and the offset distance of the leg center point on the first video frame and the second video frame; subtracting a jitter value from the coordinates of the leg center point and the leg representation area of the leg on the second video frame, and determining the coordinates of the leg center point and the leg representation area of the leg; and multiplying the deformation parameter of the first video frame by the jitter parameter to be used as a final value of the deformation parameter of the second video frame.
2. The method of claim 1, wherein after the smooth beautification processing of the representation area, the leg center point and the deformation parameter on the first video frame and the second video frame, the method further comprises:
acquiring a third video frame adjacent to the second video frame;
performing key point detection on the third video frame to obtain a hip key point and a leg key point;
performing image beautification operation on a third video frame based on the hip key point and the leg key point on a second video frame and the third video frame to form a third beautified image;
and taking the third beautified image as a video frame of the beautified video.
3. The method of claim 1, wherein the smooth beautification of the representation area, the leg center point and the deformation parameter on the first video frame and the second video frame comprises:
obtaining coordinates (x1, y1) of a first keypoint on the first video frame, and coordinates (x2, y2) of a second keypoint on a second video frame, the second keypoint corresponding to the first keypoint;
calculating the coordinates (x2 ', y 2') after the smoothing of the second key point based on a preset smoothing coefficient L, wherein
x2’＝x1*L+x2*(1-L)
y2’＝y1*L+y2*(1-L)。
4. The method of claim 1, wherein the performing of the key point detection for the hip and the leg of the target object on the first video frame and the second video frame respectively to form a hip key point and a leg key point comprises:
performing key point detection on the leg of the target object to obtain knee key points and ankle key points;
taking the knee keypoints and the ankle keypoints as the leg keypoints.
5. The method of claim 1, wherein performing keypoint detection for the hip and leg of the target subject on the first video frame and the second video frame, respectively, comprises:
performing edge detection on the target object to obtain contour points of the target object;
selecting any contour point as a starting scanning point, establishing a row pointer and a column pointer pointing to the starting scanning point, and a total pointer pointing to the right of the row pointer and pointing to the down of the column pointer;
when scanning other scanning points except the initial scanning point based on the contour point, correspondingly establishing a row pointer, a column pointer and a total pointer of other scanning points;
based on the row, column, and global pointers, hip, knee, and ankle key points of the target object are determined.
6. The method of claim 1, wherein prior to performing keypoint detection for the hip and leg of the target subject on the first video frame and the second video frame, respectively, the method further comprises:
transforming the target image into a grayscale image;
and carrying out edge detection on the gray level image to obtain an edge contour of the target object for key point detection.
7. A video beautification apparatus, comprising:
the device comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a first video frame and a second video frame which are adjacent in a target video, and the first video frame is earlier than the second video frame in the time dimension of the target video;
the detection module is used for respectively carrying out key point detection aiming at the hip and the leg of the target object on the first video frame and the second video frame to form a hip key point and a leg key point;
the determining module is used for determining the representing areas of the legs and the leg center points of the target object on the first video frame and the second video frame based on the hip key points and the leg key points;
a beautification module, configured to perform smooth beautification processing on the representation area, the leg center point, and the deformation parameter on the first video frame and the second video frame, including: when the extension direction of the leg is not consistent with the vertical direction of the video frame of the target video, establishing a corrected vertical direction of the target object in the target video by taking the central point of the shoulder key point and the central point of the hip key point on the first video frame and the second video frame as a connecting line; performing a stretching operation on a representation area of the leg portion based on the corrected vertical direction;
the beautification module is further configured to perform smooth beautification processing on the representation area, the leg center point, and the deformation parameter on the first video frame and the second video frame, and includes: respectively calculating jitter values of the representing area of the leg, the center point of the leg and the deformation parameter on the first video frame and the second video frame; determining coordinates of the representing region of the leg and the center point of the leg, and a final value of the deformation parameter, based on the jitter value; performing a stretching process on the representative region of the leg based on the determined coordinates of the representative region of the leg and the leg center point, and the final value of the deformation parameter;
wherein the jitter value and the jitter parameter are determined based on the contour of the target object, the leg representation area and the offset distance of the leg center point on the first video frame and the second video frame; subtracting a jitter value from the coordinates of the leg center point and the leg representation area of the leg on the second video frame, and determining the coordinates of the leg center point and the leg representation area of the leg; and multiplying the deformation parameter of the first video frame by the jitter parameter to be used as a final value of the deformation parameter of the second video frame.
8. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the video beautification method of any of claims 1-6.
9. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the video beautification method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910580723.6A CN110288554B (en) | 2019-06-29 | 2019-06-29 | Video beautifying method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910580723.6A CN110288554B (en) | 2019-06-29 | 2019-06-29 | Video beautifying method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110288554A CN110288554A (en) | 2019-09-27 |
| CN110288554B true CN110288554B (en) | 2022-09-16 |
Family
ID=68020173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910580723.6A Active CN110288554B (en) | 2019-06-29 | 2019-06-29 | Video beautifying method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110288554B (en) |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102262724B (en) * | 2010-05-31 | 2013-01-09 | 汉王科技股份有限公司 | Object image characteristic points positioning method and object image characteristic points positioning system |
| KR20150113751A (en) * | 2014-03-31 | 2015-10-08 | (주)트라이큐빅스 | Method and apparatus for acquiring three-dimensional face model using portable camera |
| CN106482656B (en) * | 2016-09-28 | 2019-09-20 | 成都通甲优博科技有限责任公司 | A kind of mountainous region landslide visible detection method and device for overcoming monitor supervision platform itself to shake |
| CN107341777B (en) * | 2017-06-26 | 2020-12-04 | 北京小米移动软件有限公司 | Picture processing method and device |
| CN108830783B (en) * | 2018-05-31 | 2021-07-02 | 北京市商汤科技开发有限公司 | Image processing method and device and computer storage medium |
| CN108898118B (en) * | 2018-07-04 | 2023-04-18 | 腾讯科技（深圳）有限公司 | Video data processing method, device and storage medium |
| CN109191552A (en) * | 2018-08-16 | 2019-01-11 | Oppo广东移动通信有限公司 | Threedimensional model processing method, device, electronic equipment and storage medium |
| CN109194876B (en) * | 2018-10-31 | 2020-11-13 | Oppo广东移动通信有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| CN109544620B (en) * | 2018-10-31 | 2021-03-30 | Oppo广东移动通信有限公司 | Image processing method and apparatus, computer-readable storage medium, and electronic device |
| CN109684920B (en) * | 2018-11-19 | 2020-12-11 | 腾讯科技（深圳）有限公司 | Object key point positioning method, image processing method, device and storage medium |
-
2019
- 2019-06-29 CN CN201910580723.6A patent/CN110288554B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110288554A (en) | 2019-09-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110288551B (en) | Video beautifying method and device and electronic equipment | |
| CN110058685B (en) | Virtual object display method and device, electronic equipment and computer-readable storage medium | |
| CN110288520B (en) | Image beautifying method and device and electronic equipment | |
| CN109460233B (en) | Method, device, terminal equipment and medium for updating native interface display of page | |
| CN110070495B (en) | Image processing method and device and electronic equipment | |
| CN110288553A (en) | Image beautification method, device and electronic equipment | |
| CN110298785A (en) | Image beautification method, device and electronic equipment | |
| CN110288521A (en) | Image beautification method, device and electronic equipment | |
| CN111325668B (en) | Training method and device for image processing deep learning model and electronic equipment | |
| CN110264430B (en) | Video beautifying method and device and electronic equipment | |
| CN110070482B (en) | Image processing method, apparatus and computer readable storage medium | |
| CN110288554B (en) | Video beautifying method and device and electronic equipment | |
| CN116596748A (en) | Image stylization processing method, apparatus, device, storage medium, and program product | |
| CN111696041B (en) | Image processing method and device and electronic equipment | |
| CN110390717B (en) | 3D model reconstruction method and device and electronic equipment | |
| CN111200705B (en) | Image processing method and device | |
| CN111223105B (en) | Image processing method and device | |
| CN110349109B (en) | Fisheye distortion correction method and system and electronic equipment thereof | |
| CN111292245A (en) | Image processing method and device | |
| CN111586261B (en) | Target video processing method and device and electronic equipment | |
| CN111402116A (en) | Method and device for processing human waist body beautification in picture and electronic equipment | |
| CN111415382B (en) | Method and device for processing human body arm body beautification in picture and electronic equipment | |
| CN111353929A (en) | Image processing method and device and electronic equipment | |
| CN110189279B (en) | Model training method and device, electronic equipment and storage medium | |
| CN115937010B (en) | Image processing method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |