WO2023029379A1 - Image special effect generation method and apparatus - Google Patents

Image special effect generation method and apparatus Download PDF

Info

Publication number
WO2023029379A1
WO2023029379A1 PCT/CN2022/075194 CN2022075194W WO2023029379A1 WO 2023029379 A1 WO2023029379 A1 WO 2023029379A1 CN 2022075194 W CN2022075194 W CN 2022075194W WO 2023029379 A1 WO2023029379 A1 WO 2023029379A1
Authority
WO
WIPO (PCT)
Prior art keywords
stroke key
sub
key point
texture material
stroke
Prior art date
Application number
PCT/CN2022/075194
Other languages
French (fr)
Chinese (zh)
Inventor
颜敏炜
Original Assignee
北京达佳互联信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京达佳互联信息技术有限公司 filed Critical 北京达佳互联信息技术有限公司
Publication of WO2023029379A1 publication Critical patent/WO2023029379A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/40—Filling a planar surface by adding surface attributes, e.g. colour or texture
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream

Definitions

  • the embodiments of the present application relate to the technical field of image processing, and in particular, to a method and device for generating image special effects, electronic equipment, and a storage medium.
  • the stroke special effect is a special effect that people often choose.
  • the Edge Effect creates a special effect pattern on the edge of the outline of the target object in the video.
  • the outline stroke key point of the target object is usually obtained first, and at the position of each outline stroke key point, an independent rectangle is constructed with the stroke key point as the center, forming Multiple independent rectangles surrounding the target object, and then fill all the constructed independent rectangles with the texture material map to form a stroke texture effect around the target object.
  • Embodiments of the present application provide a method, device, electronic device, and storage medium for generating an image special effect.
  • the embodiment of the present application provides a method for generating image special effects, the method including:
  • the video frame of the target video contains the target object
  • the establishment of two extension points respectively located on both sides of the stroke key point on any straight line passing through the stroke key point includes:
  • the adjacent stroke key point refers to the Stroke key points adjacent to the key point
  • the establishment of two extension points on the straight line of the extension point with equal distances from the stroke key point includes:
  • a plurality of the stroke key points are formed into a stroke key point sequence
  • the method also includes:
  • the target distance scaling factor corresponding to the non-initial stroke key point determines the target distance scaling factor corresponding to the non-initial stroke key point from the magnification corresponding relationship, and the magnification corresponding relationship is used To characterize the correspondence between the sequence of non-initial stroke key points in the sequence of stroke key points and the distance scaling factor;
  • the product of the target distance scaling factor and the first distance is determined as the second distance corresponding to the non-initial stroke key point.
  • the use of a preset texture material map to fill the filling area of multiple video frames in the target video includes:
  • each of the quadrilateral areas in the filling area is divided into two filling sub-areas along a diagonal line in turn, to obtain the predetermined Set the number of filled sub-regions;
  • the texture material sub-block is filled into the corresponding filling sub-region.
  • the dividing the texture material map into texture material sub-blocks with a preset number of triangles includes:
  • the texture material map is divided into equal widths to obtain quadrilateral texture material blocks equal to the number of quadrilateral regions;
  • each of the texture material blocks is sequentially divided into two texture material sub-blocks along a diagonal line.
  • the establishment of a one-to-one correspondence between the texture material sub-block and the filling sub-region includes:
  • the generation order of the texture material sub-blocks and the generation order of the filling sub-regions establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions, wherein the texture material sub-blocks in the same order are generated Blocks correspond to filled subregions.
  • the embodiment of the present application provides a device for generating image special effects, the device comprising:
  • An acquisition module configured to acquire a target video, where a video frame of the target video contains a target object
  • a key point module configured to determine stroke key points located outside the target object in the video frame
  • the filling area module is configured to establish two extension points respectively located on both sides of the stroke key point on any straight line passing through the stroke key point, and according to each two adjacent stroke key points and The corresponding extension points form a quadrilateral area, and a filled area formed by connecting multiple quadrilateral areas is obtained;
  • the filling module is configured to use a preset texture material map to fill the filling area of the video frame to obtain the filled video frame;
  • the combination module is configured to combine the filled video frames in time order to obtain special effect video.
  • the filled region module includes:
  • the extension point straight line sub-module is configured to determine an extension point straight line that passes through the stroke key point and is perpendicular to a connecting line formed by the stroke key point and an adjacent stroke key point, and the adjacent stroke key point A key point refers to a stroke key point adjacent to the stroke key point;
  • the extension point submodule is configured to establish two extension points on the extension point straight line with equal distances from the stroke key point.
  • the extension point submodule includes:
  • the key point sequence submodule is configured to form a plurality of stroke key points into a stroke key point sequence according to the arrangement order of the stroke key points outside the target object;
  • the start key point sub-module is configured to establish two distances from the start stroke key point at the first distance on the extension point line corresponding to the start stroke key point in the stroke point sequence. extension point;
  • the non-initial key point sub-module is configured to establish two distances from the non-initial stroke key point on the straight line corresponding to the non-initial stroke key point in the stroke point sequence. Two-distance extension point.
  • the device also includes:
  • the scaling sub-module is configured to determine the target distance zoom magnification corresponding to the non-initial stroke key point from the magnification correspondence relationship according to the order of the non-initial stroke key point in the stroke key point sequence , the magnification correspondence is used to characterize the correspondence between the order of the non-initial stroke key points in the stroke key point sequence and the distance zoom magnification;
  • the second distance sub-module is configured to determine the product of the target distance scaling factor and the first distance as the second distance corresponding to the non-initial stroke key point.
  • the filling module includes:
  • the material cutting sub-module is configured to divide the texture material map into texture material sub-blocks with a preset number of triangles
  • the filling area cutting sub-module is configured to divide each quadrilateral area into two parts along a diagonal line in sequence starting from one end of the filling area according to the order of arrangement of the quadrilateral areas in the filling area. filling sub-regions, obtaining the preset number of filling sub-regions;
  • a correspondence sub-module configured to establish a one-to-one correspondence between the texture material sub-block and the filling sub-region
  • the matching submodule is configured to deform the texture material sub-block, so that the texture material sub-block matches the shape of the corresponding filling sub-region;
  • the filling sub-module is configured to fill the texture material sub-block into the corresponding filling sub-area according to the one-to-one correspondence between the texture material sub-block and the filling sub-area.
  • the material cutting sub-module includes:
  • the first cutting module is configured to perform equal-width segmentation on the texture material map from one end to the other end of the texture material map, so as to obtain quadrilateral texture material blocks equal to the number of the quadrilateral regions;
  • the second sub-cutting module is configured to sequentially cut each texture material block into two texture material sub-blocks along a diagonal line according to the generation order of the texture material blocks.
  • the correspondence submodule includes:
  • the relationship establishment submodule is configured to establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions according to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, wherein , the texture material sub-blocks with the same generation order correspond to the filling sub-regions.
  • the embodiment of the present application also provides an electronic device, including a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to realize the image How to generate special effects.
  • the embodiment of the present application also provides a computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can execute the image special effect. generate method.
  • the embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above-mentioned method for generating an image special effect is implemented.
  • the key points of the stroke by extracting the key points of the stroke from the acquired video frames containing the target object, the key points of the stroke outside the target object are obtained, and then each key point of the stroke is expanded.
  • Build a connecting line around a stroke key point and pass through two extension points of the stroke key point so that the expansion points corresponding to all adjacent stroke key points can form a quadrilateral area, and then all quadrilateral areas can form a fill around the target object area, and use the texture map to fill the filled area of the video frame, and finally combine the filled video frames according to the playback time sequence to obtain the special effect video.
  • the shape structure makes the stroke texture special effect generated by filling the video frame more smooth and smooth, and improves the rendering effect of the stroke texture special effect in the special effect video.
  • FIG. 1 is a flow chart of the steps of a method for generating an image special effect provided by an embodiment of the present application
  • Fig. 2 is a partially enlarged view of a filling area provided by the embodiment of the present application.
  • FIG. 3 is a flow chart of the steps of another method for generating an image special effect provided by an embodiment of the present application
  • FIG. 4 is a schematic diagram of a texture material map segmentation provided by an embodiment of the present application.
  • Fig. 5 is a block diagram of a device for implementing stroke texture special effects provided by an embodiment of the present application
  • FIG. 6 is a logical block diagram of an electronic device according to an embodiment of the present application.
  • Fig. 7 is a logic block diagram of an electronic device according to another embodiment of the present application.
  • Fig. 1 is a flow chart of the steps of a method for generating image special effects provided by an embodiment of the present application. , medical equipment, fitness equipment, personal digital assistants, etc. As shown in Figure 1, the method may include:
  • Step 101 acquire a target video, and a video frame of the target video includes a target object.
  • the stroke texture effect can add a texture effect around the edge contour of the target object.
  • various special effects around the edge contour of the target object can be produced, such as Fluorescent special effects, rainbow special effects, etc., help to highlight the target object in the image and increase the interest of the image.
  • the stroke texture effect can be applied to a single image or to a video containing multiple video frames. When stroking the target object in the video, it is necessary to add a stroke to the target object in the multi-frame image in the video. Edge texture effects to add stroke texture effects to the target object in the video.
  • the target video may be a video picture being taken by the user, for example, a video picture captured by a camera presented in a viewfinder frame during the shooting process of the user using a device with a video shooting function such as a mobile terminal. It may also be a video file selected by the user, for example, a video file downloaded by the user through the network.
  • the target object does not necessarily exist in each frame of the target video, the target object may move out of the frame range at some moments, and no further processing is required for the video frame without the target object. Therefore, when obtaining the video frame in the target video Finally, the video frame can be detected first to determine whether there is a target object in the video frame, so as to avoid the waste of computing power caused by subsequent processing of the video frame without the target object.
  • the target object may be any object, such as a human body, a plant, a pet, an article, and the like.
  • the video frame in the target video can be input into the object recognition model, the object recognition is performed on the video frame, and the corresponding object recognition result is output, if the object recognition result output by the object recognition model matches the target object, then A video frame may be determined to contain a target object.
  • the network structure of the above object recognition model can be flexibly designed according to actual needs.
  • the object recognition model may include but not limited to: convolutional layer, Relu layer, pooling layer and fully connected layer, etc., the more layers the object recognition model contains, the higher the recognition accuracy; another example, the object recognition model
  • the network structure can be adopted but not limited to: ALexNet or deep residual network and other network structures.
  • the target video can be the video screen that the user is playing
  • object detection can be performed on the video frames in the target video through the object recognition model set on the local device to meet the requirements of adding The demand for real-time stroke texture effects.
  • the video frames in the target video can be detected through the object recognition model, and the video frames containing the target object can be determined, which avoids subsequent processing of the video frames without the target object, and saves calculations. resource.
  • the object recognition model By setting the object recognition model locally, the response speed of object detection for video frames is improved, and the data security of users is also guaranteed.
  • the user before adding the stroke texture effect, can determine the target object to be added with the stroke texture effect.
  • the method of determining the target object can be that the user selects an object in the video as the target object when the video is playing. For example, when watching a video, the user may click or select character A in the video, so as to determine character A as the target object. It is also possible for the user to pre-select the target object category to which the stroke texture effect will be added through a list, menu, etc. Pets in a video file selected by the user are identified as target objects.
  • both the target object and the target object category can include multiple types.
  • the user can determine the human body and pets as the target object category, and then can add a stroke texture effect to both the human body and the pet in the target video, or in the Select character A and character B in the video, and then add stroke texture effects to character A and character B at the same time.
  • the user can select the target object to add the stroke texture special effect in the video screen, or pre-determine the target object category to add the texture special effect, and can also adjust the number of target objects and target object categories to improve It provides users with the flexibility and convenience of adding stroke texture effects to videos.
  • Step 102 in the video frame, determine stroke key points located outside the target object.
  • the embodiment of the present application needs to add a stroke texture effect to the target object, in order to make the stroke texture effect can be drawn on the edge position of the outline of the target object, it is necessary to further determine in the video frame after determining the video frame where the target object exists Multiple stroke keys around the outside of the target object.
  • the higher the distribution density of the determined stroke key points in the video frame the more accurate the stroke texture effect will be, and the lower the density will be, the more it will help to improve the calculation speed. Therefore, it can be flexible according to actual needs Adjusts the resulting stroke key density.
  • the video frame containing the target object in the target video can be input into the stroke key point detection model, the stroke key points distributed on the outline of the target object in the video frame can be determined, and each stroke key point is located in the video frame Position coordinates in the frame.
  • the network structure of the above stroke key point detection model can be flexibly designed according to actual needs.
  • the stroke key point detection model can include but not limited to: convolutional layer, Relu layer, pooling layer and fully connected layer, etc. The more layers the stroke key point detection model contains, the higher the recognition accuracy;
  • the network structure of the object recognition model may adopt, but is not limited to: ALexNet or deep residual network and other network structures.
  • different stroke key point detection models can be used to determine the stroke key points on its outline, so as to improve the efficiency and accuracy of determining the stroke key points.
  • the human body stroke key point detection model is used to determine the stroke key points on the outline of the human body in the video frame;
  • the plant stroke key point detection model is used to determine Stroke keys on plant silhouettes in video frames.
  • the stroke texture effects need to be displayed in real time. However, due to device performance limitations, it takes a certain amount of processing time to generate stroke texture effects for video frames. If the time is too long, it may cause the recording screen or video playback screen to freeze.
  • each frame in the target video may be processed, or video frames in the target video may be processed at preset frame intervals to reduce the impact on device performance.
  • the preset frame interval is 0, each frame in the target video is processed, and when the preset frame interval is 1 frame, the video frames in the target video are processed at 1 frame intervals.
  • the processing frame interval time between the video frames to be processed can be determined according to the frame rate of the recording picture or the video playback picture, and the processing time for adding stroke texture special effects to the video frame can be monitored at the same time.
  • frame interval time increase the number of preset frame intervals so that the processing time is not greater than the processing frame interval time.
  • the frame interval of the video can be calculated as 1/30 second by dividing 1 second by the frame rate.
  • the video frames are continuously processed, and the preset frame interval is 0.
  • the processing frame interval time is also 1/30 second. If it is monitored that the processing time of each frame is 1/20 second, it can be determined that the processing time 1/20 is greater than the processing frame interval time 1/30, and the preset frame interval needs to be increased.
  • the video frame is processed at an interval of 1 frame, so the processing frame interval is increased to 1/15 seconds, and the processing frame interval is 1/ 15 is less than 1/20 of the processing time, which can ensure the smooth playback of the video, and will not cause the target video to freeze due to the addition of stroke texture effects.
  • Step 103 On any straight line passing through the stroke key point, establish two extension points respectively located on both sides of the stroke key point, and according to each two adjacent stroke key points and their corresponding The extension points form a quadrilateral area, and a filled area formed by connecting multiple quadrilateral areas is obtained.
  • a filled area around the target object can be constructed in the video frame through the aforementioned stroke keypoints around the target object.
  • the transition between the two quadrilaterals is smoother, so multiple stroke keys around the target object can be used to generate multiple quadrilateral regions and make adjacent quadrilaterals Regions share the same edge so that all quadrilateral regions form a smoother fill region around the target object.
  • two extension points can be extended on any straight line passing through it, and one of the two extension points is located on the side of the stroke key point.
  • another expansion point is located on the other side of the stroke key point, that is, the two expansion points corresponding to a stroke key point are located on both sides of the stroke key point, and the connection The line passes through this stroke key.
  • the two stroke key points extended from each stroke key point can form a quadrilateral area with the two stroke key points extended from adjacent stroke key points, since every two adjacent quadrilateral areas share The connection line of two extension points extended by the same stroke key point, so that all quadrilateral areas can form a smoother filled area.
  • FIG. 2 it shows a partial enlarged view of a filling area provided by an embodiment of the present application.
  • the partial enlarged view includes four stroke key points O1 to O4, and two extension points P1 and P2 are established around O1.
  • establish two extension points P3 and P4 around O2 establish two extension points P5 and P6 around O3, and establish two extension points P7 and P8 around O4, then the quadrilateral area formed by P1, P2, P3, and P4 is the same as
  • the quadrilateral area composed of P3, P4, P5, and P6 shares the line segment P3-P4 as the shared side of the two, and the quadrilateral area composed of P3, P4, P5, and P6 shares the line segment P5-P5 with the quadrilateral area composed of P5, P6, P7, and P8.
  • P6 is used as the common side of the two, so that the quadrilateral area formed by P1, P2, P3, and P4, the quadrilateral area formed by P3, P4, P5, and P6, and the quadrilateral area formed by P5, P6, P7, and P8 are filled together.
  • the outline of the region is smoother.
  • Step 104 Fill the filled area of the video frame by using a preset texture material map to obtain a filled video frame.
  • the filling area of each video frame may be filled with a special effect pattern to obtain a filled video frame.
  • the effect of the stroke texture effect is related to the texture material map used, for example, the fluorescent stroke effect can be obtained by using the texture material map of the fluorescent style to fill the filled area, and the texture material map of the rainbow style is used to fill the filled area. Filling can obtain rainbow stroke special effect, therefore, according to the stroke texture effect to be achieved, different texture material maps can be selected to fill the filled area.
  • the texture material map when used to fill the filled area, the texture material map may be directly filled to the entire filled area, so as to achieve a fast filling effect. In some other embodiments, when filling the filled area, the texture material map may be filled into each quadrilateral area respectively, so as to achieve a better filling effect.
  • Step 105 Combine the filled video frames according to time sequence to obtain special effect video.
  • the video frames are arranged in a certain order in the target video, when obtaining video frames from the target video, the video frames can be obtained in the order of the video frames in the target video, and after filling the video frames to obtain the filled video frames , according to the acquisition sequence of the video frames, the filled video frames are combined to obtain the special effect video.
  • the video frames are combined so that the padded video frames are arranged in the same order as the corresponding video frames in the target video.
  • the embodiment of the present application provides a method for generating image special effects.
  • the key points of the stroke By extracting the key points of the stroke from the acquired video frame containing the target object, the key points of the stroke outside the target object are obtained.
  • Each stroke key point is expanded, and a connection line is constructed around each stroke key point to pass through two extension points of the stroke key point, so that the expansion points corresponding to all adjacent stroke key points can form a quadrilateral area, Furthermore, all quadrilateral areas can form a filling area around the target object, and the filling area of the video frame is filled with texture maps, and the filled video frames are combined in order of playing time to obtain a special effect video.
  • the shape structure makes the stroke texture special effect generated by filling the video frame more smooth and smooth, and improves the rendering effect of the stroke texture special effect in the special effect video.
  • Fig. 3 is a flow chart of the steps of another method for generating image special effects provided by an embodiment of the present application.
  • Electronic device implementations for equipment, medical equipment, fitness equipment, personal digital assistants, etc. As shown in Figure 3, the method may include:
  • Step 201 Obtain a target video, where a video frame of the target video includes a target object.
  • Step 202 In the video frame, determine stroke key points located outside the target object.
  • step 102 The implementation of this step is similar to the implementation process of step 102 above, and will not be described in detail here.
  • Step 203 On any straight line passing through the stroke key point, establish two extension points respectively located on both sides of the stroke key point, and according to each two adjacent stroke key points and their corresponding The extension points form a quadrilateral area, and a filled area formed by connecting multiple quadrilateral areas is obtained.
  • Step 203 may also include:
  • Sub-step 2031 determine the extension point straight line passing through the stroke key point and perpendicular to the connection line formed by the stroke key point and an adjacent stroke key point, the adjacent stroke key point refers to the The stroke key adjacent to the stroke key.
  • An adjacent stroke key corresponding to a stroke key can be the previous stroke key of this stroke key, or it can be the next stroke key of this stroke key.
  • the key point connecting lines are virtual straight lines used to determine the direction in which the expansion point is located at the corresponding stroked key point.
  • the stroke key point sequence can be established according to the distribution order of the stroke key points on the video frame, and the previous stroke key point or the next stroke key point of each stroke key point can be determined according to the adjacent relationship in each stroke key point sequence Edge key points.
  • the stroke key points on the outside of the target object can be connected to form an open curve around the target object, Causes stroke keys at the ends of open curves to have no previous or next stroke keys.
  • the vertical direction of the connection line between the endpoint stroke key point and any adjacent stroke key point can be determined as the extension point corresponding to the endpoint stroke key point. direction.
  • the stroke key points in the stroke key point sequence are arranged according to the distribution sequence of the stroke key points in the video frame, therefore, the stroke key points of any stroke key point in the stroke key point sequence
  • the key points are all adjacent in the video frame.
  • a stroke key point vector can be determined according to each stroke key point and its adjacent stroke key points in the stroke key point sequence, and the directions of all stroke key point vectors are pointed from the stroke key point to the stroke Neighboring stroke keys of the key.
  • the vector is the stroke key point vector, where the stroke key point O2 and the adjacent stroke key point O1 form a vector
  • a vector is formed between the stroke key point O3 and its adjacent stroke key point O2
  • a vector is formed between the stroke key point O4 and its adjacent stroke key point O3
  • the orthogonal unit vector orthogonal to the stroke key point vector can be calculated according to the properties of the orthogonal vectors.
  • the direction of the key point line corresponding to the stroke key point can be further determined.
  • Orthogonal unit vectors can reflect the direction of the keypoint line.
  • the above-mentioned orthogonal unit vector can also be directly calculated according to the coordinates of the stroke key point and the adjacent stroke key points, as shown in Figure 2, the stroke key point adjacent to the stroke key point O2 is O1,
  • the coordinates of the stroke key point O2 in the video frame are (X2, Y2)
  • the coordinates of the stroke key point O1 adjacent to the stroke key point O2 in the video frame are (X1, Y1), constructed according to O1 and O2
  • the stroke keypoint vector of is (X2-X1, Y2-Y1), then according to the nature of the orthogonal vector, the key point vector of the stroke can be calculated
  • a vertical orthogonal vector (Y2-Y1, X1-X2), the orthogonal vector is unitized, that is, the orthogonal vector is divided by its own modulus, and the orthogonal unit vector of the stroke key point O2 is obtained.
  • Sub-step 2032 Establish two extension points on the straight line of the extension point with equal distances from the stroke key point.
  • the line between the two extension points corresponding to each stroke key point passes through the stroke key point, and the two extension points corresponding to each stroke key point reach the stroke key point , that is, each stroke key is located at the midpoint of the line between the two extension points corresponding to the stroke key.
  • the filling area enclosed by the extension points of each stroke key point can surround the target object, and form a filling area according to the distribution of stroke key points, so the effect of the filling area fitting the target object is better.
  • Substep 2032 may also include:
  • Sub-step A1 According to the arrangement order of the stroke key points outside the target object, a plurality of the stroke key points are formed into a stroke key point sequence.
  • the distribution of the stroke key points in the stroke key point sequence needs to be determined according to the adjacent relationship of the stroke key points in the video frame, that is, the stroke key points in the stroke key point sequence are determined according to their position on the edge of the target object in order of distribution.
  • the stroke key points on the outline of the target object can form a line around the target object.
  • the stroke keypoints on the outline of the target object can form a closed curve around the target object, and any stroke keypoint can be used as The initial stroke key point is added to the stroke key point sequence, and the rest of the stroke key points are added to the stroke key point sequence in turn according to the arrangement order of the stroke key points on the curve.
  • Sub-step A2 on the key point line corresponding to the initial stroke key point in the stroke point sequence, establish two extension points whose distance to the initial stroke key point is the first distance.
  • the vertical direction of the connection line direction formed by the stroke key point and the adjacent stroke key point can be determined as the extension direction of the extension point line corresponding to the stroke key point, and one end of the extension point line corresponding to the stroke key point can be determined is the first direction, and the other end of the extension point line corresponding to the stroke key point is determined as the second direction.
  • the first distance is half of the lateral width of the filled area at the key point of the initial stroke. Therefore, by setting different first distances, the width of the filled area can be adjusted.
  • the first distance can be input or selected by the user through the graphical interface, and the first distance can also be preset by the system.
  • the stroke texture effect displayed on high-resolution devices may be too wide, while the stroke texture effect displayed on low-resolution devices may be too wide.
  • the edge texture special effect is too narrow, therefore, the first distance may be determined according to the screen resolution of the terminal device.
  • a first preset coefficient can be set, and the horizontal resolution or vertical resolution of the screen of the terminal device is multiplied by the first preset coefficient to obtain the first distance corresponding to the terminal device.
  • the first The distance can be the number of screen pixels. For example, if the horizontal resolution of the terminal device is 1000, and the first preset coefficient is 0.05, then the calculation result of the first distance is 50 screen pixels.
  • the first distance may also be determined according to the resolutions of the target videos themselves.
  • a second preset coefficient can be set, and the horizontal resolution or vertical resolution of the target video is multiplied by the second preset coefficient to obtain the first distance corresponding to the target video.
  • the first distance Can be the number of pixels in the target video. For example, if the horizontal resolution of the target video is 1000 and the second preset coefficient is 0.06, then the calculation result of the first distance is 60 pixels of the target video.
  • one of the expansion points is located in the first direction of the start stroke key, and the other first expansion point is located at the start stroke key The opposite direction of the first direction, that is, the second direction.
  • each initial stroke key point can be moved toward the orthogonal unit vector in the coordinate system corresponding to the video frame Move the first distance in the indicated direction to get an extension point corresponding to each starting stroke key point, and move each starting stroke key point in the opposite direction of the orthogonal unit vector in the coordinate system corresponding to the video frame The first distance to get another extension point corresponding to each starting stroke keypoint.
  • the coordinates of another first extension point corresponding to the starting stroke key point can be calculated by the following formula through the coordinates of the starting stroke key point, the orthogonal unit vector and the first distance:
  • the first distance can be automatically set according to the resolution of the terminal screen or target video, so that the generated stroke texture effect is always presented with an appropriate width on different terminal screens or different target videos, which significantly improves the The scope of application and display effect of the stroke texture special effect generated by the embodiment of the application.
  • Sub-step A3 according to the order of the non-initial stroke key points in the stroke key point sequence, determine the target distance scaling factor corresponding to the non-initial stroke key point from the magnification correspondence relationship, and the magnification corresponds to The relationship is used to characterize the correspondence between the order of the non-initial stroke key points in the stroke key point sequence and the distance scaling factor.
  • Each non-initial stroke key point corresponds to a second distance, and the second distance corresponding to a non-initial stroke key point is half of the horizontal width of the filled area at the position of the non-initial stroke key point , therefore, setting different second distances for different non-initial stroke key points can make the width of the filled area different at different stroke key points, thereby making the width of the stroke texture effect different at different positions, which helps To improve the expressiveness of the stroke texture effect, it enriches the shape of the stroke texture effect.
  • a magnification correspondence can be preset, and the magnification correspondence is used to characterize the correspondence between the order of the non-initial stroke key points in the stroke key point sequence and the distance zooming magnification. By querying the magnification corresponding relationship, determine the distance zoom magnification corresponding to each non-initial stroke key point. It is also possible to input the order of the non-initial stroke key points in the stroke key points into the preset function.
  • the preset function can be binary functions such as trigonometric functions, logarithmic functions, exponential functions, or other types The function. The default function outputs the distance scaling factor for non-initial stroke keys in different orders.
  • Sub-step A4 Determine the product of the target distance scaling factor and the first distance as the second distance corresponding to the non-initial stroke key point.
  • the second distance corresponding to each non-initial stroke key point may also be determined in other ways, which is not limited in this embodiment of the present application.
  • the non-initial stroke key points in the middle of the stroke key point sequence correspond to a larger distance scaling factor, and the second distance is larger than the first distance
  • the non-initial stroke key points at both ends of the stroke key point sequence The corresponding distance zoom ratio is small, and the second distance is smaller than the first distance, so a filling area with varying width can be formed, and finally a stroke texture effect with a gradient effect in width can be obtained, which improves the vividness of the stroke texture effect , for example, it can form a crescent-shaped stroke texture effect with thin ends and a thick middle, a wavy stroke texture effect with periodic changes in thickness, etc.
  • Sub-step A5 On the extension point straight line corresponding to the non-initial stroke key point in the stroke point sequence, establish two extension points whose distance to the non-initial stroke key point is the second distance.
  • each non-initial stroke key point After determining the orthogonal unit vector corresponding to the non-initial stroke key point, that is, the direction of the extension point straight line of the non-initial stroke key point, and the second distance, each non-initial stroke key point can be placed at In the coordinate system corresponding to the video frame, move the second distance in the direction indicated by the orthogonal unit vector to obtain an extension point corresponding to each non-initial stroke key point, and each non-initial stroke key point can be placed in the video frame Moving the second distance in the opposite direction of the orthogonal unit vector in the corresponding coordinate system to obtain another extension point corresponding to each non-initial stroke key point.
  • the second distance calculates the coordinates of a first extension point corresponding to a non-initial stroke key point by the following formula:
  • the coordinates of another first extension point corresponding to the non-initial stroke key point can be calculated by the following formula:
  • Step 204 dividing the texture material map into texture material sub-blocks with a preset number of triangles.
  • Different texture material maps can be preset for different stroke texture effects, and the corresponding texture material maps can be selected when generating stroke texture effects.
  • multiple texture material map libraries can be established, and the corresponding relationship between the texture material library and different types of stroke texture special effects can be constructed.
  • multiple texture material maps can be stored in each texture material map library, and multiple texture material maps in the corresponding texture material map library can be obtained when generating stroke texture effects.
  • different texture material maps can be filled for different quadrilateral areas in the filled area, so that the final stroke texture effect can present a richer effect.
  • the user may also specify the texture material map to be filled before generating the texture special effect.
  • the user can select any one or more pictures on the local device or the server, and use them as texture material maps to fill the filling area. In order to produce stroke texture effects that are more varied and more in line with user intentions.
  • Step 204 may also include:
  • Sub-step 2041 from one end of the texture material map to the other end, divide the texture material map into equal widths to obtain quadrilateral texture material blocks equal in number to the quadrilateral regions.
  • the number of texture material maps may be small, for example, when only one texture material map is obtained to generate a stroke texture effect, if the overall image of the texture material is filled in each quadrilateral area, the generated stroke texture effect will be in the quadrilateral area
  • the junction transitions are not natural. Therefore, in order to fill the filled area composed of multiple quadrangular areas and achieve a better filling effect, it is necessary to segment the texture material map and generate texture material blocks with the same number as the quadrilateral area, so that the filled area can be filled as a whole. Restore the overall effect of the texture material map, so that the transition after the texture material is filled into the quadrilateral area is more natural.
  • the texture material can be divided from one side of the texture material map to the opposite side according to the same spacing, and the texture material map can be divided into texture material blocks with the same width as the number of quadrilateral regions.
  • FIG. 4 it shows a schematic diagram of a texture material map segmentation provided by an embodiment of the present application, wherein, from the left side of the texture material map to the right side of the texture material map, the texture material map is sequentially cut into n texture materials blocks, each texture material block has the same width.
  • Sub-step 2042 Sequentially divide each texture material block into two texture material sub-blocks along a diagonal line according to the generation sequence of the texture material blocks.
  • each texture material block is in the shape of a quadrilateral, a diagonal line of each texture material block divides it into two triangles, therefore, each texture material block can be divided into two triangles along a diagonal line Texture material subblock.
  • the texture material block A1A2B2B1 can be divided into the texture material sub-block A1A2B1 and the texture material sub-block A2B1B2 along the diagonal formed by the corner point A2 and the corner point B1.
  • the generation order of the texture material block can reflect the distribution of patterns in the texture material map, and when the texture material block is divided, it is in accordance with The generated sequence of the texture material block is segmented, and the obtained texture material sub-blocks are arranged according to the generated sequence, which can restore the pattern in the texture material map.
  • Step 205 According to the arrangement order of the quadrilateral areas in the filling area, starting from one end of the filling area, sequentially divide each quadrilateral area into two filling sub-areas along a diagonal line, to obtain The preset number of filled sub-regions.
  • Each quadrilateral area is divided along one of its diagonals, and two filling sub-areas are generated for each quadrilateral area for filling the triangular material sub-block.
  • the quadrilateral area P1P2P3P4 is cut along the diagonal line connecting P2P3 to obtain the filled sub-area P1P2P3 and the filled sub-area P2P3P4.
  • Step 206 establishing a one-to-one correspondence between the texture material sub-block and the filling sub-region.
  • each texture material block is divided into two texture material sub-blocks, and each quadrilateral area is divided into two filling sub-areas, therefore, the number of texture material sub-blocks is the same as The number of filled subregions is the same. A one-to-one correspondence between all texture material sub-blocks and all filling sub-regions may be further established.
  • the entire filled area can restore the overall effect of the texture material map, making the transition of the stroke texture effect formed by the sub-block of the texture material filled to the filled sub-area more natural.
  • Step 206 may also include:
  • Sub-step 2061 Establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions according to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, wherein the generation order is the same
  • the texture material sub-blocks correspond to the filled sub-regions.
  • the texture material blocks are obtained by cutting from one side to the opposite side, the adjacent texture material blocks are adjacent in the texture material map, and the transition is the most natural, and the texture material sub-blocks are also textured according to the generation order of the texture material blocks.
  • the material blocks are obtained by segmenting, so the adjacent texture material sub-blocks in the generation order are also adjacent in the texture material map, and the transition is the most natural.
  • the filled sub-area is obtained by dividing each quadrilateral area according to the arrangement order of the quadrilateral areas in the filled area, so the filled sub-areas that are generated in adjacent order are also adjacent in the filled area, and the first generated filled sub-area is located in One end of the filled region, and the last generated filled subregion at the other end of the filled region.
  • the texture material at one end of the texture material block A subblock corresponds to a padding subarea at one end of the padding area.
  • the overall effect of the texture material map can be restored in the filled area, so that the transition after the texture material is filled into the quadrilateral area is more natural.
  • Step 207 Perform deformation processing on the texture material sub-block, so that the texture material sub-block matches the shape of the corresponding filling sub-region.
  • the shape of the texture material sub-block may not be exactly the same as the corresponding filling sub-area, before filling the texture material sub-block into the corresponding filling sub-area, it is also necessary to deform the texture material sub-block to make it consistent with The shape of the corresponding filled subregion matches.
  • the coordinates of each material vertex of the texture material sub-block and the coordinates of each filling vertex of the corresponding filling sub-region can be obtained, and a corresponding relationship between each material vertex coordinate and each filling vertex coordinate can be established. Then, each vertex of the texture material sub-block is adjusted so that the coordinates of each material vertex of the texture material sub-block are consistent with the corresponding filling vertex coordinates, and the deformed filling sub-region is obtained.
  • Step 208 According to the one-to-one correspondence between the texture material sub-block and the filling sub-area, fill the texture material sub-block into the corresponding filling sub-area.
  • an expansion point array containing all expansion points corresponding to stroke key points, the first index used to indicate all filled sub-areas in the filled area, and the first index used to indicate all filled sub-areas can be established in advance.
  • the second index of the texture material sub-block, and the extension point array, the first index and the second index are input into OpenGL, graphics drawing models such as Web Graphics Library (Web Graphics Library, WebGL), to fill the filled area of the video frame, Get the filled video frame. In this way, the filling of the entire filling area can be completed with only one call to the graphics drawing model, which improves the filling efficiency.
  • the operation steps are as follows:
  • Sub-step B1 Add the coordinates of the two extension points corresponding to each stroke key point to the extension point array in turn according to the order of the stroke key points in the stroke key point sequence, so as to obtain the constructed extension point array, Among them, the stroke key point sequence is constructed according to the adjacent relationship of the stroke key points.
  • the extension point array is used to store all the extension points generated by the stroke key point, including the first extension point generated by the start stroke key point and the second extension point generated by the non-start stroke key point.
  • the extension points corresponding to all the stroke key points can be sequentially generated according to the sequence of the stroke key points in the stroke key point sequence. And, when generating two first extension points corresponding to the initial stroke key point, the first first extension point generated first is recorded as the initial extension point. When generating two second extension points corresponding to non-initial stroke key points, first generate the second extension point on the same side of the fill area as the initial extension point. For example, as shown in Figure 2, the expansion points P1, P3, P5, and P7 are located on one side of the filled area, and the expanded points P2, P4, P6, and P8 are located on the other side of the filled area.
  • extension points corresponding to each stroke key point When generating the extension points corresponding to each stroke key point, all the first extension points and the second extension points are sequentially added to the extension point array according to the generation sequence of the above extension points. So that the extension points in the extension point array follow their order in the extension point array, and every three adjacent extension points indicate a filled sub-area divided by a quadrilateral area. It should be noted that when the extension point is added to the extension point array, the coordinates corresponding to the extension point are added.
  • the formed extension point array is (P1, P2, P3, P4, P5, P6, P7, P8), where the extension point P1 , P2, P3 form a filling sub-area P1P2P3, expansion points P2, P3, P4 form a filling sub-area P2P3P4, and so on, every three adjacent expansion points in the expansion point array can correspond to a filling sub-area.
  • Sub-step B2 starting from one end of the extension point array, select all extension point groups in sequence, and the extension point group includes three consecutive extension point coordinates.
  • extension point array three adjacent extension points that fill a sub-area may be indicated to form an extension point group.
  • (P1, P2, P3) is sequentially selected to form an extension point group
  • (P2, P3, P4) is selected to form an extension point group.
  • sub-step B3 sequentially constructing the identifiers of the coordinates of the three extension points of each extension point group as a first index, and adding the first index to the first index array, so as to obtain the constructed first index array.
  • the identifier of the extension point coordinates can be the arrangement order of the extension point coordinates in the extension point array, for example, the first extension point coordinate in the extension point array corresponds to 1, and the second extension point coordinate in the extension point array corresponds to The ID of is 2, and so on. It can also be other identifiers that can indicate the coordinates in the extension point array.
  • extension point arrays (P1, P2, P3) and (P2, P3, P4) if the extension point P1 is identified as 1, P2 is identified as 2, P3 is identified as 3, and P4 is identified as 4, then
  • the first index corresponding to the extension point array (P1, P2, P3) is (1, 2, 3), and the first index corresponding to the extension point array (P2, P3, P4) is (2, 3, 4).
  • the first indexes corresponding to all the extension point groups are sequentially added to the first index array according to the arrangement order of the extension point groups in the extension point array, and the construction of the first index array is completed.
  • Sub-step B4 starting from one end of the texture material map, sequentially divide the texture material map into equal widths to obtain multiple texture material blocks.
  • Sub-step B5 sequentially divide each texture material block into two texture material sub-blocks.
  • Sub-step B6 sequentially construct the coordinates of the three corner points corresponding to each texture material sub-block in the texture material map as a second index, and add the second index to the second index array to obtain the constructed The second index array.
  • the second index array is used as a texture coordinate system, wherein one corner of the texture material map is the origin (0,0) of the texture map constituting the coordinate system, and the opposite corner of the corner where the origin is located is (1,1 ).
  • the corner point of each texture material block corresponds to a top corner coordinate in the texture material map. Since the texture material map is divided into n blocks, the coordinates of A1 in the texture coordinate system are (0,0), the coordinates of A2 in the texture coordinate system are (1/n,0), and so on.
  • Sub-step B7 input the first index array, the second index array and the extension point array into the preset drawing model, and obtain the drawing result of stroke texture special effect output by the drawing model.
  • the drawing model can include graphics drawing models such as OpenGL and Web Graphics Library (Web Graphics Library, WebGL), to draw the stroke texture special effect, and output the drawing result. Take OpenGL as an example to illustrate the graphics drawing process.
  • OpenGL Web Graphics Library, WebGL
  • the extension point array, the first index array and the second index array are constructed, the extension point array, the first index array and the second index array are passed into the shader (shader). Then by calling the glDrawElements() function in OpenGL once in GL_TRIANGLES drawing mode, the filled sub-area indicated by each first index in the first index array and the texture indicated by each second index in the second index array The material sub-blocks are matched, and all the texture material sub-blocks are filled into the corresponding filled sub-areas, and the drawing of the stroke texture effect is completed efficiently and quickly at one time.
  • the drawing of the entire stroke texture effect can be completed through only one model call, which can greatly improve the drawing stroke texture effect.
  • the efficiency helps save computing resources of the terminal device, speeds up the drawing speed of stroke texture effects, and greatly improves the user experience of adding stroke texture effects to videos or pictures.
  • Step 209 Combine the filled video frames according to time sequence to obtain special effect video.
  • step 105 For this step, reference may be made to step 105, which will not be repeated here.
  • each stroke key point is expanded.
  • the texture map is used to fill the filled area of the video frame, and the filled video frames are combined in order of playing time to obtain a special effect video.
  • the shape structure makes the stroke texture special effect generated by filling the video frame more smooth and smooth, and improves the rendering effect of the stroke texture special effect in the special effect video.
  • Fig. 5 is a block diagram of an implementation device for stroke texture effects provided by an embodiment of the present application, as shown in Fig. 5 , including:
  • the obtaining module 301 is configured to obtain a target video, the video frame of which contains a target object;
  • the key point module 302 is configured to, in the video frame, determine a stroke key point located outside the target object;
  • the fill area module 303 is configured to establish two extension points respectively located on both sides of the stroke key point on any straight line passing through the stroke key point, and according to each two adjacent stroke key points and their respective corresponding extension points form a quadrilateral area, and obtain a filled area formed by connecting multiple quadrilateral areas;
  • the filling module 304 is configured to use a preset texture material map to fill the filling area of the video frame to obtain a filled video frame;
  • the combination module 305 is configured to combine the filled video frames in time sequence to obtain special effect video.
  • the filled area module 303 includes:
  • the extension point straight line sub-module is configured to determine an extension point straight line that passes through the stroke key point and is perpendicular to a connecting line formed by the stroke key point and an adjacent stroke key point, and the adjacent stroke key point A key point refers to a stroke key point adjacent to the stroke key point;
  • the extension point submodule is configured to establish two extension points on the extension point straight line with equal distances from the stroke key point.
  • the extension point submodule includes:
  • the key point sequence submodule is configured to form a plurality of stroke key points into a stroke key point sequence according to the arrangement order of the stroke key points outside the target object;
  • the start key point sub-module is configured to establish two distances from the start stroke key point at the first distance on the extension point line corresponding to the start stroke key point in the stroke point sequence. extension point;
  • the non-initial key point sub-module is configured to establish two distances from the non-initial stroke key point on the straight line corresponding to the non-initial stroke key point in the stroke point sequence. Two-distance extension point.
  • the device also includes:
  • the scaling sub-module is configured to determine the target distance zoom magnification corresponding to the non-initial stroke key point from the magnification correspondence relationship according to the order of the non-initial stroke key point in the stroke key point sequence , the magnification correspondence is used to characterize the correspondence between the order of the non-initial stroke key points in the stroke key point sequence and the distance zoom magnification;
  • the second distance sub-module is configured to determine the product of the target distance scaling factor and the first distance as the second distance corresponding to the non-initial stroke key point.
  • the filling module 304 includes:
  • the material cutting sub-module is configured to divide the texture material map into texture material sub-blocks with a preset number of triangles
  • the filling area cutting sub-module is configured to divide each quadrilateral area into two parts along a diagonal line in sequence starting from one end of the filling area according to the order of arrangement of the quadrilateral areas in the filling area. filling sub-regions, obtaining the preset number of filling sub-regions;
  • a correspondence sub-module configured to establish a one-to-one correspondence between the texture material sub-block and the filling sub-region
  • the matching submodule is configured to deform the texture material sub-block, so that the texture material sub-block matches the shape of the corresponding filling sub-region;
  • the filling sub-module is configured to fill the texture material sub-block into the corresponding filling sub-area according to the one-to-one correspondence between the texture material sub-block and the filling sub-area.
  • the material cutting sub-module includes:
  • the first split module is configured to perform equal-width segmentation on the texture material map from one end to the other end of the texture material map, so as to obtain quadrilateral texture material blocks having the same number as the quadrilateral area;
  • the second sub-cutting module is configured to sequentially cut each texture material block into two texture material sub-blocks along a diagonal line according to the generation order of the texture material blocks.
  • the correspondence submodule includes:
  • the relationship establishment submodule is configured to establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions according to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, wherein , the texture material sub-blocks with the same generation order correspond to the filling sub-regions.
  • each stroke key point is expanded, and each stroke key point is Build a connection line around the edge key point and pass through two extension points of the stroke key point, so that the extension points corresponding to all adjacent stroke key points can form a quadrilateral area, and then all quadrilateral areas can form a filled area around the target object,
  • the texture map is used to fill the filling area of the video frame, and finally the filled video frames are combined according to the playing time sequence to obtain the special effect video.
  • the shape structure makes the stroke texture special effect generated by filling the video frame more smooth and smooth, and improves the rendering effect of the stroke texture special effect in the special effect video.
  • Fig. 6 is a block diagram of an electronic device 600 according to an exemplary embodiment.
  • the electronic device 600 may execute the method for generating an image special effect in the embodiment of the present application.
  • the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.
  • electronic device 600 may include one or more of the following components: processing component 602, memory 604, power component 606, multimedia component 608, audio component 610, input/output (I/O) interface 612, sensor component 614 , and the communication component 616.
  • the processing component 602 generally controls the overall operations of the electronic device 600, such as those associated with display, telephone calls, data communications, camera operations, and recording operations.
  • the processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 602 may include one or more modules that facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602 .
  • the memory 604 is used to store various types of data to support operations in the electronic device 600 . Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phonebook data, messages, pictures, videos, and the like.
  • the memory 604 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM erasable Programmable Read Only Memory
  • PROM Programmable Read Only Memory
  • ROM Read Only Memory
  • Magnetic Memory Flash Memory
  • Magnetic or Optical Disk Magnetic Disk
  • the power supply component 606 provides power to various components of the electronic device 600 .
  • Power components 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 600 .
  • the multimedia component 608 includes a screen providing an output interface between the electronic device 600 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user.
  • the touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action.
  • the multimedia component 608 includes a front camera and/or a rear camera. When the electronic device 600 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.
  • the audio component 610 is used to output and/or input audio signals.
  • the audio component 610 includes a microphone (MIC), which is used to receive external audio signals when the electronic device 600 is in operation modes, such as calling mode, recording mode and voice recognition mode. Received audio signals may be further stored in memory 604 or sent via communication component 616 .
  • the audio component 610 also includes a speaker for outputting audio signals.
  • the I/O interface 612 provides an interface between the processing component 602 and a peripheral interface module.
  • the peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.
  • Sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for electronic device 600 .
  • the sensor assembly 614 can detect the open/close state of the electronic device 600, the relative positioning of the components, such as the display and the keypad of the electronic device 600, the sensor assembly 614 can also detect the electronic device 600 or one of the electronic device 600
  • the position of components changes, the presence or absence of user contact with the electronic device 600 , the orientation or acceleration/deceleration of the electronic device 600 and the temperature of the electronic device 600 change.
  • the sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact.
  • Sensor assembly 614 may also include optical sensors, such as CMOS or CCD image sensors, for use in imaging applications.
  • the sensor component 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.
  • the communication component 616 is used to facilitate wired or wireless communication between the electronic device 600 and other devices.
  • the electronic device 600 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof.
  • the communication component 616 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID Radio Frequency Identification
  • IrDA Infrared Data Association
  • UWB Ultra Wide Band
  • Bluetooth Bluetooth
  • the electronic device 600 may be programmed by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Realized by a gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, for realizing the method for generating an image special effect provided by the embodiment of the present application.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable Realized by a gate array
  • controller a controller
  • microcontroller a microcontroller
  • microprocessor or other electronic components
  • non-transitory computer storage medium including instructions, such as the memory 604 including instructions, which can be executed by the processor 620 of the electronic device 600 to complete the above method.
  • the non-transitory storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, among others.
  • Fig. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment.
  • the electronic device 600 may execute the method for generating an image special effect in the embodiment of the present application.
  • the electronic device 700 may be provided as a server.
  • electronic device 700 includes processing component 722 , which further includes one or more processors, and a memory resource represented by memory 732 for storing instructions executable by processing component 722 , such as application programs.
  • the application program stored in memory 732 may include one or more modules each corresponding to a set of instructions.
  • the processing component 722 is configured to execute instructions to execute the method for generating an image special effect provided by the embodiment of the present application.
  • Electronic device 700 may also include a power supply component 726 configured to perform power management of electronic device 700, a wired or wireless network interface 750 configured to connect electronic device 700 to a network, and an input-output (I/O) interface 758 .
  • the electronic device 700 can operate based on an operating system stored in the memory 732, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.
  • An embodiment of the present application further provides a computer program product, including a computer program/instruction, and when the computer program/instruction is executed by a processor, the above method for generating an image special effect is implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)

Abstract

An image special effect generation method and apparatus. The method comprises: in a video frame, determining stroking key points located on an outer side of a target object; establishing, on any straight line that passes through the stroking key points, two extension points respectively located on two sides of each stroking key point, and forming a quadrilateral area according to every two adjacent stroking key points and respective corresponding extension points thereof, wherein a plurality of quadrilateral areas are connected to form a filling area; and filling the filling area using a texture material map, so as to obtain a special effect video.

Description

图像特效的生成方法及装置Method and device for generating image special effects
相关申请的交叉引用Cross References to Related Applications
本申请基于申请日为2021年8月31日、申请号为202111023554.X号的中国专利申请，并要求该中国专利申请的优先权，该中国专利申请的全部内容通过引用结合在本公开中。This application is based on the Chinese patent application with the application date of August 31, 2021 and the application number 202111023554.X, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is incorporated in this disclosure by reference.
技术领域technical field
本申请实施例涉及图像处理技术领域，尤其涉及一种图像特效的生成方法及装置、电子设备及存储介质。The embodiments of the present application relate to the technical field of image processing, and in particular, to a method and device for generating image special effects, electronic equipment, and a storage medium.
背景技术Background technique
随着手机和移动设备的普及，越来越多的人喜欢用视频来记录自己的生活，并对拍摄到的视频增加各种特效，其中，描边特效是人们经常选择的一种特效，描边特效可以在视频中目标对象的轮廓边缘产生特效图案。With the popularization of mobile phones and mobile devices, more and more people like to use videos to record their lives and add various special effects to the captured videos. Among them, the stroke special effect is a special effect that people often choose. The Edge Effect creates a special effect pattern on the edge of the outline of the target object in the video.
在相关技术中，对于每一帧视频画面，通常先获取目标对象的轮廓描边关键点，并在每个轮廓描边关键点的位置，以该描边关键点为中心构建一个独立矩形，形成围绕目标对象的多个独立矩形，再通过纹理素材贴图对构建出的所有独立矩形进行填充，以形成围绕目标对象的描边纹理特效。In the related art, for each frame of video picture, the outline stroke key point of the target object is usually obtained first, and at the position of each outline stroke key point, an independent rectangle is constructed with the stroke key point as the center, forming Multiple independent rectangles surrounding the target object, and then fill all the constructed independent rectangles with the texture material map to form a stroke texture effect around the target object.
发明内容Contents of the invention
本申请实施例提供一种图像特效的生成方法、装置、电子设备及存储介质。Embodiments of the present application provide a method, device, electronic device, and storage medium for generating an image special effect.
第一方面，本申请实施例提供了一种图像特效的生成方法，该方法包括：In the first aspect, the embodiment of the present application provides a method for generating image special effects, the method including:
获取目标视频，所述目标视频的视频帧包含目标对象；Obtain the target video, the video frame of the target video contains the target object;
在所述视频帧中，确定位于所述目标对象外侧的描边关键点；In the video frame, determine stroke key points located outside the target object;
在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，并根据每两个相邻描边关键点及其各自对应的扩展点构成一个四边形区域，得到多个所述四边形区域连接组成的填充区域；On any straight line passing through the stroke key point, establish two extension points respectively located on both sides of the stroke key point, and form A quadrilateral area, obtaining a filled area formed by connecting a plurality of said quadrilateral areas;
采用预设的纹理素材贴图，对所述视频帧的填充区域进行填充，得到填充后的视频帧；Filling the filled area of the video frame by using a preset texture material map to obtain the filled video frame;
将所述填充后的视频帧按照时间顺序组合，得到特效视频。Combining the filled video frames in chronological order to obtain a special effect video.
在一些实施例中，所述在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，包括：In some embodiments, the establishment of two extension points respectively located on both sides of the stroke key point on any straight line passing through the stroke key point includes:
确定经过所述描边关键点，且垂直于所述描边关键点与一个相邻描边关键点构成的连接线的扩展点直线，所述相邻描边关键点是指与所述描边关键点相邻的描边关键点；Determine the extension point straight line passing through the stroke key point and perpendicular to the connection line formed by the stroke key point and an adjacent stroke key point, the adjacent stroke key point refers to the Stroke key points adjacent to the key point;
在所述扩展点直线上建立两个到所述描边关键点距离相等的扩展点。Two expansion points with equal distances from the stroke key point are established on the expansion point straight line.
在一些实施例中，所述在所述扩展点直线上建立两个到所述描边关键点距离相等的扩展点，包括：In some embodiments, the establishment of two extension points on the straight line of the extension point with equal distances from the stroke key point includes:
按照所述描边关键点在所述目标对象外侧的排列顺序，将多个所述描边关键点构成描边关键点序列；According to the arrangement order of the stroke key points outside the target object, a plurality of the stroke key points are formed into a stroke key point sequence;
在所述描边点序列中的起始描边关键点对应的扩展点直线上，建立两个到所述起始描边关键点的距离为第一距离的扩展点；On the extension point straight line corresponding to the initial stroke key point in the stroke point sequence, establish two extension points whose distance to the initial stroke key point is a first distance;
在所述描边点序列中的非起始描边关键点对应的扩展点直线上，建立两个到所述非起始描边关键点的距离为第二距离的扩展点。On the extension point line corresponding to the non-initial stroke key point in the stroke point sequence, establish two extension points whose distance to the non-initial stroke key point is a second distance.
在一些实施例中，所述方法还包括：In some embodiments, the method also includes:
根据所述非起始描边关键点在所述描边关键点序列中的次序，从倍率对应关系中确定所述非起始描边关键点对应的目标距离缩放倍率，所述倍率对应关系用于表征非起始描边关键点在所述描边关键点序列中的次序和距离缩放倍率之间的对应关系；According to the order of the non-initial stroke key points in the stroke key point sequence, determine the target distance scaling factor corresponding to the non-initial stroke key point from the magnification corresponding relationship, and the magnification corresponding relationship is used To characterize the correspondence between the sequence of non-initial stroke key points in the sequence of stroke key points and the distance scaling factor;
将所述目标距离缩放倍率与所述第一距离的乘积，确定为所述非起始描边关键点对应 的第二距离。The product of the target distance scaling factor and the first distance is determined as the second distance corresponding to the non-initial stroke key point.
在一些实施例中，所述采用预设的纹理素材贴图，对所述目标视频中多个视频帧的填充区域进行填充，包括：In some embodiments, the use of a preset texture material map to fill the filling area of multiple video frames in the target video includes:
将所述纹理素材贴图切分为预设数量个三角形的纹理素材子块；Dividing the texture material map into texture material sub-blocks with a preset number of triangles;
按照所述四边形区域在所述填充区域中的排列顺序，从所述填充区域的一端开始，依次将每个所述四边形区域沿一条对角线切分为两个填充子区域，得到所述预设数量个填充子区域；According to the arrangement order of the quadrilateral areas in the filling area, starting from one end of the filling area, each of the quadrilateral areas is divided into two filling sub-areas along a diagonal line in turn, to obtain the predetermined Set the number of filled sub-regions;
建立所述纹理素材子块与所述填充子区域之间的一一对应关系；Establishing a one-to-one correspondence between the texture material sub-block and the filling sub-region;
对所述纹理素材子块进行变形处理，使得所述纹理素材子块与对应的填充子区域的形状匹配；deforming the texture material sub-block, so that the texture material sub-block matches the shape of the corresponding filling sub-region;
根据所述纹理素材子块与所述填充子区域之间的一一对应关系，将所述纹理素材子块填充至对应的填充子区域中。According to the one-to-one correspondence between the texture material sub-block and the filling sub-region, the texture material sub-block is filled into the corresponding filling sub-region.
在一些实施例中，所述将所述纹理素材贴图切分为预设数量个三角形的纹理素材子块，包括：In some embodiments, the dividing the texture material map into texture material sub-blocks with a preset number of triangles includes:
从所述纹理素材贴图的一端到另一端，对所述纹理素材贴图进行等宽切分，得到与所述四边形区域的数量相同个四边形的纹理素材块；From one end of the texture material map to the other end, the texture material map is divided into equal widths to obtain quadrilateral texture material blocks equal to the number of quadrilateral regions;
按照所述纹理素材块的生成次序，依次将每个所述纹理素材块沿一条对角线切分为两个纹理素材子块。According to the generation sequence of the texture material blocks, each of the texture material blocks is sequentially divided into two texture material sub-blocks along a diagonal line.
在一些实施例中，所述建立所述纹理素材子块与所述填充子区域之间的一一对应关系，包括：In some embodiments, the establishment of a one-to-one correspondence between the texture material sub-block and the filling sub-region includes:
按照所述纹理素材子块的生成次序以及所述填充子区域的生成次序，建立所述纹理素材子块和所述填充子区域之间的一一对应关系，其中，生成次序相同的纹理素材子块和填充子区域相对应。According to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions, wherein the texture material sub-blocks in the same order are generated Blocks correspond to filled subregions.
第二方面，本申请实施例提供了一种图像特效的生成装置，该装置包括：In the second aspect, the embodiment of the present application provides a device for generating image special effects, the device comprising:
获取模块，被配置为获取目标视频，所述目标视频的视频帧包含目标对象；An acquisition module configured to acquire a target video, where a video frame of the target video contains a target object;
关键点模块，被配置为在所述视频帧中，确定位于所述目标对象外侧的描边关键点；a key point module configured to determine stroke key points located outside the target object in the video frame;
填充区域模块，被配置为在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，并根据每两个相邻描边关键点及其各自对应的扩展点构成一个四边形区域，得到多个所述四边形区域连接组成的填充区域；The filling area module is configured to establish two extension points respectively located on both sides of the stroke key point on any straight line passing through the stroke key point, and according to each two adjacent stroke key points and The corresponding extension points form a quadrilateral area, and a filled area formed by connecting multiple quadrilateral areas is obtained;
填充模块，被配置为采用预设的纹理素材贴图，对所述视频帧的填充区域进行填充，得到填充后的视频帧；The filling module is configured to use a preset texture material map to fill the filling area of the video frame to obtain the filled video frame;
组合模块，被配置为将所述填充后的视频帧按照时间顺序组合，得到特效视频。The combination module is configured to combine the filled video frames in time order to obtain special effect video.
在一些实施例中，所述填充区域模块包括：In some embodiments, the filled region module includes:
扩展点直线子模块，被配置为确定经过所述描边关键点，且垂直于所述描边关键点与一个相邻描边关键点构成的连接线的扩展点直线，所述相邻描边关键点是指与所述描边关键点相邻的描边关键点；The extension point straight line sub-module is configured to determine an extension point straight line that passes through the stroke key point and is perpendicular to a connecting line formed by the stroke key point and an adjacent stroke key point, and the adjacent stroke key point A key point refers to a stroke key point adjacent to the stroke key point;
扩展点子模块，被配置为在所述扩展点直线上建立两个到所述描边关键点距离相等的扩展点。The extension point submodule is configured to establish two extension points on the extension point straight line with equal distances from the stroke key point.
在一些实施例中，所述扩展点子模块包括：In some embodiments, the extension point submodule includes:
关键点序列子模块，被配置为按照所述描边关键点在所述目标对象外侧的排列顺序，将多个所述描边关键点构成描边关键点序列；The key point sequence submodule is configured to form a plurality of stroke key points into a stroke key point sequence according to the arrangement order of the stroke key points outside the target object;
起始关键点子模块，被配置为在所述描边点序列中的起始描边关键点对应的扩展点直线上，建立两个到所述起始描边关键点的距离为第一距离的扩展点；The start key point sub-module is configured to establish two distances from the start stroke key point at the first distance on the extension point line corresponding to the start stroke key point in the stroke point sequence. extension point;
非起始关键点子模块，被配置为在所述描边点序列中的非起始描边关键点对应的扩展点直线上，建立两个到所述非起始描边关键点的距离为第二距离的扩展点。The non-initial key point sub-module is configured to establish two distances from the non-initial stroke key point on the straight line corresponding to the non-initial stroke key point in the stroke point sequence. Two-distance extension point.
在一些实施例中，所述装置还包括：In some embodiments, the device also includes:
缩放子模块，被配置为根据所述非起始描边关键点在所述描边关键点序列中的次序，从倍率对应关系中确定所述非起始描边关键点对应的目标距离缩放倍率，所述倍率对应关系用于表征非起始描边关键点在所述描边关键点序列中的次序和距离缩放倍率之间的对应关系；The scaling sub-module is configured to determine the target distance zoom magnification corresponding to the non-initial stroke key point from the magnification correspondence relationship according to the order of the non-initial stroke key point in the stroke key point sequence , the magnification correspondence is used to characterize the correspondence between the order of the non-initial stroke key points in the stroke key point sequence and the distance zoom magnification;
第二距离子模块，被配置为将所述目标距离缩放倍率与所述第一距离的乘积，确定为所述非起始描边关键点对应的第二距离。The second distance sub-module is configured to determine the product of the target distance scaling factor and the first distance as the second distance corresponding to the non-initial stroke key point.
在一些实施例中，所述填充模块包括：In some embodiments, the filling module includes:
素材切分子模块，被配置为将所述纹理素材贴图切分为预设数量个三角形的纹理素材子块；The material cutting sub-module is configured to divide the texture material map into texture material sub-blocks with a preset number of triangles;
填充区域切分子模块，被配置为按照所述四边形区域在所述填充区域中的排列顺序，从所述填充区域的一端开始，依次将每个所述四边形区域沿一条对角线切分为两个填充子区域，得到所述预设数量个填充子区域；The filling area cutting sub-module is configured to divide each quadrilateral area into two parts along a diagonal line in sequence starting from one end of the filling area according to the order of arrangement of the quadrilateral areas in the filling area. filling sub-regions, obtaining the preset number of filling sub-regions;
对应关系子模块，被配置为建立所述纹理素材子块与所述填充子区域之间的一一对应关系；a correspondence sub-module configured to establish a one-to-one correspondence between the texture material sub-block and the filling sub-region;
匹配子模块，被配置为对所述纹理素材子块进行变形处理，使得所述纹理素材子块与对应的填充子区域的形状匹配；The matching submodule is configured to deform the texture material sub-block, so that the texture material sub-block matches the shape of the corresponding filling sub-region;
填充子模块，被配置为根据所述纹理素材子块与所述填充子区域之间的一一对应关系，将所述纹理素材子块填充至对应的填充子区域中。The filling sub-module is configured to fill the texture material sub-block into the corresponding filling sub-area according to the one-to-one correspondence between the texture material sub-block and the filling sub-area.
在一些实施例中，所述素材切分子模块包括：In some embodiments, the material cutting sub-module includes:
第一切分子模块，被配置为从所述纹理素材贴图的一端到另一端，对所述纹理素材贴图进行等宽切分，得到与所述四边形区域的数量相同个四边形的纹理素材块；The first cutting module is configured to perform equal-width segmentation on the texture material map from one end to the other end of the texture material map, so as to obtain quadrilateral texture material blocks equal to the number of the quadrilateral regions;
第二切分子模块，被配置为按照所述纹理素材块的生成次序，依次将每个所述纹理素材块沿一条对角线切分为两个纹理素材子块。The second sub-cutting module is configured to sequentially cut each texture material block into two texture material sub-blocks along a diagonal line according to the generation order of the texture material blocks.
在一些实施例中，所述对应关系子模块包括：In some embodiments, the correspondence submodule includes:
关系建立子模块，被配置为按照所述纹理素材子块的生成次序以及所述填充子区域的生成次序，建立所述纹理素材子块和所述填充子区域之间的一一对应关系，其中，生成次序相同的纹理素材子块和填充子区域相对应。The relationship establishment submodule is configured to establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions according to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, wherein , the texture material sub-blocks with the same generation order correspond to the filling sub-regions.
第三方面，本申请实施例还提供了一种电子设备，包括用于存储所述处理器可执行指令的存储器；其中，所述处理器被配置为执行所述指令，以实现所述的图像特效的生成方法。In a third aspect, the embodiment of the present application also provides an electronic device, including a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to realize the image How to generate special effects.
第四方面，本申请实施例还提供了一种计算机可读存储介质，当所述计算机可读存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行所述的图像特效的生成方法。In the fourth aspect, the embodiment of the present application also provides a computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can execute the image special effect. generate method.
第五方面，本申请实施例还提供了一种计算机程序产品，包括计算机程序，所述计算机程序被处理器执行时实现所述的图像特效的生成方法。In a fifth aspect, the embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above-mentioned method for generating an image special effect is implemented.
在本申请实施例中，通过对获取的目标视频中包含目标对象的视频帧进行描边关键点提取，获得目标对象外侧的描边关键点，再对每个描边关键点进行扩展，在每个描边关键点周围构建出连接线经过该描边关键点的两个扩展点，使得所有相邻描边关键点对应的扩展点可以构成四边形区域，进而所有四边形区域可以形成围绕目标对象的填充区域，并采用纹理贴图对视频帧的填充区域进行填充，最终将填充后的视频帧按照播放时间顺序组合得到特效视频。由于相邻四边形区域之间使用同一个描边关键点对应的两个扩展点连接线作为共用边，因此相邻四边形区域之间可以形成较为平滑的过渡，形成的填充区域不会存在明显的锯齿状结构，使得对视频帧填充生成的描边纹理特效更为流畅平滑，提升了特效视频中描边纹理特效的呈现效果。In the embodiment of the present application, by extracting the key points of the stroke from the acquired video frames containing the target object, the key points of the stroke outside the target object are obtained, and then each key point of the stroke is expanded. Build a connecting line around a stroke key point and pass through two extension points of the stroke key point, so that the expansion points corresponding to all adjacent stroke key points can form a quadrilateral area, and then all quadrilateral areas can form a fill around the target object area, and use the texture map to fill the filled area of the video frame, and finally combine the filled video frames according to the playback time sequence to obtain the special effect video. Since two expansion point connecting lines corresponding to the same stroke key point are used as shared edges between adjacent quadrilateral areas, a relatively smooth transition can be formed between adjacent quadrilateral areas, and there will be no obvious jaggedness in the formed filled area The shape structure makes the stroke texture special effect generated by filling the video frame more smooth and smooth, and improves the rendering effect of the stroke texture special effect in the special effect video.
附图说明Description of drawings
通过阅读下文实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出实施方式的目的，而并不认为是对本申请的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the embodiments. The drawings are only for the purpose of illustrating the embodiments and are not to be considered as limiting the application. Also throughout the drawings, the same reference numerals are used to designate the same parts. In the attached picture:
图1是本申请实施例提供的一种图像特效的生成方法的步骤流程图；FIG. 1 is a flow chart of the steps of a method for generating an image special effect provided by an embodiment of the present application;
图2是本申请实施例提供的一种填充区域局部放大图；Fig. 2 is a partially enlarged view of a filling area provided by the embodiment of the present application;
图3是本申请实施例提供的另一种图像特效的生成方法的步骤流程图；FIG. 3 is a flow chart of the steps of another method for generating an image special effect provided by an embodiment of the present application;
图4是本申请实施例提供的一种纹理素材贴图切分示意图；FIG. 4 is a schematic diagram of a texture material map segmentation provided by an embodiment of the present application;
图5是本申请实施例提供的一种描边纹理特效的实现装置的框图；Fig. 5 is a block diagram of a device for implementing stroke texture special effects provided by an embodiment of the present application;
图6是本申请一个实施例的电子设备的逻辑框图；FIG. 6 is a logical block diagram of an electronic device according to an embodiment of the present application;
图7是本申请另一个实施例的电子设备的逻辑框图。Fig. 7 is a logic block diagram of an electronic device according to another embodiment of the present application.
具体实施方式Detailed ways
下面将参照附图更详细地描述本申请的示例性实施例。虽然附图中显示了本申请的示例性实施例，然而应当理解，可以以各种形式实现本申请而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本申请，并且能够将本申请的范围完整的传达给本领域的技术人员。Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present application can be more thoroughly understood, and the scope of the present application can be fully conveyed to those skilled in the art.
图1是本申请实施例提供的一种图像特效的生成方法的步骤流程图，该方法可以由诸如服务器、处理器、移动电话、计算机、数字广播终端、消息收发设备、游戏控制台、平板设备、医疗设备、健身设备、个人数字助理等的电子设备执行。如图1所示，该方法可以包括：Fig. 1 is a flow chart of the steps of a method for generating image special effects provided by an embodiment of the present application. , medical equipment, fitness equipment, personal digital assistants, etc. As shown in Figure 1, the method may include:
步骤101，获取目标视频，所述目标视频的视频帧包含目标对象。 Step 101, acquire a target video, and a video frame of the target video includes a target object.
在本申请实施例中，描边纹理特效可以对目标对象增加环绕其边缘轮廓的纹理特效，通过选取不同的纹理素材贴图对轮廓进行描绘，可以产生环绕目标对象边缘轮廓的多种特效效果，诸如荧光特效、彩虹特效等，有助于突出图像中的目标对象并增加图像的趣味性。描边纹理特效可以应用于单张图像，也可以应用于包含多个视频帧的视频中，在对视频中的目标对象进行描边时，需要对视频中的多帧图像中的目标对象添加描边纹理特效，以实现对视频中目标对象添加描边纹理特效。In the embodiment of this application, the stroke texture effect can add a texture effect around the edge contour of the target object. By selecting different texture material maps to describe the contour, various special effects around the edge contour of the target object can be produced, such as Fluorescent special effects, rainbow special effects, etc., help to highlight the target object in the image and increase the interest of the image. The stroke texture effect can be applied to a single image or to a video containing multiple video frames. When stroking the target object in the video, it is necessary to add a stroke to the target object in the multi-frame image in the video. Edge texture effects to add stroke texture effects to the target object in the video.
目标视频可以是用户正在拍摄的视频画面，例如，用户使用移动终端等具备视频拍摄功能的设备在拍摄过程中，取景框内所呈现的摄像头采集到的视频画面。也可以是用户选择的视频文件，例如，用户通过网络所下载的视频文件。The target video may be a video picture being taken by the user, for example, a video picture captured by a camera presented in a viewfinder frame during the shooting process of the user using a device with a video shooting function such as a mobile terminal. It may also be a video file selected by the user, for example, a video file downloaded by the user through the network.
由于目标视频中的每一帧并不一定均存在目标对象，目标对象有可能在一些时刻移出画面范围，对于不存在目标对象的视频帧无需进行进一步处理，因此，在获取目标视频中的视频帧后，可以先对视频帧进行检测，以确定视频帧中是否存在目标物体，避免对不存在目标对象的视频帧进行后续处理而造成算力浪费。其中，目标物体可以是任何物体，例如人体、植物、宠物、物品等。Since the target object does not necessarily exist in each frame of the target video, the target object may move out of the frame range at some moments, and no further processing is required for the video frame without the target object. Therefore, when obtaining the video frame in the target video Finally, the video frame can be detected first to determine whether there is a target object in the video frame, so as to avoid the waste of computing power caused by subsequent processing of the video frame without the target object. Wherein, the target object may be any object, such as a human body, a plant, a pet, an article, and the like.
在一些实施例中，可以将目标视频中的视频帧输入物体识别模型，对视频帧进行物体识别，并输出相对应的物体识别结果，若物体识别模型输出的物体识别结果与目标对象匹配，则可以确定视频帧包含目标对象。上述物体识别模型的网络结构可以根据实际需求灵活设计。例如，物体识别模型可以包括但不限于：卷积层、Relu层、池化层和全连接层等，物体识别模型所包含的层数越多，则识别精度越高；再例如，物体识别模型的网络结构可以采用但不限于：ALexNet或深度残差网络等网络结构。In some embodiments, the video frame in the target video can be input into the object recognition model, the object recognition is performed on the video frame, and the corresponding object recognition result is output, if the object recognition result output by the object recognition model matches the target object, then A video frame may be determined to contain a target object. The network structure of the above object recognition model can be flexibly designed according to actual needs. For example, the object recognition model may include but not limited to: convolutional layer, Relu layer, pooling layer and fully connected layer, etc., the more layers the object recognition model contains, the higher the recognition accuracy; another example, the object recognition model The network structure can be adopted but not limited to: ALexNet or deep residual network and other network structures.
由于目标视频可以是用户正在播放的视频画面，为了保证视频播放的流畅性，可以通过设置在本地设备上的物体识别模型对目标视频中的视频帧进行物体检测，以满足为正在 播放的视频添加实时描边纹理特效的需求。Since the target video can be the video screen that the user is playing, in order to ensure the smoothness of video playback, object detection can be performed on the video frames in the target video through the object recognition model set on the local device to meet the requirements of adding The demand for real-time stroke texture effects.
因此，在一些实施例中，可以通过物体识别模型对目标视频中的视频帧进行检测，能够确定出包含目标对象的视频帧，避免了对不存在目标对象的视频帧进行后续处理，节省了运算资源。还通过将物体识别模型设置在本地，提升了对视频帧进行物体检测的响应速度，同时还保证了用户的数据安全。Therefore, in some embodiments, the video frames in the target video can be detected through the object recognition model, and the video frames containing the target object can be determined, which avoids subsequent processing of the video frames without the target object, and saves calculations. resource. By setting the object recognition model locally, the response speed of object detection for video frames is improved, and the data security of users is also guaranteed.
在一些实施例中，用户可以在添加描边纹理特效之前，确定所要添加描边纹理特效的目标对象，确定目标对象的方式可以是由用户在视频播放时进行选取视频中的物体作为目标对象，例如，用户可以在观看视频时点击或框选视频中的人物A，以将人物A确定为目标对象。也可以由用户预先通过列表、菜单等方式选择要添加描边纹理特效的目标对象类别，例如，用户选择了宠物作为目标对象类别，则在拍摄时将拍摄到的宠物确定为目标对象，或将用户选择的视频文件中的宠物确定为目标对象。In some embodiments, before adding the stroke texture effect, the user can determine the target object to be added with the stroke texture effect. The method of determining the target object can be that the user selects an object in the video as the target object when the video is playing. For example, when watching a video, the user may click or select character A in the video, so as to determine character A as the target object. It is also possible for the user to pre-select the target object category to which the stroke texture effect will be added through a list, menu, etc. Pets in a video file selected by the user are identified as target objects.
需要说明的是，目标对象和目标对象类别均可以包括多个，例如，用户可以将人体和宠物确定为目标对象类别，进而可以对目标视频中的人体和宠物均添加描边纹理特效，或在视频中选择人物A和人物B，进而为人物A和人物B同时添加描边纹理特效。It should be noted that both the target object and the target object category can include multiple types. For example, the user can determine the human body and pets as the target object category, and then can add a stroke texture effect to both the human body and the pet in the target video, or in the Select character A and character B in the video, and then add stroke texture effects to character A and character B at the same time.
因此，在一些实施例中，用户可以在视频画面中选取要添加描边纹理特效的目标对象，或预先确定所要添加纹理特效的目标对象类别，还可以调整目标对象和目标对象类别的数量，提升了用户为视频添加描边纹理特效的灵活性和便利性。Therefore, in some embodiments, the user can select the target object to add the stroke texture special effect in the video screen, or pre-determine the target object category to add the texture special effect, and can also adjust the number of target objects and target object categories to improve It provides users with the flexibility and convenience of adding stroke texture effects to videos.
步骤102、在所述视频帧中，确定位于所述目标对象外侧的描边关键点。 Step 102, in the video frame, determine stroke key points located outside the target object.
由于本申请实施例需要对目标对象添加描边纹理特效，为了使描边纹理特效可以绘制在目标对象的轮廓边缘位置，需要在确定出存在目标对象的视频帧后，进一步在视频帧中确定出围绕目标对象外侧的多个描边关键点。确定出的描边关键点在视频帧中分布的密度越高，则有助于生成越精确的描边纹理特效，密度越低，越有有助于提升运算速度，因此，可以根据实际需要灵活调整所生成的描边关键点密度。Since the embodiment of the present application needs to add a stroke texture effect to the target object, in order to make the stroke texture effect can be drawn on the edge position of the outline of the target object, it is necessary to further determine in the video frame after determining the video frame where the target object exists Multiple stroke keys around the outside of the target object. The higher the distribution density of the determined stroke key points in the video frame, the more accurate the stroke texture effect will be, and the lower the density will be, the more it will help to improve the calculation speed. Therefore, it can be flexible according to actual needs Adjusts the resulting stroke key density.
在一些实施例中，可以将目标视频中包含目标对象的视频帧输入描边关键点检测模型，确定分布在视频帧中目标对象轮廓上的描边关键点，并输出各个描边关键点位于视频帧中的位置坐标。上述描边关键点检测模型的网络结构可以根据实际需求灵活设计。例如，描边关键点检测模型可以包括但不限于：卷积层、Relu层、池化层和全连接层等，描边关键点检测模型所包含的层数越多，则识别精度越高；再例如，物体识别模型的网络结构可以采用但不限于：ALexNet或深度残差网络等网络结构。In some embodiments, the video frame containing the target object in the target video can be input into the stroke key point detection model, the stroke key points distributed on the outline of the target object in the video frame can be determined, and each stroke key point is located in the video frame Position coordinates in the frame. The network structure of the above stroke key point detection model can be flexibly designed according to actual needs. For example, the stroke key point detection model can include but not limited to: convolutional layer, Relu layer, pooling layer and fully connected layer, etc. The more layers the stroke key point detection model contains, the higher the recognition accuracy; For another example, the network structure of the object recognition model may adopt, but is not limited to: ALexNet or deep residual network and other network structures.
需要说明的是，对于不同的目标对象，可以采用不同的描边关键点检测模型确定其轮廓上的描边关键点，以提升确定描边关键点的效率和准确性。例如，对于目标对象为人体的视频帧，采用人体描边关键点检测模型确定视频帧中人体轮廓上的描边关键点；对于目标对象为植物的视频帧，采用植物描边关键点检测模型确定视频帧中植物轮廓上的描边关键点。It should be noted that, for different target objects, different stroke key point detection models can be used to determine the stroke key points on its outline, so as to improve the efficiency and accuracy of determining the stroke key points. For example, for a video frame where the target object is a human body, the human body stroke key point detection model is used to determine the stroke key points on the outline of the human body in the video frame; for a video frame where the target object is a plant, the plant stroke key point detection model is used to determine Stroke keys on plant silhouettes in video frames.
若用户在录制视频或播放视频过程中添加描边纹理特效，则需要对描边纹理特效进行实时显示，但由于设备性能的限制，为视频帧生成描边纹理特效需要一定的处理时间，当处理时间过长时，可能导致录制画面或视频播放画面卡顿。If the user adds stroke texture effects during video recording or playback, the stroke texture effects need to be displayed in real time. However, due to device performance limitations, it takes a certain amount of processing time to generate stroke texture effects for video frames. If the time is too long, it may cause the recording screen or video playback screen to freeze.
因此，在对目标视频添加描边纹理特效时，可以是对目标视频中的每一帧进行处理，也可以是以预设帧间隔对目标视频中的视频帧进行处理，以降低对设备性能的要求，其中预设帧间隔为0时，对目标视频中的每一帧进行处理，预设帧间隔为1帧时，以1帧为间隔对目标视频中的视频帧进行。Therefore, when adding a stroke texture effect to the target video, each frame in the target video may be processed, or video frames in the target video may be processed at preset frame intervals to reduce the impact on device performance. Requirements, wherein when the preset frame interval is 0, each frame in the target video is processed, and when the preset frame interval is 1 frame, the video frames in the target video are processed at 1 frame intervals.
在一些实施例中，可以根据录制画面或视频播放画面的帧率确定要处理的视频帧之间的处理帧间隔时间，同时监控为视频帧添加描边纹理特效的处理时间，在处理时间大于处理帧间隔时间的情况下，增加预设帧间隔的数量，以使处理时间不大于处理帧间隔时间。In some embodiments, the processing frame interval time between the video frames to be processed can be determined according to the frame rate of the recording picture or the video playback picture, and the processing time for adding stroke texture special effects to the video frame can be monitored at the same time. In the case of frame interval time, increase the number of preset frame intervals so that the processing time is not greater than the processing frame interval time.
例如，视频播放帧率为30帧每秒，可以通过1秒除以帧率计算出该视频的帧间隔时 间为1/30秒，此时对视频帧进行连续处理，预设帧间隔为0，则处理帧间隔时间同样为1/30秒，若监控到每一帧的处理时间为1/20秒，可以确定处理时间1/20大于处理帧间隔时间1/30，需要增加预设帧间隔，以保证视频流畅播放，若此时将预设帧间隔增加到1，即以1帧为间隔对视频帧进行处理，所以处理帧间隔时间增加到了1/15秒，此时处理帧间隔时间1/15小于处理时间1/20，可以确保视频流畅播放，不会因为添加描边纹理特效而造成目标视频播放卡顿。For example, if the video playback frame rate is 30 frames per second, the frame interval of the video can be calculated as 1/30 second by dividing 1 second by the frame rate. At this time, the video frames are continuously processed, and the preset frame interval is 0. Then the processing frame interval time is also 1/30 second. If it is monitored that the processing time of each frame is 1/20 second, it can be determined that the processing time 1/20 is greater than the processing frame interval time 1/30, and the preset frame interval needs to be increased. To ensure smooth video playback, if the preset frame interval is increased to 1 at this time, the video frame is processed at an interval of 1 frame, so the processing frame interval is increased to 1/15 seconds, and the processing frame interval is 1/ 15 is less than 1/20 of the processing time, which can ensure the smooth playback of the video, and will not cause the target video to freeze due to the addition of stroke texture effects.
步骤103、在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，并根据每两个相邻描边关键点及其各自对应的扩展点构成一个四边形区域，得到多个所述四边形区域连接组成的填充区域。Step 103: On any straight line passing through the stroke key point, establish two extension points respectively located on both sides of the stroke key point, and according to each two adjacent stroke key points and their corresponding The extension points form a quadrilateral area, and a filled area formed by connecting multiple quadrilateral areas is obtained.
在本申请实施例中，为了实现描边纹理特效，需要使用纹理素材对视频帧中目标对象边缘的填充区域进行填充，以使目标对象周围产生与纹理素材类似的描边纹理特效。因此，可通过上述围绕目标对象的描边关键点在视频帧中构建围绕目标对象的填充区域。In the embodiment of the present application, in order to realize the stroke texture special effect, it is necessary to use the texture material to fill the filling area at the edge of the target object in the video frame, so that the stroke texture special effect similar to the texture material is generated around the target object. Therefore, a filled area around the target object can be constructed in the video frame through the aforementioned stroke keypoints around the target object.
当两个不同的四边形共用一条边时，这两个四边形之间的过渡会较为平滑，因此，可以通过目标对象周围的多个描边关键点，生成多个四边形区域，并使相邻的四边形区域之间共用同一条边，这样所有四边形区域可以在目标对象周围形成围绕目标对象的较为平滑的填充区域。When two different quadrilaterals share an edge, the transition between the two quadrilaterals is smoother, so multiple stroke keys around the target object can be used to generate multiple quadrilateral regions and make adjacent quadrilaterals Regions share the same edge so that all quadrilateral regions form a smoother fill region around the target object.
在一些实施例中，针对每一个描边关键点，可以在经过其的任一直线上，分别扩展出两个扩展点，这两个扩展点中的一个扩展点位于描边关键点一侧的直线上，另一个扩展点位于描边关键点的另一侧的直线上，即，一个描边关键点对应的两个扩展点分别位于该描边关键点两侧，且两个扩展点的连线经过该描边关键点。这样，每一个描边关键点扩展出的两个描边关键点可以和相邻描边关键点扩展出的两个描边关键点构成一个四边形区域，由于每两个相邻的四边形区域均共用由同一个描边关键点扩展出的两个扩展点的连接线，因此所有四边形区域可以组成一个较为平滑的填充区域。In some embodiments, for each stroke key point, two extension points can be extended on any straight line passing through it, and one of the two extension points is located on the side of the stroke key point. On the straight line, another expansion point is located on the other side of the stroke key point, that is, the two expansion points corresponding to a stroke key point are located on both sides of the stroke key point, and the connection The line passes through this stroke key. In this way, the two stroke key points extended from each stroke key point can form a quadrilateral area with the two stroke key points extended from adjacent stroke key points, since every two adjacent quadrilateral areas share The connection line of two extension points extended by the same stroke key point, so that all quadrilateral areas can form a smoother filled area.
参照图2，其示出了本申请实施例提供的一种填充区域局部放大图，该局部放大图中包括O1至O4这四个描边关键点，在O1周围建立两个扩展点P1和P2，在O2周围建立两个扩展点P3和P4，在O3周围建立两个扩展点P5和P6，在O4周围建立两个扩展点P7和P8，则P1、P2、P3、P4构成的四边形区域与P3、P4、P5、P6构成的四边形区域共用线段P3-P4作为二者的共用边，P3、P4、P5、P6构成的四边形区域与P5、P6、P7、P8构成的四边形区域共用线段P5-P6作为二者的共用边，这样，P1、P2、P3、P4构成的四边形区域、P3、P4、P5、P6构成的四边形区域、和P5、P6、P7、P8构成的四边形区域共同组成的填充区域的轮廓较为平滑。Referring to FIG. 2 , it shows a partial enlarged view of a filling area provided by an embodiment of the present application. The partial enlarged view includes four stroke key points O1 to O4, and two extension points P1 and P2 are established around O1. , establish two extension points P3 and P4 around O2, establish two extension points P5 and P6 around O3, and establish two extension points P7 and P8 around O4, then the quadrilateral area formed by P1, P2, P3, and P4 is the same as The quadrilateral area composed of P3, P4, P5, and P6 shares the line segment P3-P4 as the shared side of the two, and the quadrilateral area composed of P3, P4, P5, and P6 shares the line segment P5-P5 with the quadrilateral area composed of P5, P6, P7, and P8. P6 is used as the common side of the two, so that the quadrilateral area formed by P1, P2, P3, and P4, the quadrilateral area formed by P3, P4, P5, and P6, and the quadrilateral area formed by P5, P6, P7, and P8 are filled together. The outline of the region is smoother.
步骤104、采用预设的纹理素材贴图，对所述视频帧的填充区域进行填充，得到填充后的视频帧。Step 104: Fill the filled area of the video frame by using a preset texture material map to obtain a filled video frame.
在确定出视频帧的填充区域后，可以使用特效图案对每个视频帧的填充区域进行填充，得到填充后的视频帧。After the filling area of the video frame is determined, the filling area of each video frame may be filled with a special effect pattern to obtain a filled video frame.
由于描边纹理特效的效果呈现与所采用的纹理素材贴图相关，例如，采用荧光样式的纹理素材贴图对填充区域进行填充可以获得荧光描边特效，而采用彩虹样式的纹理素材贴图对填充区域进行填充可以获得彩虹描边特效，因此，可以根据所要实现的描边纹理特效效果的不同，选用不同的纹理素材贴图对填充区域进行填充。Because the effect of the stroke texture effect is related to the texture material map used, for example, the fluorescent stroke effect can be obtained by using the texture material map of the fluorescent style to fill the filled area, and the texture material map of the rainbow style is used to fill the filled area. Filling can obtain rainbow stroke special effect, therefore, according to the stroke texture effect to be achieved, different texture material maps can be selected to fill the filled area.
在一些实施例中，在采用纹理素材贴图对填充区域进行填充时，可以将纹理素材贴图直接填充到整个填充区域，以达到快速填充的效果。在一些其他实施例中，对填充区域进行填充时，可以将纹理素材贴图分别填充到每个四边形区域中，以实现更好的填充效果。In some embodiments, when the texture material map is used to fill the filled area, the texture material map may be directly filled to the entire filled area, so as to achieve a fast filling effect. In some other embodiments, when filling the filled area, the texture material map may be filled into each quadrilateral area respectively, so as to achieve a better filling effect.
步骤105、将所述填充后的视频帧按照时间顺序组合，得到特效视频。Step 105: Combine the filled video frames according to time sequence to obtain special effect video.
由于视频帧在目标视频中按照一定顺序排列，因此在从目标视频中获取视频帧时，可以按照视频帧在目标视频中的顺序获取视频帧，并在对视频帧填充得到填充后的视频帧后， 按照视频帧的获取顺序，将填充后的视频帧组合，得到特效视频。Since the video frames are arranged in a certain order in the target video, when obtaining video frames from the target video, the video frames can be obtained in the order of the video frames in the target video, and after filling the video frames to obtain the filled video frames , according to the acquisition sequence of the video frames, the filled video frames are combined to obtain the special effect video.
也可以在从目标视频中获取视频帧时，同时获取视频帧在目标视频中的次序信息，并得到填充后的视频帧之后，将填充后的视频帧按照对应视频帧的次序信息，对填充后的视频帧进行组合，使填充后的视频帧排列顺序与其对应的视频帧在目标视频中的播放顺序相同。It is also possible to obtain the order information of the video frames in the target video while obtaining the video frames from the target video, and after obtaining the filled video frames, the filled video frames are processed according to the order information of the corresponding video frames. The video frames are combined so that the padded video frames are arranged in the same order as the corresponding video frames in the target video.
综上所述，本申请实施例提供的一种图像特效的生成方法，通过对获取的目标视频中包含目标对象的视频帧进行描边关键点提取，获得目标对象外侧的描边关键点，对每个描边关键点进行扩展，在每个描边关键点周围构建出连接线经过该描边关键点的两个扩展点，使得所有相邻描边关键点对应的扩展点可以构成四边形区域，进而所有四边形区域可以形成围绕目标对象的填充区域，并采用纹理贴图对视频帧的填充区域进行填充，将填充后的视频帧按照播放时间顺序组合得到特效视频。由于相邻四边形区域之间使用同一个描边关键点对应的两个扩展点连接线作为共用边，因此相邻四边形区域之间可以形成较为平滑的过渡，形成的填充区域不会存在明显的锯齿状结构，使得对视频帧填充生成的描边纹理特效更为流畅平滑，提升了特效视频中描边纹理特效的呈现效果。To sum up, the embodiment of the present application provides a method for generating image special effects. By extracting the key points of the stroke from the acquired video frame containing the target object, the key points of the stroke outside the target object are obtained. Each stroke key point is expanded, and a connection line is constructed around each stroke key point to pass through two extension points of the stroke key point, so that the expansion points corresponding to all adjacent stroke key points can form a quadrilateral area, Furthermore, all quadrilateral areas can form a filling area around the target object, and the filling area of the video frame is filled with texture maps, and the filled video frames are combined in order of playing time to obtain a special effect video. Since two expansion point connecting lines corresponding to the same stroke key point are used as shared edges between adjacent quadrilateral areas, a relatively smooth transition can be formed between adjacent quadrilateral areas, and there will be no obvious jaggedness in the formed filled area The shape structure makes the stroke texture special effect generated by filling the video frame more smooth and smooth, and improves the rendering effect of the stroke texture special effect in the special effect video.
图3是本申请实施例提供的另一种图像特效的生成方法的步骤流程图，该方法可以由诸如服务器、处理器、移动电话、计算机、数字广播终端、消息收发设备、游戏控制台、平板设备、医疗设备、健身设备、个人数字助理等的电子设备执行。如图3所示，该方法可以包括：Fig. 3 is a flow chart of the steps of another method for generating image special effects provided by an embodiment of the present application. Electronic device implementations for equipment, medical equipment, fitness equipment, personal digital assistants, etc. As shown in Figure 3, the method may include:
步骤201、获取目标视频，所述目标视频的视频帧包含目标对象。Step 201. Obtain a target video, where a video frame of the target video includes a target object.
本步骤的实现方式与上述步骤101的实现过程类似，在此不再详述。The implementation of this step is similar to the implementation process of step 101 above, and will not be described in detail here.
步骤202、在所述视频帧中，确定位于所述目标对象外侧的描边关键点。Step 202. In the video frame, determine stroke key points located outside the target object.
本步骤的实现方式与上述步骤102的实现过程类似，在此不再详述。The implementation of this step is similar to the implementation process of step 102 above, and will not be described in detail here.
步骤203、在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，并根据每两个相邻描边关键点及其各自对应的扩展点构成一个四边形区域，得到多个所述四边形区域连接组成的填充区域。Step 203: On any straight line passing through the stroke key point, establish two extension points respectively located on both sides of the stroke key point, and according to each two adjacent stroke key points and their corresponding The extension points form a quadrilateral area, and a filled area formed by connecting multiple quadrilateral areas is obtained.
步骤203，还可以包括：Step 203 may also include:
子步骤2031、确定经过所述描边关键点，且垂直于所述描边关键点与一个相邻描边关键点构成的连接线的扩展点直线，所述相邻描边关键点是指与所述描边关键点相邻的描边关键点。Sub-step 2031, determine the extension point straight line passing through the stroke key point and perpendicular to the connection line formed by the stroke key point and an adjacent stroke key point, the adjacent stroke key point refers to the The stroke key adjacent to the stroke key.
描边关键点对应的一个相邻描边关键点可以是该描边关键点的上一个描边关键点，也可以是该描边关键点的下一个描边关键点。在实际应用过程中，为了使最终生成的四边形区域之间连接更为平滑，可以统一使描边关键点对应的扩展点直线垂直于描边关键点与上一描边关键点之间的连线，或统一使描边关键点对应的扩展点直线垂直于描边关键点与下一描边关键点之间的连线。在一些实施例中，关键点连接线是虚拟直线，其用于确定扩展点位于对应的描边关键点的方向。An adjacent stroke key corresponding to a stroke key can be the previous stroke key of this stroke key, or it can be the next stroke key of this stroke key. In the actual application process, in order to make the connection between the finally generated quadrilateral regions smoother, you can uniformly make the extension point line corresponding to the stroke key point perpendicular to the connection line between the stroke key point and the previous stroke key point , or uniformly make the extension point line corresponding to the stroke key point perpendicular to the line between the stroke key point and the next stroke key point. In some embodiments, the key point connecting lines are virtual straight lines used to determine the direction in which the expansion point is located at the corresponding stroked key point.
可以按照描边关键点在视频帧上的分布顺序建立描边关键点序列，并按照各个描边关键点序列中的相邻关系确定各个描边关键点的上一描边关键点或下一描边关键点。The stroke key point sequence can be established according to the distribution order of the stroke key points on the video frame, and the previous stroke key point or the next stroke key point of each stroke key point can be determined according to the adjacent relationship in each stroke key point sequence Edge key points.
需要说明的是，对于视频帧中的一个目标对象，目标对象的一部分位于视频帧中，例如半身人像，此时目标对象外侧上的描边关键点可以连成一条围绕该目标对象的开放曲线，导致开放曲线两端的描边关键点不存在上一描边关键点或下一描边关键点。此时，对于开放曲线两端的端点描边关键点，可以将端点描边关键点与任一相邻描边关键点的连线的垂直方向，确定为端点描边关键点对应的扩展点直线延伸方向。It should be noted that, for a target object in a video frame, a part of the target object is located in the video frame, such as a half-length portrait, at this time, the stroke key points on the outside of the target object can be connected to form an open curve around the target object, Causes stroke keys at the ends of open curves to have no previous or next stroke keys. At this time, for the endpoint stroke key points at both ends of the open curve, the vertical direction of the connection line between the endpoint stroke key point and any adjacent stroke key point can be determined as the extension point corresponding to the endpoint stroke key point. direction.
在一些实施例中，描边关键点序列中的描边关键点按照描边关键点在视频帧中的分布顺序排列，因此，描边关键点序列中任一描边关键点的相邻描边关键点，在视频帧中均相邻。可以根据每一个描边关键点与其在描边关键点序列中的相邻描边关键点确定出一个描 边关键点向量，所有描边关键点向量的方向均由描边关键点指向该描边关键点的相邻描边关键点。In some embodiments, the stroke key points in the stroke key point sequence are arranged according to the distribution sequence of the stroke key points in the video frame, therefore, the stroke key points of any stroke key point in the stroke key point sequence The key points are all adjacent in the video frame. A stroke key point vector can be determined according to each stroke key point and its adjacent stroke key points in the stroke key point sequence, and the directions of all stroke key point vectors are pointed from the stroke key point to the stroke Neighboring stroke keys of the key.
如图2所示，向量
Figure PCTCN2022075194-appb-000001
为描边关键点向量，其中，描边关键点O2和相邻描边关键点O1之间构成向量
Figure PCTCN2022075194-appb-000002
描边关键点O3和其相邻描边关键点O2之间构成向量
Figure PCTCN2022075194-appb-000003
描边关键点O4和其相邻描边关键点O3之间构成向量
Figure PCTCN2022075194-appb-000004
As shown in Figure 2, the vector
Figure PCTCN2022075194-appb-000001
is the stroke key point vector, where the stroke key point O2 and the adjacent stroke key point O1 form a vector
Figure PCTCN2022075194-appb-000002
A vector is formed between the stroke key point O3 and its adjacent stroke key point O2
Figure PCTCN2022075194-appb-000003
A vector is formed between the stroke key point O4 and its adjacent stroke key point O3
Figure PCTCN2022075194-appb-000004
由于相互垂直的向量为正交向量，而正交向量之间的点乘结果为0，因此，可以根据正交向量的性质，计算出与描边关键点向量正交的正交单位向量。通过正交单位向量，可以进一步确定描边关键点对应的关键点直线的方向。针对每个描边关键点向量，计算与描边关键点向量点乘结果为0的正交向量，再对该正交单位向量进行单位化，进而得到与每个描边关键点向量对应的正交单位向量。正交单位向量可以反映关键点直线的方向。Since the vectors perpendicular to each other are orthogonal vectors, and the point product between the orthogonal vectors is 0, the orthogonal unit vector orthogonal to the stroke key point vector can be calculated according to the properties of the orthogonal vectors. Through the orthogonal unit vector, the direction of the key point line corresponding to the stroke key point can be further determined. For each stroke key point vector, calculate the orthogonal vector whose dot multiplication result with the stroke key point vector is 0, and then unitize the orthogonal unit vector, and then obtain the orthogonal vector corresponding to each stroke key point vector Intersection unit vector. Orthogonal unit vectors can reflect the direction of the keypoint line.
进一步的，还可以根据描边关键点与相邻描边关键点的坐标，直接计算上述正交单位向量，如图2所示，与描边关键点O2相邻的描边关键点为O1，描边关键点O2在视频帧中的坐标为(X2,Y2)，与描边关键点O2相邻的描边关键点O1在视频帧中的坐标为(X1,Y1)，根据O1和O2构建的描边关键点向量
Figure PCTCN2022075194-appb-000005
为(X2-X1,Y2-Y1)，则根据正交向量的性质，可以计算出与描边关键点向量
Figure PCTCN2022075194-appb-000006
垂直的一个正交向量(Y2-Y1,X1-X2)，对该正交向量进行单位化，即将正交向量与自身的模相除，得到描边关键点O2的正交单位向量。 Further, the above-mentioned orthogonal unit vector can also be directly calculated according to the coordinates of the stroke key point and the adjacent stroke key points, as shown in Figure 2, the stroke key point adjacent to the stroke key point O2 is O1, The coordinates of the stroke key point O2 in the video frame are (X2, Y2), and the coordinates of the stroke key point O1 adjacent to the stroke key point O2 in the video frame are (X1, Y1), constructed according to O1 and O2 The stroke keypoint vector of
Figure PCTCN2022075194-appb-000005
is (X2-X1, Y2-Y1), then according to the nature of the orthogonal vector, the key point vector of the stroke can be calculated
Figure PCTCN2022075194-appb-000006
A vertical orthogonal vector (Y2-Y1, X1-X2), the orthogonal vector is unitized, that is, the orthogonal vector is divided by its own modulus, and the orthogonal unit vector of the stroke key point O2 is obtained.
子步骤2032、在所述扩展点直线上建立两个到所述描边关键点距离相等的扩展点。Sub-step 2032: Establish two extension points on the straight line of the extension point with equal distances from the stroke key point.
在一些实施例中，每个描边关键点对应的两个扩展点之间的连线都经过该描边关键点，且每个描边关键点对应的两个扩展点到该描边关键点的距离相等，也就是说，每个描边关键点均位于该描边关键点对应的两个扩展点之间连线的中点上。使得各个描边关键点的扩展点围成的填充区域可以围绕目标对象，并按照描边关键点的分布形成填充区域，因此填充区域贴合目标对象的效果较好。In some embodiments, the line between the two extension points corresponding to each stroke key point passes through the stroke key point, and the two extension points corresponding to each stroke key point reach the stroke key point , that is, each stroke key is located at the midpoint of the line between the two extension points corresponding to the stroke key. The filling area enclosed by the extension points of each stroke key point can surround the target object, and form a filling area according to the distribution of stroke key points, so the effect of the filling area fitting the target object is better.
子步骤2032、还可以包括：Substep 2032 may also include:
子步骤A1、按照所述描边关键点在所述目标对象外侧的排列顺序，将多个所述描边关键点构成描边关键点序列。Sub-step A1: According to the arrangement order of the stroke key points outside the target object, a plurality of the stroke key points are formed into a stroke key point sequence.
描边关键点在描边关键点序列中的分布情况，需要按照描边关键点在视频帧中的相邻关系所确定，即描边关键点序列中的描边关键点按照其在目标对象边缘的分布顺序依次排列。在一些实施例中，对于视频帧中的一个目标对象，在一种情况下，目标对象的一部分位于视频帧中，例如半身人像，此时目标对象轮廓上的描边关键点可以构成一条围绕该目标对象的开放曲线，描边关键点从该曲线的一端到另一端依次排列，可以将位于该曲线一端的描边关键点作为起始描边关键点加入描边关键点序列，并根据描边关键点在该曲线上的排列顺序将其余描边关键点依次加入描边关键点序列。在另一种情况下，目标对象完全处于视频帧中，例如全身人像，此时目标对象轮廓上的描边关键点可以构成一条围绕该目标对象的封闭曲线，可以将任一描边关键点作为起始描边关键点加入描边关键点序列，并根据描边关键点在该曲线上的排列顺序将其余描边关键点依次加入描边关键点序列。The distribution of the stroke key points in the stroke key point sequence needs to be determined according to the adjacent relationship of the stroke key points in the video frame, that is, the stroke key points in the stroke key point sequence are determined according to their position on the edge of the target object in order of distribution. In some embodiments, for a target object in a video frame, in one case, a part of the target object is located in the video frame, such as a bust portrait, at this time, the stroke key points on the outline of the target object can form a line around the target object. The open curve of the target object, the stroke key points are arranged sequentially from one end of the curve to the other end, the stroke key point at one end of the curve can be added to the stroke key point sequence as the starting stroke key point, and according to the stroke The order in which keys are arranged on this curve Adds the rest of the stroke keys to the sequence of stroke keys in turn. In another case, where the target object is completely within the video frame, such as a full-body portrait, the stroke keypoints on the outline of the target object can form a closed curve around the target object, and any stroke keypoint can be used as The initial stroke key point is added to the stroke key point sequence, and the rest of the stroke key points are added to the stroke key point sequence in turn according to the arrangement order of the stroke key points on the curve.
子步骤A2、在所述描边点序列中的起始描边关键点对应的关键点直线上，建立两个到所述起始描边关键点的距离为第一距离的扩展点。Sub-step A2, on the key point line corresponding to the initial stroke key point in the stroke point sequence, establish two extension points whose distance to the initial stroke key point is the first distance.
可以将描边关键点与相邻描边关键点构成的连接线方向的垂直方向确定为描边关键点对应的扩展点直线的延伸方向，并将描边关键点对应的扩展点直线的一端确定为第一方向，将描边关键点对应的扩展点直线的另一端确定为第二方向。The vertical direction of the connection line direction formed by the stroke key point and the adjacent stroke key point can be determined as the extension direction of the extension point line corresponding to the stroke key point, and one end of the extension point line corresponding to the stroke key point can be determined is the first direction, and the other end of the extension point line corresponding to the stroke key point is determined as the second direction.
由于通过一个点确定另一个点的位置需要两个参数，即方向和距离，因此，在确定出第一方向和第二方向后，还需要确定第一距离，以确定第一扩展点相较于起始描边关键点的方向和距离。应当理解，第一距离为填充区域在起始描边关键点位置的横向宽度的一半，因此，通过设置不同的第一距离，可以调节填充区域的宽度。可以由用户通过图形交互界 面输入或选择第一距离，还可以由***预设第一距离。Since determining the position of another point through one point requires two parameters, namely direction and distance, after determining the first direction and the second direction, it is also necessary to determine the first distance to determine the distance between the first expansion point and The direction and distance of the starting stroke key. It should be understood that the first distance is half of the lateral width of the filled area at the key point of the initial stroke. Therefore, by setting different first distances, the width of the filled area can be adjusted. The first distance can be input or selected by the user through the graphical interface, and the first distance can also be preset by the system.
由于播放目标视频的终端设备的屏幕分辨率各不相同，如果对所有设备套用同一个第一距离，可能会导致高分辨率设备显示的描边纹理特效过宽，而低分辨率设备显示的描边纹理特效过窄，因此，可以根据终端设备的屏幕分辨率确定第一距离。Since the screen resolutions of the terminal devices playing the target video are different, if the same first distance is applied to all devices, the stroke texture effect displayed on high-resolution devices may be too wide, while the stroke texture effect displayed on low-resolution devices may be too wide. The edge texture special effect is too narrow, therefore, the first distance may be determined according to the screen resolution of the terminal device.
在一些实施例中，可以设置第一预设系数，将终端设备的屏幕横向分辨率或纵向分辨率与该第一预设系数相乘，得到该终端设备对应的第一距离，此时第一距离可以为屏幕像素的数量。例如，终端设备的横向分辨率1000，第一预设系数为0.05，则第一距离的计算结果为50个屏幕像素。In some embodiments, a first preset coefficient can be set, and the horizontal resolution or vertical resolution of the screen of the terminal device is multiplied by the first preset coefficient to obtain the first distance corresponding to the terminal device. At this time, the first The distance can be the number of screen pixels. For example, if the horizontal resolution of the terminal device is 1000, and the first preset coefficient is 0.05, then the calculation result of the first distance is 50 screen pixels.
由于目标视频本身分辨率各不相同，因此，还可以根据目标视频本身的分辨率确定第一距离。Since the resolutions of the target videos themselves are different, the first distance may also be determined according to the resolutions of the target videos themselves.
在一些实施例中，可以设置第二预设系数，将目标视频的横向分辨率或纵向分辨率与该第二预设系数相乘，得到该目标视频对应的第一距离，此时第一距离可以为目标视频中像素的数量。例如，目标视频的横向分辨率1000，第二预设系数为0.06，则第一距离的计算结果为60个目标视频像素。In some embodiments, a second preset coefficient can be set, and the horizontal resolution or vertical resolution of the target video is multiplied by the second preset coefficient to obtain the first distance corresponding to the target video. At this time, the first distance Can be the number of pixels in the target video. For example, if the horizontal resolution of the target video is 1000 and the second preset coefficient is 0.06, then the calculation result of the first distance is 60 pixels of the target video.
在确定出第一方向和第一距离后，将起始描边关键点向第一方向移动第一距离，得到起始描边关键点对应的一个第一扩展点，也就是说，将距离起始描边关键点第一距离且相对起始描边关键点方向为第一方向的点确定为一个第一扩展点。After determining the first direction and the first distance, move the starting stroke key point to the first direction for the first distance to obtain a first extension point corresponding to the starting stroke key point, that is to say, the starting stroke key point A point that is at the first distance from the key point of the initial stroke and whose direction is the first direction relative to the key point of the initial stroke is determined as a first extension point.
由于起始描边关键点在生成的两个第一扩展点之间，因此，其中一个扩展点位于起始描边关键点的第一方向，另一个第一扩展点位于起始描边关键点的第一方向的反方向，即第二方向。Since the start stroke key is between the two generated first expansion points, one of the expansion points is located in the first direction of the start stroke key, and the other first expansion point is located at the start stroke key The opposite direction of the first direction, that is, the second direction.
在确定出第二方向和第一距离后，将起始描边关键点向第二方向移动第一距离，得到起始描边关键点对应的另一个第一扩展点，也就是说，将距离起始描边关键点第一距离且相对起始描边关键点方向为第二方向的点确定为另一个第一扩展点。After determining the second direction and the first distance, move the initial stroke key point to the second direction by the first distance to obtain another first extension point corresponding to the initial stroke key point, that is, set the distance The point at the first distance from the starting stroke key point and in the second direction relative to the starting stroke key point is determined as another first extension point.
在一些实施例中，在确定出起始描边关键点对应的正交单位向量和第一距离后，可以将每个起始描边关键点在视频帧对应的坐标系中朝正交单位向量指示的方向移动第一距离，得到每个起始描边关键点对应的一个扩展点，可以将每个起始描边关键点在视频帧对应的坐标系中朝正交单位向量的反方向移动第一距离，得到每个起始描边关键点对应的另一个扩展点。In some embodiments, after determining the orthogonal unit vector and the first distance corresponding to the initial stroke key point, each initial stroke key point can be moved toward the orthogonal unit vector in the coordinate system corresponding to the video frame Move the first distance in the indicated direction to get an extension point corresponding to each starting stroke key point, and move each starting stroke key point in the opposite direction of the orthogonal unit vector in the coordinate system corresponding to the video frame The first distance to get another extension point corresponding to each starting stroke keypoint.
由于每个起始描边关键点的坐标可以表示为一个由坐标原点指向该起始描边关键点的向量，因此可以通过起始描边关键点的坐标、正交单位向量和第一距离通过如下公式计算起始描边关键点对应的一个第一扩展点的坐标：Since the coordinates of each starting stroke key point can be expressed as a vector pointing from the origin of the coordinates to the starting stroke key point, it is possible to pass the coordinates of the starting stroke key point, the orthogonal unit vector and the first distance The following formula calculates the coordinates of a first extension point corresponding to the starting stroke key point:
Figure PCTCN2022075194-appb-000007
Figure PCTCN2022075194-appb-000007
可以通过起始描边关键点的坐标、正交单位向量和第一距离通过如下公式计算起始描边关键点对应的另一个第一扩展点的坐标：The coordinates of another first extension point corresponding to the starting stroke key point can be calculated by the following formula through the coordinates of the starting stroke key point, the orthogonal unit vector and the first distance:
Figure PCTCN2022075194-appb-000008
Figure PCTCN2022075194-appb-000008
其中，
Figure PCTCN2022075194-appb-000009
表示起始描边关键点对应的描边关键点向量，
Figure PCTCN2022075194-appb-000010
标识起始描边关键点对应的正交单位向量，d 1表示第一距离，P 1表示起始描边关键点对应的一个扩展点，P 2表示起始描边关键点对应的另一个扩展点。 in,
Figure PCTCN2022075194-appb-000009
Indicates the stroke key point vector corresponding to the starting stroke key point,
Figure PCTCN2022075194-appb-000010
Identify the orthogonal unit vector corresponding to the key point of the initial stroke, d 1 represents the first distance, P 1 represents an extension point corresponding to the key point of the initial stroke, P 2 represents another extension corresponding to the key point of the initial stroke point.
在一些实施例中，可以根据终端屏幕或目标视频的分辨率自动设置第一距离，使得生成的描边纹理特效，在不同终端屏幕或不同目标视频中始终以合适的宽度呈现，显著提升了本申请实施例生成的描边纹理特效的适用范围与显示效果。In some embodiments, the first distance can be automatically set according to the resolution of the terminal screen or target video, so that the generated stroke texture effect is always presented with an appropriate width on different terminal screens or different target videos, which significantly improves the The scope of application and display effect of the stroke texture special effect generated by the embodiment of the application.
子步骤A3、根据非起始描边关键点在所述描边关键点序列中的次序，从倍率对应关系中确定所述非起始描边关键点对应的目标距离缩放倍率，所述倍率对应关系用于表征非起始描边关键点在所述描边关键点序列中的次序和距离缩放倍率之间的对应关系。Sub-step A3, according to the order of the non-initial stroke key points in the stroke key point sequence, determine the target distance scaling factor corresponding to the non-initial stroke key point from the magnification correspondence relationship, and the magnification corresponds to The relationship is used to characterize the correspondence between the order of the non-initial stroke key points in the stroke key point sequence and the distance scaling factor.
每个非起始描边关键点均对应一个第二距离，且某一非起始描边关键点对应的第二距离，为填充区域在该非起始描边关键点位置的横向宽度的一半，因此，针对不同的非起始描边关键点设置不同的第二距离，可以使填充区域的宽度在不同描边关键点位置不同，进而使描边纹理特效在不同位置的宽度不同，有助于提升描边纹理特效的表现力，丰富了描边纹理特效的形态。Each non-initial stroke key point corresponds to a second distance, and the second distance corresponding to a non-initial stroke key point is half of the horizontal width of the filled area at the position of the non-initial stroke key point , therefore, setting different second distances for different non-initial stroke key points can make the width of the filled area different at different stroke key points, thereby making the width of the stroke texture effect different at different positions, which helps To improve the expressiveness of the stroke texture effect, it enriches the shape of the stroke texture effect.
在一些实施例中，可以预设有倍率对应关系，倍率对应关系用于表征非起始描边关键点在描边关键点序列中的次序和距离缩放倍率之间的对应关系。通过查询倍率对应关系，确定各个非起始描边关键点对应的距离缩放倍率。也可以将非起始描边关键点在描边关键点中的次序，输入到预设函数中，预设函数可以是三角函数、对数函数、指数函数等二元函数，也可以为其他类型的函数。预设函数输出不同次序的非起始描边关键点对应的距离缩放倍率。In some embodiments, a magnification correspondence can be preset, and the magnification correspondence is used to characterize the correspondence between the order of the non-initial stroke key points in the stroke key point sequence and the distance zooming magnification. By querying the magnification corresponding relationship, determine the distance zoom magnification corresponding to each non-initial stroke key point. It is also possible to input the order of the non-initial stroke key points in the stroke key points into the preset function. The preset function can be binary functions such as trigonometric functions, logarithmic functions, exponential functions, or other types The function. The default function outputs the distance scaling factor for non-initial stroke keys in different orders.
子步骤A4、将所述目标距离缩放倍率与所述第一距离的乘积，确定为所述非起始描边关键点对应的第二距离。Sub-step A4. Determine the product of the target distance scaling factor and the first distance as the second distance corresponding to the non-initial stroke key point.
计算每个非起始描边关键点对应的距离缩放倍率与第一距离的乘积，得到每个非起始描边关键点对应的第二距离。也可以通过其他方式确定每个非起始描边关键点对应的第二距离，本申请实施例不做限定。由于位于描边关键点序列中部的非起始描边关键点对应的距离缩放倍率较大，进而第二距离相对第一距离较大，位于描边关键点序列两端的非起始描边关键点对应的距离缩放倍率较小，进而第二距离相对第一距离较小，因此可以形成宽度变化的填充区域，最终可以获得宽度具备渐变效果的描边纹理特效，提高了描边纹理特效的生动性，例如可以形成两头细中间粗的月牙形描边纹理特效、粗细周期性变化的波浪形描边纹理特效等。Calculate the product of the distance scaling factor corresponding to each non-initial stroke key point and the first distance to obtain the second distance corresponding to each non-initial stroke key point. The second distance corresponding to each non-initial stroke key point may also be determined in other ways, which is not limited in this embodiment of the present application. Since the non-initial stroke key points in the middle of the stroke key point sequence correspond to a larger distance scaling factor, and the second distance is larger than the first distance, the non-initial stroke key points at both ends of the stroke key point sequence The corresponding distance zoom ratio is small, and the second distance is smaller than the first distance, so a filling area with varying width can be formed, and finally a stroke texture effect with a gradient effect in width can be obtained, which improves the vividness of the stroke texture effect , for example, it can form a crescent-shaped stroke texture effect with thin ends and a thick middle, a wavy stroke texture effect with periodic changes in thickness, etc.
子步骤A5、在所述描边点序列中的非起始描边关键点对应的扩展点直线上，建立两个到所述非起始描边关键点的距离为第二距离的扩展点。Sub-step A5: On the extension point straight line corresponding to the non-initial stroke key point in the stroke point sequence, establish two extension points whose distance to the non-initial stroke key point is the second distance.
在确定出非起始描边关键点对应的正交单位向量，即非起始描边关键点的扩展点直线的方向，以及第二距离后，可以将每个非起始描边关键点在视频帧对应的坐标系中朝正交单位向量指示的方向移动第二距离，得到每个非起始描边关键点对应的一个扩展点，可以将每个非起始描边关键点在视频帧对应的坐标系中朝正交单位向量的反方向移动第二距离，得到每个非起始描边关键点对应的另一个扩展点。After determining the orthogonal unit vector corresponding to the non-initial stroke key point, that is, the direction of the extension point straight line of the non-initial stroke key point, and the second distance, each non-initial stroke key point can be placed at In the coordinate system corresponding to the video frame, move the second distance in the direction indicated by the orthogonal unit vector to obtain an extension point corresponding to each non-initial stroke key point, and each non-initial stroke key point can be placed in the video frame Moving the second distance in the opposite direction of the orthogonal unit vector in the corresponding coordinate system to obtain another extension point corresponding to each non-initial stroke key point.
由于每个非起始描边关键点的坐标可以表示为一个由坐标原点指向该非起始描边关键点的向量，因此可以通过非起始描边关键点的坐标、正交单位向量和第二距离通过如下公式计算非起始描边关键点对应的一个第一扩展点的坐标：Since the coordinates of each non-initial stroke key point can be expressed as a vector pointing from the origin of the coordinates to the non-initial stroke key point, the coordinates of the non-initial stroke key point, the orthogonal unit vector and the first The second distance calculates the coordinates of a first extension point corresponding to a non-initial stroke key point by the following formula:
Figure PCTCN2022075194-appb-000011
Figure PCTCN2022075194-appb-000011
可以通过非起始描边关键点的坐标、正交单位向量和第二距离通过如下公式计算非起始描边关键点对应的另一个第一扩展点的坐标：The coordinates of another first extension point corresponding to the non-initial stroke key point can be calculated by the following formula:
Figure PCTCN2022075194-appb-000012
Figure PCTCN2022075194-appb-000012
其中，
Figure PCTCN2022075194-appb-000013
表示非起始描边关键点对应的描边关键点向量，
Figure PCTCN2022075194-appb-000014
标识非起始描边关键点对应的正交单位向量，d 2表示第二距离，P 1表示非起始描边关键点对应的一个扩展点，P 2表示非起始描边关键点对应的另一个扩展点。 in,
Figure PCTCN2022075194-appb-000013
Indicates the stroke keypoint vector corresponding to the non-starting stroke keypoint,
Figure PCTCN2022075194-appb-000014
Identify the orthogonal unit vector corresponding to the non-initial stroke key point, d 2 represents the second distance, P 1 represents an extension point corresponding to the non-initial stroke key point, P 2 represents the non-initial stroke key point corresponding to Another extension point.
步骤204、将所述纹理素材贴图切分为预设数量个三角形的纹理素材子块。Step 204 , dividing the texture material map into texture material sub-blocks with a preset number of triangles.
可以针对不同的描边纹理特效预置不同的纹理素材贴图，并在生成描边纹理特效时选择对应的纹理素材贴图。在一些实施例中，可以建立多个纹理素材贴图库，并构建纹理素材库与不同种类的描边纹理特效的对应关系。在生成描边纹理特效时，根据所要生成的描边纹理特效种类确定对应的描边纹理特效库，并从对应的纹理特效库中获取纹理素材贴图。需要说明的是，每个纹理素材贴图库中可以存储有多张纹理素材贴图，生成描边纹理特效 时可以获取对应的纹理素材贴图库中的多张纹理素材贴图。进而，可以为填充区域中的不同四边形区域填充不同的纹理素材贴图，以使最终生成的描边纹理特效可以呈现更为丰富的效果。Different texture material maps can be preset for different stroke texture effects, and the corresponding texture material maps can be selected when generating stroke texture effects. In some embodiments, multiple texture material map libraries can be established, and the corresponding relationship between the texture material library and different types of stroke texture special effects can be constructed. When generating the stroke texture effect, determine the corresponding stroke texture effect library according to the type of the stroke texture effect to be generated, and obtain the texture material map from the corresponding texture effect library. It should be noted that multiple texture material maps can be stored in each texture material map library, and multiple texture material maps in the corresponding texture material map library can be obtained when generating stroke texture effects. Furthermore, different texture material maps can be filled for different quadrilateral areas in the filled area, so that the final stroke texture effect can present a richer effect.
在一些实施例中，还可以由用户在生成纹理特效之前指定所要填充的纹理素材贴图。在一些实施例中，用户可以选取本地设备或服务器上的任意一张或多张图片，作为纹理素材贴图对填充区域进行填充。以产生效果更为多变且更加符合用户意图的描边纹理特效。In some embodiments, the user may also specify the texture material map to be filled before generating the texture special effect. In some embodiments, the user can select any one or more pictures on the local device or the server, and use them as texture material maps to fill the filling area. In order to produce stroke texture effects that are more varied and more in line with user intentions.
由于某些绘图引擎仅能对填充子区域进行绘制，例如开放图形库或者(Open Graphics Library，OpenGL)等。因此，在获得纹理素材贴图后，还需要对其再次切分，以获得用于对填充区域进行填充的三角形的纹理素材子块。Because some drawing engines can only draw the filled sub-region, such as Open Graphics Library or (Open Graphics Library, OpenGL) and so on. Therefore, after the texture material map is obtained, it needs to be divided again to obtain triangular texture material sub-blocks for filling the filled area.
步骤204、还可以包括：Step 204 may also include:
子步骤2041、从所述纹理素材贴图的一端到另一端，对所述纹理素材贴图进行等宽切分，得到与所述四边形区域的数量相同数量的四边形的纹理素材块。Sub-step 2041, from one end of the texture material map to the other end, divide the texture material map into equal widths to obtain quadrilateral texture material blocks equal in number to the quadrilateral regions.
由于纹理素材贴图的数量可能较少，例如仅获取一张纹理素材贴图生成描边纹理特效时，若在每个四边形区域中填充该纹理素材的整体图，则生成的描边纹理特效在四边形区域的交界处过渡并不自然。因此，为了对多个四边形区域构成的填充区域进行填充，并取得较好的填充效果，需要对纹理素材贴图进行切分，并生成与四边形区域数量一致的纹理素材块，以使填充区域整体可以还原纹理素材贴图整体所呈现的效果，使纹理素材被填充到四边形区域后过渡更为自然。在一些实施例中，可以由纹理素材贴图的一边向对边，按照相同的间距对纹理素材进行切分，将纹理素材贴图切分成与四边形区域数量一致的宽度相等的纹理素材块。Since the number of texture material maps may be small, for example, when only one texture material map is obtained to generate a stroke texture effect, if the overall image of the texture material is filled in each quadrilateral area, the generated stroke texture effect will be in the quadrilateral area The junction transitions are not natural. Therefore, in order to fill the filled area composed of multiple quadrangular areas and achieve a better filling effect, it is necessary to segment the texture material map and generate texture material blocks with the same number as the quadrilateral area, so that the filled area can be filled as a whole. Restore the overall effect of the texture material map, so that the transition after the texture material is filled into the quadrilateral area is more natural. In some embodiments, the texture material can be divided from one side of the texture material map to the opposite side according to the same spacing, and the texture material map can be divided into texture material blocks with the same width as the number of quadrilateral regions.
参见图4，其示出了本申请实施例提供的一种纹理素材贴图切分示意图，其中，从纹理素材贴图左侧向纹理素材贴图的右侧，依次将纹理素材贴图切割成n个纹理素材块，每个纹理素材块的宽度相同。Referring to FIG. 4 , it shows a schematic diagram of a texture material map segmentation provided by an embodiment of the present application, wherein, from the left side of the texture material map to the right side of the texture material map, the texture material map is sequentially cut into n texture materials blocks, each texture material block has the same width.
子步骤2042、按照所述纹理素材块的生成次序，依次将每个所述纹理素材块沿一条对角线切分为两个纹理素材子块。Sub-step 2042: Sequentially divide each texture material block into two texture material sub-blocks along a diagonal line according to the generation sequence of the texture material blocks.
由于每个纹理素材块均为四边形形状，每个纹理素材块的一条对角线将其分割为两个三角形，因此，可沿一条对角线将每个纹理素材块切分为两个三角形的纹理素材子块。例如，如图4所示，可沿角点A2和角点B1构成的对角线，将纹理素材块A1A2B2B1切分为纹理素材子块A1A2B1和纹理素材子块A2B1B2。Since each texture material block is in the shape of a quadrilateral, a diagonal line of each texture material block divides it into two triangles, therefore, each texture material block can be divided into two triangles along a diagonal line Texture material subblock. For example, as shown in FIG. 4 , the texture material block A1A2B2B1 can be divided into the texture material sub-block A1A2B1 and the texture material sub-block A2B1B2 along the diagonal formed by the corner point A2 and the corner point B1.
由于纹理素材块是纹理素材贴图从一端向另一端切分生成的，因此纹理素材块的生成顺序可以体现出纹理素材贴图中图案的分布情况，并且在对纹理素材块进行切分时，是按照纹理素材块的生成次序进行切分的，得到的纹理素材子块按照生成顺序排列，可以还原纹理素材贴图中的图案。Since the texture material block is generated by splitting the texture material map from one end to the other end, the generation order of the texture material block can reflect the distribution of patterns in the texture material map, and when the texture material block is divided, it is in accordance with The generated sequence of the texture material block is segmented, and the obtained texture material sub-blocks are arranged according to the generated sequence, which can restore the pattern in the texture material map.
步骤205、按照所述四边形区域在所述填充区域中的排列顺序，从所述填充区域的一端开始，依次将每个所述四边形区域沿一条对角线切分为两个填充子区域，得到所述预设数量个填充子区域。Step 205: According to the arrangement order of the quadrilateral areas in the filling area, starting from one end of the filling area, sequentially divide each quadrilateral area into two filling sub-areas along a diagonal line, to obtain The preset number of filled sub-regions.
对每个四边形区域均沿其一条对角线进行切分，针对每个四边形区域生成两个用于填充三角形素材子块的填充子区域。例如，如图2所示，针对四边形区域P1P2P3P4，将四边形区域P1P2P3P4沿P2P3连接而成的对角线切开，得到填充子区域P1P2P3和填充子区域P2P3P4。Each quadrilateral area is divided along one of its diagonals, and two filling sub-areas are generated for each quadrilateral area for filling the triangular material sub-block. For example, as shown in FIG. 2 , for the quadrilateral area P1P2P3P4, the quadrilateral area P1P2P3P4 is cut along the diagonal line connecting P2P3 to obtain the filled sub-area P1P2P3 and the filled sub-area P2P3P4.
步骤206、建立所述纹理素材子块与所述填充子区域之间的一一对应关系。Step 206, establishing a one-to-one correspondence between the texture material sub-block and the filling sub-region.
由于纹理素材块与四边形区域的数量一致，且每个纹理素材块被切分为两个纹理素材子块，每个四边形区域被切分两个填充子区域，因此，纹理素材子块的数量与填充子区域的数量相同。可进一步建立所有纹理素材子块与所有填充子区域之间的一一对应关系。Since the number of texture material blocks is the same as that of the quadrilateral area, and each texture material block is divided into two texture material sub-blocks, and each quadrilateral area is divided into two filling sub-areas, therefore, the number of texture material sub-blocks is the same as The number of filled subregions is the same. A one-to-one correspondence between all texture material sub-blocks and all filling sub-regions may be further established.
需要注意的是，所有纹理素材子块与所有填充子区域之间的一一对应关系中，需要使 纹理素材贴图中两个相邻的纹理素材子块分别对应的填充子区域，在填充区域中也是相邻的。It should be noted that in the one-to-one correspondence between all texture material sub-blocks and all filling sub-regions, it is necessary to make the filling sub-regions corresponding to two adjacent texture material sub-blocks in the texture material map, in the filling area is also adjacent.
这样，可以使填充区域整体可以还原纹理素材贴图整体所呈现的效果，使纹理素材子块被填充到填充子区域所形成的描边纹理特效过渡更为自然。In this way, the entire filled area can restore the overall effect of the texture material map, making the transition of the stroke texture effect formed by the sub-block of the texture material filled to the filled sub-area more natural.
步骤206、还可以包括：Step 206 may also include:
子步骤2061、按照所述纹理素材子块的生成次序以及所述填充子区域的生成次序，建立所述纹理素材子块和所述填充子区域之间的一一对应关系，其中，生成次序相同的纹理素材子块和填充子区域相对应。Sub-step 2061: Establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions according to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, wherein the generation order is the same The texture material sub-blocks correspond to the filled sub-regions.
由于纹理素材块按照从一边向对边的顺序切分得到，因此相邻的纹理素材块在纹理素材贴图中相邻，过渡最为自然，而纹理素材子块也是按照纹理素材块的生成顺序对纹理素材块进行切分得到，因此生成次序相邻的纹理素材子块之间也在纹理素材贴图中相邻，过渡也最为自然。Since the texture material blocks are obtained by cutting from one side to the opposite side, the adjacent texture material blocks are adjacent in the texture material map, and the transition is the most natural, and the texture material sub-blocks are also textured according to the generation order of the texture material blocks. The material blocks are obtained by segmenting, so the adjacent texture material sub-blocks in the generation order are also adjacent in the texture material map, and the transition is the most natural.
而填充子区域是按照四边形区域在填充区域中的排列顺序对各个四边形区域切分得到的，因此生成次序相邻的填充子区域在填充区域中也是相邻的，最先生成的填充子区域位于填充区域的一端，而最后生成的填充子区域位于填充区域的另一端。The filled sub-area is obtained by dividing each quadrilateral area according to the arrangement order of the quadrilateral areas in the filled area, so the filled sub-areas that are generated in adjacent order are also adjacent in the filled area, and the first generated filled sub-area is located in One end of the filled region, and the last generated filled subregion at the other end of the filled region.
可以按照纹理素材子块的生成次序，以及填充子区域的生成次序，建立纹理素材子块和所述填充子区域之间的一一对应关系，在该对应关系中，纹理素材块一端的纹理素材子块与填充区域一端的填充子区域对应。According to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions can be established. In this correspondence, the texture material at one end of the texture material block A subblock corresponds to a padding subarea at one end of the padding area.
因此，按照上述一一对应关系将纹理素材子块填充到填充子区域后，可以在填充区域中还原纹理素材贴图整体所呈现的效果，使纹理素材被填充到四边形区域后过渡更为自然。Therefore, after the texture material sub-blocks are filled into the filled sub-area according to the above-mentioned one-to-one correspondence, the overall effect of the texture material map can be restored in the filled area, so that the transition after the texture material is filled into the quadrilateral area is more natural.
步骤207、对所述纹理素材子块进行变形处理，使得所述纹理素材子块与对应的填充子区域的形状匹配。Step 207: Perform deformation processing on the texture material sub-block, so that the texture material sub-block matches the shape of the corresponding filling sub-region.
由于纹理素材子块与对应的填充子区域的形状可能并不完全相同，因此在将纹理素材子块填充到对应的填中子区域前，还需要对纹理素材子块进行变形处理，使其与对应的填充子区域的形状向匹配。Since the shape of the texture material sub-block may not be exactly the same as the corresponding filling sub-area, before filling the texture material sub-block into the corresponding filling sub-area, it is also necessary to deform the texture material sub-block to make it consistent with The shape of the corresponding filled subregion matches.
在一些实施例中，可以获取纹理素材子块的各个素材顶点坐标，以及对应的填充子区域的各个填充顶点坐标，并建立各个素材顶点坐标和各个填充顶点坐标的对应关系。再对纹理素材子块的各个顶点进行调整，使纹理素材子块的各个素材顶点坐标与对应的填充顶点坐标相一致，得到变形后的填充子区域。In some embodiments, the coordinates of each material vertex of the texture material sub-block and the coordinates of each filling vertex of the corresponding filling sub-region can be obtained, and a corresponding relationship between each material vertex coordinate and each filling vertex coordinate can be established. Then, each vertex of the texture material sub-block is adjusted so that the coordinates of each material vertex of the texture material sub-block are consistent with the corresponding filling vertex coordinates, and the deformed filling sub-region is obtained.
步骤208、根据所述纹理素材子块与所述填充子区域之间的一一对应关系，将所述纹理素材子块填充至对应的填充子区域中。Step 208: According to the one-to-one correspondence between the texture material sub-block and the filling sub-area, fill the texture material sub-block into the corresponding filling sub-area.
在一些实施例中，在进行填充操作时，可以预先建立包含所有描边关键点对应的扩展点的扩展点数组、用于指示填充区域中所有填充子区域的第一索引、以及用于指示所有纹理素材子块的第二索引，并将扩展点数组、第一索引和第二索引输入OpenGL，Web图形库(Web Graphics Library，WebGL)等图形绘制模型，以对视频帧的填充区域进行填充，得到填充后的视频帧。这样，仅通过对图形绘制模型的一次调用即可完成对整个填充区域的填充，提高了填充效率。In some embodiments, when the filling operation is performed, an expansion point array containing all expansion points corresponding to stroke key points, the first index used to indicate all filled sub-areas in the filled area, and the first index used to indicate all filled sub-areas can be established in advance. The second index of the texture material sub-block, and the extension point array, the first index and the second index are input into OpenGL, graphics drawing models such as Web Graphics Library (Web Graphics Library, WebGL), to fill the filled area of the video frame, Get the filled video frame. In this way, the filling of the entire filling area can be completed with only one call to the graphics drawing model, which improves the filling efficiency.
在一些实施例中，操作步骤如下：In some embodiments, the operation steps are as follows:
子步骤B1、按照描边关键点在描边关键点序列中的顺序，依次将每个描边关键点对应的两个扩展点的坐标添加至扩展点数组，从而得到构建好的扩展点数组，其中，描边关键点序列按照描边关键点的相邻关系构建。Sub-step B1. Add the coordinates of the two extension points corresponding to each stroke key point to the extension point array in turn according to the order of the stroke key points in the stroke key point sequence, so as to obtain the constructed extension point array, Among them, the stroke key point sequence is constructed according to the adjacent relationship of the stroke key points.
扩展点数组用于存放由描边关键点生成的所有扩展点，包括由起始描边关键点生成的第一扩展点，以及由非起始描边关键点生成的第二扩展点。The extension point array is used to store all the extension points generated by the stroke key point, including the first extension point generated by the start stroke key point and the second extension point generated by the non-start stroke key point.
可以按照描边关键点在描边关键点序列中的顺序，依次生成所有描边关键点对应的扩展点。并且，在生成起始描边关键点对应的两个第一扩展点时，将最先生成的一个第一扩 展点记作起始扩展点。在生成非起始描边关键点对应的两个第二扩展点时，先生成与起始扩展点在填充区域同侧的第二扩展点。例如，如图2所示，扩展点P1、P3、P5、P7位于填充区域的一侧，扩展点P2、P4、P6、P8位于填充区域的另一侧，假设在生成起始描边关键点O1对应的第一扩展点P1和P2时，先生成的第一扩展点为P1，则将第一扩展点P1作为起始扩展点，由于描边关键点O2对应的第二扩展点P3和P4中，第二扩展点P3与起始扩展点P1在填充区域的同一侧，则在生成描边关键点O2对应的第二扩展点时，先生成第二扩展点P3，后生成第二扩展点P4。The extension points corresponding to all the stroke key points can be sequentially generated according to the sequence of the stroke key points in the stroke key point sequence. And, when generating two first extension points corresponding to the initial stroke key point, the first first extension point generated first is recorded as the initial extension point. When generating two second extension points corresponding to non-initial stroke key points, first generate the second extension point on the same side of the fill area as the initial extension point. For example, as shown in Figure 2, the expansion points P1, P3, P5, and P7 are located on one side of the filled area, and the expanded points P2, P4, P6, and P8 are located on the other side of the filled area. When O1 corresponds to the first extension point P1 and P2, the first extension point generated earlier is P1, then the first extension point P1 is used as the initial extension point, because the stroke key point O2 corresponds to the second extension point P3 and P4 , the second extension point P3 is on the same side of the filling area as the initial extension point P1, then when generating the second extension point corresponding to the stroke key point O2, first generate the second extension point P3, and then generate the second extension point P4.
在生成每个描边关键点对应的扩展点时，按照上述扩展点的生成顺序，依次将所有第一扩展点和第二扩展点加入扩展点数组。以使扩展点数组中的扩展点按照其在扩展点数组中的顺序，每三个相邻的扩展点指示由一个四边形区域切分而成的填充子区域。需要说明的是，将扩展点加入扩展点数组时，加入的是扩展点对应的坐标。When generating the extension points corresponding to each stroke key point, all the first extension points and the second extension points are sequentially added to the extension point array according to the generation sequence of the above extension points. So that the extension points in the extension point array follow their order in the extension point array, and every three adjacent extension points indicate a filled sub-area divided by a quadrilateral area. It should be noted that when the extension point is added to the extension point array, the coordinates corresponding to the extension point are added.
例如，如图2所示，将扩展点P1至P8加入扩展点数组后，所形成的扩展点数组为(P1,P2,P3,P4,P5,P6,P7,P8),其中，扩展点P1、P2、P3构成填充子区域P1P2P3，扩展点P2、P3、P4构成填充子区域P2P3P4，以此类推，该扩展点数组中的每三个相邻的扩展点均可对应与一个填充子区域。For example, as shown in Figure 2, after the extension points P1 to P8 are added to the extension point array, the formed extension point array is (P1, P2, P3, P4, P5, P6, P7, P8), where the extension point P1 , P2, P3 form a filling sub-area P1P2P3, expansion points P2, P3, P4 form a filling sub-area P2P3P4, and so on, every three adjacent expansion points in the expansion point array can correspond to a filling sub-area.
子步骤B2、从扩展点数组的一端开始，依次选取所有扩展点组，扩展点组包括三个连续的扩展点坐标。Sub-step B2, starting from one end of the extension point array, select all extension point groups in sequence, and the extension point group includes three consecutive extension point coordinates.
在扩展点数组中，可以指示一个填充子区域的三个相邻的扩展点，构成一个扩展点组。例如，在子步骤B1中形成的扩展点数组中，依次选取(P1,P2，P3)构成一个扩展点组，(P2，P3，P4)构成一个扩展点组。In the extension point array, three adjacent extension points that fill a sub-area may be indicated to form an extension point group. For example, in the extension point array formed in sub-step B1, (P1, P2, P3) is sequentially selected to form an extension point group, and (P2, P3, P4) is selected to form an extension point group.
子步骤B3、依次将每个扩展点组的三个扩展点坐标的标识构建为一个第一索引，并将第一索引添加至第一索引数组中，从而得到构建好的第一索引数组。In sub-step B3 , sequentially constructing the identifiers of the coordinates of the three extension points of each extension point group as a first index, and adding the first index to the first index array, so as to obtain the constructed first index array.
扩展点坐标的标识可以是扩展点坐标在扩展点数组中的排列次序，例如，扩展点数组中的第一个扩展点坐标对应的标识为1，扩展点数组中的第二个扩展点坐标对应的标识为2，以此类推。也可以是其他能够指示扩展点数组中坐标的标识。The identifier of the extension point coordinates can be the arrangement order of the extension point coordinates in the extension point array, for example, the first extension point coordinate in the extension point array corresponds to 1, and the second extension point coordinate in the extension point array corresponds to The ID of is 2, and so on. It can also be other identifiers that can indicate the coordinates in the extension point array.
例如，对于扩展点数组(P1,P2,P3)和(P2,P3,P4)，若扩展点P1的标识为1，P2的标识为2，P3的标识为3，P4的标识为4，则扩展点数组(P1,P2,P3)对应的第一索引为(1,2,3)，扩展点数组(P2,P3,P4)对应的第一索引为(2,3,4)。For example, for the extension point arrays (P1, P2, P3) and (P2, P3, P4), if the extension point P1 is identified as 1, P2 is identified as 2, P3 is identified as 3, and P4 is identified as 4, then The first index corresponding to the extension point array (P1, P2, P3) is (1, 2, 3), and the first index corresponding to the extension point array (P2, P3, P4) is (2, 3, 4).
将所有扩展点组对应的第一索引按照扩展点组在扩展点数组中的排列顺序依次加入第一索引数组，完成对第一索引数组的构建。The first indexes corresponding to all the extension point groups are sequentially added to the first index array according to the arrangement order of the extension point groups in the extension point array, and the construction of the first index array is completed.
子步骤B4、从纹理素材贴图的一端开始，对纹理素材贴图依次进行等宽切分得到多个纹理素材块。Sub-step B4, starting from one end of the texture material map, sequentially divide the texture material map into equal widths to obtain multiple texture material blocks.
子步骤B5、依次将每个纹理素材块切分为两个纹理素材子块。Sub-step B5, sequentially divide each texture material block into two texture material sub-blocks.
子步骤B6、依次将每个纹理素材子块在纹理素材贴图中对应的三个角点坐标，构建为一个第二索引，并将第二索引添加至第二索引数组中，从而得到构建好的第二索引数组。Sub-step B6, sequentially construct the coordinates of the three corner points corresponding to each texture material sub-block in the texture material map as a second index, and add the second index to the second index array to obtain the constructed The second index array.
如图4所示，将第二索引数组作为一个纹理坐标系，其中，纹理素材贴图的一个角为纹理贴图构成坐标系的原点(0,0)，原点所在角的对角为(1,1)。则对纹理素材贴图进行切分后，每一个纹理素材块的角点在纹理素材贴图中对应一个顶角坐标。由于纹理素材贴图被切分为n块，则A1在纹理坐标系中的坐标为(0,0)，A2在纹理坐标系中的坐标为(1/n，0)，以此类推。As shown in Figure 4, the second index array is used as a texture coordinate system, wherein one corner of the texture material map is the origin (0,0) of the texture map constituting the coordinate system, and the opposite corner of the corner where the origin is located is (1,1 ). After the texture material map is segmented, the corner point of each texture material block corresponds to a top corner coordinate in the texture material map. Since the texture material map is divided into n blocks, the coordinates of A1 in the texture coordinate system are (0,0), the coordinates of A2 in the texture coordinate system are (1/n,0), and so on.
将纹理素材块在纹理素材贴图中的角点坐标依次加入第二索引数组中，使得角点坐标在第二索引数组中不重复，且第二索引数组中一组相邻的三个顶角坐标唯一对应于纹理素材贴图中的一个纹理素材子块。Add the corner point coordinates of the texture material block in the texture material map to the second index array in turn, so that the corner point coordinates are not repeated in the second index array, and a group of three adjacent corner coordinates in the second index array Uniquely corresponds to a texture material sub-block in the texture material map.
子步骤B7、将第一索引数组、第二索引数组和扩展点数组输入预设的绘制模型，得到绘制模型输出的描边纹理特效绘制结果。Sub-step B7, input the first index array, the second index array and the extension point array into the preset drawing model, and obtain the drawing result of stroke texture special effect output by the drawing model.
绘制模型可以包括OpenGL，Web图形库(Web Graphics Library，WebGL)等图形绘制模型，以对描边纹理特效进行绘制，并输出绘制结果。以OpenGL为例对图形绘制过程进行说明。The drawing model can include graphics drawing models such as OpenGL and Web Graphics Library (Web Graphics Library, WebGL), to draw the stroke texture special effect, and output the drawing result. Take OpenGL as an example to illustrate the graphics drawing process.
在扩展点数组、第一索引数组和第二索引数组构建完成后，将扩展点数组、第一索引数组和第二索引数组传入着色器(shader)中。再通过调用一次OpenGL中的glDrawElements()函数在GL_TRIANGLES绘制模式下，将第一索引数组中的每个第一索引指示的填充子区域，与第二索引数组中的每个第二索引指示的纹理素材子块进行匹配，并将所有纹理素材子块填充至对应的填充子区域中，一次性高效快速完成对描边纹理特效的绘制工作。After the extension point array, the first index array and the second index array are constructed, the extension point array, the first index array and the second index array are passed into the shader (shader). Then by calling the glDrawElements() function in OpenGL once in GL_TRIANGLES drawing mode, the filled sub-area indicated by each first index in the first index array and the texture indicated by each second index in the second index array The material sub-blocks are matched, and all the texture material sub-blocks are filled into the corresponding filled sub-areas, and the drawing of the stroke texture effect is completed efficiently and quickly at one time.
在本申请实施例中，通过构建扩展点数组、第一索引数组和第二索引数组，仅通过一次模型调用，即可完成对整个描边纹理特效的绘制工作，可以大大提高绘制描边纹理特效的效率，有助于节省终端设备的运算资源，加快描边纹理特效的绘制速度，极大改善了用户为视频或图片添加描边纹理特效的使用体验。In the embodiment of this application, by constructing the extension point array, the first index array, and the second index array, the drawing of the entire stroke texture effect can be completed through only one model call, which can greatly improve the drawing stroke texture effect. The efficiency helps save computing resources of the terminal device, speeds up the drawing speed of stroke texture effects, and greatly improves the user experience of adding stroke texture effects to videos or pictures.
步骤209、将所述填充后的视频帧按照时间顺序组合，得到特效视频。Step 209: Combine the filled video frames according to time sequence to obtain special effect video.
此步骤可参见步骤105，在此不再赘述。For this step, reference may be made to step 105, which will not be repeated here.
综上所述，通过对获取的目标视频中包含目标对象的视频帧进行描边关键点提取，获得目标对象外侧的描边关键点，对每个描边关键点进行扩展，在每个描边关键点周围构建出连接线经过该描边关键点的两个扩展点，使得所有相邻描边关键点对应的扩展点可以构成四边形区域，进而所有四边形区域可以形成围绕目标对象的填充区域，并采用纹理贴图对视频帧的填充区域进行填充，将填充后的视频帧按照播放时间顺序组合得到特效视频。由于相邻四边形区域之间使用同一个描边关键点对应的两个扩展点连接线作为共用边，因此相邻四边形区域之间可以形成较为平滑的过渡，形成的填充区域不会存在明显的锯齿状结构，使得对视频帧填充生成的描边纹理特效更为流畅平滑，提升了特效视频中描边纹理特效的呈现效果。To sum up, by extracting the stroke key points from the video frames containing the target object in the acquired target video, the stroke key points outside the target object are obtained, and each stroke key point is expanded. Build a connection line around the key point and pass through two extension points of the stroke key point, so that the extension points corresponding to all adjacent stroke key points can form a quadrilateral area, and then all quadrilateral areas can form a filled area around the target object, and The texture map is used to fill the filled area of the video frame, and the filled video frames are combined in order of playing time to obtain a special effect video. Since two expansion point connecting lines corresponding to the same stroke key point are used as shared edges between adjacent quadrilateral areas, a relatively smooth transition can be formed between adjacent quadrilateral areas, and there will be no obvious jaggedness in the formed filled area The shape structure makes the stroke texture special effect generated by filling the video frame more smooth and smooth, and improves the rendering effect of the stroke texture special effect in the special effect video.
图5是本申请实施例提供的一种描边纹理特效的实现装置的框图，如图5所示，包括：Fig. 5 is a block diagram of an implementation device for stroke texture effects provided by an embodiment of the present application, as shown in Fig. 5 , including:
获取模块301，被配置为获取目标视频，所述目标视频的视频帧包含目标对象；The obtaining module 301 is configured to obtain a target video, the video frame of which contains a target object;
关键点模块302，被配置为在所述视频帧中，确定位于所述目标对象外侧的描边关键点；The key point module 302 is configured to, in the video frame, determine a stroke key point located outside the target object;
填充区域模块303，被配置为在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，并根据每两个相邻描边关键点及其各自对应的扩展点构成一个四边形区域，得到多个所述四边形区域连接组成的填充区域；The fill area module 303 is configured to establish two extension points respectively located on both sides of the stroke key point on any straight line passing through the stroke key point, and according to each two adjacent stroke key points and their respective corresponding extension points form a quadrilateral area, and obtain a filled area formed by connecting multiple quadrilateral areas;
填充模块304，被配置为采用预设的纹理素材贴图，对所述视频帧的填充区域进行填充，得到填充后的视频帧；The filling module 304 is configured to use a preset texture material map to fill the filling area of the video frame to obtain a filled video frame;
组合模块305，被配置为将所述填充后的视频帧按照时间顺序组合，得到特效视频。The combination module 305 is configured to combine the filled video frames in time sequence to obtain special effect video.
在一些实施例中，所述填充区域模块303包括：In some embodiments, the filled area module 303 includes:
扩展点直线子模块，被配置为确定经过所述描边关键点，且垂直于所述描边关键点与一个相邻描边关键点构成的连接线的扩展点直线，所述相邻描边关键点是指与所述描边关键点相邻的描边关键点；The extension point straight line sub-module is configured to determine an extension point straight line that passes through the stroke key point and is perpendicular to a connecting line formed by the stroke key point and an adjacent stroke key point, and the adjacent stroke key point A key point refers to a stroke key point adjacent to the stroke key point;
扩展点子模块，被配置为在所述扩展点直线上建立两个到所述描边关键点距离相等的扩展点。The extension point submodule is configured to establish two extension points on the extension point straight line with equal distances from the stroke key point.
在一些实施例中，所述扩展点子模块包括：In some embodiments, the extension point submodule includes:
关键点序列子模块，被配置为按照所述描边关键点在所述目标对象外侧的排列顺序，将多个所述描边关键点构成描边关键点序列；The key point sequence submodule is configured to form a plurality of stroke key points into a stroke key point sequence according to the arrangement order of the stroke key points outside the target object;
起始关键点子模块，被配置为在所述描边点序列中的起始描边关键点对应的扩展点直 线上，建立两个到所述起始描边关键点的距离为第一距离的扩展点；The start key point sub-module is configured to establish two distances from the start stroke key point at the first distance on the extension point line corresponding to the start stroke key point in the stroke point sequence. extension point;
非起始关键点子模块，被配置为在所述描边点序列中的非起始描边关键点对应的扩展点直线上，建立两个到所述非起始描边关键点的距离为第二距离的扩展点。The non-initial key point sub-module is configured to establish two distances from the non-initial stroke key point on the straight line corresponding to the non-initial stroke key point in the stroke point sequence. Two-distance extension point.
在一些实施例中，所述装置还包括：In some embodiments, the device also includes:
缩放子模块，被配置为根据所述非起始描边关键点在所述描边关键点序列中的次序，从倍率对应关系中确定所述非起始描边关键点对应的目标距离缩放倍率，所述倍率对应关系用于表征非起始描边关键点在所述描边关键点序列中的次序和距离缩放倍率之间的对应关系；The scaling sub-module is configured to determine the target distance zoom magnification corresponding to the non-initial stroke key point from the magnification correspondence relationship according to the order of the non-initial stroke key point in the stroke key point sequence , the magnification correspondence is used to characterize the correspondence between the order of the non-initial stroke key points in the stroke key point sequence and the distance zoom magnification;
第二距离子模块，被配置为将所述目标距离缩放倍率与所述第一距离的乘积，确定为所述非起始描边关键点对应的第二距离。The second distance sub-module is configured to determine the product of the target distance scaling factor and the first distance as the second distance corresponding to the non-initial stroke key point.
在一些实施例中，所述填充模块304包括：In some embodiments, the filling module 304 includes:
素材切分子模块，被配置为将所述纹理素材贴图切分为预设数量个三角形的纹理素材子块；The material cutting sub-module is configured to divide the texture material map into texture material sub-blocks with a preset number of triangles;
填充区域切分子模块，被配置为按照所述四边形区域在所述填充区域中的排列顺序，从所述填充区域的一端开始，依次将每个所述四边形区域沿一条对角线切分为两个填充子区域，得到所述预设数量个填充子区域；The filling area cutting sub-module is configured to divide each quadrilateral area into two parts along a diagonal line in sequence starting from one end of the filling area according to the order of arrangement of the quadrilateral areas in the filling area. filling sub-regions, obtaining the preset number of filling sub-regions;
对应关系子模块，被配置为建立所述纹理素材子块与所述填充子区域之间的一一对应关系；a correspondence sub-module configured to establish a one-to-one correspondence between the texture material sub-block and the filling sub-region;
匹配子模块，被配置为对所述纹理素材子块进行变形处理，使得所述纹理素材子块与对应的填充子区域的形状匹配；The matching submodule is configured to deform the texture material sub-block, so that the texture material sub-block matches the shape of the corresponding filling sub-region;
填充子模块，被配置为根据所述纹理素材子块与所述填充子区域之间的一一对应关系，将所述纹理素材子块填充至对应的填充子区域中。The filling sub-module is configured to fill the texture material sub-block into the corresponding filling sub-area according to the one-to-one correspondence between the texture material sub-block and the filling sub-area.
在一些实施例中，所述素材切分子模块包括：In some embodiments, the material cutting sub-module includes:
第一切分子模块，被配置为从所述纹理素材贴图的一端到另一端，对所述纹理素材贴图进行等宽切分，得到与所述四边形区域的数量相同数量的四边形的纹理素材块；The first split module is configured to perform equal-width segmentation on the texture material map from one end to the other end of the texture material map, so as to obtain quadrilateral texture material blocks having the same number as the quadrilateral area;
第二切分子模块，被配置为按照所述纹理素材块的生成次序，依次将每个所述纹理素材块沿一条对角线切分为两个纹理素材子块。The second sub-cutting module is configured to sequentially cut each texture material block into two texture material sub-blocks along a diagonal line according to the generation order of the texture material blocks.
在一些实施例中，所述对应关系子模块包括：In some embodiments, the correspondence submodule includes:
关系建立子模块，被配置为按照所述纹理素材子块的生成次序以及所述填充子区域的生成次序，建立所述纹理素材子块和所述填充子区域之间的一一对应关系，其中，生成次序相同的纹理素材子块和填充子区域相对应。The relationship establishment submodule is configured to establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions according to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, wherein , the texture material sub-blocks with the same generation order correspond to the filling sub-regions.
综上所述，通过对获取的目标视频中包含目标对象的视频帧进行描边关键点提取，获得目标对象外侧的描边关键点，再对每个描边关键点进行扩展，在每个描边关键点周围构建出连接线经过该描边关键点的两个扩展点，使得所有相邻描边关键点对应的扩展点可以构成四边形区域，进而所有四边形区域可以形成围绕目标对象的填充区域，并采用纹理贴图对视频帧的填充区域进行填充，最终将填充后的视频帧按照播放时间顺序组合得到特效视频。由于相邻四边形区域之间使用同一个描边关键点对应的两个扩展点连接线作为共用边，因此相邻四边形区域之间可以形成较为平滑的过渡，形成的填充区域不会存在明显的锯齿状结构，使得对视频帧填充生成的描边纹理特效更为流畅平滑，提升了特效视频中描边纹理特效的呈现效果。To sum up, by extracting the stroke key points from the acquired video frame containing the target object, the stroke key points outside the target object are obtained, and then each stroke key point is expanded, and each stroke key point is Build a connection line around the edge key point and pass through two extension points of the stroke key point, so that the extension points corresponding to all adjacent stroke key points can form a quadrilateral area, and then all quadrilateral areas can form a filled area around the target object, And the texture map is used to fill the filling area of the video frame, and finally the filled video frames are combined according to the playing time sequence to obtain the special effect video. Since two expansion point connecting lines corresponding to the same stroke key point are used as shared edges between adjacent quadrilateral areas, a relatively smooth transition can be formed between adjacent quadrilateral areas, and there will be no obvious jaggedness in the formed filled area The shape structure makes the stroke texture special effect generated by filling the video frame more smooth and smooth, and improves the rendering effect of the stroke texture special effect in the special effect video.
图6是根据一示例性实施例示出的一种电子设备600的框图。该电子设备600可以执行本申请实施例中的图像特效的生成方法。例如，电子设备600可以是移动电话、计算机、数字广播终端、消息收发设备、游戏控制台、平板设备、医疗设备、健身设备、个人数字助理等。Fig. 6 is a block diagram of an electronic device 600 according to an exemplary embodiment. The electronic device 600 may execute the method for generating an image special effect in the embodiment of the present application. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.
参照图6，电子设备600可以包括以下一个或多个组件：处理组件602，存储器604， 电力组件606，多媒体组件608，音频组件610，输入/输出(I/O)的接口612，传感器组件614，以及通信组件616。6, electronic device 600 may include one or more of the following components: processing component 602, memory 604, power component 606, multimedia component 608, audio component 610, input/output (I/O) interface 612, sensor component 614 , and the communication component 616.
处理组件602通常控制电子设备600的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件602可以包括一个或多个处理器620来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件602可以包括一个或多个模块，便于处理组件602和其他组件之间的交互。例如，处理组件602可以包括多媒体模块，以方便多媒体组件608和处理组件602之间的交互。The processing component 602 generally controls the overall operations of the electronic device 600, such as those associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 602 may include one or more modules that facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602 .
存储器604用于存储各种类型的数据以支持在电子设备600的操作。这些数据的示例包括用于在电子设备600上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器604可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。The memory 604 is used to store various types of data to support operations in the electronic device 600 . Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phonebook data, messages, pictures, videos, and the like. The memory 604 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.
电源组件606为电子设备600的各种组件提供电力。电源组件606可以包括电源管理***，一个或多个电源，及其他与为电子设备600生成、管理和分配电力相关联的组件。The power supply component 606 provides power to various components of the electronic device 600 . Power components 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 600 .
多媒体组件608包括在所述电子设备600和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件608包括一个前置摄像头和/或后置摄像头。当电子设备600处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜***或具有焦距和光学变焦能力。The multimedia component 608 includes a screen providing an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 608 includes a front camera and/or a rear camera. When the electronic device 600 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.
音频组件610用于输出和/或输入音频信号。例如，音频组件610包括一个麦克风(MIC)，当电子设备600处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风用于接收外部音频信号。所接收的音频信号可以被进一步存储在存储器604或经由通信组件616发送。在一些实施例中，音频组件610还包括一个扬声器，用于输出音频信号。The audio component 610 is used to output and/or input audio signals. For example, the audio component 610 includes a microphone (MIC), which is used to receive external audio signals when the electronic device 600 is in operation modes, such as calling mode, recording mode and voice recognition mode. Received audio signals may be further stored in memory 604 or sent via communication component 616 . In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.
I/O接口612为处理组件602和***接口模块之间提供接口，上述***接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。The I/O interface 612 provides an interface between the processing component 602 and a peripheral interface module. The peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.
传感器组件614包括一个或多个传感器，用于为电子设备600提供各个方面的状态评估。例如，传感器组件614可以检测到电子设备600的打开/关闭状态，组件的相对定位，例如所述组件为电子设备600的显示器和小键盘，传感器组件614还可以检测电子设备600或电子设备600一个组件的位置改变，用户与电子设备600接触的存在或不存在，电子设备600方位或加速/减速和电子设备600的温度变化。传感器组件614可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件614还可以包括光传感器，如CMOS或CCD图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件614还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。 Sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for electronic device 600 . For example, the sensor assembly 614 can detect the open/close state of the electronic device 600, the relative positioning of the components, such as the display and the keypad of the electronic device 600, the sensor assembly 614 can also detect the electronic device 600 or one of the electronic device 600 The position of components changes, the presence or absence of user contact with the electronic device 600 , the orientation or acceleration/deceleration of the electronic device 600 and the temperature of the electronic device 600 change. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 614 may also include optical sensors, such as CMOS or CCD image sensors, for use in imaging applications. In some embodiments, the sensor component 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.
通信组件616用于便于电子设备600和其他设备之间有线或无线方式的通信。电子设备600可以接入基于通信标准的无线网络，如WiFi，运营商网络(如2G、3G、4G或5G)，或它们的组合。在一个示例性实施例中，通信组件616经由广播信道接收来自外部广播管理***的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件616还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技 术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。The communication component 616 is used to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 616 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.
在一些实施例中，电子设备600可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于实现本申请实施例提供的图像特效的生成方法。In some embodiments, the electronic device 600 may be programmed by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Realized by a gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, for realizing the method for generating an image special effect provided by the embodiment of the present application.
在一些实施例中，还提供了一种包括指令的非临时性计算机存储介质，例如包括指令的存储器604，上述指令可由电子设备600的处理器620执行以完成上述方法。例如，所述非临时性存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。In some embodiments, there is also provided a non-transitory computer storage medium including instructions, such as the memory 604 including instructions, which can be executed by the processor 620 of the electronic device 600 to complete the above method. For example, the non-transitory storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, among others.
图7是根据一示例性实施例示出的一种电子设备700的框图。该电子设备600可以执行本申请实施例中的图像特效的生成方法。例如，电子设备700可以被提供为一服务器。参照图7，电子设备700包括处理组件722，其进一步包括一个或多个处理器，以及由存储器732所代表的存储器资源，用于存储可由处理组件722的执行的指令，例如应用程序。存储器732中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件722被配置为执行指令，以执行本申请实施例提供的图像特效的生成方法。Fig. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. The electronic device 600 may execute the method for generating an image special effect in the embodiment of the present application. For example, the electronic device 700 may be provided as a server. Referring to FIG. 7 , electronic device 700 includes processing component 722 , which further includes one or more processors, and a memory resource represented by memory 732 for storing instructions executable by processing component 722 , such as application programs. The application program stored in memory 732 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 722 is configured to execute instructions to execute the method for generating an image special effect provided by the embodiment of the present application.
电子设备700还可以包括一个电源组件726被配置为执行电子设备700的电源管理，一个有线或无线网络接口750被配置为将电子设备700连接到网络，和一个输入输出(I/O)接口758。电子设备700可以操作基于存储在存储器732的操作***，例如Windows ServerTM，Mac OS XTM，UnixTM，LinuxTM，FreeBSDTM或类似。 Electronic device 700 may also include a power supply component 726 configured to perform power management of electronic device 700, a wired or wireless network interface 750 configured to connect electronic device 700 to a network, and an input-output (I/O) interface 758 . The electronic device 700 can operate based on an operating system stored in the memory 732, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.
本申请实施例还提供一种计算机程序产品，包括计算机程序/指令，所述计算机程序/指令被处理器执行时实现所述的图像特效的生成方法。An embodiment of the present application further provides a computer program product, including a computer program/instruction, and when the computer program/instruction is executed by a processor, the above method for generating an image special effect is implemented.
本公开的所有实施例均可以单独被执行，也可以与其他实施例相结合被执行，均视为本公开要求的保护范围。All the embodiments of the present disclosure can be implemented independently or in combination with other embodiments, which are all regarded as the scope of protection required by the present disclosure.
本领域技术人员在考虑说明书及实践这里公开的申请后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由下面的权利要求指出。Other embodiments of the application will be readily apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any modification, use or adaptation of the application, these modifications, uses or adaptations follow the general principles of the application and include common knowledge or conventional technical means in the technical field not disclosed in this disclosure . The specification and examples are to be considered exemplary only, with a true scope and spirit of the application indicated by the following claims.
应当理解的是，本申请并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变。本申请的范围仅由所附的权利要求来限制。It should be understood that the present application is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (23)

  1. 一种图像特效的生成方法，其特征在于，包括：A method for generating image special effects, comprising:
    获取目标视频，所述目标视频的视频帧包含目标对象；Obtain the target video, the video frame of the target video contains the target object;
    在所述视频帧中，确定位于所述目标对象外侧的描边关键点；In the video frame, determine stroke key points located outside the target object;
    在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，并根据每两个相邻描边关键点及其各自对应的扩展点构成一个四边形区域，得到多个所述四边形区域连接组成的填充区域；On any straight line passing through the stroke key point, establish two extension points respectively located on both sides of the stroke key point, and form A quadrilateral area, obtaining a filled area formed by connecting a plurality of said quadrilateral areas;
    采用预设的纹理素材贴图，对所述视频帧的填充区域进行填充，得到填充后的视频帧；Filling the filled area of the video frame by using a preset texture material map to obtain the filled video frame;
    将所述填充后的视频帧按照时间顺序组合，得到特效视频。Combining the filled video frames in chronological order to obtain a special effect video.
  2. 根据权利要求1所述的方法，其特征在于，所述在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，包括：The method according to claim 1, wherein, on any straight line passing through the stroke key point, establishing two extension points respectively located on both sides of the stroke key point comprises:
    确定经过所述描边关键点，且垂直于所述描边关键点与一个相邻描边关键点构成的连接线的扩展点直线，所述相邻描边关键点是指与所述描边关键点相邻的描边关键点；Determine the extension point straight line passing through the stroke key point and perpendicular to the connection line formed by the stroke key point and an adjacent stroke key point, the adjacent stroke key point refers to the Stroke key points adjacent to the key point;
    在所述扩展点直线上建立两个到所述描边关键点距离相等的扩展点。Two expansion points with equal distances from the stroke key point are established on the expansion point straight line.
  3. 根据权利要求2所述的方法，其特征在于，所述在所述扩展点直线上建立两个到所述描边关键点距离相等的扩展点，包括：The method according to claim 2, wherein the establishment of two extension points on the straight line of the extension point with equal distances from the stroke key point comprises:
    按照所述描边关键点在所述目标对象外侧的排列顺序，将多个所述描边关键点构成描边关键点序列；According to the arrangement order of the stroke key points outside the target object, a plurality of the stroke key points are formed into a stroke key point sequence;
    在所述描边点序列中的起始描边关键点对应的扩展点直线上，建立两个到所述起始描边关键点的距离为第一距离的扩展点；On the extension point straight line corresponding to the initial stroke key point in the stroke point sequence, establish two extension points whose distance to the initial stroke key point is a first distance;
    在所述描边点序列中的非起始描边关键点对应的扩展点直线上，建立两个到所述非起始描边关键点的距离为第二距离的扩展点。On the extension point line corresponding to the non-initial stroke key point in the stroke point sequence, establish two extension points whose distance to the non-initial stroke key point is a second distance.
  4. 根据权利要求3所述的方法，其特征在于，所述方法还包括：The method according to claim 3, further comprising:
    根据所述非起始描边关键点在所述描边关键点序列中的次序，从倍率对应关系中确定所述非起始描边关键点对应的目标距离缩放倍率，所述倍率对应关系用于表征非起始描边关键点在所述描边关键点序列中的次序和距离缩放倍率之间的对应关系；According to the order of the non-initial stroke key points in the stroke key point sequence, determine the target distance scaling factor corresponding to the non-initial stroke key point from the magnification corresponding relationship, and the magnification corresponding relationship is used To characterize the correspondence between the sequence of non-initial stroke key points in the sequence of stroke key points and the distance scaling factor;
    将所述目标距离缩放倍率与所述第一距离的乘积，确定为所述非起始描边关键点对应的第二距离。The product of the target distance scaling factor and the first distance is determined as the second distance corresponding to the non-initial stroke key point.
  5. 根据权利要求1所述的方法，其特征在于，所述采用预设的纹理素材贴图，对所述目标视频中多个视频帧的填充区域进行填充，包括：The method according to claim 1, wherein the filling area of a plurality of video frames in the target video is filled by using a preset texture material map, comprising:
    将所述纹理素材贴图切分为预设数量个三角形的纹理素材子块；Dividing the texture material map into texture material sub-blocks with a preset number of triangles;
    按照所述四边形区域在所述填充区域中的排列顺序，从所述填充区域的一端开始，依次将每个所述四边形区域沿一条对角线切分为两个填充子区域，得到所述预设数量个填充子区域；According to the arrangement order of the quadrilateral areas in the filling area, starting from one end of the filling area, each of the quadrilateral areas is divided into two filling sub-areas along a diagonal line in turn, to obtain the predetermined Set the number of filled sub-regions;
    建立所述纹理素材子块与所述填充子区域之间的一一对应关系；Establishing a one-to-one correspondence between the texture material sub-block and the filling sub-region;
    对所述纹理素材子块进行变形处理，使得所述纹理素材子块与对应的填充子区域的形状匹配；deforming the texture material sub-block, so that the texture material sub-block matches the shape of the corresponding filling sub-region;
    根据所述纹理素材子块与所述填充子区域之间的一一对应关系，将所述纹理素材子块填充至对应的填充子区域中。According to the one-to-one correspondence between the texture material sub-block and the filling sub-region, the texture material sub-block is filled into the corresponding filling sub-region.
  6. 根据权利要求5所述的方法，其特征在于，所述将所述纹理素材贴图切分为预设数量个三角形的纹理素材子块，包括：The method according to claim 5, wherein said dividing said texture material map into texture material sub-blocks with a preset number of triangles comprises:
    从所述纹理素材贴图的一端到另一端，对所述纹理素材贴图进行等宽切分，得到与所述四边形区域的数量相同个四边形的纹理素材块；From one end of the texture material map to the other end, the texture material map is divided into equal widths to obtain quadrilateral texture material blocks equal to the number of quadrilateral regions;
    按照所述纹理素材块的生成次序，依次将每个所述纹理素材块沿一条对角线切分为两个纹理素材子块。According to the generation sequence of the texture material blocks, each of the texture material blocks is sequentially divided into two texture material sub-blocks along a diagonal line.
  7. 根据权利要求6所述的方法，其特征在于，所述建立所述纹理素材子块与所述填充子区域之间的一一对应关系，包括：The method according to claim 6, wherein the establishment of the one-to-one correspondence between the texture material sub-block and the filling sub-region comprises:
    按照所述纹理素材子块的生成次序以及所述填充子区域的生成次序，建立所述纹理素材子块和所述填充子区域之间的一一对应关系，其中，生成次序相同的纹理素材子块和填充子区域相对应。According to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions, wherein the texture material sub-blocks in the same order are generated Blocks correspond to filled subregions.
  8. 一种图像特效的生成装置，其特征在于，包括：A device for generating special image effects, comprising:
    获取模块，被配置为获取目标视频，所述目标视频的视频帧包含目标对象；An acquisition module configured to acquire a target video, where a video frame of the target video contains a target object;
    关键点模块，被配置为在所述视频帧中，确定位于所述目标对象外侧的描边关键点；a key point module configured to determine stroke key points located outside the target object in the video frame;
    填充区域模块，被配置为在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，并根据每两个相邻描边关键点及其各自对应的扩展点构成一个四边形区域，得到多个所述四边形区域连接组成的填充区域；The filling area module is configured to establish two extension points respectively located on both sides of the stroke key point on any straight line passing through the stroke key point, and according to each two adjacent stroke key points and The corresponding extension points form a quadrilateral area, and a filled area formed by connecting multiple quadrilateral areas is obtained;
    填充模块，被配置为采用预设的纹理素材贴图，对所述视频帧的填充区域进行填充，得到填充后的视频帧；The filling module is configured to use a preset texture material map to fill the filling area of the video frame to obtain the filled video frame;
    组合模块，被配置为将所述填充后的视频帧按照时间顺序组合，得到特效视频。The combination module is configured to combine the filled video frames in time order to obtain special effect video.
  9. 根据权利要求8所述的装置，其特征在于，所述填充区域模块包括：The device according to claim 8, wherein the filling area module comprises:
    扩展点直线子模块，被配置为确定经过所述描边关键点，且垂直于所述描边关键点与一个相邻描边关键点构成的连接线的扩展点直线，所述相邻描边关键点是指与所述描边关键点相邻的描边关键点；The extension point straight line sub-module is configured to determine an extension point straight line that passes through the stroke key point and is perpendicular to a connecting line formed by the stroke key point and an adjacent stroke key point, and the adjacent stroke key point A key point refers to a stroke key point adjacent to the stroke key point;
    扩展点子模块，被配置为在所述扩展点直线上建立两个到所述描边关键点距离相等的扩展点。The extension point submodule is configured to establish two extension points on the extension point straight line with equal distances from the stroke key point.
  10. 根据权利要求9所述的装置，其特征在于，所述扩展点子模块包括：The device according to claim 9, wherein the extension point submodule comprises:
    关键点序列子模块，被配置为按照所述描边关键点在所述目标对象外侧的排列顺序，将多个所述描边关键点构成描边关键点序列；The key point sequence submodule is configured to form a plurality of stroke key points into a stroke key point sequence according to the arrangement order of the stroke key points outside the target object;
    起始关键点子模块，被配置为在所述描边点序列中的起始描边关键点对应的扩展点直线上，建立两个到所述起始描边关键点的距离为第一距离的扩展点；The start key point sub-module is configured to establish two distances from the start stroke key point at the first distance on the extension point line corresponding to the start stroke key point in the stroke point sequence. extension point;
    非起始关键点子模块，被配置为在所述描边点序列中的非起始描边关键点对应的扩展点直线上，建立两个到所述非起始描边关键点的距离为第二距离的扩展点。The non-initial key point sub-module is configured to establish two distances from the non-initial stroke key point on the straight line corresponding to the non-initial stroke key point in the stroke point sequence. Two-distance extension point.
  11. 根据权利要求10所述的装置，其特征在于，所述装置还包括：The device according to claim 10, further comprising:
    缩放子模块，被配置为根据所述非起始描边关键点在所述描边关键点序列中的次序，从倍率对应关系中确定所述非起始描边关键点对应的目标距离缩放倍率，所述倍率对应关系用于表征非起始描边关键点在所述描边关键点序列中的次序和距离缩放倍率之间的对应关系；The scaling sub-module is configured to determine the target distance zoom magnification corresponding to the non-initial stroke key point from the magnification correspondence relationship according to the order of the non-initial stroke key point in the stroke key point sequence , the magnification correspondence is used to characterize the correspondence between the order of the non-initial stroke key points in the stroke key point sequence and the distance zoom magnification;
    第二距离子模块，被配置为将所述目标距离缩放倍率与所述第一距离的乘积，确定为所述非起始描边关键点对应的第二距离。The second distance sub-module is configured to determine the product of the target distance scaling factor and the first distance as the second distance corresponding to the non-initial stroke key point.
  12. 根据权利要求8所述的装置，其特征在于，所述填充模块包括：The device according to claim 8, wherein the filling module comprises:
    素材切分子模块，被配置为将所述纹理素材贴图切分为预设数量个三角形的纹理素材子块；The material cutting sub-module is configured to divide the texture material map into texture material sub-blocks with a preset number of triangles;
    填充区域切分子模块，被配置为按照所述四边形区域在所述填充区域中的排列顺序，从所述填充区域的一端开始，依次将每个所述四边形区域沿一条对角线切分为两个填充子区域，得到所述预设数量个填充子区域；The filling area cutting sub-module is configured to divide each quadrilateral area into two parts along a diagonal line in sequence starting from one end of the filling area according to the order of arrangement of the quadrilateral areas in the filling area. filling sub-regions, obtaining the preset number of filling sub-regions;
    对应关系子模块，被配置为建立所述纹理素材子块与所述填充子区域之间的一一对应关系；a correspondence sub-module configured to establish a one-to-one correspondence between the texture material sub-block and the filling sub-region;
    匹配子模块，被配置为对所述纹理素材子块进行变形处理，使得所述纹理素材子块与对应的填充子区域的形状匹配；The matching submodule is configured to deform the texture material sub-block, so that the texture material sub-block matches the shape of the corresponding filling sub-region;
    填充子模块，被配置为根据所述纹理素材子块与所述填充子区域之间的一一对应关系，将所述纹理素材子块填充至对应的填充子区域中。The filling sub-module is configured to fill the texture material sub-block into the corresponding filling sub-area according to the one-to-one correspondence between the texture material sub-block and the filling sub-area.
  13. 根据权利要求12所述的装置，其特征在于，所述素材切分子模块包括：The device according to claim 12, wherein the material cutting sub-module comprises:
    第一切分子模块，被配置为从所述纹理素材贴图的一端到另一端，对所述纹理素材贴图进行等宽切分，得到与所述四边形区域的数量相同个四边形的纹理素材块；The first cutting module is configured to perform equal-width segmentation on the texture material map from one end to the other end of the texture material map, so as to obtain quadrilateral texture material blocks equal to the number of the quadrilateral regions;
    第二切分子模块，被配置为按照所述纹理素材块的生成次序，依次将每个所述纹理素材块沿一条对角线切分为两个纹理素材子块。The second sub-cutting module is configured to sequentially cut each texture material block into two texture material sub-blocks along a diagonal line according to the generation order of the texture material blocks.
  14. 根据权利要求13所述的装置，其特征在于，所述对应关系子模块包括：The device according to claim 13, wherein the correspondence submodule comprises:
    关系建立子模块，被配置为按照所述纹理素材子块的生成次序以及所述填充子区域的生成次序，建立所述纹理素材子块和所述填充子区域之间的一一对应关系，其中，生成次序相同的纹理素材子块和填充子区域相对应。The relationship establishment submodule is configured to establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions according to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, wherein , the texture material sub-blocks with the same generation order correspond to the filling sub-regions.
  15. 一种电子设备，其特征在于，包括：An electronic device, characterized in that it comprises:
    处理器；以及processor; and
    用于存储所述处理器可执行指令的存储器；memory for storing said processor-executable instructions;
    其中，所述处理器被配置为执行所述指令，以实现以下步骤：Wherein, the processor is configured to execute the instructions to achieve the following steps:
    获取目标视频，所述目标视频的视频帧包含目标对象；Obtain the target video, the video frame of the target video contains the target object;
    在所述视频帧中，确定位于所述目标对象外侧的描边关键点；In the video frame, determine stroke key points located outside the target object;
    在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，并根据每两个相邻描边关键点及其各自对应的扩展点构成一个四边形区域，得到多个所述四边形区域连接组成的填充区域；On any straight line passing through the stroke key point, establish two extension points respectively located on both sides of the stroke key point, and form A quadrilateral area, obtaining a filled area formed by connecting a plurality of said quadrilateral areas;
    采用预设的纹理素材贴图，对所述视频帧的填充区域进行填充，得到填充后的视频帧；Filling the filled area of the video frame by using a preset texture material map to obtain the filled video frame;
    将所述填充后的视频帧按照时间顺序组合，得到特效视频。Combining the filled video frames in chronological order to obtain a special effect video.
  16. 根据权利要求15所述的电子设备，其特征在于，所述在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，包括：The electronic device according to claim 15, wherein, on any straight line passing through the stroke key point, establishing two extension points respectively located on both sides of the stroke key point comprises:
    确定经过所述描边关键点，且垂直于所述描边关键点与一个相邻描边关键点构成的连接线的扩展点直线，所述相邻描边关键点是指与所述描边关键点相邻的描边关键点；Determine the extension point straight line passing through the stroke key point and perpendicular to the connection line formed by the stroke key point and an adjacent stroke key point, the adjacent stroke key point refers to the Stroke key points adjacent to the key point;
    在所述扩展点直线上建立两个到所述描边关键点距离相等的扩展点。Two expansion points with equal distances from the stroke key point are established on the expansion point straight line.
  17. 根据权利要求16所述的电子设备，其特征在于，所述在所述扩展点直线上建立两个到所述描边关键点距离相等的扩展点，包括：The electronic device according to claim 16, wherein the establishment of two expansion points on the straight line of the expansion point with equal distances from the stroke key point comprises:
    按照所述描边关键点在所述目标对象外侧的排列顺序，将多个所述描边关键点构成描边关键点序列；According to the arrangement order of the stroke key points outside the target object, a plurality of the stroke key points are formed into a stroke key point sequence;
    在所述描边点序列中的起始描边关键点对应的扩展点直线上，建立两个到所述起始描边关键点的距离为第一距离的扩展点；On the extension point straight line corresponding to the initial stroke key point in the stroke point sequence, establish two extension points whose distance to the initial stroke key point is a first distance;
    在所述描边点序列中的非起始描边关键点对应的扩展点直线上，建立两个到所述非起始描边关键点的距离为第二距离的扩展点。On the extension point line corresponding to the non-initial stroke key point in the stroke point sequence, establish two extension points whose distance to the non-initial stroke key point is a second distance.
  18. 根据权利要求17所述的电子设备，其特征在于，所述处理器还被配置为执行所述指令以实现以下步骤：The electronic device according to claim 17, wherein the processor is further configured to execute the instructions to achieve the following steps:
    根据所述非起始描边关键点在所述描边关键点序列中的次序，从倍率对应关系中确定所述非起始描边关键点对应的目标距离缩放倍率，所述倍率对应关系用于表征非起始描边关键点在所述描边关键点序列中的次序和距离缩放倍率之间的对应关系；According to the order of the non-initial stroke key points in the stroke key point sequence, determine the target distance scaling factor corresponding to the non-initial stroke key point from the magnification corresponding relationship, and the magnification corresponding relationship is used To characterize the correspondence between the sequence of non-initial stroke key points in the sequence of stroke key points and the distance scaling factor;
    将所述目标距离缩放倍率与所述第一距离的乘积，确定为所述非起始描边关键 点对应的第二距离。The product of the target distance scaling factor and the first distance is determined as the second distance corresponding to the non-initial stroke key point.
  19. 根据权利要求15所述的电子设备，其特征在于，所述采用预设的纹理素材贴图，对所述目标视频中多个视频帧的填充区域进行填充，包括：The electronic device according to claim 15, wherein the filling area of a plurality of video frames in the target video is filled by using a preset texture material map, comprising:
    将所述纹理素材贴图切分为预设数量个三角形的纹理素材子块；Dividing the texture material map into texture material sub-blocks with a preset number of triangles;
    按照所述四边形区域在所述填充区域中的排列顺序，从所述填充区域的一端开始，依次将每个所述四边形区域沿一条对角线切分为两个填充子区域，得到所述预设数量个填充子区域；According to the arrangement order of the quadrilateral areas in the filling area, starting from one end of the filling area, each of the quadrilateral areas is divided into two filling sub-areas along a diagonal line in turn, to obtain the predetermined Set the number of filled sub-regions;
    建立所述纹理素材子块与所述填充子区域之间的一一对应关系；Establishing a one-to-one correspondence between the texture material sub-block and the filling sub-region;
    对所述纹理素材子块进行变形处理，使得所述纹理素材子块与对应的填充子区域的形状匹配；deforming the texture material sub-block, so that the texture material sub-block matches the shape of the corresponding filling sub-region;
    根据所述纹理素材子块与所述填充子区域之间的一一对应关系，将所述纹理素材子块填充至对应的填充子区域中。According to the one-to-one correspondence between the texture material sub-block and the filling sub-region, the texture material sub-block is filled into the corresponding filling sub-region.
  20. 根据权利要求19所述的电子设备，其特征在于，所述将所述纹理素材贴图切分为预设数量个三角形的纹理素材子块，包括：The electronic device according to claim 19, wherein said dividing said texture material map into texture material sub-blocks with a preset number of triangles comprises:
    从所述纹理素材贴图的一端到另一端，对所述纹理素材贴图进行等宽切分，得到与所述四边形区域的数量相同个四边形的纹理素材块；From one end of the texture material map to the other end, the texture material map is divided into equal widths to obtain quadrilateral texture material blocks equal to the number of quadrilateral regions;
    按照所述纹理素材块的生成次序，依次将每个所述纹理素材块沿一条对角线切分为两个纹理素材子块。According to the generation sequence of the texture material blocks, each of the texture material blocks is sequentially divided into two texture material sub-blocks along a diagonal line.
  21. 根据权利要求20所述的电子设备，其特征在于，所述建立所述纹理素材子块与所述填充子区域之间的一一对应关系，包括：The electronic device according to claim 20, wherein the establishment of the one-to-one correspondence between the texture material sub-block and the filling sub-region comprises:
    按照所述纹理素材子块的生成次序以及所述填充子区域的生成次序，建立所述纹理素材子块和所述填充子区域之间的一一对应关系，其中，生成次序相同的纹理素材子块和填充子区域相对应。According to the generation order of the texture material sub-blocks and the generation order of the filling sub-regions, establish a one-to-one correspondence between the texture material sub-blocks and the filling sub-regions, wherein the texture material sub-blocks in the same order are generated Blocks correspond to filled subregions.
  22. 一种计算机存储介质，其特征在于，当所述计算机可读存储介质中的指令由电子设备的处理器执行时，使得电子设备能够执行以下步骤：A computer storage medium, characterized in that when the instructions in the computer-readable storage medium are executed by a processor of the electronic device, the electronic device can perform the following steps:
    获取目标视频，所述目标视频的视频帧包含目标对象；Obtain the target video, the video frame of the target video contains the target object;
    在所述视频帧中，确定位于所述目标对象外侧的描边关键点；In the video frame, determine stroke key points located outside the target object;
    在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，并根据每两个相邻描边关键点及其各自对应的扩展点构成一个四边形区域，得到多个所述四边形区域连接组成的填充区域；On any straight line passing through the stroke key point, establish two extension points respectively located on both sides of the stroke key point, and form A quadrilateral area, obtaining a filled area formed by connecting a plurality of said quadrilateral areas;
    采用预设的纹理素材贴图，对所述视频帧的填充区域进行填充，得到填充后的视频帧；Filling the filled area of the video frame by using a preset texture material map to obtain the filled video frame;
    将所述填充后的视频帧按照时间顺序组合，得到特效视频。Combining the filled video frames in chronological order to obtain a special effect video.
  23. 一种计算机程序产品，包括计算机程序，其特征在于，所述计算机程序被处理器执行时实现以下步骤：A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the following steps are implemented:
    获取目标视频，所述目标视频的视频帧包含目标对象；Obtain the target video, the video frame of the target video contains the target object;
    在所述视频帧中，确定位于所述目标对象外侧的描边关键点；In the video frame, determine stroke key points located outside the target object;
    在经过所述描边关键点的任一直线上，建立两个分别位于所述描边关键点两侧的扩展点，并根据每两个相邻描边关键点及其各自对应的扩展点构成一个四边形区域，得到多个所述四边形区域连接组成的填充区域；On any straight line passing through the stroke key point, establish two extension points respectively located on both sides of the stroke key point, and form A quadrilateral area, obtaining a filled area formed by connecting a plurality of said quadrilateral areas;
    采用预设的纹理素材贴图，对所述视频帧的填充区域进行填充，得到填充后的视频帧；Filling the filled area of the video frame by using a preset texture material map to obtain the filled video frame;
    将所述填充后的视频帧按照时间顺序组合，得到特效视频。Combining the filled video frames in chronological order to obtain a special effect video.
PCT/CN2022/075194 2021-08-31 2022-01-30 Image special effect generation method and apparatus WO2023029379A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111023554.XA CN114125320B (en) 2021-08-31 2021-08-31 Method and device for generating special effects of image
CN202111023554.X 2021-08-31

Publications (1)

Publication Number Publication Date
WO2023029379A1 true WO2023029379A1 (en) 2023-03-09

Family

ID=80441168

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/075194 WO2023029379A1 (en) 2021-08-31 2022-01-30 Image special effect generation method and apparatus

Country Status (2)

Country Link
CN (1) CN114125320B (en)
WO (1) WO2023029379A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116777940A (en) * 2023-08-18 2023-09-19 腾讯科技（深圳）有限公司 Data processing method, device, equipment and storage medium

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114596383A (en) * 2022-03-08 2022-06-07 脸萌有限公司 Line special effect processing method and device, electronic equipment, storage medium and product
CN117274432B (en) * 2023-09-20 2024-05-14 书行科技(北京)有限公司 Method, device, equipment and readable storage medium for generating image edge special effect
CN117435110B (en) * 2023-10-11 2024-06-18 书行科技(北京)有限公司 Picture processing method and device, electronic equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6300955B1 (en) * 1997-09-03 2001-10-09 Mgi Software Corporation Method and system for mask generation
CN101764936A (en) * 2008-11-04 2010-06-30 新奥特（北京）视频技术有限公司 Method for confirming shortest distance of pixel space mask code matrix from pixel to boundary
CN101950427A (en) * 2010-09-08 2011-01-19 东莞电子科技大学电子信息工程研究院 Vector line segment contouring method applicable to mobile terminal
CN108399654A (en) * 2018-02-06 2018-08-14 北京市商汤科技开发有限公司 It retouches in the generation of special efficacy program file packet and special efficacy generation method and device when retouching
CN110070554A (en) * 2018-10-19 2019-07-30 北京微播视界科技有限公司 Image processing method, device, hardware device
CN112581620A (en) * 2020-11-30 2021-03-30 北京达佳互联信息技术有限公司 Image processing method, image processing device, electronic equipment and storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110675310B (en) * 2019-07-02 2020-10-02 北京达佳互联信息技术有限公司 Video processing method and device, electronic equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6300955B1 (en) * 1997-09-03 2001-10-09 Mgi Software Corporation Method and system for mask generation
CN101764936A (en) * 2008-11-04 2010-06-30 新奥特（北京）视频技术有限公司 Method for confirming shortest distance of pixel space mask code matrix from pixel to boundary
CN101950427A (en) * 2010-09-08 2011-01-19 东莞电子科技大学电子信息工程研究院 Vector line segment contouring method applicable to mobile terminal
CN108399654A (en) * 2018-02-06 2018-08-14 北京市商汤科技开发有限公司 It retouches in the generation of special efficacy program file packet and special efficacy generation method and device when retouching
CN110070554A (en) * 2018-10-19 2019-07-30 北京微播视界科技有限公司 Image processing method, device, hardware device
CN112581620A (en) * 2020-11-30 2021-03-30 北京达佳互联信息技术有限公司 Image processing method, image processing device, electronic equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116777940A (en) * 2023-08-18 2023-09-19 腾讯科技（深圳）有限公司 Data processing method, device, equipment and storage medium
CN116777940B (en) * 2023-08-18 2023-11-21 腾讯科技（深圳）有限公司 Data processing method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN114125320B (en) 2023-05-09
CN114125320A (en) 2022-03-01

Similar Documents

Publication Publication Date Title
WO2023029379A1 (en) Image special effect generation method and apparatus
US11114130B2 (en) Method and device for processing video
CN109068166B (en) Video synthesis method, device, equipment and storage medium
CN107680033B (en) Picture processing method and device
CN107977934B (en) Image processing method and device
CN107818543B (en) Image processing method and device
WO2020007241A1 (en) Image processing method and apparatus, electronic device, and computer-readable storage medium
US11030733B2 (en) Method, electronic device and storage medium for processing image
CN107657590B (en) Picture processing method and device and storage medium
CN111258519B (en) Screen split implementation method, device, terminal and medium
CN112614228B (en) Method, device, electronic equipment and storage medium for simplifying three-dimensional grid
CN112463084A (en) Split screen display method and device, terminal equipment and computer readable storage medium
CN112508773A (en) Image processing method and device, electronic device and storage medium
US11908096B2 (en) Stereoscopic image acquisition method, electronic device and storage medium
CN112750190B (en) Three-dimensional thermodynamic diagram generation method, device, equipment and storage medium
CN116071377A (en) Map labeling method and device and map labeling device
CN113902869A (en) Three-dimensional head grid generation method and device, electronic equipment and storage medium
CN107680045B (en) Picture stretching method and device
CN114693514A (en) Image processing method, image processing device, electronic equipment and storage medium
CN113645414B (en) Method and device for generating water ripple special effect video, electronic equipment and storage medium
KR101955280B1 (en) Apparatus and Method for Providing Zoom Effect for Face in Image
WO2021243955A1 (en) Dominant hue extraction method and apparatus
CN117422611A (en) Image processing method, device, electronic equipment and storage medium
CN116126133A (en) Virtual object interaction method, device, equipment, storage medium and program product
CN115423896A (en) Curve generation method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22862530

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE