CN108111909A - Method of video image processing and computer storage media, terminal - Google Patents

Method of video image processing and computer storage media, terminal Download PDF

Info

Publication number
CN108111909A
CN108111909A CN201711353131.8A CN201711353131A CN108111909A CN 108111909 A CN108111909 A CN 108111909A CN 201711353131 A CN201711353131 A CN 201711353131A CN 108111909 A CN108111909 A CN 108111909A
Authority
CN
China
Prior art keywords
beat point
music
point
special efficacy
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711353131.8A
Other languages
Chinese (zh)
Inventor
危文
袁少龙
周宇涛
丘智鉴
颜乐驹
魏启征
李敬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Baiguoyuan Information Technology Co Ltd
Original Assignee
Guangzhou Baiguoyuan Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Baiguoyuan Information Technology Co Ltd filed Critical Guangzhou Baiguoyuan Information Technology Co Ltd
Priority to CN201711353131.8A priority Critical patent/CN108111909A/en
Publication of CN108111909A publication Critical patent/CN108111909A/en
Priority to PCT/CN2018/119266 priority patent/WO2019114582A1/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/51Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
    • G10L25/57Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for processing of video signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60Network streaming of media packets
    • H04L65/75Media network packet handling
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439Processing of audio elementary streams
    • H04N21/4394Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The present invention provides a kind of method of video image processing, and this method includes:The music signal for being intended to the music played in video is played is obtained, the beat point of the music is detected according to the music signal;Determine, when playing music in playing video, beat point occur in current play position;Obtain the corresponding special efficacy of the beat point;It is handled according to the special efficacy playing the image in video, obtains the video image for including the special efficacy.Method of video image processing provided by the invention, the beat point tight association of the special efficacy and selected video playing music shown on video image can be achieved, and then to improve the music that plays in video in the sense of hearing and visual appeal, increase the interest of Video Applications and improve the satisfaction of user experience.

Description

Method of video image processing and computer storage media, terminal
Technical field
The present invention relates to technical field of image processing, and specifically, the present invention relates to a kind of method of video image processing.
Background technology
With Internet technology and audio, the continuous development of image processing techniques, in videos such as short-sighted frequency, net casts In application field, it will usually play music in video is played, the music played can be selected by user, seen to give user It sees the hearing enjoying during video, and then improves the experience satisfaction of user.
However, in traditional technology, the video image in video and the music played are played often without relevance, music Only it is simply to be added to play in video, the music played in video is played is in the sense of hearing and visually without enough senses Power is contaminated, and then affects the experience satisfaction of user.
The content of the invention
To overcome more than technical problem, particularly the prior art, which can not be realized, plays video image in video and plays sound The problem of happy tight association, spy propose following technical scheme:
The embodiment of the present invention provides a kind of method of video image processing according on one side, including:
The music signal for being intended to the music played in video is played is obtained, the music is detected according to the music signal Beat point;
Determine, when playing music in playing video, beat point occur in current play position;
Obtain the corresponding special efficacy of the beat point;
It is handled according to the special efficacy playing the image in video, obtains the video image for including the special efficacy.
Preferably, the beat point includes strong beat point and weak beat point;
The beat point that the music is detected according to the music signal, including:
The music signal is filtered, short time discrete Fourier transform is carried out after filtering, obtains frequency spectrum;
According to the frequency spectrum, the energy change value of test point is determined;
According to energy change value, detect that strong beat point or weak beat point occurs in test point.
Preferably, the beat point includes strong beat point and weak beat point;
The beat point that the music is detected according to the music signal, including:
Processing is weighted to the music signal, the music signal after being weighted;
According to the energy intensity value of the music signal after the weighting, detect that strong beat point or weak beat occurs in test point Point.
Preferably, the beat point includes strong beat point;
The beat point that the music is detected according to the music signal, including:
Candidate's beat point is obtained according to the energy intensity value of the music signal;
According to each candidate's beat point, count each two neighboring candidate's beat point time interval between frames;
According to the time interval, detect that candidate's beat point corresponds to test point and strong beat point occurs.
Preferably, the beat point includes strong beat point;
The beat point that the music is detected according to the music signal, including:
According to the energy variation difference of the music signal of test point, candidate's beat point is obtained;
According to candidate's beat point, two sections of music letters are intercepted using each two neighboring candidate's beat point as signal starting point Number;
According to the comparing result of two sections of music signals, detect that candidate's beat point corresponds to test point and strong beat point occurs.
Preferably, it is described to obtain the music signal for being intended to the music played in video is played, it is examined according to the music signal The beat point of the music is measured, including:
The music signal for being intended to the music played in video is played is obtained, judges to detect strong beat point or the strong section of detection Clap point and weak beat point;
If detecting strong beat point, judge using intensity value detection or changing value detection;
It is detected according to intensity value, candidate's beat point is obtained according to the energy intensity value of the music signal, according to each time Select beat point, count each two neighboring candidate's beat point time interval between frames, according to the time interval, detect Candidate's beat point corresponds to test point and strong beat point occurs;
It is detected according to changing value, according to the energy variation difference of the music signal of test point, obtains candidate's beat point, root According to candidate's beat point, two sections of music signals, root are intercepted using each two neighboring candidate's beat point as signal starting point According to the comparison consistent results of two sections of music signals, detect that candidate's beat point corresponds to test point and strong beat point occurs;
If detecting strong beat point and weak beat point, judge using intensity value detection or changing value detection;
It is detected according to intensity value, processing is weighted to the music signal, the music signal after being weighted, according to The energy intensity value of music signal after the weighting detects that strong beat point or weak beat point occurs in test point;
It is detected according to changing value, the music signal is filtered, short time discrete Fourier transform is carried out after filtering, obtained Frequency spectrum according to the frequency spectrum, determines the energy change value of test point, according to energy change value, detects that weak section occurs in test point Clap point or strong candidate's beat point.
Further, it is described to judge to detect strong beat point or detect strong beat point and weak beat point, including:
Special efficacy type needed for video is obtained, the special efficacy type according to needed for video judges to detect strong beat point or detection is strong Beat point and weak beat point;
If described detect strong beat point, judge using intensity value detection or changing value detection, including:
If detecting strong beat point, the type of the music is obtained, judges to examine using intensity value or changing value according to type It surveys;
If described detect strong beat point and weak beat point, judge using intensity value detection or changing value detection, including:
If detecting strong beat point, the type of the music is obtained, judges to detect or change using intensity value according to type Value detection.
Preferably, after the beat point that the music is detected according to the music signal, further include, described in record Music position and the correspondence of beat point;
It is described to determine, when playing music in playing video, beat point occur in current play position, including:
According to the correspondence, determine, when playing music in playing video, beat point occur in current play position.
Preferably, it is described to be handled according to the special efficacy playing the image in video, it obtains and includes the special efficacy Video image, including:
The material in the special efficacy is obtained, material and the image in the broadcasting video are closed in a manner of map overlay Into acquisition includes the video image of the special efficacy.
Preferably, the video is live video;
It is described obtain the corresponding special efficacy of the beat point before, further include:
Judge whether the direct broadcasting room of the live video specifies special efficacy group;If so, obtain the special efficacy group specified;Otherwise, obtain The corresponding scene of direct broadcasting room is taken, temperature is highest in the current live server obtained from direct broadcast server with the scene matching The special efficacy group of preset quantity;
Judge whether the direct broadcasting room opens music type automatic adaptation special effective function;If so, identify the music Type obtains a special efficacy group of the type adaptation from the special efficacy group of the preset quantity;Otherwise, to client send from The request of special efficacy group is selected in the special efficacy group of the preset quantity;
Judge whether to receive the selection information according to the request feedback from client;If so, according to the selection information A special efficacy group is obtained from the special efficacy group of the preset quantity, otherwise, is obtained at random from the special efficacy group of the preset quantity One special efficacy group;
It is described to obtain the corresponding special efficacy of the beat point, including:
The corresponding special efficacy of beat point is obtained from the special efficacy group obtained;
According to the special efficacy to play video in image handle, obtain comprising the special efficacy video image it Afterwards, further include:
Treated live video is sent to client.
The embodiment of the present invention additionally provides a kind of computer readable storage medium, the calculating according on the other hand Computer program is stored on machine readable storage medium storing program for executing, which realizes above-mentioned Computer Vision side when being executed by processor Method.
The embodiment of the present invention additionally provides a kind of terminal, the terminal includes one or more according to another aspect Processor;Memory;One or more application program, wherein one or more of application programs are stored in the memory In and be configured as being performed by one or more of processors, one or more of programs are configured to:It performs above-mentioned Method of video image processing.
Compared with prior art, the present invention it has the advantages that:
Method of video image processing provided by the invention, by carrying out beat point to the music for being intended to play in video is played Detection, and the corresponding spy of the beat point is shown in video image according to detected beat point during music Effect, so that the beat point tight association of the special efficacy shown on video image and selected video playing music, and then to improve The music played in video increases the interest of Video Applications and improves user experience in the sense of hearing and visual appeal Satisfaction.
In addition, method of video image processing provided by the invention, with reference to multiple beat point detecting methods, is realized quick, accurate Really detect the beat point of selected video playing music;And the present invention can according to needed for video the class of special efficacy type and music Type obtains standard come the type of detection beat point needed for selecting and the method for detecting beat point to realize using suitable method True beat point, and operand can be reduced, shorten detection duration, be further ensured that the special efficacy shown on video image and select Video playing music beat point close association, further improve the satisfaction of user experience.
In addition, when method of video image processing provided by the invention is applied to live video field, by successively according to master Wish, music type, spectators' wish selection special efficacy are broadcast, disclosure satisfy that the requirement of live streaming, sets off live streaming atmosphere, and more preferable promotion master by contrast The interaction with spectators is broadcast, further improves the satisfaction of user experience.
The additional aspect of the present invention and advantage will be set forth in part in the description, these will become from the following description It obtains substantially or is recognized by the practice of the present invention.
Description of the drawings
Above-mentioned and/or additional aspect and advantage of the invention will become from the following description of the accompanying drawings of embodiments Substantially and it is readily appreciated that, wherein:
Fig. 1 is a kind of method flow diagram of method of video image processing of the embodiment of the present invention;
Fig. 2 is a kind of structure diagram of terminal of the embodiment of the present invention.
Specific embodiment
The embodiment of the present invention is described below in detail, the example of the embodiment is shown in the drawings, wherein from beginning to end Same or similar label represents same or similar element or has the function of same or like element.Below with reference to attached The embodiment of figure description is exemplary, and is only used for explaining the present invention, and is not construed as limiting the claims.
Embodiment one
An embodiment of the present invention provides a kind of method of video image processing, as shown in Figure 1, this method includes:
Step S101, the music signal for being intended to the music played in video is played is obtained, is detected according to the music signal Go out the beat point of the music.
For the present embodiment, the music for being intended to play in video is played can be from the music that prestores in Video Applications Selected music or the music loaded in Video Applications is selected from the music that prestores of user terminal, can also be The live music played using other equipment obtained in Video Applications by microphone, the source of the music is in the present embodiment In do not limit.
For the present embodiment, the action executing for obtaining music signal is selected in user is intended to what is played in video is played After music, and the action for selecting the music can perform before video is played, and can also be performed in video is played.
For example, when being applied to net cast field, the music can be selected before net cast or in net cast, terminal The music signal of selected music can be obtained immediately, and the beat point of the music is detected according to the music signal.
In another example during recording applied to short-sighted frequency, the music is selected before video is played that is, before video record, Terminal can obtain the music signal of selected music at once after selected music, and detect the sound according to the music signal Happy beat point.
It is described to detect that a variety of methods can be used in the beat point of the music according to the music signal for the present embodiment It realizes.
For example, for this method, the beat point includes strong beat point and weak beat point;This method includes:To the sound Music signal is filtered, and short time discrete Fourier transform is carried out after filtering, obtains frequency spectrum;According to the frequency spectrum, the energy of test point is determined Measure changing value;According to energy change value, detect that strong beat point or weak beat point occurs in test point.
In another example for this method, the beat point includes strong beat point and weak beat point;This method includes:To described Music signal is weighted processing, the music signal after being weighted;According to the energy intensity of the music signal after the weighting Value detects that strong beat point or weak beat point occurs in test point.
In another example for this method, the beat point includes strong beat point;This method includes:According to the music signal Energy intensity value obtain candidate's beat point;According to each candidate's beat point, frame where each two neighboring candidate's beat point of statistics it Between time interval;According to the time interval, detect that candidate's beat point corresponds to test point and strong beat point occurs.
For another example for this method, the beat point includes strong beat point;This method includes:According to the music of test point The energy variation difference of signal obtains candidate's beat point;According to candidate's beat point, made with each two neighboring candidate's beat point Two sections of music signals are intercepted for signal starting point;According to the comparing result of two sections of music signals, detect that candidate's beat point corresponds to There is strong beat point in test point.
Step S102, determine, when playing music in playing video, beat point occur in current play position.
For the present embodiment, each beat point detected in step S101 corresponds to the different of the music and plays position It puts, when playing music in playing video, if music current play position is corresponding with the beat point that step S101 is detected, really The fixed beat point is strong beat point or weak beat point.
Step S103, the corresponding special efficacy of the beat point is obtained.
For the present embodiment, the special efficacy can be the materials such as fireworks, love, snowflake, the specific manifestation shape of the material Formula does not limit in the present embodiment.Same material can set different resemblance parameters, and the resemblance parameter includes Dimensional parameters, color parameter etc., do not limit in the present embodiment.
For the present embodiment, the point of beat described in a piece of music includes strong beat point and weak beat point, the strong beat point The material in special efficacy corresponding with weak beat point is identical, but the resemblance parameter of material differs.
Step S104, handled according to the special efficacy playing the image in video, obtain regarding comprising the special efficacy Frequency image..
For the present embodiment, by obtaining the material in the special efficacy, and by the material and institute in a manner of map overlay It states the image played in video to be synthesized, obtains including the video image of the material in the special efficacy.In other embodiment In, it can also use and special efficacy and image are subjected to Data Integration or image etc. is changed according to the resemblance parameter of material in special efficacy Other modes realize the video image for including the special efficacy described in acquisition.
For example, in net cast field, can the data of special efficacy and image data be subjected to Data Integration and obtain video Image data bag, and the data packet is sent to client, so that client shows the video image for including the special efficacy.
In another example it can be contracted by obtaining the resemblance parameter of material in special efficacy according to the parameter to image Processing is put, to obtain the video image that special effect can be achieved and highlight.
Method of video image processing provided by the invention, by carrying out beat point to the music for being intended to play in video is played Detection, and the corresponding spy of the beat point is shown in video image according to detected beat point during music Effect, so that the beat point tight association of the special efficacy shown on video image and selected video playing music, and then to improve The music played in video increases the interest of Video Applications and improves user experience in the sense of hearing and visual appeal Satisfaction.
In addition, method of video image processing provided by the invention, with reference to multiple beat point detecting methods, is realized quick, accurate Really detect the beat point of selected video playing music, be further ensured that the special efficacy that is shown on video image with it is selected The beat point close association of video playing music further improves the satisfaction of user experience.
Embodiment two
The alternatively possible realization method of the embodiment of the present invention on the basis of shown in embodiment one, further includes implementation Step shown in example two, wherein,
Step S101 includes:
The music signal for being intended to the music played in video is played is obtained, judges to detect strong beat point or the strong section of detection Clap point and weak beat point;
If detecting strong beat point, judge using intensity value detection or changing value detection;
It is detected according to intensity value, candidate's beat point is obtained according to the energy intensity value of the music signal, according to each time Select beat point, count each two neighboring candidate's beat point time interval between frames, according to the time interval, detect Candidate's beat point corresponds to test point and strong beat point occurs;
It is detected according to changing value, according to the energy variation difference of the music signal of test point, obtains candidate's beat point, root According to candidate's beat point, two sections of music signals, root are intercepted using each two neighboring candidate's beat point as signal starting point According to the comparison consistent results of two sections of music signals, detect that candidate's beat point corresponds to test point and strong beat point occurs;
If detecting strong beat point and weak beat point, judge using intensity value detection or changing value detection;
It is detected according to intensity value, processing is weighted to the music signal, the music signal after being weighted, according to The energy intensity value of music signal after the weighting detects that strong beat point or weak beat point occurs in test point;
It is detected according to changing value, the music signal is filtered, short time discrete Fourier transform is carried out after filtering, obtained Frequency spectrum according to the frequency spectrum, determines the energy change value of test point, according to energy change value, detects that weak section occurs in test point Clap point or strong candidate's beat point.
For the present embodiment, for the type and examination criteria of different required detection beat points, it is corresponding with different Beat detection methods.
Wherein, it is described to judge to detect strong beat point or detect strong beat point and weak beat point, including:
Special efficacy type needed for video is obtained, the special efficacy type according to needed for video judges to detect strong beat point or detection is strong Beat point and weak beat point;
If described detect strong beat point, judge using intensity value detection or changing value detection, including:
If detecting strong beat point, the type of the music is obtained, judges to examine using intensity value or changing value according to type It surveys;
If described detect strong beat point and weak beat point, judge to detect using intensity value or changing value, including:
If detecting strong beat point, the type of the music is obtained, judges to examine using intensity value or changing value according to type It surveys.
It is the type that required detection beat point is judged by special efficacy type needed for the video of acquisition for the present embodiment 's.Special efficacy type needed for the video is user's selection or Video Applications acquiescence special efficacy type.For example, user wishes broadcasting The special efficacy for having in video and emerging in an endless stream is put, therefore the special efficacy type according to needed for its video judges that strong beat point should be detected, and also Detect weak beat point.
It is that examination criteria is judged by the type of the music of acquisition for the present embodiment.For example, acquired music Type for rock and roll, the music signal of the music type often has very high intensity value, but its changing value unobvious, thus according to The selection of its type detects the beat of music point by detection intensity value.
For the present embodiment, can according to needed for video the type of special efficacy type and music come detection beat point needed for selecting Type and the method for detecting beat point are obtained accurate beat point using suitable method to realize, and can reduce computing Amount shortens detection duration, is further ensured that the beat point of the special efficacy and selected video playing music shown on video image Close association further improves the satisfaction of user experience.
Embodiment three
The alternatively possible realization method of the embodiment of the present invention on the basis of shown in embodiment one, further includes implementation Step shown in example three, wherein,
After step slol, further include:Record the correspondence of the music position and beat point.
For the present embodiment, after the beat point of the music is detected according to the music signal, by the beat Correspondence is established in the corresponding music position of point, and records the correspondence.The foundation side of the correspondence Method does not limit in the present embodiment, can be the mode for adding label information.
For example, the signal data in the corresponding music position of the beat point adds label information, the label information Carry the information for representing that there is beat point in the music position and the beat point is strong beat point or weak beat point.
Step S102 includes:According to the correspondence, determine when playing music in playing video, in currently playing position It puts and beat point occurs.
For the present embodiment, when playing music in playing video, in current play position, there are corresponding with beat point During record, obtain the music position and the correspondence of beat point records, extract the correspondence, and then according to institute It states correspondence and determines that current play position beat point occurs and determines that the beat point is strong beat point or weak beat point.
For the present embodiment, by recording the music position with the correspondence of beat point, it can be achieved that simple, fast It is strong beat point or weak beat point to determine whether current play position beat point and beat point occurs fastly.
Example IV
For the present embodiment, the video is live video, i.e. method in the present embodiment is mainly used in net cast Field.
The alternatively possible realization method of the embodiment of the present invention on the basis of shown in embodiment one, further includes implementation Step shown in example four, wherein,
Before step S103, further include:
Judge whether the direct broadcasting room of the live video specifies special efficacy group;If so, obtain the special efficacy group specified;Otherwise, obtain The corresponding scene of direct broadcasting room is taken, temperature is highest in the current live server obtained from direct broadcast server with the scene matching The special efficacy group of preset quantity;
Judge whether the direct broadcasting room opens music type automatic adaptation special effective function;If so, identify the music Type obtains a special efficacy group of the type adaptation from the special efficacy group of the preset quantity;Otherwise, to client send from The request of special efficacy group is selected in the special efficacy group of the preset quantity;
Judge whether to receive the selection information according to the request feedback from client;If so, according to the selection information A special efficacy group is obtained from the special efficacy group of the preset quantity, otherwise, is obtained at random from the special efficacy group of the preset quantity One special efficacy group.
Step S103 includes:The corresponding special efficacy of beat point is obtained from the special efficacy group obtained.
For the present embodiment, the material of the special efficacy of the special efficacy group is identical, but the resemblance parameter of material differs.
For the present embodiment, the special efficacy group specified can be specified by the main broadcaster or user of the direct broadcasting room, the finger Fixed special efficacy group is corresponding with main broadcaster's live streaming feature of the main broadcaster, such as the live streaming of the main broadcaster is the theme with a certain animation, and The main broadcaster, which preassigns, uses the special efficacy group that is the theme with the animation, thus the special efficacy shown in the live video of the direct broadcasting room from It is obtained in the special efficacy group specified.
For the present embodiment, the corresponding scene of the direct broadcasting room can be by extracting the corresponding place of direct broadcasting room, weather, scape The scene characteristics such as sight determine that the scene can be coffee-house, sunny outdoor, seashore etc., not limit in the present embodiment.With The special efficacy group and the scene characteristic of the scene of the scene matching have correlation.
For the present embodiment, the preset quantity is the numerical value more than or equal to 2, can be the numerical value such as 3,5,6, in this reality It applies in example and does not limit.
For the present embodiment, the type of the music includes rock and roll, prevalence etc., and the type of the music is according to acquired The beat point of the music determines.
After step S104, further include:
Treated live video is sent to client.
When method of video image processing provided by the invention is applied to live video field, by being anticipated successively according to main broadcaster Hope, music type, spectators' wish selection special efficacy, disclosure satisfy that the requirement of live streaming, set off live streaming atmosphere by contrast, and it is more preferable promote main broadcaster with The interaction of spectators further improves the satisfaction of user experience.
In addition, an embodiment of the present invention provides a kind of computer readable storage medium, deposited on computer readable storage medium Computer program is contained, the Computer Vision side described in above example one to four is realized when which is executed by processor Method.Wherein, the computer readable storage medium includes but not limited to any kind of disk (including floppy disk, hard disk, CD, CD- ROM and magneto-optic disk), ROM (Read-Only Memory, read-only memory), RAM (Random AcceSS Memory, immediately Memory), EPROM (EraSable Programmable Read-Only Memory, Erarable Programmable Read only Memory), (Electrically EraSable Programmable Read-Only Memory, electric erazable programmable is read-only to be deposited EEPROM Reservoir), flash memory, magnetic card or light card.It is, storage device is included by equipment (for example, computer, mobile phone) with energy Any medium for the form storage or transmission information enough read, can be read-only memory, disk or CD etc..
A kind of computer readable storage medium provided in an embodiment of the present invention is, it can be achieved that by being intended to broadcast in video is played The music put carries out beat point detection, and is shown during music according to detected beat point in video image The corresponding special efficacy of beat point, so that the beat point of the special efficacy shown on video image and selected video playing music is close Association, and then to improve the music that plays in video in the sense of hearing and visual appeal, increase the entertaining of Video Applications Property simultaneously improves the satisfaction of user experience.In addition, computer readable storage medium provided by the invention, can also be achieved with reference to multiple Beat point detecting method realizes the beat point for quickly and accurately detecting selected video playing music;It and can be according to video institute The method for needing the type of special efficacy type and music to select the type of required detection beat point and detect beat point, is adopted with realizing Accurate beat point is obtained with suitable method, and operand can be reduced, shortens detection duration, is further ensured that video image The beat point close association of the special efficacy of upper display and selected video playing music, further improves the satisfaction of user experience Degree.In addition, when computer readable storage medium provided by the invention is applied to live video field, can also be achieved through root successively According to main broadcaster's wish, music type, spectators' wish selection special efficacy, the requirement of live streaming disclosure satisfy that, set off live streaming atmosphere, and more preferable rush by contrast Into the interaction of main broadcaster and spectators, the satisfaction of user experience is further improved.
Computer readable storage medium provided in an embodiment of the present invention can realize the embodiment of the method for above-mentioned offer, specifically Function realization refers to the explanation in embodiment of the method, and details are not described herein.
In addition, the embodiment of the present invention additionally provides a kind of terminal, as shown in Fig. 2, the terminal can include one or More than one processor 201, further include memory 202, WiFi (wireless fidelity, Wireless Fidelity) circuit 203, RF (Radio Frequency, radio frequency) circuit 204, voicefrequency circuit 205, sensor 206, output equipment 207, input equipment 204th, power supply 209, processor 201 are the control centres of terminal, utilize various interfaces and connection more than each several part.This field Technical staff is appreciated that the restriction of terminal structure shown in Figure 2 not structure paired terminal, can include than illustrate it is more or Less component either combines some components or different components arrangement.
WiFi circuitry 203 can provide WLAN or internet access to the user;It may include antenna, WiFi module Deng.RF circuits 204 can receive and send messages or signal sends and receivees in communication process;It may include antenna, at least one puts Big device, tuner, one or more oscillators, coupler, duplexer etc..The voice data that voicefrequency circuit 205 can will receive Electric signal is converted into, is transferred to loud speaker, also the voice signal that microphone is collected voice data can be converted to, issue processor 201 processing;It can set loud speaker, microphone, earphone interface etc..Sensor 206 can be used for sensing outer signals, and issue place Reason device 201 is handled;It may include motion sensor, optical sensor etc..Output equipment 207 can be used for showing various signals;It can For using LCD (Liquid Crystal Display, liquid crystal display), OLED (Organic Light-Emitting Diode, Organic Light Emitting Diode) etc. forms configure display panel.Input equipment 204 can be used for the letters such as input number and character Breath;It can be physical button, touch panel etc..Power supply 209 can be that terminal each several part is powered, and pass through power-supply management system and processing 209 logical connection of device;It may include one or more direct current or AC power, charging system, power supply status indicator Wait components.Memory 202 can be used for storage software program and module;It can be computer readable storage medium, be specifically Hard disk, flash memory etc..Processor is the control centre of terminal, by running or performing the software program being stored in memory 202 And/or module and calling are stored in the data of memory 202, perform terminal various functions, processing terminal data.
As one embodiment, terminal includes:One or more processors 201, memory 202, one or more application Program, wherein one or more of application programs are stored in memory 202 and are configured as by one or more of Processor 201 performs, and one or more of programs are configured to carry out at the video image described in above example one to four Reason method.
A kind of terminal provided in an embodiment of the present invention is, it can be achieved that by saving the music for being intended to play in video is played Point detection is clapped, and shows that the beat point is corresponding in video image according to detected beat point during music Special efficacy, so that the beat point tight association of the special efficacy shown on video image and selected video playing music, and then to carry The music that plays in video of height increases the interest of Video Applications and improves user's body in the sense of hearing and visual appeal The satisfaction tested.In addition, terminal provided by the invention is realized quickly and accurately, it can be achieved that with reference to multiple beat point detecting methods Detect the beat point of selected video playing music;And can according to needed for video the type of special efficacy type and music to select The method that the type of beat point need to be detected and detect beat point obtains accurate beat using suitable method to realize Point, and operand can be reduced, shorten detection duration, be further ensured that the special efficacy shown on video image is broadcast with selected video The beat point close association put the music on further improves the satisfaction of user experience.In addition, terminal applies provided by the invention When live video field, it can be achieved that by selecting special efficacy, Neng Gouman according to main broadcaster's wish, music type, spectators' wish successively The requirement being broadcast live enough sets off live streaming atmosphere, and the more preferable interaction for promoting main broadcaster and spectators by contrast, further improves the satisfaction of user experience Degree.
Terminal provided in an embodiment of the present invention can realize the embodiment of the method for above-mentioned offer, and concrete function realization refers to Explanation in embodiment of the method, details are not described herein.
The above is only some embodiments of the present invention, it is noted that for the ordinary skill people of the art For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications also should It is considered as protection scope of the present invention.

Claims (12)

1. a kind of method of video image processing, which is characterized in that include the following steps:
The music signal for being intended to the music played in video is played is obtained, the section of the music is detected according to the music signal Clap point;
Determine, when playing music in playing video, beat point occur in current play position;
Obtain the corresponding special efficacy of the beat point;
It is handled according to the special efficacy playing the image in video, obtains the video image for including the special efficacy.
2. method of video image processing according to claim 1, which is characterized in that the beat point include strong beat point and Weak beat point;
The beat point that the music is detected according to the music signal, including:
The music signal is filtered, short time discrete Fourier transform is carried out after filtering, obtains frequency spectrum;
According to the frequency spectrum, the energy change value of test point is determined;
According to energy change value, detect that strong beat point or weak beat point occurs in test point.
3. method of video image processing according to claim 1, which is characterized in that the beat point include strong beat point and Weak beat point;
The beat point that the music is detected according to the music signal, including:
Processing is weighted to the music signal, the music signal after being weighted;
According to the energy intensity value of the music signal after the weighting, detect that strong beat point or weak beat point occurs in test point.
4. method of video image processing according to claim 1, which is characterized in that the beat point includes strong beat point;
The beat point that the music is detected according to the music signal, including:
Candidate's beat point is obtained according to the energy intensity value of the music signal;
According to each candidate's beat point, count each two neighboring candidate's beat point time interval between frames;
According to the time interval, detect that candidate's beat point corresponds to test point and strong beat point occurs.
5. method of video image processing according to claim 1, which is characterized in that the beat point includes strong beat point;
The beat point that the music is detected according to the music signal, including:
According to the energy variation difference of the music signal of test point, candidate's beat point is obtained;
According to candidate's beat point, two sections of music signals are intercepted using each two neighboring candidate's beat point as signal starting point;
According to the comparing result of two sections of music signals, detect that candidate's beat point corresponds to test point and strong beat point occurs.
6. method of video image processing according to claim 1, which is characterized in that the acquisition is intended to broadcast in video is played The music signal for the music put detects the beat point of the music according to the music signal, including:
The music signal for being intended to the music played in video is played is obtained, judges to detect strong beat point or the strong beat point of detection With weak beat point;
If detecting strong beat point, judge using intensity value detection or changing value detection;
It is detected according to intensity value, candidate's beat point is obtained according to the energy intensity value of the music signal, is saved according to each candidate Clap point, count each two neighboring candidate's beat point time interval between frames, according to the time interval, detect candidate Beat point corresponds to test point and strong beat point occurs;
It is detected according to changing value, according to the energy variation difference of the music signal of test point, candidate's beat point is obtained, according to institute Candidate's beat point is stated, two sections of music signals are intercepted using each two neighboring candidate's beat point as signal starting point, according to two The comparison consistent results of section music signal detect that candidate's beat point corresponds to test point and strong beat point occurs;
If detecting strong beat point and weak beat point, judge using intensity value detection or changing value detection;
It is detected according to intensity value, processing is weighted to the music signal, the music signal after being weighted, according to described The energy intensity value of music signal after weighting detects that strong beat point or weak beat point occurs in test point;
It is detected according to changing value, the music signal is filtered, short time discrete Fourier transform is carried out after filtering, obtain frequency Spectrum according to the frequency spectrum, determines the energy change value of test point, according to energy change value, detects that weak beat occurs in test point Point or strong candidate's beat point.
7. method of video image processing according to claim 6, which is characterized in that it is described to judge to detect strong beat point, also It is to detect strong beat point and weak beat point, including:
Special efficacy type needed for video is obtained, the special efficacy type according to needed for video judges to detect strong beat point or the strong beat of detection Point and weak beat point;
If described detect strong beat point, judge using intensity value detection or changing value detection, including:
If detecting strong beat point, the type of the music is obtained, judges to detect using intensity value or changing value according to type;
If described detect strong beat point and weak beat point, judge using intensity value detection or changing value detection, including:
If detecting strong beat point, the type of the music is obtained, judges to examine using intensity value detection or changing value according to type It surveys.
8. method of video image processing according to claim 1, which is characterized in that described to be detected according to the music signal Go out after the beat point of the music, further include, record the correspondence of the music position and beat point;
It is described to determine, when playing music in playing video, beat point occur in current play position, including:
According to the correspondence, determine, when playing music in playing video, beat point occur in current play position.
9. method of video image processing according to claim 1, which is characterized in that described to be regarded according to the special efficacy to broadcasting Image in frequency is handled, and obtains the video image for including the special efficacy, including:
The material in the special efficacy is obtained, material and the image in the broadcasting video are synthesized in a manner of map overlay, Obtain the video image for including the special efficacy.
10. method of video image processing according to claim 1, which is characterized in that the video is live video;
It is described obtain the corresponding special efficacy of the beat point before, further include:
Judge whether the direct broadcasting room of the live video specifies special efficacy group;If so, obtain the special efficacy group specified;Otherwise, obtain straight It is highest preset to obtain temperature in current live server with the scene matching from direct broadcast server for corresponding scene between broadcasting The special efficacy group of quantity;
Judge whether the direct broadcasting room opens music type automatic adaptation special effective function;If so, identify the type of the music, A special efficacy group of the type adaptation is obtained from the special efficacy group of the preset quantity;Otherwise, sent to client from described The request of special efficacy group is selected in the special efficacy group of preset quantity;
Judge whether to receive the selection information according to the request feedback from client;If so, information is selected from institute according to described One special efficacy group of acquisition in the special efficacy group of preset quantity is stated, otherwise, one is obtained at random from the special efficacy group of the preset quantity Special efficacy group;
It is described to obtain the corresponding special efficacy of the beat point, including:
The corresponding special efficacy of beat point is obtained from the special efficacy group obtained;
It is handled according to the special efficacy playing the image in video, after obtaining the video image comprising the special efficacy, also Including:
Treated live video is sent to client.
11. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium Program realizes claims 1 to 10 any one of them method of video image processing when the program is executed by processor.
12. a kind of terminal, which is characterized in that it includes:
One or more processors;
Memory;
One or more application program, wherein one or more of application programs are stored in the memory and are configured To be performed by one or more of processors, one or more of programs are configured to:It performs according to claims 1 to 10 Any one of them method of video image processing.
CN201711353131.8A 2017-12-15 2017-12-15 Method of video image processing and computer storage media, terminal Pending CN108111909A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201711353131.8A CN108111909A (en) 2017-12-15 2017-12-15 Method of video image processing and computer storage media, terminal
PCT/CN2018/119266 WO2019114582A1 (en) 2017-12-15 2018-12-05 Video image processing method and computer storage medium and terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711353131.8A CN108111909A (en) 2017-12-15 2017-12-15 Method of video image processing and computer storage media, terminal

Publications (1)

Publication Number Publication Date
CN108111909A true CN108111909A (en) 2018-06-01

Family

ID=62217388

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711353131.8A Pending CN108111909A (en) 2017-12-15 2017-12-15 Method of video image processing and computer storage media, terminal

Country Status (1)

Country Link
CN (1) CN108111909A (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108810597A (en) * 2018-06-25 2018-11-13 百度在线网络技术(北京)有限公司 Special video effect processing method and processing device
CN109120875A (en) * 2018-09-27 2019-01-01 乐蜜有限公司 Video Rendering method and device
CN109495786A (en) * 2018-12-20 2019-03-19 北京微播视界科技有限公司 Method for pre-configuration, device and the electronic equipment of video processing parameter information
WO2019114582A1 (en) * 2017-12-15 2019-06-20 广州市百果园信息技术有限公司 Video image processing method and computer storage medium and terminal
CN110069641A (en) * 2019-03-27 2019-07-30 北京字节跳动网络技术有限公司 Image processing method, device and electronic equipment
CN111277761A (en) * 2020-03-05 2020-06-12 北京达佳互联信息技术有限公司 Video shooting method, device and system, electronic equipment and storage medium
CN111489769A (en) * 2019-01-25 2020-08-04 北京字节跳动网络技术有限公司 Image processing method, device and hardware device
CN111857923A (en) * 2020-07-17 2020-10-30 北京字节跳动网络技术有限公司 Special effect display method and device, electronic equipment and computer readable medium
CN112235631A (en) * 2019-07-15 2021-01-15 北京字节跳动网络技术有限公司 Video processing method and device, electronic equipment and storage medium
CN112565882A (en) * 2019-09-26 2021-03-26 北京字节跳动网络技术有限公司 Video generation method and device, electronic equipment and computer readable medium
CN112804578A (en) * 2021-01-28 2021-05-14 广州虎牙科技有限公司 Atmosphere special effect generation method and device, electronic equipment and storage medium
CN113573442A (en) * 2021-07-16 2021-10-29 广东和音元视电子科技有限公司 Method, apparatus, and computer-readable storage medium for audio control of scenes
WO2022194097A1 (en) * 2021-03-15 2022-09-22 北京字跳网络技术有限公司 Object control method and device

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101123086A (en) * 2006-08-09 2008-02-13 株式会社河合乐器制作所 Tempo detection apparatus and tempo-detection computer program
CN101165773A (en) * 2006-10-20 2008-04-23 索尼株式会社 Signal processing apparatus and method, program, and recording medium
CN104103300A (en) * 2014-07-04 2014-10-15 厦门美图之家科技有限公司 Method for automatically processing video according to music beats
CN104239389A (en) * 2013-06-10 2014-12-24 宏达国际电子股份有限公司 Methods and systems for media file management
US20150007708A1 (en) * 2009-05-27 2015-01-08 Microsoft Corporation Detecting beat information using a diverse set of correlations
CN104822095A (en) * 2015-02-08 2015-08-05 苏州乐聚一堂电子科技有限公司 Composite beat special effect system and composite beat special effect processing method
CN105872838A (en) * 2016-04-28 2016-08-17 徐文波 Sending method and device of special media effects of real-time videos
CN107103917A (en) * 2017-03-17 2017-08-29 福建星网视易信息***有限公司 Music rhythm detection method and its system
CN107124624A (en) * 2017-04-21 2017-09-01 腾讯科技(深圳)有限公司 The method and apparatus of video data generation

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101123086A (en) * 2006-08-09 2008-02-13 株式会社河合乐器制作所 Tempo detection apparatus and tempo-detection computer program
CN101165773A (en) * 2006-10-20 2008-04-23 索尼株式会社 Signal processing apparatus and method, program, and recording medium
US20150007708A1 (en) * 2009-05-27 2015-01-08 Microsoft Corporation Detecting beat information using a diverse set of correlations
CN104239389A (en) * 2013-06-10 2014-12-24 宏达国际电子股份有限公司 Methods and systems for media file management
CN104103300A (en) * 2014-07-04 2014-10-15 厦门美图之家科技有限公司 Method for automatically processing video according to music beats
CN104822095A (en) * 2015-02-08 2015-08-05 苏州乐聚一堂电子科技有限公司 Composite beat special effect system and composite beat special effect processing method
CN105872838A (en) * 2016-04-28 2016-08-17 徐文波 Sending method and device of special media effects of real-time videos
CN107103917A (en) * 2017-03-17 2017-08-29 福建星网视易信息***有限公司 Music rhythm detection method and its system
CN107124624A (en) * 2017-04-21 2017-09-01 腾讯科技(深圳)有限公司 The method and apparatus of video data generation

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019114582A1 (en) * 2017-12-15 2019-06-20 广州市百果园信息技术有限公司 Video image processing method and computer storage medium and terminal
CN108810597A (en) * 2018-06-25 2018-11-13 百度在线网络技术(北京)有限公司 Special video effect processing method and processing device
CN108810597B (en) * 2018-06-25 2021-08-17 百度在线网络技术(北京)有限公司 Video special effect processing method and device
CN109120875A (en) * 2018-09-27 2019-01-01 乐蜜有限公司 Video Rendering method and device
WO2020124679A1 (en) * 2018-12-20 2020-06-25 北京微播视界科技有限公司 Method and apparatus for pre-configuring video processing parameter information, and electronic device
CN109495786A (en) * 2018-12-20 2019-03-19 北京微播视界科技有限公司 Method for pre-configuration, device and the electronic equipment of video processing parameter information
CN111489769A (en) * 2019-01-25 2020-08-04 北京字节跳动网络技术有限公司 Image processing method, device and hardware device
CN111489769B (en) * 2019-01-25 2022-07-12 北京字节跳动网络技术有限公司 Image processing method, device and hardware device
CN110069641A (en) * 2019-03-27 2019-07-30 北京字节跳动网络技术有限公司 Image processing method, device and electronic equipment
CN110069641B (en) * 2019-03-27 2021-04-20 北京字节跳动网络技术有限公司 Image processing method and device and electronic equipment
CN112235631B (en) * 2019-07-15 2022-05-03 北京字节跳动网络技术有限公司 Video processing method and device, electronic equipment and storage medium
US11978485B2 (en) 2019-07-15 2024-05-07 Beijing Bytedance Network Technology Co., Ltd. Video processing method and apparatus, and electronic device and storage medium
CN112235631A (en) * 2019-07-15 2021-01-15 北京字节跳动网络技术有限公司 Video processing method and device, electronic equipment and storage medium
CN112565882A (en) * 2019-09-26 2021-03-26 北京字节跳动网络技术有限公司 Video generation method and device, electronic equipment and computer readable medium
US11710510B2 (en) 2019-09-26 2023-07-25 Beijing Bytedance Network Technology Co., Ltd. Video generation method and apparatus, electronic device, and computer readable medium
CN111277761A (en) * 2020-03-05 2020-06-12 北京达佳互联信息技术有限公司 Video shooting method, device and system, electronic equipment and storage medium
CN111277761B (en) * 2020-03-05 2022-03-01 北京达佳互联信息技术有限公司 Video shooting method, device and system, electronic equipment and storage medium
CN111857923A (en) * 2020-07-17 2020-10-30 北京字节跳动网络技术有限公司 Special effect display method and device, electronic equipment and computer readable medium
CN112804578A (en) * 2021-01-28 2021-05-14 广州虎牙科技有限公司 Atmosphere special effect generation method and device, electronic equipment and storage medium
WO2022194097A1 (en) * 2021-03-15 2022-09-22 北京字跳网络技术有限公司 Object control method and device
CN113573442A (en) * 2021-07-16 2021-10-29 广东和音元视电子科技有限公司 Method, apparatus, and computer-readable storage medium for audio control of scenes

Similar Documents

Publication Publication Date Title
CN108111909A (en) Method of video image processing and computer storage media, terminal
CN108259983A (en) A kind of method of video image processing, computer readable storage medium and terminal
CN108322802A (en) Stick picture disposing method, computer readable storage medium and the terminal of video image
CN108259925A (en) Music gifts processing method, storage medium and terminal in net cast
CN108259984A (en) Method of video image processing, computer readable storage medium and terminal
CN106162369B (en) It is a kind of to realize the method, apparatus and system interacted in virtual scene
CN104133851B (en) The detection method and detection device of audio similarity, electronic equipment
CN107396137A (en) The method, apparatus and system of online interaction
CN108810576A (en) The even method, apparatus and storage medium of wheat live streaming
CN110267067A (en) Method, apparatus, equipment and the storage medium that direct broadcasting room is recommended
CN109151593A (en) Main broadcaster's recommended method, device storage medium
US9596538B2 (en) Wearable audio mixing
CN108509660A (en) A kind of broadcasting object recommendation method and terminal device
CN109448761B (en) Method and device for playing songs
CN106155623A (en) A kind of audio collocation method, system and relevant device
CN109040297A (en) User's portrait generation method and device
CN106303733B (en) Method and device for playing live special effect information
CN108401124A (en) The method and apparatus of video record
CN108683927A (en) Main broadcaster recommends method, apparatus and storage medium
CN106713840B (en) Virtual information display methods and device
CN108419126A (en) Abnormal main broadcaster's recognition methods, storage medium and the terminal of platform is broadcast live
CN107343085A (en) Method for playing music and Related product
CN110267054B (en) Method and device for recommending live broadcast room
CN110290392A (en) Live information display methods, device, equipment and storage medium
CN109271038A (en) Candidate words recommending method, terminal and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20180601

RJ01 Rejection of invention patent application after publication