CN111105669A - Teaching sounding simulation display method and device - Google Patents
Teaching sounding simulation display method and device Download PDFInfo
- Publication number
- CN111105669A CN111105669A CN201911338619.2A CN201911338619A CN111105669A CN 111105669 A CN111105669 A CN 111105669A CN 201911338619 A CN201911338619 A CN 201911338619A CN 111105669 A CN111105669 A CN 111105669A
- Authority
- CN
- China
- Prior art keywords
- singing
- singer
- information
- sound source
- organ
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B15/00—Teaching music
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- Processing Or Creating Images (AREA)
Abstract
The embodiment of the application discloses a teaching sounding simulation display method and device, which are used for generating a three-dimensional mannequin, wherein the mannequin at least comprises a mouth, a nose, a tongue, sound source organs, relevant organs of a tuning area and main muscle groups of the relevant organs; the method comprises the following steps: at least acquiring mouth shape information, opening and closing state information of sound source organs and muscle stress state information of organs related to a tuning area when a singer sings; recording voice information of a singer, and adjusting the mouth shape of the model based on the mouth shape information of the singer during singing to simulate the mouth shape of the singer; adjusting the stress state of a sound source organ muscle group of the model based on the opening and closing state information of the sound source organ when the singer sings, and simulating the opening and closing state of the sound source organ of the singer; based on the muscle stress state information of the organs related to the tone tuning area when the singer sings, the muscle stress state of the organs related to the tone tuning area of the singer is simulated, the shape state of the organs related to the tone tuning area of the singer is simulated, and a video file is generated and output or uploaded to a server.
Description
Technical Field
The embodiment of the application relates to a teaching sounding simulation display method and device.
Background
The traditional vocal music teaching mode still mostly adopts 'one-to-one' teaching, and is born by demonstrating and simulating the mode of the oral cardioverter, and in the vocal music teaching and learning process, the explanation and demonstration of teachers occupy extremely important status and leading role, so that the traditional vocal music learning needs several years to dozens of years. Although the traditional music teaching and learning is an excellent education form, the practical application has a plurality of limitations: vocal music learning is the process that students slowly experience vocal music skills and continuously make progress through vocal training. The teacher is the most important guide in this process, guiding by teaching the students the necessary vocalization skills. In fact, the teacher is unlikely to appear in every training of the student, and therefore, it is important for the student to learn autonomously. Since learners lack sharp self-listening ability at first, the learners cannot accurately judge and grasp their own voice production states and cannot distinguish whether their own voice production methods are correct, and therefore, learners tend to enter wrong areas during self-exercise. Also, the conventional teaching cannot freely select a place and a learning time. Traditional vocal music teaching is face-to-face teaching, and teachers teach according to different conditions and sound characteristics of each student according to the conditions, however, because the vocal music teaching has such strict requirements on the actual teaching environment, a lot of students who do not have conditions to learn face-to-face with teachers lose learning opportunities. Most vocal music teachers do not have sufficient understanding and attention on body structure. However, since the 'musical instruments' of the body cannot be seen and touched, most of the vocal music students cannot know the body structure of the students, so that the students can go around many curved roads on the learning road. In the traditional vocal music teaching, a teacher needs to make necessary guidance on not only a sound production method but also work analysis and the grasp of singing style. In the original teaching mode, the explanation and analysis of the works by the vocal teacher become the most important way for the students to understand the works, but the ability of the students to understand the works and the ability of learning are restricted to a certain extent, and the dependence and laziness of the students on teachers are also caused.
Disclosure of Invention
In order to solve the above technical problem, the embodiments of the present application are expected to provide a method and an apparatus for simulating and displaying a teaching vocalization.
The technical scheme of the invention is realized as follows:
the embodiment of the application provides a teaching sounding simulation display method, which is used for generating a three-dimensional mannequin, wherein the three-dimensional mannequin at least comprises a mouth, a nose, a tongue, sound source organs, relevant organs of a tuning area and main muscle groups of the relevant organs; the method comprises the following steps:
at least acquiring mouth shape information, opening and closing state information of sound source organs and muscle stress state information of organs related to a tuning area when a singer sings;
recording the voice information of the singer, and adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singer during singing to simulate the mouth shape information of the singer; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singer sings, and simulating the opening and closing state of the sound source organ of the singer; based on the muscle stress state information of the organs related to the tone tuning area when the singer sings, simulating the muscle stress state of the organs related to the tone tuning area of the singer, and simulating the shape state of the organs related to the tone tuning area of the singer;
generating a video file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ and the shape state of the organ related to the sound tuning area;
outputting or uploading the video file to a server based on a call instruction for the video file.
As an implementation, the method further comprises:
acquiring mouth shape information, opening and closing state information of sound source organs and muscle stress state information of organs related to a tuning area when a singing expert sings a set song;
recording the voice information of the singing expert, and adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singing expert during singing to simulate the mouth shape information of the singing expert; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singing expert sings, and simulating the opening and closing state of the sound source organ of the singing expert; based on the muscle stress state information of the organs related to the tuning area when the singing expert sings, simulating the muscle stress state of the organs related to the tuning area of the singing expert and simulating the shape state of the organs related to the tuning area of the singing expert;
generating a video teaching file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ and the shape state of the relevant organ of the sound tuning area;
uploading the video teaching file to a server for downloading.
As an implementation, the method further comprises:
determining a partial segment with singing difficulty exceeding a set threshold value in the set song singing, and searching a video segment corresponding to the partial segment;
adding marking information for the mouth shape, the sound source organ and the muscle of the organ related to the tuning area of the three-dimensional mannequin in the video clip; the marking information comprises graphic and text descriptions of sounding key points, posture key points and exertion key points.
As an implementation, the method further comprises:
receiving a video file of the singer, and identifying the singing track of the singer in the video file according to the voice information of the singer in the video file;
searching the video files of the singing tracks sung by the singing experts, which are the same as the singing tracks, according to the singing tracks of the singers;
comparing the mouth shape of the three-dimensional human body model, the opening and closing state of the sound source organ and the shape state of the relevant organ of the tuning area in the video file of the singer with the mouth shape of the three-dimensional human body model, the opening and closing state of the sound source organ and the shape state of the relevant organ of the tuning area in the video file of the singing expert, determining the difference between the singing actions of the singer in the singing process and the singing expert in the singing process, and outputting the video or video screenshot corresponding to the singing action with the difference exceeding the set threshold value so as to enable the singer to watch and learn the singing difference between the singer and the singing expert.
As an implementation, the method further comprises:
receiving comment information or an on-demand request aiming at the video file of the singer, sequencing the video file of the singer based on the comment information, and playing the video file of the singer for the requester based on the on-demand request.
The embodiment of the application provides still a simulation display device of teaching pronunciation, includes:
the system comprises a first generation unit, a second generation unit and a third generation unit, wherein the first generation unit is used for generating a three-dimensional mannequin, and the three-dimensional mannequin at least comprises a mouth, a nose, a tongue, a sound source organ, a sound regulation area related organ and a main muscle group thereof;
the acquisition unit is used for at least acquiring mouth shape information, opening and closing state information of sound source organs and muscle stress state information of organs related to a tuning area when a singer sings;
a recording unit for recording the voice information of the singer;
the simulation unit is used for adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singer singing so as to simulate the mouth shape information of the singer; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singer sings, and simulating the opening and closing state of the sound source organ of the singer; based on the muscle stress state information of the organs related to the tone tuning area when the singer sings, simulating the muscle stress state of the organs related to the tone tuning area of the singer, and simulating the shape state of the organs related to the tone tuning area of the singer;
the second generation unit is used for generating a video file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ and the shape state of the organ related to the tuning area;
and the uploading output unit is used for outputting or uploading the video file to a server based on the calling instruction for the video file.
As an implementation manner, the obtaining unit is further configured to obtain mouth shape information, opening and closing state information of a sound source organ, and muscle stress state information of a relevant organ of a tuning range when a singing expert sings a set song;
the recording unit is also used for recording the voice information of the singing expert;
the simulation unit is also used for adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singing expert during singing so as to simulate the mouth shape information of the singing expert; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singing expert sings, and simulating the opening and closing state of the sound source organ of the singing expert; based on the muscle stress state information of the organs related to the tuning area when the singing expert sings, simulating the muscle stress state of the organs related to the tuning area of the singing expert and simulating the shape state of the organs related to the tuning area of the singing expert;
the second generating unit is also used for generating a video teaching file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ and the shape state of the organ related to the tuning area;
the upload output unit is further for uploading the video teaching file to a server for downloading.
As an implementation, the apparatus further includes:
the first searching unit is used for determining a partial segment with the singing difficulty exceeding a set threshold value in the set song singing and searching a video segment corresponding to the partial segment;
the marking unit is used for adding marking information to muscles of the mouth shape, the sound source organ and the organ related to the sound mixing area of the three-dimensional mannequin in the video clip; the marking information comprises graphic and text descriptions of sounding key points, posture key points and exertion key points.
As an implementation, the apparatus further includes:
a first receiving unit for receiving a video file of the singer;
the identification unit is used for identifying the singing track of the singer in the video file according to the voice information of the singer in the video file;
the second searching unit is used for searching the video files of the singing tracks sung by the singing experts, which are the same as the singing tracks, according to the singing tracks of the singer;
the determining unit is used for respectively comparing the mouth shape, the opening and closing state of a sound source organ and the shape state of a organ related to a tuning area of the three-dimensional manikin in the video file of the singer with the mouth shape, the opening and closing state of the sound source organ and the shape state of the organ related to the tuning area of the three-dimensional manikin in the video file of the singer, and determining the difference between the singing actions of the singer in the singing process and the singing actions of the singer in the singing process of the singer;
and the output unit is used for outputting the video or video screenshot corresponding to the singing action with the difference exceeding the set threshold value so that the singer can watch and learn the singing difference between the singer and the singing expert.
As an implementation, the apparatus further includes:
a second receiving unit for receiving comment information or on-demand request for the video file of the singer;
the sorting unit is used for sorting the video files of the singers based on the comment information;
and the playing unit is used for playing the video file of the singer for the requester based on the on-demand request.
Compared with the prior art, the technical scheme of the embodiment of the application has the following advantages:
according to the teaching vocalization simulation display method and the teaching vocalization simulation display device, teaching is not limited by time and space, and the learning efficiency of learners is greatly improved. The learner can select courses according to the needs of the learner wherever and whenever. Most vocal music learners, especially beginners, are not enough to know own 'musical instruments', and sound production characteristics and singing detail requirements are met in a 3D animation mode through setting a three-dimensional manikin, so that users can establish correct sound production consciousness and theoretical knowledge, and can realize teaching according to the materials and improve the learning efficiency of the users. The embodiment of the application changes the traditional teaching mode, does not adopt one-to-one oral teaching, and improves the teaching efficiency. The learner who gets on the line and gets off the vocal music training class has higher training and learning cost, and the teaching mode of the embodiment of the application has lower cost. In addition, because the high-quality education resources are unevenly distributed at present, the embodiment of the application supports remote teaching, so that learners in remote areas can make up the deficiency.
Drawings
Fig. 1 is a schematic flow chart of a teaching vocalization simulation display method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a teaching sounding simulation display device according to an embodiment of the present application.
Detailed Description
The embodiments described in the present invention can be combined without conflict.
The technical solution of the present invention will be described in detail with reference to the accompanying drawings.
Fig. 1 is a schematic flow diagram of a teaching vocalization simulation display method according to an embodiment of the present application, and as shown in fig. 1, the teaching vocalization simulation display method according to the embodiment of the present application includes the following steps:
102, at least obtaining mouth shape information, opening and closing state information of sound source organs and muscle stress state information of organs related to a tuning area when a singer sings;
the vocal organ of singing is a physiological phenomenon generated by the coordination of the vocal organs, the phenomenon is a physical phenomenon formed by breath movement and vocal cord vibration, but the vocal movement of singing is different from the vocal production of normal speaking, so the vocal production of singing is a physical acoustic and acoustics phenomenon. The singing motion can be said to be a physiological and physical behavior.
The vocal organs are formed of human organs having various physiological functions, and when they participate in vocal activities, they are called vocal organs, forming a unique system having social functions. The vocal organ of singing consists of four parts, namely a respiratory organ, a vocal organ, a resonance organ and a word biting organ, which are all material bases of singing and sounding and are main functional systems in singing and sounding movement.
The singing and sounding organs comprise a larynx and a vocal cord. The larynx is a delicate small chamber, is positioned in the middle of the front part of the neck and consists of muscles such as cartilage, ligaments and the like.
The vocal organs of the human body can be divided into three major parts: a power area, a sound source area and a tuning area.
Speech is the distinction of speech determined by the organs and their activities.
The dynamic region includes the relevant organs such as lung, diaphragm, trachea, etc.
The lungs are the moving fans that breathe the airflow, which is the motive force of the speech. The air flow exhaled from the lung
It passes through bronchial organs and reaches larynx, and acts on vocal organs such as vocal cords, pharyngeal cavity, oral cavity, nasal cavity, etc.
The sound source area comprises a vocal cord which is positioned in the middle of the larynx and is two elastic strip-shaped films. The gap between the two vocal cords is called glottis, the contraction of muscles and the activity of the arytenoid cartilage can relax or tighten the vocal cords, so that the glottis can be opened or closed, the airflow from the lung can make the sound vibrate to make a sound through the glottis, and the change of tightness of the vocal cords can be controlled to make different sounds.
The toning zone comprises related organs such as oral cavity, nasal cavity, pharyngeal cavity and the like. The back of the oral cavity (including lips, teeth and tongue) is the pharyngeal cavity, and the pharyngeal head is communicated with the oral cavity, the nasal cavity and the lower laryngeal joint. The oral and nasal cavities are separated by the soft palate and uvula. When the soft palate and the uvula rise, the nasal cavity is closed, the oral cavity is unblocked, and the sound emitted is resonated in the oral cavity and called as the mouth sound. The soft palate and the uvula are prolapsed, the oral cavity is blocked, and airflow can only be emitted from the nasal cavity, which is that emitted sound mainly resonates in the nasal cavity and is called nasal sound. If the oral cavity is not obstructed, the airflow is exhaled from the oral cavity and the nasal cavity simultaneously, and the emitted sound resonates in the oral cavity and the nasal cavity simultaneously, so that the sound is called a nasal sound (also called a half-nose sound or an oronasal sound).
The method and the device have the main idea that the singing action of a singer in the singing process is recorded, the singing level of the singer is made according to the singing action of the singer, and the singing action needing to be improved by the singer is determined according to the comparison of the singing actions between the singer and a singing expert, so that the singing level of a learner based on the singing action of the singing expert is improved better.
In the embodiment of the application, information collection can be performed on mouth shape, singing shape action and the like of a singer based on a depth camera and the like, the singing action is determined through singing blood pressure change, muscle change, opening and closing states of organs and the like of the singer, and state change indexes of the organs relevant to singing can be collected through corresponding sensors through state change of the organs relevant to singing, so that the states of the organs relevant to singing are determined.
For example, the respiratory organ can be regarded as the "source" of singing, and is composed of mouth, nose, throat, trachea, bronchi, lungs, chest cavity, diaphragm (also called diaphragm), abdominal muscle, etc. Breath is inhaled from the nose and mouth, passes through the pharynx, larynx, trachea and bronchus, and is distributed into the lung bubbles of the left and right lung lobes (the lung is composed of two leaf-shaped air boxes with sponge tissues and contains a lot of small bubbles for containing air); then, in the opposite direction, the breath is collected to the three big tracheas on the two sides from the branched trachea (bronchus) at the outlet of the lung, finally, a trachea is formed, and the breath is exhaled from the mouth and the nose through the throat. The movement of the muscle groups associated with the respiratory system, also linked to the ability to breathe, is a guarantee of the motivation and energy of singing "the source". Daily breathing is calmer and shallower, and the whole vital capacity is not used, but the breathing movement during singing is different, the inspiration action is fast, and the expiration action is slow. If a long phrase is encountered, the rest must be persisted. The high, low, strong, weak, pause and rise-suppressing changes of a song can be completed by the firm and flexible movements of the muscles of inspiration and expiration. The above actions of the respiratory organs can determine the related singing action through the motion states of related muscle groups or related organs and the like.
For example, a sound-emitting organ, i.e. an organ that emits sound. It comprises a larynx and a vocal cord. The larynx is a delicate small chamber, is positioned in the middle of the front part of the neck and consists of muscles such as cartilage, ligaments and the like. The vocal cords are positioned in the middle of the larynx, are two horizontal, left-right parallel, symmetrical and elastic white ligaments, and have very firm properties. The middle of the vocal cords, also known as the glottis, is regulated by the cartilage and muscles in the larynx. When inhaling, the two vocal cords are separated, the glottis is opened, and the breath is inhaled; when the voice is produced, the two vocal cords are closed to produce the voice. The vocal cords are relaxed and open when not sounding to allow smooth passage of breath. The vocal cords produce sound, one part is the self function, and the other part depends on the muscle group around the vocal cords to assist the vocal exercise. When vocally training, the functional functions of the muscle groups should be fully noticed, the muscles are reasonably used, good habits are developed, and the voice is prevented from being sung and damaged under incorrect vocalization habits. There is also a very important cartilage between the upper part of the throat and the root of the tongue, called the epiglottis. The epiglottis has two functions, namely, the epiglottis plays a role in protecting the glottis, and when food and water are swallowed, the glottis is automatically covered by the epiglottis to prevent the food from entering the trachea when the food passes through the trachea, so that the phenomenon that the air is choked by careless drinking of water sometimes is caused, and the action of the epiglottis is not coordinated. The second effect is that when singing, the epiglottis is upright, forming a channel to let the voice output smoothly.
The human body's resonance organs mainly include three resonance cavities of chest cavity, oral cavity and head cavity. The thorax includes the trachea, bronchi and the entire lung below the larynx. The oral cavity includes larynx, pharyngeal cavity and oral cavity. The head cavity comprises nasal cavity, maxillary sinus, frontal sinus, sphenoid sinus, etc. In singing, the ratio of using these resonance chambers is different due to the difference in tone quotient. Generally speaking, chest resonance is most pronounced when singing bass, mouth resonance is more often applied when singing midrange, and the effect is mainly played by head-cavity singing when singing treble. If the resonant cavities can be correctly and reasonably applied and are matched with each other, the music can be obtained by flashing.
The organs of biting and spitting words (i.e. the organs of language) include lips, tongue, teeth, palate, etc. The positions of these organs during their movement and the different stress sites form consonants and vowels (i.e. speech). When singing is performed, the actions of the components of the organ for biting and spitting words are more agile and exaggerated than usual speaking. The agility is to make the biting character accurate and clear, and the exaggeration is to make the beautified vowel or vowel smoothly extend and play. Therefore, the language organ is the material basis when the user spits and bites the characters and is also an important organ for learning pronunciations, introduction and rhyme when the user spits and bites the characters. Therefore, the movement of the singing organ of the singer can be acquired through a corresponding sensor, camera or the like.
103, recording the voice information of the singer, and adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singer during singing to simulate the mouth shape information of the singer; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singer sings, and simulating the opening and closing state of the sound source organ of the singer; and simulating the muscle stress state of the organ related to the tuning area of the singer and simulating the shape state of the organ related to the tuning area of the singer based on the muscle stress state information of the organ related to the tuning area when the singer sings.
The voice is the basis of singing, and the structure and the function of the involved vocal organs are firstly known to train the voice to sing. Singing is much less felt than the visual and fumbling of body movements such as piano and violin training, which requires each singer to have sharp self-feeling and to train repeatedly under the guidance of professional vocals teachers to form conditioned reflex to judge whether the voice is correct or not and to accord with the basic rule of the movement of the vocal organs.
And 104, generating a video file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ and the shape state of the organ related to the tuning area.
In the embodiment of the application, by arranging the three-dimensional human body model, according to the singing action collected by the singer in the singing process, the singing action of the singer is simulated by the three-dimensional human body model, the human body structure of the singer in the singing process is shown by the three-dimensional animation, the singer can know own 'musical instruments' more, the singer does not depend on 'feeling' to sing any more, and the singer can generate sound more scientifically by knowing the body structure, and correct sound production consciousness and aesthetic direction are established.
And 105, outputting or uploading the video file to a server based on a calling instruction for the video file.
In the embodiment of the application, after the simulated three-dimensional simulated video sung by the singer is obtained, the singer can directly watch the simulated video, and the singing action of the singer is corrected based on the related singing simulated action of the singing expert so as to improve the singing level of the singer.
In the embodiment of the application, not only the singing action acquisition is required for the simulated action of the singer, but also the singing action acquisition is required for the singing expert, so that the three-dimensional simulated singing action of the teaching video is generated, and a learner can learn by reference. The method specifically comprises the following steps:
acquiring mouth shape information, opening and closing state information of sound source organs and muscle stress state information of organs related to a tuning area when a singing expert sings a set song;
recording the voice information of the singing expert, and adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singing expert during singing to simulate the mouth shape information of the singing expert; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singing expert sings, and simulating the opening and closing state of the sound source organ of the singing expert; based on the muscle stress state information of the organs related to the tuning area when the singing expert sings, simulating the muscle stress state of the organs related to the tuning area of the singing expert and simulating the shape state of the organs related to the tuning area of the singing expert;
generating a video teaching file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ and the shape state of the relevant organ of the sound tuning area;
uploading the video teaching file to a server for downloading.
As an implementation, the method further comprises:
determining a partial segment with singing difficulty exceeding a set threshold value in the set song singing, and searching a video segment corresponding to the partial segment;
adding marking information for the mouth shape, the sound source organ and the muscle of the organ related to the tuning area of the three-dimensional mannequin in the video clip; the marking information comprises graphic and text descriptions of sounding key points, posture key points and exertion key points.
In the embodiment of the application, the singing action can be collected more finely aiming at the place where the singing action is difficult in the singing track, and corresponding pictures, sounds or text descriptions are added in the corresponding difficult singing segment in the singing simulation video, so that a learner can better master the important field of the singing action.
As an implementation, the method further comprises:
receiving a video file of the singer, and identifying the singing track of the singer in the video file according to the voice information of the singer in the video file;
searching the video files of the singing tracks sung by the singing experts, which are the same as the singing tracks, according to the singing tracks of the singers;
comparing the mouth shape of the three-dimensional human body model, the opening and closing state of the sound source organ and the shape state of the relevant organ of the tuning area in the video file of the singer with the mouth shape of the three-dimensional human body model, the opening and closing state of the sound source organ and the shape state of the relevant organ of the tuning area in the video file of the singing expert, determining the difference between the singing actions of the singer in the singing process and the singing expert in the singing process, and outputting the video or video screenshot corresponding to the singing action with the difference exceeding the set threshold value so as to enable the singer to watch and learn the singing difference between the singer and the singing expert.
The embodiment of the application also supports automatic comparison between the singing action of the singer and the singing action of the singing expert, determines the shortages of the singing action of the singer, and outputs the singing action to the singer, so that the singer can know which shortages exist in the singing action of the singer and which fragments need to improve the singing level.
As an implementation, the method further comprises:
receiving comment information or an on-demand request aiming at the video file of the singer, sequencing the video file of the singer based on the comment information, and playing the video file of the singer for the requester based on the on-demand request.
In the embodiment of the application, a singer can select corresponding singing action collection according to the gender and the voice type. After the singer sings, the singing result of the user can be fed back to the user according to the big data analysis result, the 3D animation corresponding to the singing result is provided, the singing result can be compared with the 3D animation of a famous person, and the user can more intuitively see the problem, so that the learning efficiency is improved.
The user can release the singing exercise segment of the user in the App, and other users can approve comment forwarding and the like. The App platform can release a series of videos such as singing backgrounds and work analysis aiming at works.
Based on the singing video of the singer, forums, group chatting and the like can be established, resources can be shared, learning experience can be exchanged, and learning enthusiasm is improved.
The embodiment of the application can select remote teaching, real-time teaching is carried out, teaching is carried out according to the material, and then the singing style of the song is more accurately mastered.
Fig. 2 is a schematic diagram of a structure of a teaching vocal simulation display device in an embodiment of the present application, as shown in fig. 2, the teaching vocal simulation display device in an embodiment of the present application includes:
a first generating unit 20 for generating a three-dimensional mannequin comprising at least mouth, nose, tongue, sound source organs, organs related to the toning zone and its main muscle groups;
the acquisition unit 21 is configured to at least acquire mouth shape information of a singer during singing, opening and closing state information of a sound source organ, and muscle stress state information of organs related to a tuning area;
a recording unit 22 for recording the voice information of the singer;
the simulation unit 23 is used for adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singer singing so as to simulate the mouth shape information of the singer; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singer sings, and simulating the opening and closing state of the sound source organ of the singer; based on the muscle stress state information of the organs related to the tone tuning area when the singer sings, simulating the muscle stress state of the organs related to the tone tuning area of the singer, and simulating the shape state of the organs related to the tone tuning area of the singer;
a second generating unit 24, configured to generate a video file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ, and the shape state of the organ related to the tuning area;
an upload output unit 25 for outputting or uploading the video file to a server based on a call instruction for the video file.
As an implementation manner, the obtaining unit 21 is further configured to obtain mouth shape information, opening and closing state information of a sound source organ, and muscle stress state information of organs related to a tuning range when a singing expert sings a set song;
the recording unit 22 is also used for recording the voice information of the singing expert;
the simulation unit 23 is further configured to adjust the mouth shape of the three-dimensional mannequin based on the mouth shape information of the singing expert during singing to simulate the mouth shape information of the singing expert; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singing expert sings, and simulating the opening and closing state of the sound source organ of the singing expert; based on the muscle stress state information of the organs related to the tuning area when the singing expert sings, simulating the muscle stress state of the organs related to the tuning area of the singing expert and simulating the shape state of the organs related to the tuning area of the singing expert;
the second generating unit 24 is further configured to generate a video teaching file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ, and the shape state of the organ related to the tuning area;
the upload output unit 25 is also used to upload the video teaching file to a server for downloading.
As an implementation, the apparatus further includes:
a first searching unit (not shown in the figure), configured to determine a partial segment with a singing difficulty exceeding a set threshold in the set song singing, and search for a video segment corresponding to the partial segment;
a marking unit (not shown in the figure) for adding marking information to the muscles of the mouth shape, the sound source organ and the organ related to the tuning volume of the three-dimensional mannequin in the video clip; the marking information comprises graphic and text descriptions of sounding key points, posture key points and exertion key points.
As an implementation, the apparatus further includes:
a first receiving unit (not shown in the drawings) for receiving a video file of the singer;
an identifying unit (not shown in the figure) for identifying the singing track of the singer in the video file according to the voice information of the singer in the video file;
a second search unit (not shown in the figure) for searching the video file of the singing track sung by the singing expert, which is the same as the singing track, according to the singing track of the singer;
a determining unit (not shown in the figure), configured to compare the mouth shape, the open-close state of the sound source organ, and the shape state of the organ related to the tuning area of the three-dimensional body model in the video file of the singer with the mouth shape, the open-close state of the sound source organ, and the shape state of the organ related to the tuning area of the three-dimensional body model in the video file of the singer, respectively, and determine a difference between the singing actions of the singer during the singing process and the singing actions of the singer during the singing process of the singer;
and an output unit (not shown in the figure) for outputting the video or video screenshot corresponding to the singing action with the difference exceeding the set threshold value, so that the singer can watch and learn the singing gap between the singer and the singing expert.
As an implementation, the apparatus further includes:
a second receiving unit (not shown in the figure) for receiving comment information or on-demand request for the video file of the singer;
a sorting unit (not shown in the figure) for sorting the video files of the singers based on the comment information;
a playing unit (not shown in the figure) for playing the video file of the singer for the requester based on the on-demand request.
It should be understood by those skilled in the art that the technical solutions of the teaching vocalization simulation display device of the embodiments of the present application can be understood by referring to the foregoing teaching vocalization simulation display method embodiments, wherein the processing unit can be implemented by means of software or simulation circuit.
According to the teaching vocalization simulation display method and the teaching vocalization simulation display device, teaching is not limited by time and space, and the learning efficiency of learners is greatly improved. The learner can select courses according to the needs of the learner wherever and whenever. Most vocal music learners, especially beginners, are not enough to know own 'musical instruments', and sound production characteristics and singing detail requirements are met in a 3D animation mode through setting a three-dimensional manikin, so that users can establish correct sound production consciousness and theoretical knowledge, and can realize teaching according to the materials and improve the learning efficiency of the users. The embodiment of the application changes the traditional teaching mode, does not adopt one-to-one oral teaching, and improves the teaching efficiency. The learner who gets on the line and gets off the vocal music training class has higher training and learning cost, and the teaching mode of the embodiment of the application has lower cost. In addition, because the high-quality education resources are unevenly distributed at present, the embodiment of the application supports remote teaching, so that learners in remote areas can make up the deficiency.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application.
Claims (10)
1. A simulation display method for teaching vocalization is characterized in that a three-dimensional mannequin is generated, wherein the three-dimensional mannequin at least comprises a mouth, a nose, a tongue, sound source organs, relevant organs of a tuning area and main muscle groups of the relevant organs; the method comprises the following steps:
at least acquiring mouth shape information, opening and closing state information of sound source organs and muscle stress state information of organs related to a tuning area when a singer sings;
recording the voice information of the singer, and adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singer during singing to simulate the mouth shape information of the singer; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singer sings, and simulating the opening and closing state of the sound source organ of the singer; based on the muscle stress state information of the organs related to the tone tuning area when the singer sings, simulating the muscle stress state of the organs related to the tone tuning area of the singer, and simulating the shape state of the organs related to the tone tuning area of the singer;
generating a video file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ and the shape state of the organ related to the sound tuning area;
outputting or uploading the video file to a server based on a call instruction for the video file.
2. The method for simulated presentation of instructional utterances according to claim 1, wherein the method further comprises:
acquiring mouth shape information, opening and closing state information of sound source organs and muscle stress state information of organs related to a tuning area when a singing expert sings a set song;
recording the voice information of the singing expert, and adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singing expert during singing to simulate the mouth shape information of the singing expert; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singing expert sings, and simulating the opening and closing state of the sound source organ of the singing expert; based on the muscle stress state information of the organs related to the tuning area when the singing expert sings, simulating the muscle stress state of the organs related to the tuning area of the singing expert and simulating the shape state of the organs related to the tuning area of the singing expert;
generating a video teaching file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ and the shape state of the relevant organ of the sound tuning area;
uploading the video teaching file to a server for downloading.
3. The method for simulated presentation of instructional utterances according to claim 2, wherein the method further comprises:
determining a partial segment with singing difficulty exceeding a set threshold value in the set song singing, and searching a video segment corresponding to the partial segment;
adding marking information for the mouth shape, the sound source organ and the muscle of the organ related to the tuning area of the three-dimensional mannequin in the video clip; the marking information comprises graphic and text descriptions of sounding key points, posture key points and exertion key points.
4. The method for simulated presentation of instructional utterances according to claim 3, wherein the method further comprises:
receiving a video file of the singer, and identifying the singing track of the singer in the video file according to the voice information of the singer in the video file;
searching the video files of the singing tracks sung by the singing experts, which are the same as the singing tracks, according to the singing tracks of the singers;
comparing the mouth shape of the three-dimensional human body model, the opening and closing state of the sound source organ and the shape state of the relevant organ of the tuning area in the video file of the singer with the mouth shape of the three-dimensional human body model, the opening and closing state of the sound source organ and the shape state of the relevant organ of the tuning area in the video file of the singing expert, determining the difference between the singing actions of the singer in the singing process and the singing expert in the singing process, and outputting the video or video screenshot corresponding to the singing action with the difference exceeding the set threshold value so as to enable the singer to watch and learn the singing difference between the singer and the singing expert.
5. The method of claim 4, wherein the method further comprises:
receiving comment information or an on-demand request aiming at the video file of the singer, sequencing the video file of the singer based on the comment information, and playing the video file of the singer for the requester based on the on-demand request.
6. A simulated display device for educational sound production, the device comprising:
the system comprises a first generation unit, a second generation unit and a third generation unit, wherein the first generation unit is used for generating a three-dimensional mannequin, and the three-dimensional mannequin at least comprises a mouth, a nose, a tongue, a sound source organ, a sound regulation area related organ and a main muscle group thereof;
the acquisition unit is used for at least acquiring mouth shape information, opening and closing state information of sound source organs and muscle stress state information of organs related to a tuning area when a singer sings;
a recording unit for recording the voice information of the singer;
the simulation unit is used for adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singer singing so as to simulate the mouth shape information of the singer; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singer sings, and simulating the opening and closing state of the sound source organ of the singer; based on the muscle stress state information of the organs related to the tone tuning area when the singer sings, simulating the muscle stress state of the organs related to the tone tuning area of the singer, and simulating the shape state of the organs related to the tone tuning area of the singer;
the second generation unit is used for generating a video file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ and the shape state of the organ related to the tuning area;
and the uploading output unit is used for outputting or uploading the video file to a server based on the calling instruction for the video file.
7. The instructional sound production simulated display apparatus of claim 6,
the acquisition unit is also used for acquiring the mouth shape information, the opening and closing state information of the sound source organ and the muscle stress state information of the relevant organ of the tuning area when the singing expert sings the set song;
the recording unit is also used for recording the voice information of the singing expert;
the simulation unit is also used for adjusting the mouth shape of the three-dimensional human body model based on the mouth shape information of the singing expert during singing so as to simulate the mouth shape information of the singing expert; adjusting the stress state of the sound source organ muscle group of the three-dimensional human body model based on the opening and closing state information of the sound source organ when the singing expert sings, and simulating the opening and closing state of the sound source organ of the singing expert; based on the muscle stress state information of the organs related to the tuning area when the singing expert sings, simulating the muscle stress state of the organs related to the tuning area of the singing expert and simulating the shape state of the organs related to the tuning area of the singing expert;
the second generating unit is also used for generating a video teaching file according to the sound information, the mouth shape of the three-dimensional mannequin, the opening and closing state of the sound source organ and the shape state of the organ related to the tuning area;
the upload output unit is further for uploading the video teaching file to a server for downloading.
8. The educational vocal simulation display apparatus of claim 7, wherein the apparatus further comprises:
the first searching unit is used for determining a partial segment with the singing difficulty exceeding a set threshold value in the set song singing and searching a video segment corresponding to the partial segment;
the marking unit is used for adding marking information to muscles of the mouth shape, the sound source organ and the organ related to the sound mixing area of the three-dimensional mannequin in the video clip; the marking information comprises graphic and text descriptions of sounding key points, posture key points and exertion key points.
9. The educational vocal simulation display apparatus of claim 8, wherein the apparatus further comprises:
a first receiving unit for receiving a video file of the singer;
the identification unit is used for identifying the singing track of the singer in the video file according to the voice information of the singer in the video file;
the second searching unit is used for searching the video files of the singing tracks sung by the singing experts, which are the same as the singing tracks, according to the singing tracks of the singer;
the determining unit is used for respectively comparing the mouth shape, the opening and closing state of a sound source organ and the shape state of a organ related to a tuning area of the three-dimensional manikin in the video file of the singer with the mouth shape, the opening and closing state of the sound source organ and the shape state of the organ related to the tuning area of the three-dimensional manikin in the video file of the singer, and determining the difference between the singing actions of the singer in the singing process and the singing actions of the singer in the singing process of the singer;
and the output unit is used for outputting the video or video screenshot corresponding to the singing action with the difference exceeding the set threshold value so that the singer can watch and learn the singing difference between the singer and the singing expert.
10. The educational vocal simulation display apparatus of claim 9, wherein the apparatus further comprises:
a second receiving unit for receiving comment information or on-demand request for the video file of the singer;
the sorting unit is used for sorting the video files of the singers based on the comment information;
and the playing unit is used for playing the video file of the singer for the requester based on the on-demand request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911338619.2A CN111105669A (en) | 2019-12-23 | 2019-12-23 | Teaching sounding simulation display method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911338619.2A CN111105669A (en) | 2019-12-23 | 2019-12-23 | Teaching sounding simulation display method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111105669A true CN111105669A (en) | 2020-05-05 |
Family
ID=70423289
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911338619.2A Pending CN111105669A (en) | 2019-12-23 | 2019-12-23 | Teaching sounding simulation display method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111105669A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111701217A (en) * | 2020-07-01 | 2020-09-25 | 三峡大学 | White singing tone channel training method |
| CN112381913A (en) * | 2020-10-20 | 2021-02-19 | 北京语言大学 | Dynamic pronunciation teaching model construction method based on 3D modeling and oral anatomy |
-
2019
- 2019-12-23 CN CN201911338619.2A patent/CN111105669A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111701217A (en) * | 2020-07-01 | 2020-09-25 | 三峡大学 | White singing tone channel training method |
| CN111701217B (en) * | 2020-07-01 | 2021-06-08 | 三峡大学 | White singing tone channel training method |
| CN112381913A (en) * | 2020-10-20 | 2021-02-19 | 北京语言大学 | Dynamic pronunciation teaching model construction method based on 3D modeling and oral anatomy |
| CN112381913B (en) * | 2020-10-20 | 2021-06-04 | 北京语言大学 | Dynamic pronunciation teaching model construction method based on 3D modeling and oral anatomy |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| McKinney | The diagnosis and correction of vocal faults: A manual for teachers of singing and for choir directors | |
| Marafioti | Caruso's method of voice production: the scientific culture of the voice | |
| CN111105669A (en) | Teaching sounding simulation display method and device | |
| Thomaidis | The vocal body | |
| Nair | The craft of singing | |
| Chapman | An investigation of current instruction practices for the undergraduate instrumental conducting student concerning left hand technique and facial gestures | |
| Rosine | Vocal Techniques for the Instrumentalist 2nd Edition | |
| Henderson | The art of the singer: Practical hints about vocal technics and style | |
| Fonza | Tone building strategies used for beginning high school choirs | |
| Babb | Rehearsal techniques used to build choral tone by four expert collegiate choral conductors across settings | |
| Howard | The Child-Voice in singing: Treated from a physiological and a practical standpoint and especially adapted to schools and boy choirs | |
| Hoss | Classical and Musical Theater Vocal Pedagogy for Female Collegiate Singers: An Observation and Study | |
| Bannan | First Instruments: Teaching Music Through Harmony Signing | |
| Stowe | Song and Wind: An Explanation of the Approach for Teachers of Beginning Brass Students | |
| Hary | Vocal Pedagogy-Reflections on How to Teach Proper Speaking and Singing to Students | |
| Smith et al. | Class Voice: Fundamental Skills for Lifelong Singing | |
| Mathis | Melodic sculpturing: The art and science of singing | |
| Walders | Vocal pedagogy and applications for conductors not trained in singing | |
| Xudoykulovich et al. | BRIEF RECOMMENDATIONS AND SCIENTIFIC SUMMARY FOR STUDENTS IN LEARNING VOCAL ART | |
| Asztalos | Correction of Children’s Vocal Production Problems | |
| Porter | A Physiological Exploration and Pedagogical Integration of Voice and Brass Tuba Physical Methods to Enhance Tone Timbre or" Play Like You Sing" | |
| Asztalos | The Theory and Practice of Singing Voice Production of Children | |
| Sonnenburg | Contemporary commercial music and the young singer | |
| Michaud | Teaching ‘Tone Deaf’Teens | |
| Zhai | Comprehensive Ability of Chorus Conductor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200505 |
|
| WD01 | Invention patent application deemed withdrawn after publication |