CN108388399A - The method of state management and system of virtual idol - Google Patents
The method of state management and system of virtual idol Download PDFInfo
- Publication number
- CN108388399A CN108388399A CN201810032045.5A CN201810032045A CN108388399A CN 108388399 A CN108388399 A CN 108388399A CN 201810032045 A CN201810032045 A CN 201810032045A CN 108388399 A CN108388399 A CN 108388399A
- Authority
- CN
- China
- Prior art keywords
- state
- virtual idol
- virtual
- idol
- technical ability
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000006243 chemical reaction Methods 0.000 claims abstract description 51
- 230000003993 interaction Effects 0.000 claims abstract description 27
- 230000002452 interceptive effect Effects 0.000 claims abstract description 23
- 210000004556 brain Anatomy 0.000 claims description 33
- 230000008447 perception Effects 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 11
- 230000009471 action Effects 0.000 claims description 9
- 230000002996 emotional effect Effects 0.000 claims description 4
- 230000008921 facial expression Effects 0.000 claims description 4
- 230000016776 visual perception Effects 0.000 claims description 4
- 230000005236 sound signal Effects 0.000 claims description 2
- 238000003384 imaging method Methods 0.000 abstract description 6
- 238000004891 communication Methods 0.000 description 14
- 230000004438 eyesight Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000004913 activation Effects 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000001093 holography Methods 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 208000025967 Dissociative Identity disease Diseases 0.000 description 1
- 241000238558 Eucarida Species 0.000 description 1
- 206010034719 Personality change Diseases 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000005034 decoration Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000012905 input function Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The present invention provides a kind of method of state management of virtual idol, and virtual idol has specific image characteristics, and is demonstrated out by hologram device, and method includes：Obtain multi-modal input；The intention in multi-modal input or operation are parsed, to obtain instructing for the conversion intention of condition conversion or conversion；Convert the current state of virtual idol to the new state for the virtual idol that conversion is intended to or conversion instruction indicates；New state includes：Open virtual idol required ability or technical ability module under new state.The method of state management and system of virtual idol provided by the invention provide a kind of virtual idol, can pass through the multi-modal interaction of holographic imaging completion and user.In addition, virtual idol provided by the invention also includes various states, for example, halted state, audio output state, waiting recording state, recording state, standby mode and technical ability open state, and the present invention can also be managed the state of virtual idol, improve the interactive experience of user.
Description
Technical field
The present invention relates to artificial intelligence fields, specifically, being related to a kind of method of state management and system of virtual idol.
Background technology
The exploitation of robot chat interactive system is dedicated to imitating human conversation.The relatively more extensive chat machine of early stage application
People's application program includes the received input of the processing such as siri chat robots on small i chat robots or iPhone
(including text or voice) and corresponding response is made according to input, to attempt to imitate the friendship between the mankind between context
Mutually.
But at present for, for the relevant robot of virtual idol chat interactive system exploitation it is also less perfect, still
Do not go out to be now able to carry out multi-modal interaction with user and the product of multi-modal interaction that virtual idol state can be managed.
Therefore, the present invention provides a kind of method of state management and system of virtual idol.
Invention content
To solve the above problems, the present invention provides a kind of method of state management of virtual idol, the virtual idol tool
There are specific image characteristics, and be demonstrated out by hologram device, the method comprises the steps of：
Obtain multi-modal input；
The intention in the multi-modal input or operation are parsed, to obtain referring to for the conversion intention of condition conversion or conversion
It enables；
Convert the current state of the virtual idol to the virtual idol that the conversion is intended to or conversion instruction indicates
The new state of picture；
The new state includes：Open the virtual idol required ability or technical ability module under the new state.
According to one embodiment of present invention, the state of the virtual idol is divided into dormant state, active state and waiting
State, wherein
Dormant state includes：Halted state and standby mode；
Active state includes：Recording state, audio output state and technical ability open state；
Under a halt condition, the virtual idol out of service；
In the standby state, the virtual idol described in running background；
Under recording state, multi-modal output before stopping starts to detect audio signal；
Under audio output state, the ability or technical ability mould language interactive module in the block is called to engage in the dialogue interaction；
Under technical ability open state, the ability or technical ability mould a song and dance module in the block is called to carry out a song and dance.
According to one embodiment of present invention, the wait state is to wait for recording state.
According to one embodiment of present invention, under the wait state, in conjunction with high in the clouds brain to the multi-modal input
Analysis result come determine the state to be entered be audio output state or technical ability open state, and entrance audio output shape
After state or technical ability open state, by the feedback in conjunction with the high in the clouds brain come executive capability or the multimode of technical ability module unlatching
State exports.
According to one embodiment of present invention, under any type active state, if appointing under detecting current state
When business processing terminates and any multi-modal input data is not detected, current state is converted into dormant state and is waited for
Machine state or halted state.
According to one embodiment of present invention, the highest priority of the recording state in the active state, virtual even
As being waited for waiting under recording state, acquisition user speech is so that virtual idol enters recording state.
According to another aspect of the present invention, a kind of program product is additionally provided, it includes as described above for executing
The series of instructions of either method step.
According to another aspect of the present invention, a kind of virtual idol is additionally provided, which is characterized in that the virtual idol tool
Standby specific virtual image and preset attribute, the condition conversion process of the virtual idol is executed using method as described above.
According to another aspect of the present invention, a kind of system for managing state of virtual idol, the system packet are additionally provided
Contain：
Smart machine is mounted with the virtual idol thereon, for obtaining multi-modal input, and has natural language reason
Solution, visual perception, the ability for touching perception, language voice output, emotional facial expressions action output；
Hologram device is used to obtain multi-modal input and converts the image of the virtual idol to hologram simultaneously
Show the hologram；
High in the clouds brain is used in the wait state, will be into determine according to the analysis result to the multi-modal input
The state entered is audio output state or technical ability open state, and is entering audio output state or technical ability open state
Afterwards, the multi-modal output of virtual idol described in decision.
The method of state management and system of a kind of virtual idol provided by the invention provide a kind of virtual idol, Neng Goutong
The mode for crossing holographic imaging completes multi-modal interaction with user.In addition, the condition managing system of virtual idol provided by the invention
Virtual idol in system also includes various states, for example, halted state, audio output state, waiting recording state, recording shape
State, standby mode and technical ability open state, and the present invention can also be managed the state of virtual idol, improve use
The interactive experience at family.
Other features and advantages of the present invention will be illustrated in the following description, also, partly becomes from specification
It obtains it is clear that understand through the implementation of the invention.The purpose of the present invention and other advantages can be by specification, rights
Specifically noted structure is realized and is obtained in claim and attached drawing.
Description of the drawings
Attached drawing is used to provide further understanding of the present invention, and a part for constitution instruction, the reality with the present invention
It applies example and is used together to explain the present invention, be not construed as limiting the invention.In the accompanying drawings：
Fig. 1 shows that the multi-modal interaction of the system for managing state of virtual idol according to an embodiment of the invention is shown
It is intended to；
Fig. 2 shows the structure diagram of the system for managing state of virtual idol according to an embodiment of the invention；
Fig. 3 shows the state classification figure of the system for managing state of virtual idol according to an embodiment of the invention；
Fig. 4 shows the condition conversion signal of the system for managing state of virtual idol according to an embodiment of the invention
Figure；
Fig. 5 shows the module frame chart of the system for managing state of virtual idol according to an embodiment of the invention；
Fig. 6 shows the flow chart of the method for state management of virtual idol according to an embodiment of the invention；
Fig. 7 shows another flow chart of the method for state management of virtual idol according to an embodiment of the invention；
And
Fig. 8 shows according to an embodiment of the invention in user, smart machine, hologram device and high in the clouds brain
The flow chart communicated between four directions.
Specific implementation mode
To make the object, technical solutions and advantages of the present invention clearer, the embodiment of the present invention is made below in conjunction with attached drawing
Further it is described in detail.
It is clear to state, it needs to carry out before embodiment as described below：
The virtual idol that the present invention mentions has specific image characteristics using hologram device as main presentation interface；
By supporting the smart machine of input and output and control module to realize multi-modal human-computer interaction, has natural language reason
Solution, visual perception touch the AI abilities such as perception, language voice output, emotional facial expressions action output；
Configurable social property, personality attribute, personage's technical ability etc., make user (Quadratic Finite Element enthusiast) enjoy amusement and individual character
Change the virtual portrait of Flow Experience.
The high in the clouds brain being previously mentioned is to provide the virtual idol to carry out semantic understanding (language language to the interaction demand of user
Reason and good sense solution, Action Semantic understanding, visual identity, affection computation, cognition calculate) processing capacity terminal, realize with user
Interaction, to help user to carry out decision.
Each embodiment of the present invention is described in detail below in conjunction with the accompanying drawings.
Fig. 1 shows that the multi-modal interaction of the system for managing state of virtual idol according to an embodiment of the invention is shown
It is intended to.As shown in Figure 1, carrying out multi-modal interaction needs user 101, smart machine 102, hologram device 103 and high in the clouds brain
104.Wherein, the user 101 interacted with virtual idol can be true people, the virtual idol of another virtual idol and entity
The interaction of picture, another virtual idol and the virtual idol of entity and the interactive process and single people and virtual idol of virtual idol
Process is similar.Therefore, only show the multi-modal interactive process of user (people) and virtual idol in Fig. 1.
The virtual process interacted between idol and user 101 is in Fig. 1：
Interaction required early-stage preparations or condition have, and virtual idol is carried and operated on smart machine 102, and empty
Quasi- idol has specific image characteristics.Virtual idol have natural language understanding, visual perception, touch perception, language output,
The AI abilities such as emotional facial expressions action output.In order to coordinate the touch perceptional function of virtual idol, it is also required to install on smart machine
There is the component for having and touching perceptional function.According to one embodiment of present invention, in order to promote interactive experience, virtual idol exists
It is indicated in the predeterminable area of hologram device after being activated, the overlong time for avoiding user from waiting for.
It should be noted that the image of virtual idol and dressing up and being not limited to one mode.Virtual idol can have
For different images and dress up.The image of virtual idol is generally 3D high mould animating images.Virtual idol can have difference
Appearance and decoration.Each virtual idol image can also correspond to it is a variety of different dress up, the classification dressed up can be according to season
Section classification, can also classify according to occasion.These images and dresss up and can reside in high in the clouds brain 104, there may also be
In smart machine 102, it can be called at any time when needing to call these images and dressing up.
Social property, personality attribute and the personage's technical ability of virtual idol are also not necessarily limited to a kind of or a kind of.Visual human can
To have a variety of social properties, multiple personality attribute and a variety of personage's technical ability.These social properties, personality attribute and personage
Technical ability can arrange in pairs or groups respectively, and be not secured to a kind of collocation mode, and user can select and arrange in pairs or groups as needed.
According to one embodiment of present invention, for show virtual idol hologram device 103 include communication interface, imaging
Device and output device.Wherein, communication interface receives the vivid of the virtual idol that smart machine 102 transmits and interaction number
According to.Imaging device is connect with communication interface, for converting the image of virtual idol to hologram, and hologram is shown
In predeterminable area.Output device is connect with communication interface and imaging device, and hologram and virtual idol are current for rendering
The display data of state.
It is that multi-modal interactive process obtains multi-modal input first below.Multi-modal input can be that user 101 sends out
, can also be to be inputted by perceiving environment.Multi-modal input can include text, voice, vision and perception information etc.
The information of multiple modalities.The reception device for obtaining multi-modal input is respectively mounted and is configured at smart machine or hologram device
On, these reception devices include to receive the received text device of text, receive the pronunciation receiver of voice, receive taking the photograph for vision
As head and the infrored equipment etc. of reception perception information.
Then, the intention in multi-modal input or operation are parsed, to obtain for the conversion intention of condition conversion or conversion
Instruction.In multi-modal interactive process, virtual idol can interact under various states with user 101, each state is all
Have the ability or technical ability module of different virtual idols.
In order to convert the state of virtual idol during virtual idol and user 101 interact, need to solve in real time
The intention in multi-modal input or operation are analysed, analyzes in multi-modal input and whether comprising user 101 to convert virtual idol state
Wish, with obtain for condition conversion conversion be intended to or conversion instruct.
After obtaining conversion intention or conversion instruction, it is intended to refer to next, converting the current state of virtual idol to conversion
The new state of the virtual idol shown.According to one embodiment of present invention, the state of virtual idol includes dormant state, enlivens shape
State and wait state.Wherein, dormant state includes：Halted state and standby mode；Active state includes：Recording state, audio
Output state and technical ability open state.The operating condition of virtual idol is under each state, under a halt condition, void out of service
Quasi- idol；In the standby state, in the virtual idol of running background；Under recording state, audio input signal is detected, starts ability
Or technical ability mould recording module in the block records audio input data；Under audio output state, in call capability or technical ability module
Language interactive module engage in the dialogue interaction；Under technical ability open state, call capability or technical ability mould a song and dance mould in the block
Block carries out a song and dance.
Finally, virtual idol required ability or technical ability module under new state are opened.
In one embodiment of the invention, the screen cover of smart machine 102 is to hologram device 103, and shows on the screen
Show the image of virtual idol, the image of virtual idol is the view of four angles, be respectively front view, rearview, left view with
And right view.
According to another embodiment of the invention, a kind of virtual idol, has specific virtual image and preset attribute,
The condition conversion process of virtual idol is executed using the method for state management of virtual idol provided by the invention.
Fig. 2 shows the structure diagram of the system for managing state of virtual idol according to an embodiment of the invention.Such as
Shown in Fig. 2, multi-modal interactive needs are completed by system：User 101, smart machine 102 and high in the clouds brain 104.Wherein, intelligence
Energy equipment 102 includes reception device 102A, processing unit 102B, output device 102C and attachment device 102D.High in the clouds brain
104 include communication device 1041.
It is needed in user 101, smart machine 102 and high in the clouds in the system for managing state of virtual idol provided by the invention
Unobstructed communication port is established between brain 104, so as to complete the interaction of user 101 and virtual idol.In order to complete to hand over
Mutual task, smart machine 102 and high in the clouds brain 104 are configured with the device and component for supporting completion interaction.With virtual idol
As the object of interaction can be a side, or multi-party.
Smart machine 102 includes reception device 102A, processing unit 102B, output device 102C and attachment device
102D.Wherein, reception device 102A is for receiving multi-modal input.The example of reception device 102A includes keyboard, cursor control
Equipment (mouse), for voice operating microphone, scanner, touch function (such as to detect the capacitive of physical touch
Sensor), camera (action touched is not related to using the detection of visible or nonvisible wavelength) etc..Smart machine 102 can be with
Multi-modal input is obtained by above-mentioned input equipment.Output device 102C is for exporting virtual idol and user 101
Interactive multi-modal output data, details are not described herein.
Processing unit 102B is for handling the interaction data transmitted by high in the clouds brain 104 in interactive process.Attachment device
102D is used for contacting between high in the clouds brain 104, and processing unit 102B processing reception devices 102A is pretreated multi-modal defeated
The data for entering or being transmitted by high in the clouds brain.Attachment device 102D sends call instruction to call the robot on high in the clouds brain 104
Ability.
In the wait state, high in the clouds brain 104 can be entered according to analysis result to multi-modal input to determine
State is audio output state or technical ability open state, and after entering audio output state or technical ability open state, certainly
The multi-modal output of virtual idol described in plan.
The communication device 1041 that high in the clouds brain 104 includes is for completing writing to each other between smart machine 102.Communication
It keeps in communication and contacts between attachment device 102D on device 1041 and smart machine 102, receive sending for smart machine 102
Request, and the handling result that high in the clouds brain 104 is sent out is sent, it is Jie linked up between smart machine 102 and high in the clouds brain 104
Matter.
Fig. 3 shows the state classification figure of the system for managing state of virtual idol according to an embodiment of the invention.
As shown in figure 3, virtual idol state 300 includes dormant state 301, active state 302 and wait state 303.Wherein, suspend mode
State 301 includes halted state 3011 and standby mode 3012.Active state 302 includes recording state 3021, audio output
State 3022 and technical ability open state 3023.
According to one embodiment of present invention, the ability of virtual idol state or technical ability includes, under halted state 3011,
Virtual idol out of service；Under standby mode 3012, in the virtual idol of running background；Under recording state 3021, sound is detected
Frequency input signal, starts ability or technical ability mould recording module in the block records audio input data；In voice output state 3022
Under, call capability or technical ability mould language interactive module in the block engage in the dialogue interaction；Under technical ability open state 3023, energy is called
Power or technical ability mould a song and dance module in the block carry out a song and dance.
In the system for managing state of virtual idol provided by the invention, wait state is important in virtual idol state
Component part is the bridge between recording state and audio output state or technical ability open state.One according to the present invention
Embodiment, wait state 303 can wait for recording state, as respond the state interrupted, in the case where waiting for recording state, knot
High in the clouds brain 104 is closed to the analysis result of multi-modal input to determine that the state to be entered is audio output state 3022 or skill
Energy open state 3023, and after entering audio output state 3022 or technical ability open state 3023, by combining high in the clouds big
The feedback of brain 104 carrys out the multi-modal output that executive capability or technical ability module are opened.
According to one embodiment of present invention, under any type active state 302, if under detecting current state
When task processing terminates and any multi-modal input data is not detected, current state is converted to dormant state 301
In standby mode 3012 or halted state 3011.In addition, the highest priority of the recording state 3021 in active state 302,
In the case where virtual idol is waited for 303 i.e. waiting recording state, acquisition user speech is so that virtual idol enters recording
State.
Fig. 4 shows the condition conversion signal of the system for managing state of virtual idol according to an embodiment of the invention
Figure.
Virtual idol in the system for managing state of virtual idol provided by the invention has a variety of different states, each
Virtual idol is but also with different ability or technical ability under state.Virtual idol can be when carrying out multi-modal interact with user
The state of virtual idol is converted under the guidance of user.
After interactive program in smart machine 102 is activated, virtual idol enters halted state immediately.In pause shape
Under state, virtual idol is out of service.When there is the generation of activation event, virtual idol enters waiting recording state.The present invention's
In one embodiment, activation event can be that user 101 presses the button opened and wait for recording state, i.e., smart machine 102 can
Record button or virtually wait for record button to be waited for comprising entity, user 101 press entity wait for record button or
When virtual waiting record button, the state of virtual idol is converted into waiting recording state by halted state.In addition, it is necessary to explanation
It is that activation event can also be that other forms, the present invention do not make limitation to the activation form for activating event.
After virtual idol is converted into waiting recording state by halted state, if virtual idol detects that user speaks,
Then the state of virtual idol is converted into recording state by waiting recording state.Under recording state, virtual idol detection audio is defeated
Enter signal, starts ability or technical ability mould recording module in the block records audio input data.When virtual idol is in recording state
When, and when virtual idol detects words such as " goodbyes ", virtual idol is converted into standby mode by recording state.At this point, user 101
Show the wish for terminating this recording, conversion conditions wait for the next multimode of user 101 to virtual idol to standby mode immediately
State inputs.
If virtual idol be in recording state, and detects that user speaks stopping, then virtual idol is by recording state turn
Turn to waiting recording state.In addition, being switched to audio output state by recording state if necessary to virtual idol, virtual idol is first
Waiting recording state is first converted by recording state, then audio output state is converted by waiting recording state.When virtual idol
When in audio output state, user 101 can open a dialogue interaction with virtual idol, and virtual idol can play out and user
The interactive audio of 101 interactions, at the end of interactive audio plays, the state of virtual idol is converted into pause by audio output state
State.
In addition, when virtual idol is in standby, if user 101 sends out wake-up and is intended to or instructs, virtually
Idol is converted into waiting recording state by standby mode.Wake-up herein is intended to can be the specific audio that sends out of virtual idol with
And the particular organisms feature of specific limb action or user 101.
Technical ability open state is switched to by recording state if necessary to virtual idol, virtual idol is turned by recording state first
Waiting recording state is turned to, then technical ability open state is converted by waiting recording state.When virtual idol is in technical ability opening state
When state, call capability or technical ability mould a song and dance module in the block carry out a song and dance, and a song and dance is showed user
101。
When virtual idol is in technical ability open state, and virtual idol is sung and finishes or be interrupted, then virtual idol by
Technical ability open state is converted into waiting recording state.When virtual idol is in technical ability open state, and virtual idol sings beginning,
Then virtual idol is converted into standby mode by technical ability open state.
Fig. 5 shows the module frame chart of the system for managing state of virtual idol according to an embodiment of the invention.Such as
Shown in Fig. 5, system includes acquisition module 501, is intended to module 502, block of state 503 and technical ability module 504.Wherein, it obtains
Module 501 includes text collection unit 5011, audio collection unit 5012, vision collecting unit 5013 and perception collecting unit
5014。
Acquisition module 501 is for obtaining multi-modal input.Wherein, text collection unit 5011 is used for acquiring text message.
Audio collection unit 5012 is used for acquiring audio-frequency information.Vision collecting unit 5013 is used for acquiring visual information.Perception acquisition is single
Member 5014 is used for acquiring perception information.The example of acquisition module 501 includes keyboard, cursor control device (mouse), is used for voice
The microphone of operation, scanner, touch function (such as to detect the capacitance type transducers of physical touch), camera, sensing control
Equipment, such as use visible or nonvisible wavelength ray, signal, environmental data.Above-mentioned input equipment can be passed through
To obtain multi-modal input data.Multi-modal input can include one kind in text, audio, vision and perception data,
Can include a variety of, the present invention restricts not to this.
It is intended to module 502 to be used to parse intention or the operation in multi-modal input, to obtain the conversion for condition conversion
It is intended to or conversion instructs.It includes resolution unit 5021 to be intended to module 502, and resolution unit 5021 is used to parse multi-modal input, with
It obtains the conversion for including in multi-modal input and is intended to or converts instruction.Conversion is intended to or conversion instruction can be used in instructing virtual idol
As the conversion between various states.
Block of state 503 is used to convert the current state of virtual idol to the new shape for the virtual idol that conversion is intended to refer to
State.According to one embodiment of present invention, virtual idol includes various states, for example, dormant state, active state and waiting shape
State.Wherein, dormant state includes halted state and standby mode.Active state include recording state, audio output state and
Technical ability open state.Intermediate active state includes to wait for recording state.Block of state 503 includes conversion unit 5031, at one
In embodiment, conversion unit 5031 can convert the state of virtual idol to active state by dormant state, also can will be empty
The state of quasi- idol is converted into dormant state by active state.
Technical ability module 504 is for opening virtual idol required ability or technical ability module under new state.Technical ability module
504 include opening unit 5041, and after virtual idol is converted into new state, it is corresponding that opening unit 5041 opens new state immediately
The ability or technical ability of virtual idol.
Fig. 6 shows the flow chart of the method for state management of virtual idol according to an embodiment of the invention.
As shown in fig. 6, in step s 601, obtaining multi-modal input.In this step, smart machine 102 or holography are set
Standby 103 can obtain multi-modal input, and multi-modal input can be that user 101 inputs, and can also be its for having input function
What his equipment inputted.Smart machine 102 and hologram device 103 can be configured with the related device for obtaining multi-modal input.Multimode
State input can be the input of the forms such as text input, audio input and perception input.
Then, in step S602, the intention in multi-modal input or operation are parsed, to obtain turning for condition conversion
Change and is intended to or converts instruction.It is needed comprising much information in order to know the interaction intent information of user 101 in multi-modal input
The intention in multi-modal input or operation are parsed, obtains being intended to for the conversion of condition conversion according to intention or operation or conversion refers to
It enables.
Then, in step S603, the current state of virtual idol is converted to the new shape for the virtual idol being intended to refer to
State.According to one embodiment of present invention, virtual idol includes various states, for example, dormant state, active state and waiting shape
State.Wherein, dormant state includes halted state and standby mode.Active state include recording state, audio output state and
Technical ability open state.Intermediate active state includes to wait for recording state.Knowing for the conversion intention of condition conversion or conversion
After instruction, in this step it converts the current state of virtual idol to the new state for the virtual idol being intended to refer to.
Finally, virtual idol enters new state, in step s 604, opens virtual idol required energy under new state
Power or technical ability module.Each state of virtual idol includes ability or technical ability module under state.According to the present invention one
A embodiment, under a halt condition, virtual idol out of service；In the standby state, in the virtual idol of running background；It is recording
Under state, audio input signal is detected, starts ability or technical ability mould recording module in the block records audio input data；In audio
Under output state, call capability or technical ability mould language interactive module in the block engage in the dialogue interaction；Under technical ability open state, adjust
A song and dance is carried out with ability or technical ability mould a song and dance module in the block.
In addition, the system for managing state of virtual idol provided by the invention can also coordinate a kind of program product, it includes
Series of instructions for executing the method for state management step for completing virtual idol.
Fig. 7 shows another flow chart of the method for state management of virtual idol according to an embodiment of the invention.
As shown in fig. 7, in step s 701, smart machine 102 sends out request to high in the clouds brain 104.Later, in step
In S702, smart machine 102 is constantly in the state interacted with high in the clouds brain 104.In interactive process, 102 meeting of smart machine
Clocked operation is carried out to returned data the time it takes.
In step S703, if the reply data not returned for a long time, for example, being more than scheduled time span
5S, then smart machine 102 can select to carry out local reply, generate local common reply data.Then, defeated in step S704
Go out the animation with local common response cooperation, and voice playing equipment is called to carry out speech play.
Fig. 8 shows according to an embodiment of the invention in user, smart machine, hologram device and high in the clouds brain
The flow chart communicated between four directions.
In order to realize the multi-modal interaction between smart machine 102 and user 101, need user 101, smart machine 102,
Communication connection is set up between hologram device 103 and high in the clouds brain 104.This communication connection should be it is real-time, unobstructed,
It can ensure interactive impregnable.
In order to complete to interact, some conditions or premise are needed to have.These conditions or premise include smart machine
Virtual idol is loaded and run in 102, and smart machine 102 has the hardware facility of perception and control function.In addition, complete
Breath equipment 103 can receive the image of the virtual idol of the transmission of smart machine 102, and convert the image of virtual idol to holography
Hologram is included in predeterminable area by image.
Complete early-stage preparations after, smart machine 102 start with user 101 be unfolded interact, first, smart machine 102 and/or
Hologram device 103 obtains multi-modal input, and multi-modal input can be that user 101 sends out, and can also be that miscellaneous equipment is sent out
's.At this point, two sides that expanding data transmits are user 101 and smart machine 102 and/or hologram device 103.Then, it parses more
Intention in mode input or operation, to obtain instructing for the conversion intention of condition conversion or conversion.
Then, when virtual idol is in wait state, smart machine 102 sends to high in the clouds brain 104 and asks, high in the clouds brain
104 according to the analysis result of multi-modal input come determine the state to be entered be audio output state or technical ability open state,
And after entering audio output state or technical ability open state, high in the clouds brain 104 replys smart machine 102, empty described in decision
The multi-modal output of quasi- idol.At this point, two sides of expansion communication are smart machine 102 and high in the clouds brain 104.
It is transported after smart machine 102 receives data and the instruction of the transmission of high in the clouds brain 104 or in smart machine 102
After the current state of capable virtual idol is converted into the new state for the virtual idol that conversion is intended to refer to, smart machine 102 can incite somebody to action
The display data of the vivid and virtual idol current state of virtual idol is transmitted to hologram device 103.Hologram device 103 can incite somebody to action
The image of virtual idol is converted to hologram, and the hologram by virtual idol includes in the preset areas of hologram device 103
Domain.At this point, two sides of expansion communication are smart machine 102 and hologram device 103.
Finally, hologram device 103 can be by the display data of the hologram of virtual idol and virtual idol current state
Output, shows user 101.Two sides that communication is unfolded at this time are hologram device 103 and user 101.
The method of state management and system of a kind of virtual idol provided by the invention provide a kind of virtual idol, Neng Goutong
The mode for crossing holographic imaging completes multi-modal interaction with user.In addition, the condition managing system of virtual idol provided by the invention
Virtual idol in system also includes various states, for example, halted state, audio output state, waiting recording state, recording shape
State, standby mode and technical ability open state, and the present invention can also be managed the state of virtual idol, improve use
The interactive experience at family.
It should be understood that disclosed embodiment of this invention is not limited to specific structure disclosed herein, processing step
Or material, and the equivalent substitute for these features that those of ordinary skill in the related art are understood should be extended to.It should also manage
Solution, term as used herein is used only for the purpose of describing specific embodiments, and is not intended to limit.
" one embodiment " or " embodiment " mentioned in specification means the special characteristic described in conjunction with the embodiments, structure
Or characteristic includes at least one embodiment of the present invention.Therefore, the phrase " reality that specification various places throughout occurs
Apply example " or " embodiment " the same embodiment might not be referred both to.
While it is disclosed that embodiment content as above but described only to facilitate understanding the present invention and adopting
Embodiment is not limited to the present invention.Any those skilled in the art to which this invention pertains are not departing from this
Under the premise of the disclosed spirit and scope of invention, any modification and change can be made in the implementing form and in details,
But the scope of patent protection of the present invention, still should be subject to the scope of the claims as defined in the appended claims.
Claims (9)
1. a kind of method of state management of virtual idol, which is characterized in that the virtual idol has specific image characteristics, and
It is demonstrated out by hologram device, the method comprises the steps of：
Obtain multi-modal input；
The intention in the multi-modal input or operation are parsed, to obtain instructing for the conversion intention of condition conversion or conversion；
Convert the current state of the virtual idol to the virtual idol that the conversion is intended to or conversion instruction indicates
New state；
The new state includes：Open the virtual idol required ability or technical ability module under the new state.
2. the method for state management of virtual idol as described in claim 1, which is characterized in that the state of the virtual idol point
For dormant state, active state and wait state, wherein
Dormant state includes：Halted state and standby mode；
Active state includes：Recording state, audio output state and technical ability open state；
Under a halt condition, the virtual idol out of service；
In the standby state, the virtual idol described in running background；
Under recording state, multi-modal output before stopping starts to detect audio signal；
Under audio output state, the ability or technical ability mould language interactive module in the block is called to engage in the dialogue interaction；
Under technical ability open state, the ability or technical ability mould a song and dance module in the block is called to carry out a song and dance.
3. the method for state management of virtual idol as claimed in claim 2, which is characterized in that wait state shape for before
Waiting recording state at the end of state.
4. the method for state management of the virtual idol as described in claim 1-3, which is characterized in that
Under the waiting recording state, the analysis result of the multi-modal input is entered to determine in conjunction with high in the clouds brain
State is audio output state or technical ability open state, and after entering audio output state or technical ability open state, is led to
The feedback crossed in conjunction with the high in the clouds brain carrys out the multi-modal output that executive capability or technical ability module are opened.
5. the method for state management of the virtual idol such as claim 2-3, which is characterized in that
Under any type active state, if the task processing under detecting current state terminates and is not detected to appoint
When what multi-modal input data, current state is converted to standby mode or halted state into dormant state.
6. the method for state management of virtual idol as described in any of claims 5, which is characterized in that the active state
In recording state highest priority, virtual idol be waited for i.e. wait for recording state under, acquire user speech
So that virtual idol enters recording state.
7. a kind of program product, it includes for executing a series of of the method and step as described in any one of claim 1-6
Instruction.
8. a kind of virtual idol, which is characterized in that the virtual idol has specific virtual image and preset attribute, using such as
Method described in claim 1-6 executes the condition conversion process of the virtual idol.
9. a kind of system for managing state of virtual idol, which is characterized in that the system includes：
Smart machine is mounted with virtual idol as claimed in claim 8 thereon, for obtaining multi-modal input, and has certainly
Right language understanding, visual perception, the ability for touching perception, language voice output, emotional facial expressions action output；
Hologram device is used to obtain multi-modal input and converts the image of virtual idol as claimed in claim 8 to
Hologram simultaneously shows the hologram；
High in the clouds brain, is used in the wait state, to be entered to determine according to analysis result to the multi-modal input
State is audio output state or technical ability open state, and after entering audio output state or technical ability open state, certainly
The multi-modal output of plan virtual idol as claimed in claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810032045.5A CN108388399B (en) | 2018-01-12 | 2018-01-12 | Virtual idol state management method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810032045.5A CN108388399B (en) | 2018-01-12 | 2018-01-12 | Virtual idol state management method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108388399A true CN108388399A (en) | 2018-08-10 |
| CN108388399B CN108388399B (en) | 2021-04-06 |
Family
ID=63076699
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810032045.5A Active CN108388399B (en) | 2018-01-12 | 2018-01-12 | Virtual idol state management method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108388399B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110850971A (en) * | 2019-10-25 | 2020-02-28 | 智亮君 | Handshake interaction method and system between hand model and intelligent mirror and storage medium |
| CN111290682A (en) * | 2018-12-06 | 2020-06-16 | 阿里巴巴集团控股有限公司 | Interaction method and device and computer equipment |
| CN113362263A (en) * | 2021-05-27 | 2021-09-07 | 百度在线网络技术（北京）有限公司 | Method, apparatus, medium, and program product for changing the image of a virtual idol |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016206643A1 (en) * | 2015-06-26 | 2016-12-29 | 北京贝虎机器人技术有限公司 | Method and device for controlling interactive behavior of robot and robot thereof |
| CN106863319A (en) * | 2017-01-17 | 2017-06-20 | 北京光年无限科技有限公司 | A kind of robot awakening method and device |
| CN107197384A (en) * | 2017-05-27 | 2017-09-22 | 北京光年无限科技有限公司 | The multi-modal exchange method of virtual robot and system applied to net cast platform |
| CN107294837A (en) * | 2017-05-22 | 2017-10-24 | 北京光年无限科技有限公司 | Engaged in the dialogue interactive method and system using virtual robot |
| CN107340859A (en) * | 2017-06-14 | 2017-11-10 | 北京光年无限科技有限公司 | The multi-modal exchange method and system of multi-modal virtual robot |
| CN107340865A (en) * | 2017-06-29 | 2017-11-10 | 北京光年无限科技有限公司 | Multi-modal virtual robot exchange method and system |
-
2018
- 2018-01-12 CN CN201810032045.5A patent/CN108388399B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016206643A1 (en) * | 2015-06-26 | 2016-12-29 | 北京贝虎机器人技术有限公司 | Method and device for controlling interactive behavior of robot and robot thereof |
| CN106863319A (en) * | 2017-01-17 | 2017-06-20 | 北京光年无限科技有限公司 | A kind of robot awakening method and device |
| CN107294837A (en) * | 2017-05-22 | 2017-10-24 | 北京光年无限科技有限公司 | Engaged in the dialogue interactive method and system using virtual robot |
| CN107197384A (en) * | 2017-05-27 | 2017-09-22 | 北京光年无限科技有限公司 | The multi-modal exchange method of virtual robot and system applied to net cast platform |
| CN107340859A (en) * | 2017-06-14 | 2017-11-10 | 北京光年无限科技有限公司 | The multi-modal exchange method and system of multi-modal virtual robot |
| CN107340865A (en) * | 2017-06-29 | 2017-11-10 | 北京光年无限科技有限公司 | Multi-modal virtual robot exchange method and system |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111290682A (en) * | 2018-12-06 | 2020-06-16 | 阿里巴巴集团控股有限公司 | Interaction method and device and computer equipment |
| CN110850971A (en) * | 2019-10-25 | 2020-02-28 | 智亮君 | Handshake interaction method and system between hand model and intelligent mirror and storage medium |
| CN113362263A (en) * | 2021-05-27 | 2021-09-07 | 百度在线网络技术（北京）有限公司 | Method, apparatus, medium, and program product for changing the image of a virtual idol |
| CN113362263B (en) * | 2021-05-27 | 2023-09-15 | 百度在线网络技术（北京）有限公司 | Method, apparatus, medium and program product for transforming an image of a virtual idol |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108388399B (en) | 2021-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107704169B (en) | Virtual human state management method and system | |
| CN109271018A (en) | Exchange method and system based on visual human's behavioral standard | |
| CN107632706B (en) | Application data processing method and system of multi-modal virtual human | |
| CN110400251A (en) | Method for processing video frequency, device, terminal device and storage medium | |
| CN107294837A (en) | Engaged in the dialogue interactive method and system using virtual robot | |
| CN107340865A (en) | Multi-modal virtual robot exchange method and system | |
| CN105141587B (en) | Virtual doll interaction method and device | |
| CN107329990A (en) | A kind of mood output intent and dialogue interactive system for virtual robot | |
| CN109324688A (en) | Exchange method and system based on visual human's behavioral standard | |
| CN107808191A (en) | The output intent and system of the multi-modal interaction of visual human | |
| CN106796789A (en) | Interacted with the speech that cooperates with of speech reference point | |
| WO2018230160A1 (en) | Information processing system, information processing method, and program | |
| CN109343695A (en) | Exchange method and system based on visual human's behavioral standard | |
| WO2018006374A1 (en) | Function recommending method, system, and robot based on automatic wake-up | |
| CN108416420A (en) | Limbs exchange method based on visual human and system | |
| CN108388399A (en) | The method of state management and system of virtual idol | |
| CN107888965A (en) | Image present methods of exhibiting and device, terminal, system, storage medium | |
| CN107784355A (en) | The multi-modal interaction data processing method of visual human and system | |
| CN108595012A (en) | Visual interactive method and system based on visual human | |
| CN113703585A (en) | Interaction method, interaction device, electronic equipment and storage medium | |
| CN108681398A (en) | Visual interactive method and system based on visual human | |
| CN108415561A (en) | Gesture interaction method based on visual human and system | |
| WO2022121592A1 (en) | Livestreaming interaction method and apparatus | |
| CN113409805B (en) | Man-machine interaction method and device, storage medium and terminal equipment | |
| CN108646918A (en) | Visual interactive method and system based on visual human |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20230927 Address after: 100000 6198, Floor 6, Building 4, Yard 49, Badachu Road, Shijingshan District, Beijing Patentee after: Beijing Virtual Dynamic Technology Co.,Ltd. Address before: 100000 Fourth Floor Ivy League Youth Venture Studio No. 193, Yuquan Building, No. 3 Shijingshan Road, Shijingshan District, Beijing Patentee before: Beijing Guangnian Infinite Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right |