WO2017114048A1 - Mobile terminal and method for identifying contact - Google Patents

Mobile terminal and method for identifying contact Download PDF

Info

Publication number
WO2017114048A1
WO2017114048A1 PCT/CN2016/107329 CN2016107329W WO2017114048A1 WO 2017114048 A1 WO2017114048 A1 WO 2017114048A1 CN 2016107329 W CN2016107329 W CN 2016107329W WO 2017114048 A1 WO2017114048 A1 WO 2017114048A1
Authority
WO
WIPO (PCT)
Prior art keywords
contact
identified
data
information
mobile terminal
Prior art date
Application number
PCT/CN2016/107329
Other languages
French (fr)
Chinese (zh)
Inventor
张圣杰
金蓉
Original Assignee
努比亚技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 努比亚技术有限公司 filed Critical 努比亚技术有限公司
Publication of WO2017114048A1 publication Critical patent/WO2017114048A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H04M1/72439—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for image or video messaging
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/26—Devices for calling a subscriber
    • H04M1/27—Devices whereby a plurality of signals may be stored simultaneously
    • H04M1/274—Devices whereby a plurality of signals may be stored simultaneously with provision for storing more than one subscriber number at a time, e.g. using toothed disc
    • H04M1/2745—Devices whereby a plurality of signals may be stored simultaneously with provision for storing more than one subscriber number at a time, e.g. using toothed disc using static electronic memories, e.g. chips
    • H04M1/27467—Methods of retrieving data
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems
    • G10L13/02—Methods for producing synthetic speech; Speech synthesisers
    • G10L13/04—Details of speech synthesis systems, e.g. synthesiser structure or memory management
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/7243—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages
    • H04M1/72433—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality with interactive means for internal management of messages for voice messaging, e.g. dictaphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/26—Devices for calling a subscriber
    • H04M1/27—Devices whereby a plurality of signals may be stored simultaneously
    • H04M1/274—Devices whereby a plurality of signals may be stored simultaneously with provision for storing more than one subscriber number at a time, e.g. using toothed disc
    • H04M1/2745—Devices whereby a plurality of signals may be stored simultaneously with provision for storing more than one subscriber number at a time, e.g. using toothed disc using static electronic memories, e.g. chips
    • H04M1/27453—Directories allowing storage of additional subscriber data, e.g. metadata
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M2250/00—Details of telephonic subscriber devices
    • H04M2250/52—Details of telephonic subscriber devices including functional features of a camera

Definitions

  • This application relates to, but is not limited to, the field of communication technology.
  • the contact's photo and name are usually used to identify related contacts, thereby facilitating the user to query the contact information.
  • the identification of the contact is very simple and not vivid enough.
  • the user uses the contact to make a call or view the contact information, the user can only see the photo and name of the contact, and cannot see other information.
  • This paper proposes a mobile terminal and contact identification method to solve the technical problem that the identification method of contacts in the related art is very simple and not vivid.
  • a mobile terminal comprising:
  • Obtaining a module configured to: obtain a three-dimensional image and voiceprint feature information corresponding to the contact to be identified when receiving the contact identifier instruction;
  • the identifier module is configured to: identify, by using the three-dimensional image and voiceprint feature information acquired by the acquiring module, the contact to be identified;
  • the broadcast module is configured to: when viewing the contact information to be identified that is identified by the identifier module, perform a voice broadcast with the three-dimensional image and the voiceprint feature information of the contact to be identified.
  • the to-be-identified contact information includes: contact information to be identified in the contact application, or short message information sent by the contact to be identified in the short message application.
  • the broadcast module is configured to: view the identifier module
  • the voice broadcast of the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
  • the broadcasting module is configured to: view the identifier of the identifier module
  • the voice broadcast of the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
  • the broadcast module includes:
  • a synthesizing unit configured to: when viewing the contact information to be identified, determine a text to be broadcasted, and synthesize a standard pronunciation of the text to be broadcasted;
  • the modifying unit is configured to: modify the standard pronunciation synthesized by the synthesizing unit according to the voiceprint feature information of the to-be-identified contact, to obtain a sound waveform of the pronunciation feature of the contact to be identified;
  • an output unit configured to: output the sound waveform obtained by the modifying unit to perform voice broadcast, and display a three-dimensional image of the contact to be identified on a screen of the mobile terminal.
  • the acquiring module includes:
  • a shooting unit configured to: activate a panoramic shooting mode in a camera application of the mobile terminal to capture and store image data of the contact to be identified;
  • a first extracting unit configured to: extract, from the image data stored by the shooting unit, feature data for generating a three-dimensional image of the contact to be identified;
  • a generating unit configured to: according to the feature data extracted by the first extracting unit, activate a three-dimensional image engine to generate a three-dimensional image of the contact to be identified.
  • the photographing unit starts shooting of the mobile terminal
  • the panoramic shooting mode in the head application to capture and store the image data of the contact to be identified, including:
  • the first extracting unit includes:
  • the target stator unit is configured to: extract the portrait data in the image data stored by the photographing unit, and calibrate the portrait data of different orientations in the image data;
  • the method is configured to: acquire the feature data from the image data of the to-be-identified contact that is calibrated by the target stator unit in different orientations, where the feature data includes facial texture feature data and body proportion characteristics Data, hair styling data, wearing feature data, color characterization data.
  • the acquiring module further includes:
  • a copying unit configured to: when the call with the to-be-identified contact is detected, copy the received voice data of the to-be-identified contact, and use the copied voice data as the to-be-identified a sample of the voice of the contact;
  • the second extracting unit is configured to extract the voiceprint feature information from the sound samples obtained by the copying unit.
  • the copying unit is further configured to: when detecting that the recording function of the mobile terminal is enabled, copying the sound data of the to-be-identified contact transmitted by the microphone, and The copied sound data is used as a sound sample of the contact to be identified.
  • a contact identification method where the contact identification method includes:
  • the to-be-identified contact information includes: contact information to be identified in the contact application, or a short message sent by the contact to be identified in the short message application. information.
  • the viewing the to-be-identified contact identified by the identifier module is The information is broadcasted by the voice broadcast having the three-dimensional image and the voiceprint feature information of the contact to be identified, including:
  • the contact information to be identified When the contact information to be identified is viewed, the three-dimensional image of the contact to be identified is displayed on the screen of the mobile terminal, and the contact information to be identified is broadcasted by using the voiceprint feature information of the contact to be identified.
  • the viewing the identifier to be identified by the identifier module is When the contact information is obtained, a voice broadcast with the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
  • the voice broadcast having the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
  • the sound waveform is output for voice broadcast, and the three-dimensional image of the contact to be identified is displayed on the screen of the mobile terminal.
  • the obtaining a three-dimensional image corresponding to the contact to be identified includes:
  • a three-dimensional image engine is launched to generate a three-dimensional image of the contact to be identified.
  • the activating the panoramic shooting mode in the camera application of the mobile terminal to capture and store the image data of the to-be-identified contact includes:
  • the extracting the feature data for generating the three-dimensional image of the to-be-identified contact from the image data includes:
  • the feature data is acquired from the portrait data of the contact to be identified in different orientations, and the feature data includes face texture feature data, body scale feature data, hair style feature data, wearing feature data, and color feature data.
  • the acquiring the voiceprint feature corresponding to the to-be-identified contact includes:
  • Voiceprint feature information is extracted from the sound samples.
  • the obtaining the voiceprint feature corresponding to the contact to be identified further includes:
  • the sound data of the to-be-identified contact transmitted by the microphone is copied, and the copied sound data is used as the sound sample of the contact to be identified.
  • the mobile terminal and the contact identification method provided by the embodiment of the present invention when receiving a contact
  • the three-dimensional image and the voiceprint feature information of the contact to be identified are obtained, and the acquired three-dimensional image and the voiceprint feature information are used to identify the contact to be identified, thereby effectively solving the identification method of the contact in the related art. Very simple and not vivid enough.
  • the contact information of the to-be-identified contact information is broadcasted with the three-dimensional image of the contact to be identified and the contact voice feature information, thereby satisfying the increasing personality of the user. Demand, improve user experience and playability.
  • FIG. 1 is a schematic structural diagram of hardware of a mobile terminal according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram showing the electrical structure of a camera in the mobile terminal shown in FIG. 1;
  • FIG. 3 is a schematic structural diagram of a mobile terminal according to an embodiment of the present disclosure.
  • FIG. 4 is a schematic structural diagram of another mobile terminal according to an embodiment of the present disclosure.
  • FIG. 5 is a schematic structural diagram of still another mobile terminal according to an embodiment of the present disclosure.
  • FIG. 6 is a schematic structural diagram of a first extracting unit in a mobile terminal according to an embodiment of the present disclosure
  • FIG. 7 is a schematic structural diagram of still another mobile terminal according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of a panoramic shooting process of a mobile terminal according to an embodiment of the present invention.
  • FIG. 9 is a schematic flowchart of a method for identifying a contact according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic flowchart diagram of another method for identifying a contact according to an embodiment of the present disclosure.
  • FIG. 11 is a schematic flowchart diagram of still another method for identifying a contact according to an embodiment of the present disclosure.
  • FIG. 12 is a schematic flowchart of extracting feature data for generating a three-dimensional image of a contact to be identified in a method for identifying a contact according to an embodiment of the present disclosure
  • FIG. 13 is a schematic flowchart diagram of still another method for identifying a contact according to an embodiment of the present invention.
  • the mobile terminal can be implemented in various forms.
  • the terminal described in the present invention may include, for example, a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet), a PMP (Portable Multimedia Player), a navigation device, etc.
  • Mobile terminals and fixed terminals such as digital TVs, desktop computers, and the like.
  • the terminal is a mobile terminal.
  • those skilled in the art will appreciate that configurations in accordance with embodiments of the present invention can be applied to fixed type terminals in addition to components that are specifically for mobile purposes.
  • FIG. 1 is a schematic structural diagram of hardware of a mobile terminal according to an embodiment of the present invention.
  • the mobile terminal 100 may include a wireless communication unit 110, an A/V (Audio/Video) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, and a controller 170.
  • Figure 1 illustrates a mobile terminal having various components, but it should be understood that not all illustrated components are required to be implemented. More or fewer components can be implemented instead. The elements of the mobile terminal will be described in detail below.
  • the mobile communication module 111 transmits radio signals to and/or receives radio signals from at least one of a base station (e.g., an access point, a Node B, etc.), an external terminal, and a server.
  • a base station e.g., an access point, a Node B, etc.
  • Such radio signals may include voice call signals, video call signals, or various types of data transmitted and/or received in accordance with text and/or multimedia messages.
  • the A/V input unit 120 is for receiving an audio or video signal.
  • the A/V input unit 120 may include a camera 121 and a microphone 122, the camera 121 being imaged in a video capture mode or an image capture mode
  • the image data of the still picture or video obtained by the capture device is processed.
  • the processed image frame can be displayed on the display module 151.
  • the image frames processed by the camera 121 may be stored in the memory 160 (or other storage medium) or transmitted via the wireless communication unit 110, and two or more cameras 121 may be provided according to the configuration of the mobile terminal.
  • the microphone 122 can receive sound (audio data) via a microphone in an operation mode of a telephone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound as audio data.
  • the processed audio (voice) data can be converted to a format output that can be transmitted to the mobile communication base station via the mobile communication module 112 in the case of a telephone call mode.
  • the microphone 122 can implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated during the process of receiving and transmitting audio signals.
  • the user input unit 130 may generate key input data according to a command input by the user to control various operations of the mobile terminal.
  • the user input unit 130 allows the user to input various types of information, and may include a keyboard, a pot, a touch pad (eg, a touch sensitive component that detects changes in resistance, pressure, capacitance, etc. due to contact), a scroll wheel , rocker, etc.
  • a touch screen can be formed.
  • the sensing unit 140 detects the current state of the mobile terminal 100 (eg, the open or closed state of the mobile terminal 100), the location of the mobile terminal 100, the presence or absence of contact (ie, touch input) by the user with the mobile terminal 100, and the mobile terminal.
  • the sensing unit 140 can sense whether the slide type phone is turned on or off.
  • the sensing unit 140 includes an accelerometer 141 for detecting real-time acceleration of the mobile terminal 100 to derive a moving direction of the mobile terminal 100, and a gyroscope 142 for detecting the mobile terminal 100 with respect to a plane thereof slope.
  • Output unit 150 is configured to provide an output signal (eg, an audio signal, a video signal, an alarm signal, a vibration signal, etc.) in a visual, audio, and/or tactile manner.
  • the output unit 150 may include a display module 151, an audio output module 152, an alarm module 153, and the like.
  • the display module 151 can display information processed in the mobile terminal 100. For example, when the mobile terminal 100 is in a phone call mode, the display module 151 can display a user interface (UI) or graphical user interface (GUI) associated with a call or other communication (eg, text messaging, multimedia file download, etc.). When the mobile terminal 100 is in a video call mode or an image capture mode, the display module 151 A captured image and/or a received image, a UI or GUI showing a video or image and related functions, and the like can be displayed.
  • UI user interface
  • GUI graphical user interface
  • the display module 151 can function as an input device and an output device.
  • the display module 151 may include at least one of a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), an organic light emitting diode (OLED) display, a flexible display, a three-dimensional (3D) display, and the like.
  • LCD liquid crystal display
  • TFT-LCD thin film transistor LCD
  • OLED organic light emitting diode
  • a flexible display a three-dimensional (3D) display, and the like.
  • 3D three-dimensional
  • Some of these displays may be configured to be transparent to allow a user to view from the outside, which may be referred to as a transparent display, and a typical transparent display may be, for example, a TOLED (Transparent Organic Light Emitting Diode) display or the like.
  • TOLED Transparent Organic Light Emitting Diode
  • the mobile terminal 100 may include two or more display units (or other display devices), for example, the mobile terminal may include an external display unit (not shown) and an internal display unit (not shown) .
  • the touch screen can be used to detect touch input pressure as well as touch input position and touch input area.
  • the audio output module 152 may convert audio data received by the wireless communication unit 110 or stored in the memory 160 when the mobile terminal is in a call signal receiving mode, a call mode, a recording mode, a voice recognition mode, a broadcast receiving mode, and the like.
  • the audio signal is output as sound.
  • the audio output module 152 can provide audio output (eg, call signal reception sound, message reception sound, etc.) associated with a particular function performed by the mobile terminal 100.
  • the audio output module 152 can include a pickup, a buzzer, and the like.
  • the alert module 153 can provide an output to notify the mobile terminal 100 of the occurrence of an event. Typical events may include call reception, message reception, key signal input, touch input, and the like. In addition to audio or video output, the alert module 153 can provide an output in a different manner to notify of the occurrence of an event. For example, the alarm module 153 can provide an output in the form of vibrations that, when receiving a call, message, or some other incoming communication, can provide a haptic output (ie, vibration) to notify the user of it. By providing such a tactile output, the user is able to recognize the occurrence of various events even when the user's mobile phone is in the user's pocket. The alarm module 153 can also provide an output of the notification event occurrence via the display module 151 or the audio output module 152.
  • a haptic output ie, vibration
  • the memory 160 may store a software program or the like for processing and control operations performed by the controller 180, or may temporarily store data (for example, a phone book, a message, a still image, a video, etc.) that has been output or is to be output. Moreover, the memory 160 can store information about when a touch is applied to the touch screen When outputting various ways of vibration and audio signal data.
  • the memory 160 may include at least one type of storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (eg, SD or DX memory, etc.), a random access memory (RAM), a static random access memory ( SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disk, optical disk, and the like.
  • the mobile terminal 100 can cooperate with a network storage device that performs a storage function of the memory 160 through a network connection.
  • the controller 180 typically controls the overall operation of the mobile terminal. For example, the controller 180 performs the control and processing associated with voice calls, data communications, video calls, and the like.
  • the controller 180 may include a multimedia module 181 for reproducing (or playing back) multimedia data, which may be constructed within the controller 180 or may be configured to be separate from the controller 180.
  • the controller 180 may perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as a character or an image.
  • the various embodiments described herein can be implemented in a computer readable medium using, for example, computer software, hardware, or any combination thereof.
  • the embodiments described herein may be through the use of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays ( An FPGA, a processor, a controller, a microcontroller, a microprocessor, at least one of the electronic units designed to perform the functions described herein, in some cases, such an embodiment may be at the controller 180 Implemented in the middle.
  • implementations such as procedures or functions may be implemented with separate software modules that permit the execution of at least one function or operation.
  • the software code can be implemented by a software application (or program) written in any suitable programming language, which can be stored in memory 160 and executed by
  • the mobile terminal has been described in terms of its function.
  • a slide type mobile terminal among various types of mobile terminals such as a folding type, a bar type, a swing type, a slide type mobile terminal, and the like will be described as an example. Therefore, the present invention can be applied to any type of mobile terminal, and is not limited to a slide type mobile terminal.
  • FIG. 2 is a schematic diagram showing the electrical structure of a camera in the mobile terminal shown in FIG. 1.
  • the photographic lens 1211 may include a plurality of optical lenses forming an object image, and the optical lens may be Think of a single focus lens or zoom lens.
  • the photographic lens 1211 is movable in the optical axis direction under the control of the lens driver 1221, and the lens driver 1221 controls the focus position of the photographic lens 1211 in accordance with a control signal from the lens driving control circuit 1222, and can also be controlled in the case of the zoom lens. Focus distance.
  • the lens drive control circuit 1222 performs drive control of the lens driver 1221 in accordance with a control command from the microcomputer 1217.
  • An imaging element 1212 is disposed on the optical axis of the photographic lens 1211 near the position of the subject image formed by the photographic lens 1211.
  • the imaging element 1212 is provided to image the subject image and acquire captured image data.
  • Photodiodes constituting each pixel are arranged two-dimensionally and in a matrix on the imaging element 1212. Each photodiode generates a photoelectric conversion current corresponding to the amount of received light, which is subjected to charge accumulation by a capacitor connected to each photodiode.
  • the front surface of each pixel is provided with a Bayer array of red, green, blue (abbreviation: RGB) color filters.
  • the imaging element 1212 is connected to an imaging circuit 1213 that performs charge accumulation control and image signal readout control in the imaging element 1212, and reduces the reset noise after the read image signal (for example, an analog image signal).
  • the shaping is performed, and the gain is increased to obtain an appropriate signal level.
  • the imaging circuit 1213 is connected to an analog-to-digital conversion (A/D) converter 1214 that performs analog-to-digital conversion on the analog image signal and outputs a digital image signal to the bus 1227 (hereinafter referred to as It is image data).
  • A/D analog-to-digital conversion
  • the bus 1227 is provided to transmit a transmission path of various data read or generated inside the camera.
  • the A/D converter 1214 is connected to the bus 1227, and an image processor 1215, a JPEG processor 1216, a microcomputer 1217, and a Synchronous Dynamic Random Access Memory (SDRAM) 1218 are connected.
  • a memory interface hereinafter referred to as a memory I/F
  • a liquid crystal display (LCD) driver 1220 a memory interface
  • the image processor 1215 performs output buffering (Output Buffer, abbreviated as: OB) subtraction processing, white balance adjustment, color matrix calculation, gamma conversion, color difference signal processing, noise removal processing, and the image data based on the output of the imaging element 1212.
  • OB output buffering
  • the JPEG processor 1216 compresses the image data read out from the SDRAM 1218 in accordance with the JPEG compression method when the image data is recorded on the recording medium 1225.
  • the JPEG processor 1216 is for The line image reproduction display is performed to decompress the JPEG image data.
  • the file recorded on the recording medium 1225 is read, and after the compression processing is performed in the JPEG processor 1216, the decompressed image data is temporarily stored in the SDRAM 1218 and displayed on the LCD 1226.
  • the JPEG method is adopted as the image compression/decompression method.
  • the compression/decompression method is not limited thereto, and other compression/decompression methods such as MPEG, TIFF, and H.264 may be used.
  • the microcomputer 1217 functions as a control unit of the entire camera, and collectively controls various processing sequences of the camera.
  • the microcomputer 1217 is connected to the operation unit 1223 and the flash memory 1224.
  • the operating unit 1223 includes, but is not limited to, a physical button or a virtual button, and the entity or virtual button may be a power button, a camera button, an edit button, a dynamic image button, a reproduction button, a menu button, a cross button, an OK button, a delete button, an enlarge button
  • the operation controls such as various input buttons and various input keys detect the operational state of these operation controls.
  • the detection result is output to the microcomputer 1217. Further, a touch panel is provided on the front surface of the LCD 1226 as a display, and the touch position of the user is detected, and the touch position is output to the microcomputer 1217.
  • the microcomputer 1217 executes various processing sequences corresponding to the user's operation in accordance with the detection result from the operation position of the operation unit 1223.
  • the flash memory 1224 stores programs for executing various processing sequences of the microcomputer 1217.
  • the microcomputer 1217 performs overall control of the camera in accordance with the program. Further, the flash memory 1224 stores various adjustment values of the camera, and the microcomputer 1217 reads out the adjustment value, and performs control of the camera in accordance with the adjustment value.
  • the SDRAM 1218 is provided as an electrically rewritable volatile memory that temporarily stores image data or the like.
  • the SDRAM 1218 temporarily stores image data output from the A/D converter 1214 and image data processed in the image processor 1215, the JPEG processor 1216, and the like.
  • the memory interface 1219 is connected to the recording medium 1225, and performs control for writing image data and a file header attached to the image data to the recording medium 1225 and reading out from the recording medium 1225.
  • the recording medium 1225 is, for example, a recording medium such as a memory card that can be detachably attached to the camera body.
  • the recording medium 1225 is not limited thereto, and may be a hard disk or the like built in the camera body.
  • the LCD driver 1210 is connected to the LCD 1226, and stores image data processed by the image processor 1215 in the SDRAM 1218.
  • the image data stored in the SDRAM 1218 is read and displayed on the LCD 1226, or the image data stored in the JPEG processor 1216 is compressed. Yu In the SDRAM 1218, when display is required, the JPEG processor 1216 reads the compressed image data of the SDRAM 1218, decompresses it, and displays the decompressed image data on the LCD 1226.
  • the LCD 1226 is configured to display an image on the back of the camera body.
  • the LCD 1226 may be an LCD, but is not limited thereto, and the LCD 1226 may be implemented by other display panels such as organic electroluminescence (EL), but is not limited thereto.
  • EL organic electroluminescence
  • the following embodiments of the mobile terminal provided by the embodiments of the present invention are based on the hardware structure of the mobile terminal and the electrical structure of the camera.
  • FIG. 3 is a schematic structural diagram of a mobile terminal according to an embodiment of the present invention.
  • the mobile terminal provided by the embodiment of the present invention may include: an obtaining module 10, an identifying module 20, and a broadcast module 30.
  • the obtaining module 10 is configured to: when receiving the contact identifier instruction, obtain the three-dimensional image and the voiceprint feature information corresponding to the contact to be identified.
  • the shortcut icon for identifying the contact may be provided on the mobile terminal, and the user may trigger the contact identification instruction based on the shortcut icon; or may be provided by the mobile terminal for identifying the contact.
  • the physical button is used by the user to trigger the contact identification instruction based on the physical button; or the virtual button for identifying the contact may be provided on the mobile terminal, for the user to trigger the contact identification instruction based on the virtual button.
  • the voiceprint feature information in the embodiment of the present invention may be a voice spectrum parameter, a linear prediction parameter, a wavelet feature parameter, and the like of the voice signal.
  • the acquisition module 10 When the acquisition module 10 receives the user-triggered contact identification instruction, the acquisition module 10 acquires the three-dimensional image and voiceprint feature information corresponding to the to-be-identified contact stored in the storage unit of the mobile terminal. For example, if the user triggers an instruction to identify the information of the contact person and the three-dimensional image and voiceprint feature information of Xiao Wang, the obtaining module 10 acquires the three-dimensional image of Xiao Wang pre-stored in the mobile terminal and the voiceprint feature information of Xiao Wang.
  • the obtaining module 10 obtains the three-dimensional image of Xiao Li Wang pre-stored in the mobile terminal and the sound of Xiao Wang. Pattern feature information.
  • the identifier module 20 is configured to: identify the contact to be identified by using the three-dimensional image and the voiceprint feature information acquired by the acquiring module 10.
  • the identifier module 20 may identify the contact to be identified, that is, establish a three-dimensional image and voiceprint of the contact to be identified and the contact to be identified. The mapping relationship of feature information.
  • the broadcast module 30 is configured to: when viewing the contact information to be identified identified by the identification module 20, perform a voice broadcast with the three-dimensional image and the voiceprint feature information of the contact to be identified.
  • the to-be-identified contact information that is identified by the identifier module 20 may be the contact information to be identified in the contact application, or the short message information sent by the contact to be identified in the short message application.
  • the voice broadcast of the three-dimensional image and the contact voiceprint feature information of the contact to be identified may be: when the voice broadcast is performed, the three-dimensional image of the contact to be identified is displayed on the screen of the mobile terminal, and the contact to be identified is used at the same time.
  • the voiceprint feature information is voice broadcasted to the text information to be broadcast.
  • the broadcast module 30 displays the three-dimensional image of the contact on the screen of the mobile terminal, and broadcasts the contact information using the voiceprint feature information of the contact, the contact information includes a contact phone number, The contact address, the contact work unit, and the like; for example, if the contact information to be identified is the short message information sent by the contact to be identified in the short message application, when the short message information sent by the contact to be identified is viewed, the broadcast module 30 A three-dimensional image of the contact that sent the short message is displayed on the screen of the mobile terminal, and the short message content is broadcasted using the voiceprint feature information of the contact.
  • the idea of the embodiment of the present invention can also be applied to applications such as a voice assistant.
  • a voice assistant when a user opens a voice assistant, the user can select a contact that the user chats, and when the contact is selected, the screen of the mobile terminal The three-dimensional image associated with the contact is displayed, and during the chat, the character of the voice assistant uses the voiceprint feature information associated with the contact to broadcast the content of the chat, so that the user seems to be chatting with the real contact. To improve the user experience.
  • the mobile terminal provided by the embodiment of the present invention sets the acquisition module to: obtain the three-dimensional image and the voiceprint feature information of the contact to be identified when the contact identifier instruction is received, and the identifier module is set as:
  • the three-dimensional image and the voiceprint feature information are used to identify the contact to be identified, thereby effectively solving the problem that the identification method of the contact in the related art is very simple and not vivid.
  • the broadcast module to: When the contact information to be identified is identified, the voice information of the three-dimensional image and the voiceprint feature information of the contact to be identified is performed on the identified contact information, thereby satisfying the increasing personalized demand of the user and improving the user experience and Playful.
  • the embodiment of the present invention further provides a mobile terminal, as shown in FIG. 4, which is a schematic structural diagram of another mobile terminal according to an embodiment of the present invention.
  • the broadcast module 30 may include: a synthesizing unit 31, a modifying unit 32, and an output unit 33.
  • the synthesizing unit 31 is configured to: when viewing the contact information to be identified, determine the text to be broadcasted, and synthesize the standard pronunciation of the text to be broadcasted;
  • the modifying unit 32 is configured to: modify the standard pronunciation synthesized by the synthesizing unit 31 according to the voiceprint feature information of the contact to be identified, to obtain a sound waveform of the pronunciation feature of the contact to be identified;
  • the output unit 33 is configured to: output the sound waveform obtained by the modification unit 32 for voice broadcast, and display the three-dimensional image of the contact to be identified on the screen of the mobile terminal.
  • the synthesizing unit 31 when viewing the contact information to be identified, the synthesizing unit 31 first determines the text to be broadcasted, and synthesizes the standard pronunciation of the text to be broadcasted; and then, according to the modification unit 32, according to the voiceprint of the contact to be identified
  • the feature information is modified to the standard pronunciation synthesized by the synthesizing unit 31 to obtain a sound waveform of the pronunciation feature of the contact to be identified; finally, the sound waveform obtained by the output unit 33 is outputted by the output unit 33 for voice broadcast, and at the same time on the screen of the mobile terminal.
  • the three-dimensional image of the contact to be identified is displayed, and the voice broadcast of the three-dimensional image and the voiceprint feature information is realized.
  • the sound waveform obtained by the modifying unit 32 may be sent to the audio output interface for broadcast.
  • the audio output interface may include, for example, a microphone (Microphone, MIC for short), a headset, and a high High Definition Multimedia Interface (HDMI), built-in speakers, etc.
  • the voice broadcast of the voiceprint feature information of the contact to be identified when the voice broadcast of the voiceprint feature information of the contact to be identified is performed, the three-dimensional image of the contact to be identified is simultaneously displayed, so that the process of the voice broadcast is more vivid and the user experience is improved.
  • the embodiment of the present invention further provides a mobile terminal, as shown in FIG. 5, which is another mobile terminal structure provided by an embodiment of the present invention.
  • FIG. 5 is another mobile terminal structure provided by an embodiment of the present invention.
  • the present embodiment is illustrated by taking the structure of the mobile terminal shown in FIG. 3 as an example.
  • the obtaining module 10 in this embodiment may include: a photographing unit 11, a first extracting unit 12, and a generating unit 13.
  • the shooting unit 11 is configured to: activate a panoramic shooting mode in the camera application of the mobile terminal to capture and store image data of the contact to be identified.
  • the panoramic shooting mode in the camera application is activated by the shooting unit 11, and the photographing object (ie, the contact to be identified) is focused, and then the circular motion is started around the photographing object along the same distance, clockwise or counterclockwise.
  • the image data of the 360 degree orientation of the photographing object is obtained by the camera, as shown in FIG. 8 , which is a schematic diagram of a panoramic photographing process of the mobile terminal provided by the embodiment of the present invention.
  • gravity sensor, attitude sensor, gyroscope can be used during the shooting process.
  • the shooting module 30 can also save the captured panoramic photo (ie, image data) to the storage unit of the mobile terminal, so as to facilitate subsequent generation processing of the three-dimensional image.
  • the first extracting unit 12 is configured to extract feature data for generating a three-dimensional image of the contact to be identified from the image data stored by the photographing unit 11.
  • the feature data in the embodiment of the present invention may include: facial texture of the contact to be identified, 360 degree appearance features of the hairstyle, skin color, hair color, pupil color, height, wearing, limb ratio, and the like.
  • the generating unit 13 is configured to: according to the feature data extracted by the first extracting unit 12, activate a three-dimensional image engine to generate a three-dimensional image of the contact to be identified.
  • the generating unit 13 may start the three-dimensional image engine to generate the to-be-identified contact according to the feature data extracted by the first extracting unit 12.
  • the three-dimensional image of people In an actual application, the generating unit 13 may perform the ascending dimension calculation on the extracted feature data, calculate the length, width, height, and limb ratio data of the overall character in the three-dimensional space to generate a preliminary character model, and then use the image from the panorama.
  • the feature data extracted in the corresponding image data such as facial feature data, hair styling feature data, wearing feature data, color feature data, etc., are used to render the generated character models one by one.
  • the generating unit 13 may further perform splicing processing on the image information of different orientations by using a panoramic stitching fusion technology, thereby finally generating a three-dimensional character image corresponding to the photographed contact to be identified.
  • the three-dimensional image may also be saved in the storage unit, so as to contact the three-dimensional image with the to-be-identified identifier when receiving the contact identification instruction. People are connected.
  • the three-dimensional image engine is a three-dimensional graphics engine developed for facilitating generation of a three-dimensional image in the embodiment of the present invention.
  • a three-dimensional graphics application is generally written on a microcomputer using a large development tool such as an Open Graphics Library (OpenGL) or a multimedia programming interface (Direct eXtension, referred to as DirectX), but the three-dimensional graphics are involved.
  • OpenGL Open Graphics Library
  • DirectX multimedia programming interface
  • 3D graphics development environment that encapsulates hardware operations and graphics algorithms, as well as an easy-to-use and feature-rich environment.
  • This 3D graphics development environment can be called 3D graphics. engine.
  • the three-dimensional graphics engine includes, for example, an Object-Oriented Graphics Rendering Engine (OGRE) engine, an Open Scene Graph (OSG) engine, and the like.
  • OGI Object-Oriented Graphics Rendering Engine
  • OSG Open Scene Graph
  • the mobile terminal performs extraction of feature data required for generating a three-dimensional image based on the panoramic photo of the contact to be identified, and then starts the three-dimensional image engine to generate a corresponding three-dimensional image according to the extracted feature data.
  • the embodiment of the invention can conveniently and quickly generate a three-dimensional image of the captured object, and is used to identify the corresponding contact, thereby satisfying the personalized use requirement of the user and improving the user experience.
  • the embodiment of the present invention further provides a mobile terminal, as shown in FIG. 6, which is a first extracting unit of the mobile terminal provided by the embodiment of the present invention.
  • the first extraction unit 12 may include: a target stator unit 121 and an acquisition sub-unit 122.
  • the standard stator unit 121 is configured to: extract the portrait data in the image data stored by the photographing unit 11 and calibrate the portrait data of different orientations in the image data;
  • the target stator unit 121 may first extract the portrait data of the contact to be identified from the image data separately. deal with.
  • an image edge detection algorithm may be used to distinguish the person from the background environment, and then the corresponding determined pixel edge is closed.
  • the image data is extracted to obtain the portrait data of the contact to be identified.
  • the target stator unit 121 can also perform one-to-one calibration on the portrait data in the image data of different orientations for distinguishing.
  • the human body image data may be calibrated by using a human body orientation detection algorithm, such as calibrating a human body orientation every 45 degrees with respect to the front side of the human object, and the orientation of the human object for the 360 degree orientation may be calibrated to eight orientations.
  • the feature data in the corresponding person object image in different orientations is mostly different, so the character object feature data extraction in different orientation directions can be performed.
  • the obtaining sub-unit 122 is configured to: obtain feature data from the portrait data of the contact to be identified, which is calibrated by the standard stator unit 121, in different orientations, the feature data includes facial texture feature data, body scale feature data, and hairstyle feature data. , wearing feature data, color feature data.
  • the facial image data feature is a very important distinguishing feature. Therefore, in the embodiment of the present invention, the obtaining sub-unit 122 can perform face detection on all image data in different orientations, and determine image data of the presence of the face. The position of the face image is located, and then the position of the key point of the face is located on the basis of the image, for example, including the center of the eye, the corner of the mouth, the bridge of the nose, etc., due to the different shooting distances and angles selected during the shooting, the corresponding The head size and angle orientation of the characters in the image are also different. Therefore, the face can be obtained by scaling and/or rotating and/or stretching to obtain a preset standard size normal face avatar. The extraction of facial region feature data is performed.
  • the manner of extracting the feature data of the face region is not limited, for example, The local Binary Patterns (LBP) algorithm may be used, or the feature extraction may be performed by using a Histogram of Oriented Gradient (HOG) algorithm or a Gabor filter algorithm.
  • LBP Local Binary Patterns
  • HOG Histogram of Oriented Gradient
  • the obtaining sub-unit 122 may also determine the head, the upper body, the lower body and the limbs in the different orientation image data. a region for extracting feature data (ie, feature data of the body ratio) of the length ratio of the head, the upper body, the lower body, and the limbs of the contact to be identified.
  • feature data ie, feature data of the body ratio
  • the head, upper body, lower body and limbs of the person can be distinguished from the portrait data according to the relative positions of different parts of the human body and the correlation function, and then the head and upper body are extracted according to the proportion of the occupied pixels. Characteristic data of the length ratio of the lower body and the limbs.
  • the obtaining sub-unit 122 may also extract hairstyle feature data, wearing feature data, color feature data, and the like in the different orientation-down portrait data.
  • edge detection and feature extraction may be combined to obtain 360 degree appearance feature data of the contact hairstyle to be identified; according to the upper body and the lower body region, feature detection of the contact of the identified contact is performed, thereby extracting the clothing
  • the feature data and the feature data such as the main print; optionally, the acquisition sub-unit 122 may also extract the color feature data of the contact person's hair color, skin color, pupil color, and wearing color.
  • more feature data may be extracted from the overall image data of the contact to be identified, for example, including the most distinctive facial feature data and body limbs. Proportional data, hair styling data, wearing feature data, color characterization data, etc., to provide users with a more playable three-dimensional character image.
  • the embodiment of the present invention further provides a mobile terminal, as shown in FIG. 7 , which is a schematic structural diagram of another mobile terminal according to an embodiment of the present invention.
  • the embodiment is shown on the basis of the mobile terminal shown in FIG. 5, and the obtaining module 10 further includes: a copying unit 14 and a second extracting unit 15.
  • the copying unit 14 is configured to: when detecting a call with the to-be-identified contact, copy the received voice data of the to-be-identified contact, and use the copied voice data as the to-be-identified contact Sound sample.
  • the copying unit 14 copies the voice data transmitted by the opposite end through the audio input interface, and uses the copied voice data as the to-be-identified contact.
  • the sound sample of the person is copied; or the sound data transmitted by the microphone is copied through the audio input interface when the recording function is detected, and the copied sound data is used as the sound sample of the contact to be identified.
  • a sound sample can be collected by a copying method at a pulse code modulation (PCM) audio interface, and a sound data is copied as a sound sample of the contact to be identified. Extraction of voiceprint feature information.
  • PCM pulse code modulation
  • the second extracting unit 15 is configured to extract the voiceprint feature information from the sound samples obtained by the copying unit 14.
  • the second extracting unit 15 extracts the voiceprint feature information from the sound samples obtained by the copying unit 14.
  • the technique for extracting the voiceprint feature information may be: a base audio spectrum and its contour, the energy of the pitch frame, the appearance frequency of the pitch formant and its trajectory; linear prediction cepstrum, line spectrum pair, autocorrelation and log area ratio, Mel Frequency Cepstrum Coefficient (MFCC), perceptual linear prediction, wavelet transform technology, etc.
  • the information may also be saved to the storage unit of the mobile terminal, when the mobile terminal receives the contact identification instruction.
  • the voiceprint feature information is associated with the corresponding contact.
  • the voiceprint feature information of the contact to be identified is collected, thereby providing a basis for realizing voice broadcast according to the voiceprint feature information corresponding to the voice character.
  • FIG. 9 is a schematic flowchart of a method for identifying a contact provided by an embodiment of the present invention.
  • the method for identifying a contact may include the following steps, that is, S10 to S30:
  • the shortcut for identifying the contact may be provided on the mobile terminal.
  • the voiceprint feature information in the embodiment of the present invention may be a voice spectrum parameter, a linear prediction parameter, a wavelet feature parameter, and the like of the voice signal.
  • the mobile terminal When the mobile terminal receives the user-triggered contact identification instruction, the mobile terminal acquires the three-dimensional image and voiceprint feature information corresponding to the to-be-identified contact stored in the mobile terminal storage unit. For example, if the user triggers an instruction to identify the information of the contact Xiao Wang and the three-dimensional image and the voiceprint feature information of Xiao Wang, the three-dimensional image of Xiao Wang pre-stored in the mobile terminal and the voiceprint feature information of Xiao Wang are obtained; for example, The user triggers an instruction to identify the information of the contact Xiao Li and Xiao Li's three-dimensional image and voiceprint feature information, and obtains the three-dimensional image of Xiao Li Wang pre-stored in the mobile terminal and the voiceprint feature information of Xiao Wang.
  • the contact to be identified is identified, that is, the mapping relationship between the three-dimensional image and the voiceprint feature information of the contact to be identified is established.
  • the to-be-identified contact information may be the contact information to be identified in the contact application, or the short message information sent by the contact to be identified in the short message application.
  • the voice broadcast having the three-dimensional image and the voiceprint feature information of the contact to be identified may be: displaying the three-dimensional image of the contact to be identified on the screen of the mobile terminal during the voice broadcast, and using the voice of the contact to be identified
  • the pattern information is used for voice broadcast of the text information to be broadcast.
  • the contact information to be identified is the contact information to be identified in the contact application
  • when viewing the contact information of the three-dimensional image and the voiceprint feature information of the contact to be identified for example, when viewing the contact information to be identified in the contact application
  • the contact information is displayed, the three-dimensional image of the contact is displayed on the screen of the mobile terminal, and the contact information is broadcasted by using the voiceprint feature information of the contact, the contact information includes a contact phone number and a contact address.
  • the contact work unit, etc. for example, if the contact information to be identified is the short message information sent by the contact to be identified in the short message application, then the short message sent by the contact to be identified is viewed.
  • the three-dimensional image of the contact that sent the short message is displayed on the screen of the mobile terminal, and the short message content of the contact is used to broadcast the short message content.
  • the idea of the embodiment of the present invention can also be applied to applications such as a voice assistant.
  • a voice assistant when a user opens a voice assistant, the user can select a contact that the user chats, and when the contact is selected, the screen of the mobile terminal The three-dimensional image associated with the contact is displayed, and during the chat, the character of the voice assistant uses the voiceprint feature information associated with the contact to broadcast the content of the chat, so that the user seems to be chatting with the real contact. To improve the user experience.
  • An embodiment of the present invention provides a method for identifying a contact, by acquiring a three-dimensional image and voiceprint feature information of a contact to be identified when receiving a contact identification instruction, and using the acquired three-dimensional image and voiceprint feature information
  • the identification of the contact is performed to effectively solve the problem that the identification method of the contact in the related art is very simple and not vivid.
  • the contact information of the to-be-identified contact information is broadcasted with the three-dimensional image and the voiceprint feature information of the contact to be identified, thereby satisfying the increasing personalized demand of the user. Improve user experience and playability.
  • the embodiment of the present invention further provides a contact identification method, as shown in FIG. 10 , which is another contact identifier provided by the embodiment of the present invention.
  • FIG. 10 Schematic diagram of the process.
  • S30 may include the following steps, that is, S31 to S33:
  • S33 Output a sound waveform for voice broadcast, and display a three-dimensional image of the contact to be identified on a screen of the mobile terminal.
  • the text to be broadcast is first determined, and the standard pronunciation of the text to be broadcasted is synthesized; and then, according to the voiceprint characteristic letter of the contact to be identified
  • the standard pronunciation is modified to obtain the sound waveform of the pronunciation feature of the contact to be identified; finally, the sound waveform is output for voice broadcast, and the three-dimensional image of the contact to be identified is displayed on the screen of the mobile terminal to realize the three-dimensional image and Voice broadcast of voiceprint feature information.
  • the sound waveform may be sent to an audio output interface (such as a MIC, a headphone, an HDMI, a built-in speaker, etc.) for broadcast.
  • an audio output interface such as a MIC, a headphone, an HDMI, a built-in speaker, etc.
  • the voice broadcast of the voiceprint feature information of the contact to be identified when the voice broadcast of the voiceprint feature information of the contact to be identified is performed, the three-dimensional image of the contact to be identified is simultaneously displayed, so that the process of the voice broadcast is more vivid and the user experience is improved.
  • the embodiment of the present invention further provides a method for identifying a contact, as shown in FIG. A schematic diagram of the process of the contact identification method.
  • This embodiment is shown as an example on the basis of the embodiment shown in FIG. 9.
  • S10 in this embodiment may include the following steps, namely, S11 to S13:
  • the photographing object ie, the contact to be identified
  • the circular motion is started around the photographing object along the same distance, clockwise or counterclockwise, until
  • the camera acquires the image data of the 360-degree orientation of the photographed object, and also refers to the photographing process diagram shown in FIG.
  • gravity sensor, attitude sensor, gyroscope can be used during the shooting process.
  • the captured panoramic photo ie, image data
  • the captured panoramic photo can also be saved into the storage unit of the mobile terminal, so as to facilitate subsequent generation processing of the three-dimensional image.
  • the feature data required for the 3D image of the contact may include: facial texture of the contact to be identified, 360 degree appearance features of the hairstyle, skin color, hair color, pupil color, height, wearing, limb ratio, and the like.
  • the three-dimensional image engine is activated to generate a three-dimensional image of the contact to be identified.
  • the extracted feature data is subjected to the ascending dimension calculation, and the length, width, height and limb ratio data of the overall character in the three-dimensional space are calculated to generate a preliminary character model, and then the corresponding image from the panoramic image is used.
  • the feature data extracted in the data such as facial feature data, hair styling feature data, wearing feature data, color feature data, etc., are used to render the generated character models one by one.
  • the panoramic splicing fusion technology may also be used to splicing the image information of different orientations, thereby finally generating a three-dimensional character image corresponding to the captured contact to be identified.
  • the three-dimensional image may also be saved to the storage unit, so that when the mobile terminal receives the contact identification instruction, the three-dimensional image and the to-be-identified contact are Linked up.
  • the three-dimensional image engine is a three-dimensional graphics engine developed for facilitating generation of a three-dimensional image in the embodiment of the present invention.
  • a three-dimensional graphics application is generally written on a microcomputer using a large development tool such as OpenGL or DirectX.
  • OpenGL or DirectX a large development tool
  • the three-dimensional graphics involve many algorithms and professional knowledge, it is still difficult to rapidly develop a three-dimensional application. Therefore, the development of 3D applications requires a three-dimensional graphics development environment that encapsulates hardware operations and graphics algorithms, as well as an easy-to-use and feature-rich environment.
  • This three-dimensional graphics development environment can be called a three-dimensional graphics engine.
  • the three-dimensional graphics engine includes, for example, an OGRE engine, an OSG engine, and the like.
  • the mobile terminal performs extraction of feature data required for generating a three-dimensional image based on the panoramic photo of the contact to be identified, and then starts the three-dimensional image engine to generate a corresponding three-dimensional image according to the extracted feature data.
  • the embodiment of the invention can conveniently and quickly generate a three-dimensional image of the captured object, and is used to identify the corresponding contact, thereby satisfying the personalized use requirement of the user and improving the user experience.
  • the embodiment of the present invention further provides a method for identifying a contact, as shown in FIG. 12 , which is a method for identifying a contact provided by an embodiment of the present invention.
  • FIG. 12 A flow diagram for extracting feature data for generating a three-dimensional image of a contact to be identified.
  • the step S12 includes the following steps, that is, S121 to S122:
  • the portrait data of the contact to be identified may first be extracted from the image data and processed separately.
  • an image edge detection algorithm may be used to distinguish the person from the background environment, and then the corresponding determined pixel edge is closed.
  • the image data is extracted to obtain the portrait data of the contact to be identified.
  • the image data since the image data includes image data of different orientations, it is also possible to perform one-to-one calibration of the portrait data in the image data of different orientations for distinguishing.
  • the human body image data may be calibrated by using a human body orientation detection algorithm, such as calibrating a human body orientation every 45 degrees with respect to the front side of the human object, and the orientation of the human object for the 360 degree orientation may be calibrated to eight orientations.
  • the feature data in the corresponding person object image in different orientations is mostly different, so the character object feature data extraction in different orientation directions can be performed.
  • S122 Acquire feature data from the portrait data of the contact to be identified in different orientations, where the feature data includes face texture feature data, body scale feature data, hair style feature data, wearing feature data, and color feature data.
  • the facial image data feature is a very important distinguishing feature. Therefore, in the embodiment of the present invention, face detection can be performed on all image data in different orientations, and the face in the image data of the face is determined. The position of the image is located, and then the position of the key point of the face is located on the basis of the image, for example, including the center of the eye, the corner of the mouth, the bridge of the nose, etc., due to the different shooting distances and angles selected during the shooting, the corresponding characters in the image The head size and the angle orientation are also different. Therefore, the face can be obtained by scaling and/or rotating and/or stretching to obtain a normal standard face avatar of a preset standard size. Extraction of regional feature data.
  • the method for extracting the feature data of the face region is not limited, for example, the LBP calculation may be adopted.
  • the method, or the HOG algorithm, the Gabor filter algorithm, etc. performs feature extraction of the image.
  • the head, the upper body, the lower body and the limbs in the different orientation image data may be determined to extract Characteristic data of the length ratio of the head, upper body, lower body, and limbs of the contact to be identified (ie, feature data of the body ratio).
  • the head, upper body, lower body and limbs of the person can be distinguished from the portrait data according to the relative positions of different parts of the human body and the correlation function, and then the head and upper body are extracted according to the proportion of the occupied pixels. Characteristic data of the length ratio of the lower body and the limbs.
  • hair style feature data, wearing feature data, color feature data, and the like in different orientation-oriented portrait data may also be extracted.
  • edge detection and feature extraction may be combined to obtain 360 degree appearance feature data of the contact hairstyle to be identified; according to the upper body and the lower body region, feature detection of the contact of the identified contact is performed, thereby extracting the clothing Characteristic data such as shape and main printing; optionally, color feature data such as hair color, skin color, pupil color, and wearing color of the contact person can be extracted.
  • more feature data may be extracted from the overall image data of the contact to be identified, for example, including the most distinctive facial feature data and body limbs. Proportional data, hair styling data, wearing feature data, color characterization data, etc., to provide users with a more playable three-dimensional character image.
  • the embodiment of the present invention further provides a contact identification method, as shown in FIG. 13 , which is another contact identification method provided by an embodiment of the present invention.
  • FIG. 13 Schematic diagram of the process. This embodiment is shown as an example on the basis of the embodiment shown in FIG. 11.
  • S10 in this embodiment may further include the following steps, namely, S14 to S15:
  • the voice data transmitted by the opposite end is copied through the audio input interface, and the copied sound data is used as the voice data.
  • the sound sample can be collected by copying at the PCM audio interface, and a voice data is copied as the sound sample of the contact to be identified to extract the voiceprint feature information.
  • the voiceprint feature information is extracted from the sound sample.
  • the technique for extracting the voiceprint feature information may be: a base audio spectrum and its contour, the energy of the pitch frame, the appearance frequency of the pitch formant and its trajectory; linear prediction cepstrum, line spectrum pair, autocorrelation and log area ratio, MFCC , perceptual linear prediction; wavelet transform technology.
  • the information may be saved to the storage unit of the mobile terminal, where the mobile terminal obtains the voiceprint feature when receiving the contact identifier instruction.
  • the information is associated with the corresponding contact.
  • the voiceprint feature information of the contact to be identified is collected, thereby providing a basis for realizing voice broadcast according to the voiceprint feature information corresponding to the voice character.
  • the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better.
  • Implementation Based on such understanding, the technical solution in the embodiments of the present invention may be embodied in the form of a software product in essence or in a contribution to the related art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic
  • the disc, the optical disc includes a number of instructions for causing a terminal device (which may be a cell phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods provided by various embodiments and alternative embodiments of the present invention.
  • all or part of the steps of the above embodiments may also be implemented by using an integrated circuit. These steps may be separately fabricated into individual integrated circuit modules, or multiple modules or steps may be fabricated into a single integrated circuit module. achieve.
  • the devices/function modules/functional units in the above embodiments may be implemented by a general-purpose computing device, which may be centralized on a single computing device or distributed over a network of multiple computing devices.
  • the device/function module/functional unit in the above embodiment When the device/function module/functional unit in the above embodiment is implemented in the form of a software function module and sold or used as a stand-alone product, it can be stored in a computer readable storage medium.
  • the above mentioned computer readable storage medium may be a read only memory, a magnetic disk or an optical disk or the like.
  • the embodiment When receiving the contact identifier instruction, the embodiment obtains the three-dimensional image and the voiceprint feature information of the contact to be identified, and uses the acquired three-dimensional image and voiceprint feature information to identify the contact to be identified, thereby effectively
  • the problem that the identification of the contacts in the related art is very simple and not vivid enough is solved.
  • the contact information of the to-be-identified contact information is broadcasted with the three-dimensional image and the voiceprint feature information of the contact to be identified, thereby satisfying the increasing personalized demand of the user. Improve user experience and playability.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Library & Information Science (AREA)
  • Telephone Function (AREA)

Abstract

A mobile terminal and a method for identifying a contact. The method for identifying a contact comprises: obtaining a three-dimensional image and voiceprint characteristic information that correspond to a contact to be identified when a contact identification instruction is received; identifying the contact to be identified by using the obtained three-dimensional image and voiceprint characteristic information; and performing a voice broadcast comprising the three-dimensional image and voiceprint characteristic information of the contact to be identified when information of the contact to be identified is viewed.

Description

移动终端及联系人标识方法Mobile terminal and contact identification method 技术领域Technical field
本申请涉及但不限于通信技术领域。This application relates to, but is not limited to, the field of communication technology.
背景技术Background technique
相关技术中，通常会使用联系人的照片以及名称来标识相关的联系人，从而方便用户查询联系人信息。相关技术中联系人的标识方式十分简单，且不够生动形象，用户在使用联系人进行通话或者查看联系人信息时，往往只能看到联系人的照片以及名称，不能看到其他信息。In the related art, the contact's photo and name are usually used to identify related contacts, thereby facilitating the user to query the contact information. In the related art, the identification of the contact is very simple and not vivid enough. When the user uses the contact to make a call or view the contact information, the user can only see the photo and name of the contact, and cannot see other information.
发明内容Summary of the invention
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.
本文提出一种移动终端及联系人标识方法，以解决相关技术中联系人的标识方式十分简单，且不够生动形象的技术问题。This paper proposes a mobile terminal and contact identification method to solve the technical problem that the identification method of contacts in the related art is very simple and not vivid.
一种移动终端，所述移动终端包括：A mobile terminal, the mobile terminal comprising:
获取模块，设置为：当接收到联系人标识指令时，获取待标识联系人对应的三维形象及声纹特征信息；Obtaining a module, configured to: obtain a three-dimensional image and voiceprint feature information corresponding to the contact to be identified when receiving the contact identifier instruction;
标识模块，设置为：采用所述获取模块获取的所述三维形象及声纹特征信息对所述待标识联系人进行标识；The identifier module is configured to: identify, by using the three-dimensional image and voiceprint feature information acquired by the acquiring module, the contact to be identified;
播报模块，设置为：在查看所述标识模块所标识的待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报。The broadcast module is configured to: when viewing the contact information to be identified that is identified by the identifier module, perform a voice broadcast with the three-dimensional image and the voiceprint feature information of the contact to be identified.
可选地，如上所述的移动终端中，所述待标识联系人信息包括：联系人应用中待标识联系人信息，或者，短信应用中待标识联系人发的短信信息。Optionally, in the mobile terminal, the to-be-identified contact information includes: contact information to be identified in the contact application, or short message information sent by the contact to be identified in the short message application.
可选地，如上所述的移动终端中，所述待标识联系人信息为所述联系人应用中待标识联系人信息时，所述播报模块设置为：在查看所述标识模块所 标识的待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报，包括：Optionally, in the mobile terminal as described above, when the to-be-identified contact information is the contact information to be identified in the contact application, the broadcast module is configured to: view the identifier module When the contact information to be identified is identified, the voice broadcast of the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
在查看所述待标识联系人信息时，在所述移动终端的屏幕上显示所述待标识联系人的三维形象，并使用所述待标识联系人的声纹特征信息播报所述待标识联系人信息。Displaying the three-dimensional image of the to-be-identified contact on the screen of the mobile terminal, and using the voiceprint feature information of the to-be-identified contact to broadcast the to-be-identified contact information.
可选地，如上所述的移动终端中，所述待标识联系人信息为所述短信应用中待标识联系人发的短信信息时，所述播报模块设置为：在查看所述标识模块所标识的待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报，包括：Optionally, in the mobile terminal as described above, when the to-be-identified contact information is the short message information sent by the to-be-identified contact in the short-message application, the broadcasting module is configured to: view the identifier of the identifier module When the contact information is to be identified, the voice broadcast of the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
在查看所述待标识联系人信息时，在所述移动终端的屏幕上显示发送短信的联系人的三维形象，并使用所述联系人的声纹特征信息播报所述短信的内容。When viewing the to-be-identified contact information, displaying a three-dimensional image of the contact that sends the short message on the screen of the mobile terminal, and using the voiceprint feature information of the contact to broadcast the content of the short message.
可选地，如上所述的移动终端中，所述播报模块包括：Optionally, in the mobile terminal as described above, the broadcast module includes:
合成单元，设置为：在查看所述待标识联系人信息时，确定待播报文本，并合成所述待播报文本的标准发音；a synthesizing unit, configured to: when viewing the contact information to be identified, determine a text to be broadcasted, and synthesize a standard pronunciation of the text to be broadcasted;
修饰单元，设置为：根据所述待标识联系人的声纹特征信息对所述合成单元合成的所述标准发音进行修饰，得到所述待标识联系人的发音特征的声音波形；The modifying unit is configured to: modify the standard pronunciation synthesized by the synthesizing unit according to the voiceprint feature information of the to-be-identified contact, to obtain a sound waveform of the pronunciation feature of the contact to be identified;
输出单元，设置为：输出所述修饰单元得到的所述声音波形进行语音播报，并在所述移动终端的屏幕上显示所述待标识联系人的三维形象。And an output unit, configured to: output the sound waveform obtained by the modifying unit to perform voice broadcast, and display a three-dimensional image of the contact to be identified on a screen of the mobile terminal.
可选地，如上所述的移动终端中，所述获取模块包括：Optionally, in the mobile terminal as described above, the acquiring module includes:
拍摄单元，设置为：启动所述移动终端的摄像头应用中的全景拍摄模式，以拍摄并存储所述待标识联系人的图像数据；a shooting unit, configured to: activate a panoramic shooting mode in a camera application of the mobile terminal to capture and store image data of the contact to be identified;
第一提取单元，设置为：从所述拍摄单元存储的所述图像数据中提取用于生成所述待标识联系人的三维形象的特征数据；a first extracting unit, configured to: extract, from the image data stored by the shooting unit, feature data for generating a three-dimensional image of the contact to be identified;
生成单元，设置为：根据所述第一提取单元所提取的所述特征数据，启动三维形象引擎以生成所述待标识联系人的三维形象。And a generating unit, configured to: according to the feature data extracted by the first extracting unit, activate a three-dimensional image engine to generate a three-dimensional image of the contact to be identified.
可选地，如上所述的移动终端中，所述拍摄单元启动所述移动终端的摄 像头应用中的全景拍摄模式，以拍摄并存储所述待标识联系人的图像数据，包括：Optionally, in the mobile terminal as described above, the photographing unit starts shooting of the mobile terminal The panoramic shooting mode in the head application to capture and store the image data of the contact to be identified, including:
启动所述移动终端的摄像头应用中的全景拍摄模式，对所述待标识联系人进行对焦后沿同一距离围绕所述待标识联系人进行圆周运动，直到所述摄像头获取到所述待标识联系人的360度方位的图像数据。Activating a panoramic shooting mode in the camera application of the mobile terminal, and performing focus motion on the to-be-identified contact along the same distance after the contact to be identified is focused until the camera acquires the to-be-identified contact 360-degree image data.
可选地，如上所述的移动终端中，第一提取单元包括：Optionally, in the mobile terminal as described above, the first extracting unit includes:
标定子单元，设置为：提取所述拍摄单元存储的所述图像数据中的人像数据，并对所述图像数据中不同朝向的人像数据进行标定；The target stator unit is configured to: extract the portrait data in the image data stored by the photographing unit, and calibrate the portrait data of different orientations in the image data;
获取子单元，设置为：从所述标定子单元标定的所述待标识联系人在不同朝向下的人像数据中获取所述特征数据，所述特征数据包括人脸纹理特征数据、身体比例的特征数据、发型特征数据、穿着特征数据、颜色特征数据。Obtaining the sub-unit, the method is configured to: acquire the feature data from the image data of the to-be-identified contact that is calibrated by the target stator unit in different orientations, where the feature data includes facial texture feature data and body proportion characteristics Data, hair styling data, wearing feature data, color characterization data.
可选地，如上所述的移动终端中，所述获取模块还包括：Optionally, in the mobile terminal as described above, the acquiring module further includes:
复制单元，设置为：在侦测到与所述待标识联系人的通话时，复制所接收到的所述待标识联系人的声音数据，并将复制得到的所述声音数据作为所述待标识联系人的声音样本；a copying unit, configured to: when the call with the to-be-identified contact is detected, copy the received voice data of the to-be-identified contact, and use the copied voice data as the to-be-identified a sample of the voice of the contact;
第二提取单元，设置为：从所述复制单元得到的所述声音样本中提取所述声纹特征信息。The second extracting unit is configured to extract the voiceprint feature information from the sound samples obtained by the copying unit.
可选地，如上所述的移动终端中，所述复制单元，还设置为：在侦测到所述移动终端的录音功能开启时，复制麦克风传输的所述待标识联系人的声音数据，并将复制得到的所述声音数据作为所述待标识联系人的声音样本。Optionally, in the mobile terminal as described above, the copying unit is further configured to: when detecting that the recording function of the mobile terminal is enabled, copying the sound data of the to-be-identified contact transmitted by the microphone, and The copied sound data is used as a sound sample of the contact to be identified.
一种联系人标识方法，所述联系人标识方法包括：A contact identification method, where the contact identification method includes:
当接收到联系人标识指令时，获取待标识联系人对应的三维形象及声纹特征信息；Obtaining a three-dimensional image and voiceprint feature information corresponding to the contact to be identified when receiving the contact identification instruction;
采用获取的所述三维形象及声纹特征信息对所述待标识联系人进行标识，以在查看所述待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报。And identifying the to-be-identified contact by using the acquired three-dimensional image and voiceprint feature information, to perform three-dimensional image and voiceprint feature information of the contact to be identified when viewing the to-be-identified contact information Voice broadcast.
可选地，如上所述的联系人标识方法中，所述待标识联系人信息包括：联系人应用中待标识联系人信息，或者，短信应用中待标识联系人发的短信 信息。Optionally, in the contact identification method, the to-be-identified contact information includes: contact information to be identified in the contact application, or a short message sent by the contact to be identified in the short message application. information.
可选地，如上所述的联系人标识方法中，所述待标识联系人信息为所述联系人应用中待标识联系人信息时，所述在查看所述标识模块所标识的待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报，包括：Optionally, in the contact identification method, as described above, when the to-be-identified contact information is the contact information to be identified in the contact application, the viewing the to-be-identified contact identified by the identifier module is The information is broadcasted by the voice broadcast having the three-dimensional image and the voiceprint feature information of the contact to be identified, including:
在查看所述待标识联系人信息时，在移动终端的屏幕上显示所述待标识联系人的三维形象，并使用所述待标识联系人的声纹特征信息播报所述待标识联系人信息。When the contact information to be identified is viewed, the three-dimensional image of the contact to be identified is displayed on the screen of the mobile terminal, and the contact information to be identified is broadcasted by using the voiceprint feature information of the contact to be identified.
可选地，如上所述的联系人标识方法中，所述待标识联系人信息为所述短信应用中待标识联系人发的短信信息时，所述在查看所述标识模块所标识的待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报，包括：Optionally, in the contact identification method, as described above, when the to-be-identified contact information is the short message information sent by the to-be-identified contact in the short-message application, the viewing the identifier to be identified by the identifier module is When the contact information is obtained, a voice broadcast with the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
在查看所述待标识联系人信息时，在移动终端的屏幕上显示发送短信的联系人的三维形象，并使用所述联系人的声纹特征信息播报所述短信的内容。When viewing the to-be-identified contact information, displaying a three-dimensional image of the contact that sends the short message on the screen of the mobile terminal, and using the voiceprint feature information of the contact to broadcast the content of the short message.
可选地，如上所述的联系人标识方法中，所述在查看所述待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报，包括：Optionally, in the contact identification method, as described above, when the contact information to be identified is viewed, the voice broadcast having the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
在查看所述待标识联系人信息时，确定待播报文本，并合成所述待播报文本的标准发音；When viewing the contact information to be identified, determining a text to be broadcasted, and synthesizing a standard pronunciation of the text to be broadcasted;
根据所述待标识联系人的声纹特征信息对所述标准发音进行修饰，得到所述待标识联系人的发音特征的声音波形；And modifying the standard pronunciation according to the voiceprint feature information of the to-be-identified contact, to obtain a sound waveform of the pronunciation feature of the contact to be identified;
输出所述声音波形进行语音播报，并在移动终端的屏幕上显示所述待标识联系人的三维形象。The sound waveform is output for voice broadcast, and the three-dimensional image of the contact to be identified is displayed on the screen of the mobile terminal.
可选地，如上所述的联系人标识方法中，所述获取待标识联系人对应的三维形象，包括：Optionally, in the method for identifying a contact as described above, the obtaining a three-dimensional image corresponding to the contact to be identified includes:
启动移动终端的摄像头应用中的全景拍摄模式，以拍摄并存储所述待标识联系人的图像数据； Activating a panoramic shooting mode in a camera application of the mobile terminal to capture and store image data of the contact to be identified;
从所述图像数据中提取用于生成所述待标识联系人的三维形象的特征数据；Extracting, from the image data, feature data for generating a three-dimensional image of the contact to be identified;
根据所提取的所述特征数据，启动三维形象引擎以生成所述待标识联系人的三维形象。Based on the extracted feature data, a three-dimensional image engine is launched to generate a three-dimensional image of the contact to be identified.
可选地，如上所述的联系人标识方法中，所述启动移动终端的摄像头应用中的全景拍摄模式，以拍摄并存储所述待标识联系人的图像数据，包括：Optionally, in the contact identification method as described above, the activating the panoramic shooting mode in the camera application of the mobile terminal to capture and store the image data of the to-be-identified contact includes:
启动所述移动终端的摄像头应用中的全景拍摄模式，对所述待标识联系人进行对焦后沿同一距离围绕所述待标识联系人进行圆周运动，直到所述摄像头获取到所述待标识联系人的360度方位的图像数据。Activating a panoramic shooting mode in the camera application of the mobile terminal, and performing focus motion on the to-be-identified contact along the same distance after the contact to be identified is focused until the camera acquires the to-be-identified contact 360-degree image data.
可选地，如上所述的联系人标识方法中，所述从所述图像数据中提取用于生成所述待标识联系人的三维形象的特征数据，包括：Optionally, in the contact identification method as described above, the extracting the feature data for generating the three-dimensional image of the to-be-identified contact from the image data includes:
提取所述图像数据中的人像数据，并对所述图像数据中不同朝向的人像数据进行标定；Extracting the portrait data in the image data, and calibrating the portrait data of different orientations in the image data;
从所述待标识联系人在不同朝向下的人像数据中获取所述特征数据，所述特征数据包括人脸纹理特征数据、身体比例的特征数据、发型特征数据、穿着特征数据、颜色特征数据。The feature data is acquired from the portrait data of the contact to be identified in different orientations, and the feature data includes face texture feature data, body scale feature data, hair style feature data, wearing feature data, and color feature data.
可选地，如上所述的联系人标识方法中，所述获取待标识联系人对应的声纹特征，包括：Optionally, in the contact identification method, the acquiring the voiceprint feature corresponding to the to-be-identified contact includes:
在侦测到与所述待标识联系人的通话时，复制所接收到的所述待标识联系人的声音数据，并将复制得到的所述声音数据作为所述待标识联系人的声音样本；And detecting the received voice data of the to-be-identified contact, and using the copied voice data as a sound sample of the to-be-identified contact;
从所述声音样本中提取声纹特征信息。Voiceprint feature information is extracted from the sound samples.
可选地，如上所述的联系人标识方法中，所述获取待标识联系人对应的声纹特征，还包括：Optionally, in the method for identifying a contact as described above, the obtaining the voiceprint feature corresponding to the contact to be identified further includes:
在侦测到所述移动终端的录音功能开启时，复制麦克风传输的所述待标识联系人的声音数据，并将复制得到的所述声音数据作为所述待标识联系人的声音样本。When the recording function of the mobile terminal is detected, the sound data of the to-be-identified contact transmitted by the microphone is copied, and the copied sound data is used as the sound sample of the contact to be identified.
本发明实施例提供的移动终端及联系人标识方法，通过当接收到联系人 标识指令时，获取待标识联系人的三维形象及声纹特征信息，并采用获取的三维形象及声纹特征信息对该待标识联系人进行标识，从而有效解决了相关技术中联系人的标识方式十分简单，且不够生动形象的问题。另外，还通过在查看上述待标识联系人信息时，对该待标识联系人信息进行具有该待标识联系人的三维形象及联系人声纹特征信息的语音播报，从而满足了用户日益增长的个性化需求，提高用户的体验及可玩性。The mobile terminal and the contact identification method provided by the embodiment of the present invention, when receiving a contact When the instruction is specified, the three-dimensional image and the voiceprint feature information of the contact to be identified are obtained, and the acquired three-dimensional image and the voiceprint feature information are used to identify the contact to be identified, thereby effectively solving the identification method of the contact in the related art. Very simple and not vivid enough. In addition, when viewing the contact information to be identified, the contact information of the to-be-identified contact information is broadcasted with the three-dimensional image of the contact to be identified and the contact voice feature information, thereby satisfying the increasing personality of the user. Demand, improve user experience and playability.
在阅读并理解了附图和详细描述后，可以明白其他方面。Other aspects will be apparent upon reading and understanding the drawings and detailed description.
附图概述BRIEF abstract
图1为实现本发明实施例提供的移动终端的硬件结构示意图；1 is a schematic structural diagram of hardware of a mobile terminal according to an embodiment of the present invention;
图2为图1所示的移动终端中相机的电气结构示意图；2 is a schematic diagram showing the electrical structure of a camera in the mobile terminal shown in FIG. 1;
图3为本发明实施例提供的一种移动终端的结构示意图；FIG. 3 is a schematic structural diagram of a mobile terminal according to an embodiment of the present disclosure;
图4为本发明实施例提供的另一种移动终端的结构示意图；FIG. 4 is a schematic structural diagram of another mobile terminal according to an embodiment of the present disclosure;
图5为本发明实施例提供的又一种移动终端的结构示意图；FIG. 5 is a schematic structural diagram of still another mobile terminal according to an embodiment of the present disclosure;
图6为本发明实施例提供的移动终端中一种第一提取单元的结构示意图；FIG. 6 is a schematic structural diagram of a first extracting unit in a mobile terminal according to an embodiment of the present disclosure;
图7为本发明实施例提供的再一种移动终端的结构示意图；FIG. 7 is a schematic structural diagram of still another mobile terminal according to an embodiment of the present disclosure;
图8为本发明实施例提供的移动终端的一种全景拍摄过程的示意图；FIG. 8 is a schematic diagram of a panoramic shooting process of a mobile terminal according to an embodiment of the present invention; FIG.
图9为本发明实施例提供的一种联系人标识方法的流程示意图；FIG. 9 is a schematic flowchart of a method for identifying a contact according to an embodiment of the present disclosure;
图10为本发明实施例提供的另一种联系人标识方法的流程示意图；FIG. 10 is a schematic flowchart diagram of another method for identifying a contact according to an embodiment of the present disclosure;
图11为本发明实施例提供的又一种联系人标识方法的流程示意图；FIG. 11 is a schematic flowchart diagram of still another method for identifying a contact according to an embodiment of the present disclosure;
图12为本发明实施例提供的联系人标识方法中一种提取用于生成待标识联系人的三维形象的特征数据的流程示意图；FIG. 12 is a schematic flowchart of extracting feature data for generating a three-dimensional image of a contact to be identified in a method for identifying a contact according to an embodiment of the present disclosure;
图13为本发明实施例提供的再一种联系人标识方法的流程示意图。FIG. 13 is a schematic flowchart diagram of still another method for identifying a contact according to an embodiment of the present invention.
本发明的实施方式Embodiments of the invention
应当理解，以下所描述的实施例仅仅用以解释本发明，并不用于限定本 发明。It should be understood that the embodiments described below are merely illustrative of the invention and are not intended to be limiting invention.
下文中将结合附图对本发明的实施方式进行详细说明。需要说明的是，在不冲突的情况下，本文中的实施例及实施例中的特征可以相互任意组合。Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that, in the case of no conflict, the features in the embodiments and the embodiments herein may be arbitrarily combined with each other.
在附图的流程图示出的步骤可以在诸根据一组计算机可执行指令的计算机***中执行。并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。The steps illustrated in the flowchart of the figures may be executed in a computer system in accordance with a set of computer executable instructions. Also, although logical sequences are shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than the ones described herein.
现在将参考附图描述实现本发明各个实施例的移动终端。在后续的描述中，使用用于表示元件的诸如“模块”、“部件”或“单元”的后缀仅为了有利于本发明的说明，其本身并没有特定的意义。因此，“模块”与“部件”可以混合地使用。A mobile terminal embodying various embodiments of the present invention will now be described with reference to the accompanying drawings. In the following description, the use of suffixes such as "module", "component" or "unit" for indicating an element is merely an explanation for facilitating the present invention, and does not have a specific meaning per se. Therefore, "module" and "component" can be used in combination.
移动终端可以以各种形式来实施。例如，本发明中描述的终端可以包括诸如移动电话、智能电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、导航装置等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。下面，假设终端是移动终端。然而，本领域技术人员将理解的是，除了特别用于移动目的的元件之外，根据本发明的实施方式的构造也能够应用于固定类型的终端。The mobile terminal can be implemented in various forms. For example, the terminal described in the present invention may include, for example, a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet), a PMP (Portable Multimedia Player), a navigation device, etc. Mobile terminals and fixed terminals such as digital TVs, desktop computers, and the like. In the following, it is assumed that the terminal is a mobile terminal. However, those skilled in the art will appreciate that configurations in accordance with embodiments of the present invention can be applied to fixed type terminals in addition to components that are specifically for mobile purposes.
图1为实现本发明实施例提供的移动终端的硬件结构示意图。FIG. 1 is a schematic structural diagram of hardware of a mobile terminal according to an embodiment of the present invention.
移动终端100可以包括无线通信单元110、A/V(音频/视频)输入单元120、用户输入单元130、感测单元140、输出单元150、存储器160和控制器170。图1示出了具有各种组件的移动终端，但是应理解的是，并不要求实施所有示出的组件。可以替代地实施更多或更少的组件。将在下面详细描述移动终端的元件。The mobile terminal 100 may include a wireless communication unit 110, an A/V (Audio/Video) input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, a memory 160, and a controller 170. Figure 1 illustrates a mobile terminal having various components, but it should be understood that not all illustrated components are required to be implemented. More or fewer components can be implemented instead. The elements of the mobile terminal will be described in detail below.
移动通信模块111将无线电信号发送到基站(例如，接入点、节点B等等)、外部终端以及服务器中的至少一个和/或从其接收无线电信号。这样的无线电信号可以包括语音通话信号、视频通话信号、或者根据文本和/或多媒体消息发送和/或接收的各种类型的数据。The mobile communication module 111 transmits radio signals to and/or receives radio signals from at least one of a base station (e.g., an access point, a Node B, etc.), an external terminal, and a server. Such radio signals may include voice call signals, video call signals, or various types of data transmitted and/or received in accordance with text and/or multimedia messages.
A/V输入单元120用于接收音频或视频信号。A/V输入单元120可以包括相机121和麦克风122，相机121对在视频捕获模式或图像捕获模式中由图像 捕获装置获得的静态图片或视频的图像数据进行处理。处理后的图像帧可以显示在显示模块151上。经相机121处理后的图像帧可以存储在存储器160(或其它存储介质)中或者经由无线通信单元110进行发送，可以根据移动终端的构造提供两个或更多相机121。麦克风122可以在电话通话模式、记录模式、语音识别模式等等运行模式中经由麦克风接收声音(音频数据)，并且能够将这样的声音处理为音频数据。处理后的音频(语音)数据可以在电话通话模式的情况下转换为可经由移动通信模块112发送到移动通信基站的格式输出。麦克风122可以实施各种类型的噪声消除(或抑制)算法以消除(或抑制)在接收和发送音频信号的过程中产生的噪声或者干扰。The A/V input unit 120 is for receiving an audio or video signal. The A/V input unit 120 may include a camera 121 and a microphone 122, the camera 121 being imaged in a video capture mode or an image capture mode The image data of the still picture or video obtained by the capture device is processed. The processed image frame can be displayed on the display module 151. The image frames processed by the camera 121 may be stored in the memory 160 (or other storage medium) or transmitted via the wireless communication unit 110, and two or more cameras 121 may be provided according to the configuration of the mobile terminal. The microphone 122 can receive sound (audio data) via a microphone in an operation mode of a telephone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound as audio data. The processed audio (voice) data can be converted to a format output that can be transmitted to the mobile communication base station via the mobile communication module 112 in the case of a telephone call mode. The microphone 122 can implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated during the process of receiving and transmitting audio signals.
用户输入单元130可以根据用户输入的命令生成键输入数据以控制移动终端的各种操作。用户输入单元130允许用户输入各种类型的信息，并且可以包括键盘、锅仔片、触摸板(例如，检测由于被接触而导致的电阻、压力、电容等等的变化的触敏组件)、滚轮、摇杆等等。特别地，当触摸板以层的形式叠加在显示模块151上时，可以形成触摸屏。The user input unit 130 may generate key input data according to a command input by the user to control various operations of the mobile terminal. The user input unit 130 allows the user to input various types of information, and may include a keyboard, a pot, a touch pad (eg, a touch sensitive component that detects changes in resistance, pressure, capacitance, etc. due to contact), a scroll wheel , rocker, etc. In particular, when the touch panel is superimposed on the display module 151 in the form of a layer, a touch screen can be formed.
感测单元140检测移动终端100的当前状态，(例如，移动终端100的打开或关闭状态)、移动终端100的位置、用户对于移动终端100的接触(即，触摸输入)的有无、移动终端100的取向、移动终端100的加速或将速移动和方向等等，并且生成用于控制移动终端100的操作的命令或信号。例如，当移动终端100实施为滑动型移动电话时，感测单元140可以感测该滑动型电话是打开还是关闭。感测单元140包括加速度计141和陀螺仪142，加速度计141用于检测移动终端100的实时加速度以得出移动终端100的运动方向，陀螺仪142用于检测移动终端100相对于其所在平面的倾斜角度。The sensing unit 140 detects the current state of the mobile terminal 100 (eg, the open or closed state of the mobile terminal 100), the location of the mobile terminal 100, the presence or absence of contact (ie, touch input) by the user with the mobile terminal 100, and the mobile terminal. The orientation of 100, the acceleration of the mobile terminal 100 or the speed of movement and direction, and the like, and generates a command or signal for controlling the operation of the mobile terminal 100. For example, when the mobile terminal 100 is implemented as a slide type mobile phone, the sensing unit 140 can sense whether the slide type phone is turned on or off. The sensing unit 140 includes an accelerometer 141 for detecting real-time acceleration of the mobile terminal 100 to derive a moving direction of the mobile terminal 100, and a gyroscope 142 for detecting the mobile terminal 100 with respect to a plane thereof slope.
输出单元150被构造为以视觉、音频和/或触觉方式提供输出信号(例如，音频信号、视频信号、警报信号、振动信号等等)。输出单元150可以包括显示模块151、音频输出模块152、警报模块153等等。 Output unit 150 is configured to provide an output signal (eg, an audio signal, a video signal, an alarm signal, a vibration signal, etc.) in a visual, audio, and/or tactile manner. The output unit 150 may include a display module 151, an audio output module 152, an alarm module 153, and the like.
显示模块151可以显示在移动终端100中处理的信息。例如，当移动终端100处于电话通话模式时，显示模块151可以显示与通话或其它通信(例如，文本消息收发、多媒体文件下载等等)相关的用户界面(UI)或图形用户界面(GUI)。当移动终端100处于视频通话模式或者图像捕获模式时，显示模块151 可以显示捕获的图像和/或接收的图像、示出视频或图像以及相关功能的UI或GUI等等。The display module 151 can display information processed in the mobile terminal 100. For example, when the mobile terminal 100 is in a phone call mode, the display module 151 can display a user interface (UI) or graphical user interface (GUI) associated with a call or other communication (eg, text messaging, multimedia file download, etc.). When the mobile terminal 100 is in a video call mode or an image capture mode, the display module 151 A captured image and/or a received image, a UI or GUI showing a video or image and related functions, and the like can be displayed.
同时，当显示模块151和触摸板以层的形式彼此叠加以形成触摸屏时，显示模块151可以用作输入装置和输出装置。显示模块151可以包括液晶显示器(LCD)、薄膜晶体管LCD(TFT-LCD)、有机发光二极管(OLED)显示器、柔性显示器、三维(3D)显示器等等中的至少一种。这些显示器中的一些可以被构造为透明状以允许用户从外部观看，这可以称为透明显示器，典型的透明显示器可以例如为TOLED(透明有机发光二极管)显示器等等。根据特定想要的实施方式，移动终端100可以包括两个或更多显示单元(或其它显示装置)，例如，移动终端可以包括外部显示单元(未示出)和内部显示单元(未示出)。触摸屏可用于检测触摸输入压力以及触摸输入位置和触摸输入面积。Meanwhile, when the display module 151 and the touch panel are superposed on each other in the form of layers to form a touch screen, the display module 151 can function as an input device and an output device. The display module 151 may include at least one of a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), an organic light emitting diode (OLED) display, a flexible display, a three-dimensional (3D) display, and the like. Some of these displays may be configured to be transparent to allow a user to view from the outside, which may be referred to as a transparent display, and a typical transparent display may be, for example, a TOLED (Transparent Organic Light Emitting Diode) display or the like. According to a particular desired embodiment, the mobile terminal 100 may include two or more display units (or other display devices), for example, the mobile terminal may include an external display unit (not shown) and an internal display unit (not shown) . The touch screen can be used to detect touch input pressure as well as touch input position and touch input area.
音频输出模块152可以在移动终端处于呼叫信号接收模式、通话模式、记录模式、语音识别模式、广播接收模式等等模式下时，将无线通信单元110接收的或者在存储器160中存储的音频数据转换音频信号并且输出为声音。而且，音频输出模块152可以提供与移动终端100执行的特定功能相关的音频输出(例如，呼叫信号接收声音、消息接收声音等等)。音频输出模块152可以包括拾音器、蜂鸣器等等。The audio output module 152 may convert audio data received by the wireless communication unit 110 or stored in the memory 160 when the mobile terminal is in a call signal receiving mode, a call mode, a recording mode, a voice recognition mode, a broadcast receiving mode, and the like. The audio signal is output as sound. Moreover, the audio output module 152 can provide audio output (eg, call signal reception sound, message reception sound, etc.) associated with a particular function performed by the mobile terminal 100. The audio output module 152 can include a pickup, a buzzer, and the like.
警报模块153可以提供输出以将事件的发生通知给移动终端100。典型的事件可以包括呼叫接收、消息接收、键信号输入、触摸输入等等。除了音频或视频输出之外，警报模块153可以以不同的方式提供输出以通知事件的发生。例如，警报模块153可以以振动的形式提供输出，当接收到呼叫、消息或一些其它进入通信(incoming communication)时，警报模块153可以提供触觉输出(即，振动)以将其通知给用户。通过提供这样的触觉输出，即使在用户的移动电话处于用户的口袋中时，用户也能够识别出各种事件的发生。警报模块153也可以经由显示模块151或音频输出模块152提供通知事件的发生的输出。The alert module 153 can provide an output to notify the mobile terminal 100 of the occurrence of an event. Typical events may include call reception, message reception, key signal input, touch input, and the like. In addition to audio or video output, the alert module 153 can provide an output in a different manner to notify of the occurrence of an event. For example, the alarm module 153 can provide an output in the form of vibrations that, when receiving a call, message, or some other incoming communication, can provide a haptic output (ie, vibration) to notify the user of it. By providing such a tactile output, the user is able to recognize the occurrence of various events even when the user's mobile phone is in the user's pocket. The alarm module 153 can also provide an output of the notification event occurrence via the display module 151 or the audio output module 152.
存储器160可以存储由控制器180执行的处理和控制操作的软件程序等等，或者可以暂时地存储己经输出或将要输出的数据(例如，电话簿、消息、静态图像、视频等等)。而且，存储器160可以存储关于当触摸施加到触摸屏 时输出的各种方式的振动和音频信号的数据。The memory 160 may store a software program or the like for processing and control operations performed by the controller 180, or may temporarily store data (for example, a phone book, a message, a still image, a video, etc.) that has been output or is to be output. Moreover, the memory 160 can store information about when a touch is applied to the touch screen When outputting various ways of vibration and audio signal data.
存储器160可以包括至少一种类型的存储介质，所述存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等等。而且，移动终端100可以与通过网络连接执行存储器160的存储功能的网络存储装置协作。The memory 160 may include at least one type of storage medium including a flash memory, a hard disk, a multimedia card, a card type memory (eg, SD or DX memory, etc.), a random access memory (RAM), a static random access memory ( SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disk, optical disk, and the like. Moreover, the mobile terminal 100 can cooperate with a network storage device that performs a storage function of the memory 160 through a network connection.
控制器180通常控制移动终端的总体操作。例如，控制器180执行与语音通话、数据通信、视频通话等等相关的控制和处理。另外，控制器180可以包括用于再现(或回放)多媒体数据的多媒体模块181，多媒体模块181可以构造在控制器180内，或者可以构造为与控制器180分离。控制器180可以执行模式识别处理，以将在触摸屏上执行的手写输入或者图片绘制输入识别为字符或图像。The controller 180 typically controls the overall operation of the mobile terminal. For example, the controller 180 performs the control and processing associated with voice calls, data communications, video calls, and the like. In addition, the controller 180 may include a multimedia module 181 for reproducing (or playing back) multimedia data, which may be constructed within the controller 180 or may be configured to be separate from the controller 180. The controller 180 may perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as a character or an image.
这里描述的各种实施方式可以以使用例如计算机软件、硬件或其任何组合的计算机可读介质来实施。对于硬件实施，这里描述的实施方式可以通过使用特定用途集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理装置(DSPD)、可编程逻辑装置(PLD)、现场可编程门阵列(FPGA)、处理器、控制器、微控制器、微处理器、被设计为执行这里描述的功能的电子单元中的至少一种来实施，在一些情况下，这样的实施方式可以在控制器180中实施。对于软件实施，诸如过程或功能的实施方式可以与允许执行至少一种功能或操作的单独的软件模块来实施。软件代码可以由以任何适当的编程语言编写的软件应用程序(或程序)来实施，软件代码可以存储在存储器160中并且由控制器180执行。The various embodiments described herein can be implemented in a computer readable medium using, for example, computer software, hardware, or any combination thereof. For hardware implementations, the embodiments described herein may be through the use of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays ( An FPGA, a processor, a controller, a microcontroller, a microprocessor, at least one of the electronic units designed to perform the functions described herein, in some cases, such an embodiment may be at the controller 180 Implemented in the middle. For software implementations, implementations such as procedures or functions may be implemented with separate software modules that permit the execution of at least one function or operation. The software code can be implemented by a software application (or program) written in any suitable programming language, which can be stored in memory 160 and executed by controller 180.
至此，己经按照其功能描述了移动终端。下面，为了简要起见，将描述诸如折叠型、直板型、摆动型、滑动型移动终端等等的各种类型的移动终端中的滑动型移动终端作为示例。因此，本发明能够应用于任何类型的移动终端，并且不限于滑动型移动终端。So far, the mobile terminal has been described in terms of its function. Hereinafter, for the sake of brevity, a slide type mobile terminal among various types of mobile terminals such as a folding type, a bar type, a swing type, a slide type mobile terminal, and the like will be described as an example. Therefore, the present invention can be applied to any type of mobile terminal, and is not limited to a slide type mobile terminal.
参照图2，图2为图1所示的移动终端中相机的电气结构示意图。Referring to FIG. 2, FIG. 2 is a schematic diagram showing the electrical structure of a camera in the mobile terminal shown in FIG. 1.
摄影镜头1211中可以包括形成被摄体像的多个光学镜头，该光学镜头可 以为单焦点镜头或变焦镜头。摄影镜头1211在镜头驱动器1221的控制下能够在光轴方向上移动，镜头驱动器1221根据来自镜头驱动控制电路1222的控制信号，控制摄影镜头1211的焦点位置，在变焦镜头的情况下，也可控制焦点距离。镜头驱动控制电路1222按照来自微型计算机1217的控制命令对镜头驱动器1221进行的驱动控制。The photographic lens 1211 may include a plurality of optical lenses forming an object image, and the optical lens may be Think of a single focus lens or zoom lens. The photographic lens 1211 is movable in the optical axis direction under the control of the lens driver 1221, and the lens driver 1221 controls the focus position of the photographic lens 1211 in accordance with a control signal from the lens driving control circuit 1222, and can also be controlled in the case of the zoom lens. Focus distance. The lens drive control circuit 1222 performs drive control of the lens driver 1221 in accordance with a control command from the microcomputer 1217.
在摄影镜头1211的光轴上、由摄影镜头1211形成的被摄体像的位置附近配置有摄像元件1212。摄像元件1212，设置为：对被摄体像摄像并取得摄像图像数据。在摄像元件1212上二维且呈矩阵状配置有构成每个像素的光电二极管。每个光电二极管产生与受光量对应的光电转换电流，该光电转换电流由与每个光电二极管连接的电容器进行电荷蓄积。每个像素的前表面配置有拜耳排列的红、绿、蓝(Red、Green、Blue，简称为：RGB)滤色器。An imaging element 1212 is disposed on the optical axis of the photographic lens 1211 near the position of the subject image formed by the photographic lens 1211. The imaging element 1212 is provided to image the subject image and acquire captured image data. Photodiodes constituting each pixel are arranged two-dimensionally and in a matrix on the imaging element 1212. Each photodiode generates a photoelectric conversion current corresponding to the amount of received light, which is subjected to charge accumulation by a capacitor connected to each photodiode. The front surface of each pixel is provided with a Bayer array of red, green, blue (abbreviation: RGB) color filters.
摄像元件1212与摄像电路1213连接，该摄像电路1213在摄像元件1212中进行电荷蓄积控制和图像信号读出控制，对该读出的图像信号(例如为模拟图像信号)降低重置噪声后进行波形整形，进而进行增益提高等以成为适当的信号电平。The imaging element 1212 is connected to an imaging circuit 1213 that performs charge accumulation control and image signal readout control in the imaging element 1212, and reduces the reset noise after the read image signal (for example, an analog image signal). The shaping is performed, and the gain is increased to obtain an appropriate signal level.
摄像电路1213与模数转换(Analog Digital Converter，简称为：A/D)转换器1214连接，该A/D转换器1214对模拟图像信号进行模数转换，向总线1227输出数字图像信号(以下称之为图像数据)。The imaging circuit 1213 is connected to an analog-to-digital conversion (A/D) converter 1214 that performs analog-to-digital conversion on the analog image signal and outputs a digital image signal to the bus 1227 (hereinafter referred to as It is image data).
总线1227，设置为：传送在相机的内部读出或生成的各种数据的传送路径。在总线1227连接着上述A/D转换器1214，此外还连接着图像处理器1215、JPEG处理器1216、微型计算机1217、同步动态随机存取内存(Synchronous Dynamic random access memory，简称为：SDRAM)1218、存储器接口(以下称之为存储器I/F)1219、液晶显示器(Liquid Crystal Display，简称为：LCD)驱动器1220。The bus 1227 is provided to transmit a transmission path of various data read or generated inside the camera. The A/D converter 1214 is connected to the bus 1227, and an image processor 1215, a JPEG processor 1216, a microcomputer 1217, and a Synchronous Dynamic Random Access Memory (SDRAM) 1218 are connected. A memory interface (hereinafter referred to as a memory I/F) 1219 and a liquid crystal display (LCD) driver 1220.
图像处理器1215对基于摄像元件1212的输出的图像数据进行输出缓冲(Output Buffer，简称为：OB)相减处理、白平衡调整、颜色矩阵运算、伽马转换、色差信号处理、噪声去除处理、同时化处理、边缘处理等各种图像处理。JPEG处理器1216在将图像数据记录于记录介质1225时，按照JPEG压缩方式压缩从SDRAM1218读出的图像数据。此外，JPEG处理器1216为了进 行图像再现显示而进行JPEG图像数据的解压缩。进行解压缩时，读出记录在记录介质1225中的文件，在JPEG处理器1216中实施了解压缩处理后，将解压缩的图像数据暂时存储于SDRAM1218中并在LCD1226上进行显示。另外，在本实施方式中，作为图像压缩解压缩方式采用的是JPEG方式，然而压缩解压缩方式不限于此，当然可以采用MPEG、TIFF、H.264等其他的压缩解压缩方式。The image processor 1215 performs output buffering (Output Buffer, abbreviated as: OB) subtraction processing, white balance adjustment, color matrix calculation, gamma conversion, color difference signal processing, noise removal processing, and the image data based on the output of the imaging element 1212. Various image processing such as simultaneous processing and edge processing. The JPEG processor 1216 compresses the image data read out from the SDRAM 1218 in accordance with the JPEG compression method when the image data is recorded on the recording medium 1225. In addition, the JPEG processor 1216 is for The line image reproduction display is performed to decompress the JPEG image data. At the time of decompression, the file recorded on the recording medium 1225 is read, and after the compression processing is performed in the JPEG processor 1216, the decompressed image data is temporarily stored in the SDRAM 1218 and displayed on the LCD 1226. Further, in the present embodiment, the JPEG method is adopted as the image compression/decompression method. However, the compression/decompression method is not limited thereto, and other compression/decompression methods such as MPEG, TIFF, and H.264 may be used.
微型计算机1217发挥作为该相机整体的控制部的功能，统一控制相机的各种处理序列。微型计算机1217连接着操作单元1223和闪存1224。The microcomputer 1217 functions as a control unit of the entire camera, and collectively controls various processing sequences of the camera. The microcomputer 1217 is connected to the operation unit 1223 and the flash memory 1224.
操作单元1223包括但不限于实体按键或者虚拟按键，该实体或虚拟按键可以为电源按钮、拍照键、编辑按键、动态图像按钮、再现按钮、菜单按钮、十字键、OK按钮、删除按钮、放大按钮等各种输入按钮和各种输入键等操作控件，检测这些操作控件的操作状态。The operating unit 1223 includes, but is not limited to, a physical button or a virtual button, and the entity or virtual button may be a power button, a camera button, an edit button, a dynamic image button, a reproduction button, a menu button, a cross button, an OK button, a delete button, an enlarge button The operation controls such as various input buttons and various input keys detect the operational state of these operation controls.
将检测结果向微型计算机1217输出。此外，在作为显示器的LCD1226的前表面设有触摸面板，检测用户的触摸位置，将该触摸位置向微型计算机1217输出。微型计算机1217根据来自操作单元1223的操作位置的检测结果，执行与用户的操作对应的各种处理序列。The detection result is output to the microcomputer 1217. Further, a touch panel is provided on the front surface of the LCD 1226 as a display, and the touch position of the user is detected, and the touch position is output to the microcomputer 1217. The microcomputer 1217 executes various processing sequences corresponding to the user's operation in accordance with the detection result from the operation position of the operation unit 1223.
闪存1224存储用于执行微型计算机1217的各种处理序列的程序。微型计算机1217根据该程序进行相机整体的控制。此外，闪存1224存储相机的各种调整值，微型计算机1217读出调整值，按照该调整值进行相机的控制。The flash memory 1224 stores programs for executing various processing sequences of the microcomputer 1217. The microcomputer 1217 performs overall control of the camera in accordance with the program. Further, the flash memory 1224 stores various adjustment values of the camera, and the microcomputer 1217 reads out the adjustment value, and performs control of the camera in accordance with the adjustment value.
SDRAM1218，设置为：对图像数据等进行暂时存储的可电改写的易失性存储器。该SDRAM1218暂时存储从A/D转换器1214输出的图像数据和在图像处理器1215、JPEG处理器1216等中进行了处理后的图像数据。The SDRAM 1218 is provided as an electrically rewritable volatile memory that temporarily stores image data or the like. The SDRAM 1218 temporarily stores image data output from the A/D converter 1214 and image data processed in the image processor 1215, the JPEG processor 1216, and the like.
存储器接口1219与记录介质1225连接，进行将图像数据和附加在图像数据中的文件头等数据写入记录介质1225和从记录介质1225中读出的控制。记录介质1225例如为能够在相机主体上自由拆装的存储器卡等记录介质，然而不限于此，也可以是内置在相机主体中的硬盘等。The memory interface 1219 is connected to the recording medium 1225, and performs control for writing image data and a file header attached to the image data to the recording medium 1225 and reading out from the recording medium 1225. The recording medium 1225 is, for example, a recording medium such as a memory card that can be detachably attached to the camera body. However, the recording medium 1225 is not limited thereto, and may be a hard disk or the like built in the camera body.
LCD驱动器1210与LCD1226连接，将由图像处理器1215处理后的图像数据存储于SDRAM1218，需要显示时，读取SDRAM1218存储的图像数据并在LCD1226上显示，或者，JPEG处理器1216压缩过的图像数据存储于 SDRAM1218，在需要显示时，JPEG处理器1216读取SDRAM1218的压缩过的图像数据，再进行解压缩，将解压缩后的图像数据通过LCD1226进行显示。The LCD driver 1210 is connected to the LCD 1226, and stores image data processed by the image processor 1215 in the SDRAM 1218. When display is required, the image data stored in the SDRAM 1218 is read and displayed on the LCD 1226, or the image data stored in the JPEG processor 1216 is compressed. Yu In the SDRAM 1218, when display is required, the JPEG processor 1216 reads the compressed image data of the SDRAM 1218, decompresses it, and displays the decompressed image data on the LCD 1226.
LCD1226配置在相机主体的背面进行图像显示。该LCD1226可以为LCD，然而不限于此，也可以采用有机电致发光(electroluminescent，简称为：EL)等其它显示面板实现LCD1226，然而不限于此。The LCD 1226 is configured to display an image on the back of the camera body. The LCD 1226 may be an LCD, but is not limited thereto, and the LCD 1226 may be implemented by other display panels such as organic electroluminescence (EL), but is not limited thereto.
基于上述移动终端硬件结构以及相机的电气结构示意图，本发明实施例提供的移动终端的以下实施例。The following embodiments of the mobile terminal provided by the embodiments of the present invention are based on the hardware structure of the mobile terminal and the electrical structure of the camera.
如图3所示，为本发明实施例提供的一种移动终端的结构示意图。FIG. 3 is a schematic structural diagram of a mobile terminal according to an embodiment of the present invention.
本发明实施例提供的移动终端可以包括：获取模块10、标识模块20及播报模块30。The mobile terminal provided by the embodiment of the present invention may include: an obtaining module 10, an identifying module 20, and a broadcast module 30.
其中，获取模块10，设置为：当接收到联系人标识指令时，获取待标识联系人对应的三维形象及声纹特征信息。The obtaining module 10 is configured to: when receiving the contact identifier instruction, obtain the three-dimensional image and the voiceprint feature information corresponding to the contact to be identified.
在本发明实施例中，可以通过在移动终端上提供用于标识联系人的快捷图标，供用户基于该快捷图标触发联系人标识指令；或者，也可以通过在移动终端上提供用于标识联系人的物理按键，供用户基于该物理按键触发联系人标识指令；或者，也可以通过在移动终端上提供用于标识联系人的虚拟按键，供用户基于该虚拟按键触发联系人标识指令。本发明实施例中的声纹特征信息可以为语音信号的语音频谱参数、线性预测参数、小波特征参数等。当获取模块10接收到用户触发的联系人标识指令时，获取存储于移动终端存储单元中的待标识联系人对应的三维形象及声纹特征信息。例如，用户触发将联系人小王的信息与小王的三维形象及声纹特征信息进行标识的指令，则获取模块10获取移动终端中预存的小王的三维形象及小王的声纹特征信息；又例如，用户触发将联系人小李的信息与小李的三维形象及声纹特征信息进行标识的指令，则获取模块10获取移动终端中预存的小李王的三维形象及小王的声纹特征信息。In the embodiment of the present invention, the shortcut icon for identifying the contact may be provided on the mobile terminal, and the user may trigger the contact identification instruction based on the shortcut icon; or may be provided by the mobile terminal for identifying the contact. The physical button is used by the user to trigger the contact identification instruction based on the physical button; or the virtual button for identifying the contact may be provided on the mobile terminal, for the user to trigger the contact identification instruction based on the virtual button. The voiceprint feature information in the embodiment of the present invention may be a voice spectrum parameter, a linear prediction parameter, a wavelet feature parameter, and the like of the voice signal. When the acquisition module 10 receives the user-triggered contact identification instruction, the acquisition module 10 acquires the three-dimensional image and voiceprint feature information corresponding to the to-be-identified contact stored in the storage unit of the mobile terminal. For example, if the user triggers an instruction to identify the information of the contact person and the three-dimensional image and voiceprint feature information of Xiao Wang, the obtaining module 10 acquires the three-dimensional image of Xiao Wang pre-stored in the mobile terminal and the voiceprint feature information of Xiao Wang. For example, if the user triggers an instruction to identify the information of the contact Xiao Li and Xiao Li's three-dimensional image and voiceprint feature information, the obtaining module 10 obtains the three-dimensional image of Xiao Li Wang pre-stored in the mobile terminal and the sound of Xiao Wang. Pattern feature information.
标识模块20，设置为：采用获取模块10获取的三维形象及声纹特征信息对待标识联系人进行标识。 The identifier module 20 is configured to: identify the contact to be identified by using the three-dimensional image and the voiceprint feature information acquired by the acquiring module 10.
当获取模块10获取到待标识联系人的三维形象及声纹特征信息后，可以由标识模块20对待标识联系人进行标识，即建立待标识联系人与该待标识联系人的三维形象及声纹特征信息的映射关系。After the obtaining module 10 obtains the three-dimensional image and the voiceprint feature information of the contact to be identified, the identifier module 20 may identify the contact to be identified, that is, establish a three-dimensional image and voiceprint of the contact to be identified and the contact to be identified. The mapping relationship of feature information.
播报模块30，设置为：在查看标识模块20所标识的待标识联系人信息时，进行具有该待标识联系人的三维形象及声纹特征信息的语音播报。The broadcast module 30 is configured to: when viewing the contact information to be identified identified by the identification module 20, perform a voice broadcast with the three-dimensional image and the voiceprint feature information of the contact to be identified.
在本发明实施例中，标识模块20进行标识的待标识联系人信息可以为联系人应用中待标识联系人信息，或者为短信应用中待标识联系人发的短信信息等。上述具有待标识联系人的三维形象及联系人声纹特征信息的语音播报可以为：在语音播报时，在移动终端的屏幕中显示该待标识联系人的三维形象，同时使用该待标识联系人的声纹特征信息对待播报的文本信息进行语音播报。对上述具有待标识联系人的三维形象及声纹特征信息的语音播报的实现方式，举例予以说明，例如：若待标识联系人信息为联系人应用中待标识联系人信息时，则当查看某个联系人信息时，播报模块30会在移动终端的屏幕上显示该联系人的三维形象，并使用该联系人的声纹特征信息播报该联系人信息，该联系人信息包括联系人电话号码、联系人地址、联系人工作单位等；又例如：若待标识联系人信息为短信应用中待标识联系人发的短信信息时，则当查看该待标识联系人发的短信信息时，播报模块30会在移动终端的屏幕上显示发送该短信的联系人的三维形象，并使用该联系人的声纹特征信息播报该短信内容。In the embodiment of the present invention, the to-be-identified contact information that is identified by the identifier module 20 may be the contact information to be identified in the contact application, or the short message information sent by the contact to be identified in the short message application. The voice broadcast of the three-dimensional image and the contact voiceprint feature information of the contact to be identified may be: when the voice broadcast is performed, the three-dimensional image of the contact to be identified is displayed on the screen of the mobile terminal, and the contact to be identified is used at the same time. The voiceprint feature information is voice broadcasted to the text information to be broadcast. For example, if the contact information to be identified is the contact information to be identified in the contact application, when viewing the contact information of the three-dimensional image and the voiceprint feature information of the contact to be identified, for example, when viewing the contact information to be identified in the contact application, When the contact information is used, the broadcast module 30 displays the three-dimensional image of the contact on the screen of the mobile terminal, and broadcasts the contact information using the voiceprint feature information of the contact, the contact information includes a contact phone number, The contact address, the contact work unit, and the like; for example, if the contact information to be identified is the short message information sent by the contact to be identified in the short message application, when the short message information sent by the contact to be identified is viewed, the broadcast module 30 A three-dimensional image of the contact that sent the short message is displayed on the screen of the mobile terminal, and the short message content is broadcasted using the voiceprint feature information of the contact.
在实际应用中，本发明实施例的思想也可应用于语音助手等应用中，例如，当用户打开语音助手时，用户可以选择用户聊天的联系人，当选定联系人之后，移动终端的屏幕会显示该联系人所关联的三维形象，并且在聊天时，语音助手的人物会使用该联系人关联的声纹特征信息播报聊天的内容，从而做到用户好像是和真的联系人进行聊天一样，提高用户体验。In practical applications, the idea of the embodiment of the present invention can also be applied to applications such as a voice assistant. For example, when a user opens a voice assistant, the user can select a contact that the user chats, and when the contact is selected, the screen of the mobile terminal The three-dimensional image associated with the contact is displayed, and during the chat, the character of the voice assistant uses the voiceprint feature information associated with the contact to broadcast the content of the chat, so that the user seems to be chatting with the real contact. To improve the user experience.
本发明实施例提供的移动终端，通过将获取模块设置为：当接收到联系人标识指令时，获取待标识联系人的三维形象及声纹特征信息，标识模块设置为：采用获取模块获取的所述三维形象及声纹特征信息对所述待标识联系人进行标识，从而有效解决了相关技术中联系人的标识方式十分简单，且不够生动形象的问题。另外，还通过将播报模块设置为：在查看标识模块所标 识的待标识联系人信息时，对待标识联系人信息进行具有该待标识联系人的三维形象及声纹特征信息的语音播报，从而满足了用户日益增长的个性化需求，提高用户的体验及可玩性。The mobile terminal provided by the embodiment of the present invention sets the acquisition module to: obtain the three-dimensional image and the voiceprint feature information of the contact to be identified when the contact identifier instruction is received, and the identifier module is set as: The three-dimensional image and the voiceprint feature information are used to identify the contact to be identified, thereby effectively solving the problem that the identification method of the contact in the related art is very simple and not vivid. In addition, by setting the broadcast module to: When the contact information to be identified is identified, the voice information of the three-dimensional image and the voiceprint feature information of the contact to be identified is performed on the identified contact information, thereby satisfying the increasing personalized demand of the user and improving the user experience and Playful.
可选地，基于上述图1所示实施例的移动终端，本发明实施例还提供一种移动终端，如图4所示，为本发明实施例提供的另一种移动终端的结构示意图。本实施例提供的移动终端中，播报模块30可以包括：合成单元31、修饰单元32及输出单元33。Optionally, based on the mobile terminal in the foregoing embodiment of the present invention, the embodiment of the present invention further provides a mobile terminal, as shown in FIG. 4, which is a schematic structural diagram of another mobile terminal according to an embodiment of the present invention. In the mobile terminal provided in this embodiment, the broadcast module 30 may include: a synthesizing unit 31, a modifying unit 32, and an output unit 33.
其中，合成单元31，设置为：在查看待标识联系人信息时，确定待播报文本，并合成该待播报文本的标准发音；The synthesizing unit 31 is configured to: when viewing the contact information to be identified, determine the text to be broadcasted, and synthesize the standard pronunciation of the text to be broadcasted;
修饰单元32，设置为：根据该待标识联系人的声纹特征信息对合成单元31合成的标准发音进行修饰，得到该待标识联系人的发音特征的声音波形；The modifying unit 32 is configured to: modify the standard pronunciation synthesized by the synthesizing unit 31 according to the voiceprint feature information of the contact to be identified, to obtain a sound waveform of the pronunciation feature of the contact to be identified;
输出单元33，设置为：输出修饰单元32得到的声音波形进行语音播报，并在移动终端的屏幕上显示该待标识联系人的三维形象。The output unit 33 is configured to: output the sound waveform obtained by the modification unit 32 for voice broadcast, and display the three-dimensional image of the contact to be identified on the screen of the mobile terminal.
在本发明实施例中，当查看待标识联系人信息时，合成单元31首先确定待播报文本，并合成该待播报文本的标准发音；然后，由修饰单元32根据该待标识联系人的声纹特征信息对合成单元31合成的标准发音进行修饰，得到该待标识联系人的发音特征的声音波形；最后，由输出单元33输出修饰单元32得到的声音波形进行语音播报，同时在移动终端的屏幕上显示该待标识联系人的三维形象，实现三维形象及声纹特征信息的语音播报。可选地，可在输出修饰单元32得到的声音波形时，可以将该声音波形发送至音频输出接口进行播报，该音频输出接口例如可以包括：麦克风(Microphone，简称为：MIC)、耳机、高清晰度多媒体接口(High Definition Multimedia Interface，简称为：HDMI)、自带的扬声器等。In the embodiment of the present invention, when viewing the contact information to be identified, the synthesizing unit 31 first determines the text to be broadcasted, and synthesizes the standard pronunciation of the text to be broadcasted; and then, according to the modification unit 32, according to the voiceprint of the contact to be identified The feature information is modified to the standard pronunciation synthesized by the synthesizing unit 31 to obtain a sound waveform of the pronunciation feature of the contact to be identified; finally, the sound waveform obtained by the output unit 33 is outputted by the output unit 33 for voice broadcast, and at the same time on the screen of the mobile terminal. The three-dimensional image of the contact to be identified is displayed, and the voice broadcast of the three-dimensional image and the voiceprint feature information is realized. Optionally, when the sound waveform obtained by the modifying unit 32 is output, the sound waveform may be sent to the audio output interface for broadcast. The audio output interface may include, for example, a microphone (Microphone, MIC for short), a headset, and a high High Definition Multimedia Interface (HDMI), built-in speakers, etc.
本发明实施例通过在进行具有待标识联系人的声纹特征信息的语音播报时，同时显示待标识联系人的三维形象，使得语音播报的过程更加生动形象，提高了用户体验性。 In the embodiment of the present invention, when the voice broadcast of the voiceprint feature information of the contact to be identified is performed, the three-dimensional image of the contact to be identified is simultaneously displayed, so that the process of the voice broadcast is more vivid and the user experience is improved.
可选地，基于上述图3和图4所示实施例的移动终端，本发明实施例还提供一种移动终端，如图5所示，为本发明实施例提供的又一种移动终端的结构示意图。本实施例以在图3所示移动终端的结构基础上为例予以示出，本实施例中的获取模块10可以包括：拍摄单元11、第一提取单元12及生成单元13。Optionally, based on the mobile terminal of the embodiment shown in FIG. 3 and FIG. 4, the embodiment of the present invention further provides a mobile terminal, as shown in FIG. 5, which is another mobile terminal structure provided by an embodiment of the present invention. schematic diagram. The present embodiment is illustrated by taking the structure of the mobile terminal shown in FIG. 3 as an example. The obtaining module 10 in this embodiment may include: a photographing unit 11, a first extracting unit 12, and a generating unit 13.
其中，拍摄单元11，设置为：启动移动终端的摄像头应用中的全景拍摄模式，以拍摄并存储待标识联系人的图像数据。The shooting unit 11 is configured to: activate a panoramic shooting mode in the camera application of the mobile terminal to capture and store image data of the contact to be identified.
本发明实施例中，通过拍摄单元11启动摄像头应用中的全景拍摄模式，对拍照对象(即待标识联系人)进行对焦，然后沿同一距离开始围绕拍照对象进行圆周运动，顺时针或逆时针均可，直到摄像头获取了该拍照对象的360度方位的图像数据，如图8所示，为本发明实施例提供的移动终端的一种全景拍摄过程的示意图。在拍摄全景照片的过程中，为了保证获取的不同角度及方位的图像之间有足够的重叠信息以供后续进行三维模型的拼接处理，可以在拍摄过程中可使用重力传感器、姿态传感器、陀螺仪、罗盘等对拍摄过程进行监测，以判断当前的移动终端是否处于合适的水平位置，并根据检测情况给予用户一定的语音提示；上述语音提示例如为：摄像头是否移动在一个可容忍的拍摄角度位置，摄像头转动的角度是否过大或过小等，从而保证获取的不同角度及方位的图像之间有足够的重叠信息。待拍摄完成后，拍摄模块30还可以把拍摄的全景照片(即图像数据)保存至移动终端的存储单元内，以便于后续进行三维形象的生成处理。In the embodiment of the present invention, the panoramic shooting mode in the camera application is activated by the shooting unit 11, and the photographing object (ie, the contact to be identified) is focused, and then the circular motion is started around the photographing object along the same distance, clockwise or counterclockwise. The image data of the 360 degree orientation of the photographing object is obtained by the camera, as shown in FIG. 8 , which is a schematic diagram of a panoramic photographing process of the mobile terminal provided by the embodiment of the present invention. In the process of taking a panoramic photo, in order to ensure that there are enough overlapping information between different angles and azimuth images for subsequent splicing of the three-dimensional model, gravity sensor, attitude sensor, gyroscope can be used during the shooting process. The compass, etc. monitors the shooting process to determine whether the current mobile terminal is in a proper horizontal position, and gives the user a certain voice prompt according to the detection situation; the voice prompt is, for example, whether the camera is moved at a tolerable shooting angle position. Whether the angle of rotation of the camera is too large or too small, so as to ensure sufficient overlapping information between images acquired at different angles and orientations. After the shooting is completed, the shooting module 30 can also save the captured panoramic photo (ie, image data) to the storage unit of the mobile terminal, so as to facilitate subsequent generation processing of the three-dimensional image.
第一提取单元12，设置为：从拍摄单元11存储的图像数据中提取用于生成待标识联系人的三维形象的特征数据。The first extracting unit 12 is configured to extract feature data for generating a three-dimensional image of the contact to be identified from the image data stored by the photographing unit 11.
在拍摄单元11获得待标识联系人的图像数据后，有第一提取单元12，从拍摄单元11存储的图像数据中提取用于生成待标识联系人的三维形象的特征数据。本发明实施例中的特征数据可以包括：待标识联系人的面部纹理、发型的360度外观特征、肤色、发色、瞳孔颜色、身高、穿着、四肢比例等特征数据。After the image data of the contact to be identified is obtained by the photographing unit 11, there is a first extracting unit 12 that extracts feature data for generating a three-dimensional image of the contact to be identified from the image data stored by the photographing unit 11. The feature data in the embodiment of the present invention may include: facial texture of the contact to be identified, 360 degree appearance features of the hairstyle, skin color, hair color, pupil color, height, wearing, limb ratio, and the like.
生成单元13，设置为：根据第一提取单元12所提取的所述特征数据，启动三维形象引擎以生成待标识联系人的三维形象。 The generating unit 13 is configured to: according to the feature data extracted by the first extracting unit 12, activate a three-dimensional image engine to generate a three-dimensional image of the contact to be identified.
在第一提取单元12提取到用于生成待标识联系人的三维形象的特征数据后，生成单元13可以根据该第一提取单元12所提取的特征数据，启动三维形象引擎以生成该待标识联系人的三维形象。在实际应用中，生成单元13可以将提取的特征数据进行升维计算，计算出在三维空间中整体人物的长宽高以及四肢比例数据，以生成一个初步的人物模型，然后使用从全景图片中相对应的图像数据中所提取的特征数据，比如脸部特征数据、发型特征数据、穿着特征数据、颜色特征数据等对生成的人物模型一一进行渲染。可选地，生成单元13还可采用全景拼接融合技术将不同朝向的图像信息进行拼接处理，从而最终生成与所拍摄的待标识联系人相对应的三维人物形象。可选地，在生成单元13生成该待标识联系人的三维形象后，还可以将该三维形象保存到存储单元中，以便在接收到联系人标识指令时，将该三维形象与该待标识联系人关联起来。After the first extracting unit 12 extracts the feature data for generating the three-dimensional image of the contact to be identified, the generating unit 13 may start the three-dimensional image engine to generate the to-be-identified contact according to the feature data extracted by the first extracting unit 12. The three-dimensional image of people. In an actual application, the generating unit 13 may perform the ascending dimension calculation on the extracted feature data, calculate the length, width, height, and limb ratio data of the overall character in the three-dimensional space to generate a preliminary character model, and then use the image from the panorama. The feature data extracted in the corresponding image data, such as facial feature data, hair styling feature data, wearing feature data, color feature data, etc., are used to render the generated character models one by one. Optionally, the generating unit 13 may further perform splicing processing on the image information of different orientations by using a panoramic stitching fusion technology, thereby finally generating a three-dimensional character image corresponding to the photographed contact to be identified. Optionally, after the generating unit 13 generates the three-dimensional image of the to-be-identified contact, the three-dimensional image may also be saved in the storage unit, so as to contact the three-dimensional image with the to-be-identified identifier when receiving the contact identification instruction. People are connected.
可选地，在本发明实施例中，上述三维形象引擎是为便于生成本发明实施例中的三维形象而开发的一种三维图形引擎。相关技术中一般都是在微机上使用开放图形库(Open Graphics Library，简称为：OpenGL)或多媒体编程接口(Direct eXtension，简称为：DirectX)等大型开发工具编写三维图形应用，但由于三维图形涉及到许多算法和专业知识，要快速的开发三维应用程序仍然具有一定的困难。因此，三维(3Dimensions，简称为：3D)应用程序的开发需要一个封装了硬件操作和图形算法，同时也简单易用且功能丰富的三维图形开发环境，而这个三维图形开发环境可以称作三维图形引擎。该三维图形引擎例如包括：面向对象图形渲染(Object-Oriented Graphics Rendering Engine，简称为：OGRE)引擎、开放场景图(Open Scene Graph，简称为：OSG)引擎等。Optionally, in the embodiment of the present invention, the three-dimensional image engine is a three-dimensional graphics engine developed for facilitating generation of a three-dimensional image in the embodiment of the present invention. In the related art, a three-dimensional graphics application is generally written on a microcomputer using a large development tool such as an Open Graphics Library (OpenGL) or a multimedia programming interface (Direct eXtension, referred to as DirectX), but the three-dimensional graphics are involved. To many algorithms and expertise, it is still difficult to quickly develop 3D applications. Therefore, the development of 3D (3Dmensions, 3D) applications requires a 3D graphics development environment that encapsulates hardware operations and graphics algorithms, as well as an easy-to-use and feature-rich environment. This 3D graphics development environment can be called 3D graphics. engine. The three-dimensional graphics engine includes, for example, an Object-Oriented Graphics Rendering Engine (OGRE) engine, an Open Scene Graph (OSG) engine, and the like.
本发明实施例中，移动终端基于待标识联系人的全景照片进行生成三维形象所需的特征数据的提取，进而根据所提取的特征数据，启动三维形象引擎以生成相应的三维形象。本发明实施例能够方便快捷的生成所拍摄对象的三维形象，并用于标识相应的联系人，从而满足用户的个性化使用需求，提升用户使用体验。 In the embodiment of the present invention, the mobile terminal performs extraction of feature data required for generating a three-dimensional image based on the panoramic photo of the contact to be identified, and then starts the three-dimensional image engine to generate a corresponding three-dimensional image according to the extracted feature data. The embodiment of the invention can conveniently and quickly generate a three-dimensional image of the captured object, and is used to identify the corresponding contact, thereby satisfying the personalized use requirement of the user and improving the user experience.
可选地，基于上述图5所示实施例的移动终端，本发明实施例还提供一种移动终端，如图6所示，为本发明实施例提供的移动终端中一种第一提取单元的结构示意图。本实施例提供的移动终端中，第一提取单元12可以包括：标定子单元121及获取子单元122。Optionally, based on the mobile terminal of the embodiment shown in FIG. 5, the embodiment of the present invention further provides a mobile terminal, as shown in FIG. 6, which is a first extracting unit of the mobile terminal provided by the embodiment of the present invention. Schematic. In the mobile terminal provided by this embodiment, the first extraction unit 12 may include: a target stator unit 121 and an acquisition sub-unit 122.
其中，标定子单元121，设置为：提取拍摄单元11存储的图像数据中的人像数据，并对所述图像数据中不同朝向的人像数据进行标定；The standard stator unit 121 is configured to: extract the portrait data in the image data stored by the photographing unit 11 and calibrate the portrait data of different orientations in the image data;
由于拍摄所得到的图像数据中包含有待标识联系人的人像数据以及该人物对象所在环境的图像数据，因此，标定子单元121首先可以将待标识联系人的人像数据从图像数据中提取出来单独进行处理。在实际引用中，由于图像数据中待标识联系人的人像数据为一整体的闭合区域，因此，可采用图像边缘检测算法区分人物与背景环境，进而将所检测确定的像素边缘闭合后所对应的图像数据提取出来即可得到待标识联系人的人像数据。此外，又由于图像数据包括不同朝向的图像数据，因此，标定子单元121还可以对不同朝向的图像数据中的人像数据进行一一标定以用于进行区分。例如，可采用人体朝向检测算法对所述人像数据进行标定，比如以人物对象的正面为参照，每隔45度标定一个人体朝向，则对于360度方位的人物对象的朝向可标定为八个朝向。不同朝向下对应的人物对象图像中的特征数据绝大部分都不相同，因此可进行不同朝向方向下的人物对象特征数据提取。Since the image data obtained by the shooting includes the portrait data of the contact to be identified and the image data of the environment in which the character object is located, the target stator unit 121 may first extract the portrait data of the contact to be identified from the image data separately. deal with. In the actual reference, since the portrait data of the contact to be identified in the image data is a closed area of the whole, an image edge detection algorithm may be used to distinguish the person from the background environment, and then the corresponding determined pixel edge is closed. The image data is extracted to obtain the portrait data of the contact to be identified. In addition, since the image data includes image data of different orientations, the target stator unit 121 can also perform one-to-one calibration on the portrait data in the image data of different orientations for distinguishing. For example, the human body image data may be calibrated by using a human body orientation detection algorithm, such as calibrating a human body orientation every 45 degrees with respect to the front side of the human object, and the orientation of the human object for the 360 degree orientation may be calibrated to eight orientations. . The feature data in the corresponding person object image in different orientations is mostly different, so the character object feature data extraction in different orientation directions can be performed.
获取子单元122，设置为：从标定子单元121标定的待标识联系人在不同朝向下的人像数据中获取特征数据，该特征数据包括人脸纹理特征数据、身体比例的特征数据、发型特征数据、穿着特征数据、颜色特征数据。The obtaining sub-unit 122 is configured to: obtain feature data from the portrait data of the contact to be identified, which is calibrated by the standard stator unit 121, in different orientations, the feature data includes facial texture feature data, body scale feature data, and hairstyle feature data. , wearing feature data, color feature data.
在实际应用中，脸部图像数据特征是一个非常重要的区别特征，因此，本发明实施例中获取子单元122可以对不同朝向下所有图像数据进行人脸检测，并确定存在人脸的图像数据中人脸图像所在位置区域，然后再在此基础上定位人脸关键点的位置，例如包括眼睛中心、嘴角、鼻梁等，由于拍摄过程中所选定的拍摄距离、角度的不同，因而对应的图像中人物的头部大小、角度朝向也不相同，因此，可以通过将人脸通过缩放和/或旋转和/或拉伸等处理以得到一个预设的标准大小的正常脸部头像后，再进行脸部区域特征数据的提取。本发明实施例中，对于脸部区域特征数据的提取方式不限，例如 可采用局部二值模式(Local Binary Patterns，简称为：LBP)算法，或者可以采用方向梯度直方图(Histogram of Oriented Gradient简称为：HOG)算法、Gabor滤波器算法等进行图像的特征提取。In an actual application, the facial image data feature is a very important distinguishing feature. Therefore, in the embodiment of the present invention, the obtaining sub-unit 122 can perform face detection on all image data in different orientations, and determine image data of the presence of the face. The position of the face image is located, and then the position of the key point of the face is located on the basis of the image, for example, including the center of the eye, the corner of the mouth, the bridge of the nose, etc., due to the different shooting distances and angles selected during the shooting, the corresponding The head size and angle orientation of the characters in the image are also different. Therefore, the face can be obtained by scaling and/or rotating and/or stretching to obtain a preset standard size normal face avatar. The extraction of facial region feature data is performed. In the embodiment of the present invention, the manner of extracting the feature data of the face region is not limited, for example, The local Binary Patterns (LBP) algorithm may be used, or the feature extraction may be performed by using a Histogram of Oriented Gradient (HOG) algorithm or a Gabor filter algorithm.
可选地，为使得生成的三维人物形象更为逼真，除需要提取三维人物的脸部特征数据外，获取子单元122还可以确定不同朝向下人像数据中的头部、上半身、下半身及四肢所在区域，以提取该待标识联系人的头部、上半身、下半身及四肢的长度比例的特征数据(即身体比例的特征数据)。在实际应用中，可以从人像数据中根据人体不同部位的相对位置以及相关度函数来区分出人物的头部、上半身、下半身和四肢的区域，然后根据所占像素的比例提取出头部、上半身、下半身以及四肢的长度比例的特征数据。Optionally, in order to make the generated three-dimensional character image more realistic, in addition to extracting the facial feature data of the three-dimensional character, the obtaining sub-unit 122 may also determine the head, the upper body, the lower body and the limbs in the different orientation image data. a region for extracting feature data (ie, feature data of the body ratio) of the length ratio of the head, the upper body, the lower body, and the limbs of the contact to be identified. In practical applications, the head, upper body, lower body and limbs of the person can be distinguished from the portrait data according to the relative positions of different parts of the human body and the correlation function, and then the head and upper body are extracted according to the proportion of the occupied pixels. Characteristic data of the length ratio of the lower body and the limbs.
可选地，为使得生成的三维人物形象更为逼真，获取子单元122还可以提取不同朝向下人像数据中的发型特征数据、穿着特征数据、颜色特征数据等。在实际应用中，可以采用边缘检测及特征提取相结合的方式，获取待标识联系人发型的360度外观特征数据；根据上半身与下半身区域，对待标识联系人的穿着进行特征检测，从而抽取衣着的外形款式以及主要印花等特征数据；可选地，获取子单元122还可以对待标识联系人的头发颜色、皮肤颜色、瞳孔颜色、穿着颜色等颜色特征数据进行抽取。Optionally, in order to make the generated three-dimensional character image more realistic, the obtaining sub-unit 122 may also extract hairstyle feature data, wearing feature data, color feature data, and the like in the different orientation-down portrait data. In practical applications, edge detection and feature extraction may be combined to obtain 360 degree appearance feature data of the contact hairstyle to be identified; according to the upper body and the lower body region, feature detection of the contact of the identified contact is performed, thereby extracting the clothing The feature data and the feature data such as the main print; optionally, the acquisition sub-unit 122 may also extract the color feature data of the contact person's hair color, skin color, pupil color, and wearing color.
本发明实施例中，为使得最终生成的三维人物形象更为逼真，可以从待标识联系人的整体图像数据中提取更多的特征数据，例如包括最具区别性的脸部特征数据以及身体四肢比例数据、发型特征数据、穿着特征数据、颜色特征数据等，从而为用户提供更具可玩性的三维人物形象。In the embodiment of the present invention, in order to make the finally generated three-dimensional character image more realistic, more feature data may be extracted from the overall image data of the contact to be identified, for example, including the most distinctive facial feature data and body limbs. Proportional data, hair styling data, wearing feature data, color characterization data, etc., to provide users with a more playable three-dimensional character image.
可选地，基于上述任一实施例中的移动终端，本发明实施例还提供一种移动终端，如图7所示，为本发明实施例提供的再一种移动终端的结构示意图。本实施例以在图5所示移动终端的基础上为例予以示出，获取模块10还可以包括：复制单元14及第二提取单元15。Optionally, based on the mobile terminal in any of the foregoing embodiments, the embodiment of the present invention further provides a mobile terminal, as shown in FIG. 7 , which is a schematic structural diagram of another mobile terminal according to an embodiment of the present invention. The embodiment is shown on the basis of the mobile terminal shown in FIG. 5, and the obtaining module 10 further includes: a copying unit 14 and a second extracting unit 15.
其中，复制单元14，设置为：在侦测到与该待标识联系人的通话时，复制所接收到的该待标识联系人的声音数据，并将复制得到的声音数据作为该待标识联系人的声音样本。 The copying unit 14 is configured to: when detecting a call with the to-be-identified contact, copy the received voice data of the to-be-identified contact, and use the copied voice data as the to-be-identified contact Sound sample.
在本发明实施例中，可以在侦测到与待标识联系人语音通话时，由复制单元14通过音频输入接口复制对端传输过来的声音数据，将所复制得到的声音数据作为该待标识联系人的声音样本；或者，也可以在侦测到录音功能开启时，通过音频输入接口复制麦克风传输的声音数据，将所复制得到的声音数据作为所述待标识联系人的声音样本。在实际应用中，可以通过在脉冲编码调制(Pulse Code Modulation，简称为：PCM)音频接口处，通过复制的方法进行声音样本的采集，复制一份声音数据作为该待标识联系人的声音样本进行声纹特征信息的提取。In the embodiment of the present invention, when the voice call with the to-be-identified contact is detected, the copying unit 14 copies the voice data transmitted by the opposite end through the audio input interface, and uses the copied voice data as the to-be-identified contact. The sound sample of the person is copied; or the sound data transmitted by the microphone is copied through the audio input interface when the recording function is detected, and the copied sound data is used as the sound sample of the contact to be identified. In practical applications, a sound sample can be collected by a copying method at a pulse code modulation (PCM) audio interface, and a sound data is copied as a sound sample of the contact to be identified. Extraction of voiceprint feature information.
第二提取单元15，设置为：从复制单元14得到的声音样本中提取声纹特征信息。The second extracting unit 15 is configured to extract the voiceprint feature information from the sound samples obtained by the copying unit 14.
在复制单元14获取获取到待标识联系人的声音样本后，第二提取单元15，从复制单元14得到的声音样本中提取声纹特征信息。提取该声纹特征信息的技术可以为：基音频谱及其轮廓、基音帧的能量、基音共振峰的出现频率及其轨迹；线性预测倒谱、线谱对、自相关和对数面积比、Mel频率倒谱系数(Mel Frequency Cepstrum Coefficient，简称为：MFCC)、感知线性预测；小波变换技术等。After the copying unit 14 acquires the sound sample of the contact to be identified, the second extracting unit 15 extracts the voiceprint feature information from the sound samples obtained by the copying unit 14. The technique for extracting the voiceprint feature information may be: a base audio spectrum and its contour, the energy of the pitch frame, the appearance frequency of the pitch formant and its trajectory; linear prediction cepstrum, line spectrum pair, autocorrelation and log area ratio, Mel Frequency Cepstrum Coefficient (MFCC), perceptual linear prediction, wavelet transform technology, etc.
可选地，在第二提取单元15获取到待标识联系人的声纹特征信息后，还可以将该信息保存至移动终端的存储单元中，以供移动终端在接收到联系人标识指令时，将该声纹特征信息与对应的联系人关联起来。Optionally, after the second extraction unit 15 obtains the voiceprint feature information of the contact to be identified, the information may also be saved to the storage unit of the mobile terminal, when the mobile terminal receives the contact identification instruction. The voiceprint feature information is associated with the corresponding contact.
本发明实施例通过采集待标识联系人的声纹特征信息，从而为实现根据语音角色对应的声纹特征信息进行语音播报提供了基础。In the embodiment of the present invention, the voiceprint feature information of the contact to be identified is collected, thereby providing a basis for realizing voice broadcast according to the voiceprint feature information corresponding to the voice character.
本发明实施例提供一种联系人标识方法，如图9所示，为本发明实施例提供的联系人标识方法的流程示意图。An embodiment of the present invention provides a method for identifying a contact, as shown in FIG. 9 , which is a schematic flowchart of a method for identifying a contact provided by an embodiment of the present invention.
本实施例提供的联系人标识方法可以包括以下步骤，即S10～S30：The method for identifying a contact provided in this embodiment may include the following steps, that is, S10 to S30:
S10，当接收到联系人标识指令时，获取待标识联系人对应的三维形象及声纹特征信息。S10: When receiving the contact identifier instruction, obtain the three-dimensional image and voiceprint feature information corresponding to the contact to be identified.
在本发明实施例中，可以通过在移动终端上提供用于标识联系人的快捷 图标，供用户基于该快捷图标触发联系人标识指令；或者，也可以通过在移动终端上提供用于标识联系人的物理按键，供用户基于该物理按键触发联系人标识指令；或者，也可以通过在移动终端上提供用于标识联系人的虚拟按键，供用户基于该虚拟按键触发联系人标识指令。本发明实施例中的声纹特征信息可以为语音信号的语音频谱参数、线性预测参数、小波特征参数等。当移动终端接收到用户触发的联系人标识指令时，获取存储于移动终端存储单元中的待标识联系人对应的三维形象及声纹特征信息。例如，用户触发将联系人小王的信息与小王的三维形象及声纹特征信息进行标识的指令，则获取移动终端中预存的小王的三维形象及小王的声纹特征信息；又例如，用户触发将联系人小李的信息与小李的三维形象及声纹特征信息进行标识的指令，则获取移动终端中预存的小李王的三维形象及小王的声纹特征信息。In the embodiment of the present invention, the shortcut for identifying the contact may be provided on the mobile terminal. An icon for the user to trigger a contact identification instruction based on the shortcut icon; or, by providing a physical button for identifying a contact on the mobile terminal, the user may trigger a contact identification instruction based on the physical button; or A virtual button for identifying a contact is provided on the mobile terminal for the user to trigger a contact identification instruction based on the virtual button. The voiceprint feature information in the embodiment of the present invention may be a voice spectrum parameter, a linear prediction parameter, a wavelet feature parameter, and the like of the voice signal. When the mobile terminal receives the user-triggered contact identification instruction, the mobile terminal acquires the three-dimensional image and voiceprint feature information corresponding to the to-be-identified contact stored in the mobile terminal storage unit. For example, if the user triggers an instruction to identify the information of the contact Xiao Wang and the three-dimensional image and the voiceprint feature information of Xiao Wang, the three-dimensional image of Xiao Wang pre-stored in the mobile terminal and the voiceprint feature information of Xiao Wang are obtained; for example, The user triggers an instruction to identify the information of the contact Xiao Li and Xiao Li's three-dimensional image and voiceprint feature information, and obtains the three-dimensional image of Xiao Li Wang pre-stored in the mobile terminal and the voiceprint feature information of Xiao Wang.
S20，采用获取的三维形象及声纹特征信息对待标识联系人进行标识；S20, using the acquired three-dimensional image and voiceprint feature information to identify the identified contact;
S30，在查看待标识联系人信息时，进行具有该待标识联系人的三维形象及声纹特征信息的语音播报。S30, when viewing the contact information to be identified, performing a voice broadcast with the three-dimensional image and the voiceprint feature information of the contact to be identified.
当获取到待标识联系人的三维形象及声纹特征信息后，对该待标识联系人进行标识，即建立待标识联系人与该待标识联系人的三维形象及声纹特征信息的映射关系。After the three-dimensional image and the voiceprint feature information of the contact to be identified are obtained, the contact to be identified is identified, that is, the mapping relationship between the three-dimensional image and the voiceprint feature information of the contact to be identified is established.
在本发明实施例中，待标识联系人信息可以为联系人应用中待标识联系人信息，或者为短信应用中待标识联系人发的短信信息等。上述具有待标识联系人的三维形象及声纹特征信息的语音播报可以为：在语音播报时，在移动终端的屏幕中显示该待标识联系人的三维形象，同时使用该待标识联系人的声纹特征信息对待播报的文本信息进行语音播报。对上述具有待标识联系人的三维形象及声纹特征信息的语音播报的实现方式，举例予以说明，例如：若待标识联系人信息为联系人应用中待标识联系人信息时，则当查看某个联系人信息时，会在移动终端的屏幕上显示该联系人的三维形象，并使用该联系人的声纹特征信息播报该联系人信息，该联系人信息包括联系人电话号码、联系人地址、联系人工作单位等；又例如：若待标识联系人信息为短信应用中待标识联系人发的短信信息时，则当查看该待标识联系人发的短信信 息时，会在移动终端的屏幕上显示发送该短信的联系人的三维形象，并使用该联系人的声纹特征信息播报该短信内容。In the embodiment of the present invention, the to-be-identified contact information may be the contact information to be identified in the contact application, or the short message information sent by the contact to be identified in the short message application. The voice broadcast having the three-dimensional image and the voiceprint feature information of the contact to be identified may be: displaying the three-dimensional image of the contact to be identified on the screen of the mobile terminal during the voice broadcast, and using the voice of the contact to be identified The pattern information is used for voice broadcast of the text information to be broadcast. For example, if the contact information to be identified is the contact information to be identified in the contact application, when viewing the contact information of the three-dimensional image and the voiceprint feature information of the contact to be identified, for example, when viewing the contact information to be identified in the contact application, When the contact information is displayed, the three-dimensional image of the contact is displayed on the screen of the mobile terminal, and the contact information is broadcasted by using the voiceprint feature information of the contact, the contact information includes a contact phone number and a contact address. And the contact work unit, etc.; for example, if the contact information to be identified is the short message information sent by the contact to be identified in the short message application, then the short message sent by the contact to be identified is viewed. At the time of the message, the three-dimensional image of the contact that sent the short message is displayed on the screen of the mobile terminal, and the short message content of the contact is used to broadcast the short message content.
在实际应用中，本发明实施例的思想也可应用于语音助手等应用中，例如，当用户打开语音助手时，用户可以选择用户聊天的联系人，当选定联系人之后，移动终端的屏幕会显示该联系人所关联的三维形象，并且在聊天时，语音助手的人物会使用该联系人关联的声纹特征信息播报聊天的内容，从而做到用户好像是和真的联系人进行聊天一样，提高用户体验。In practical applications, the idea of the embodiment of the present invention can also be applied to applications such as a voice assistant. For example, when a user opens a voice assistant, the user can select a contact that the user chats, and when the contact is selected, the screen of the mobile terminal The three-dimensional image associated with the contact is displayed, and during the chat, the character of the voice assistant uses the voiceprint feature information associated with the contact to broadcast the content of the chat, so that the user seems to be chatting with the real contact. To improve the user experience.
本发明实施例提供提供的联系人标识方法，通过当接收到联系人标识指令时，获取待标识联系人的三维形象及声纹特征信息，并采用获取的三维形象及声纹特征信息对该待标识联系人进行标识，从而有效解决了相关技术中联系人的标识方式十分简单，且不够生动形象的问题。另外，还通过在查看上述待标识联系人信息时，对该待标识联系人信息进行具有该待标识联系人的三维形象及声纹特征信息的语音播报，从而满足了用户日益增长的个性化需求，提高用户的体验及可玩性。An embodiment of the present invention provides a method for identifying a contact, by acquiring a three-dimensional image and voiceprint feature information of a contact to be identified when receiving a contact identification instruction, and using the acquired three-dimensional image and voiceprint feature information The identification of the contact is performed to effectively solve the problem that the identification method of the contact in the related art is very simple and not vivid. In addition, when viewing the contact information to be identified, the contact information of the to-be-identified contact information is broadcasted with the three-dimensional image and the voiceprint feature information of the contact to be identified, thereby satisfying the increasing personalized demand of the user. Improve user experience and playability.
可选地，基于上述图9所示实施例的联系人标识方法，本发明实施例还提供一种联系人标识方法，如图10所示，为本发明实施例提供的另一种联系人标识方法的流程示意图。本实施例提供的联系人标识方法中，在查看待标识联系人信息时，进行具有该待标识联系人三维形象及声纹特征信息的语音播报，S30可以包括如下步骤，即S31～S33：Optionally, based on the contact identification method in the embodiment shown in FIG. 9 , the embodiment of the present invention further provides a contact identification method, as shown in FIG. 10 , which is another contact identifier provided by the embodiment of the present invention. Schematic diagram of the process. In the contact identification method provided by the embodiment, when the contact information to be identified is viewed, the voice broadcast having the three-dimensional image and the voiceprint feature information of the to-be-identified contact is performed, and S30 may include the following steps, that is, S31 to S33:
S31，在查看待标识联系人信息时，确定待播报文本，并合成待播报文本的标准发音；S31. When viewing the contact information to be identified, determining the text to be broadcasted, and synthesizing the standard pronunciation of the text to be broadcasted;
S32，根据该待标识联系人的声纹特征信息对标准发音进行修饰，得到该待标识联系人的发音特征的声音波形；S32, modifying the standard pronunciation according to the voiceprint feature information of the to-be-identified contact, to obtain a sound waveform of the pronunciation feature of the contact to be identified;
S33，输出声音波形进行语音播报，并在移动终端的屏幕上显示该待标识联系人的三维形象。S33. Output a sound waveform for voice broadcast, and display a three-dimensional image of the contact to be identified on a screen of the mobile terminal.
在本发明实施例中，当查看待标识联系人信息时，首先确定待播报文本，并合成该待播报文本的标准发音；然后，根据该待标识联系人的声纹特征信 息对标准发音进行修饰，得到该待标识联系人的发音特征的声音波形；最后，输出声音波形进行语音播报，同时在移动终端的屏幕上显示该待标识联系人的三维形象，实现三维形象及声纹特征信息的语音播报。可选地，可在输出所述声音波形时，可以将该声音波形发送至音频输出接口(如MIC、耳机、HDMI、自带的扬声器等)进行播报。In the embodiment of the present invention, when the contact information to be identified is viewed, the text to be broadcast is first determined, and the standard pronunciation of the text to be broadcasted is synthesized; and then, according to the voiceprint characteristic letter of the contact to be identified The standard pronunciation is modified to obtain the sound waveform of the pronunciation feature of the contact to be identified; finally, the sound waveform is output for voice broadcast, and the three-dimensional image of the contact to be identified is displayed on the screen of the mobile terminal to realize the three-dimensional image and Voice broadcast of voiceprint feature information. Optionally, when the sound waveform is output, the sound waveform may be sent to an audio output interface (such as a MIC, a headphone, an HDMI, a built-in speaker, etc.) for broadcast.
本发明实施例通过在进行具有待标识联系人的声纹特征信息的语音播报时，同时显示待标识联系人的三维形象，使得语音播报的过程更加生动形象，提高了用户体验性。In the embodiment of the present invention, when the voice broadcast of the voiceprint feature information of the contact to be identified is performed, the three-dimensional image of the contact to be identified is simultaneously displayed, so that the process of the voice broadcast is more vivid and the user experience is improved.
可选地，基于上述图9和图10所示实施例的联系人标识方法，本发明实施例还提供一种联系人标识方法，如图11所示，为本发明实施例提供的又一种联系人标识方法的流程示意图。本实施例以在图9所示实施例的基础上为例予以示出，本实施例中的S10可以包括如下步骤，即S11～S13：Optionally, based on the method for the contact identification of the embodiment shown in FIG. 9 and FIG. 10, the embodiment of the present invention further provides a method for identifying a contact, as shown in FIG. A schematic diagram of the process of the contact identification method. This embodiment is shown as an example on the basis of the embodiment shown in FIG. 9. S10 in this embodiment may include the following steps, namely, S11 to S13:
S11，启动移动终端的摄像头应用中的全景拍摄模式，以拍摄并存储待标识联系人的图像数据。S11. Start a panoramic shooting mode in the camera application of the mobile terminal to capture and store image data of the contact to be identified.
本发明实施例中，通过启动摄像头应用中的全景拍摄模式，对拍照对象(即待标识联系人)进行对焦，然后沿同一距离开始围绕拍照对象进行圆周运动，顺时针或逆时针均可，直到摄像头获取了该拍照对象的360度方位的图像数据，同样参考图8所示的拍摄过程示意图。在拍摄全景照片的过程中，为了保证获取的不同角度及方位的图像之间有足够的重叠信息以供后续进行三维模型的拼接处理，可以在拍摄过程中可使用重力传感器、姿态传感器、陀螺仪、罗盘等对拍摄过程进行监测，以判断当前的移动终端是否处于合适的水平位置，并根据检测情况给予用户一定的语音提示；上述语音提示例如为：摄像头是否移动在一个可容忍的拍摄角度位置，摄像头转动的角度是否过大或过小等，从而保证获取的不同角度及方位的图像之间有足够的重叠信息。待拍摄完成后，还可以把拍摄的全景照片(即图像数据)保存至移动终端的存储单元内，以便于后续进行三维形象的生成处理。In the embodiment of the present invention, by starting the panoramic shooting mode in the camera application, the photographing object (ie, the contact to be identified) is focused, and then the circular motion is started around the photographing object along the same distance, clockwise or counterclockwise, until The camera acquires the image data of the 360-degree orientation of the photographed object, and also refers to the photographing process diagram shown in FIG. In the process of taking a panoramic photo, in order to ensure that there are enough overlapping information between different angles and azimuth images for subsequent splicing of the three-dimensional model, gravity sensor, attitude sensor, gyroscope can be used during the shooting process. The compass, etc. monitors the shooting process to determine whether the current mobile terminal is in a proper horizontal position, and gives the user a certain voice prompt according to the detection situation; the voice prompt is, for example, whether the camera is moved at a tolerable shooting angle position. Whether the angle of rotation of the camera is too large or too small, so as to ensure sufficient overlapping information between images acquired at different angles and orientations. After the shooting is completed, the captured panoramic photo (ie, image data) can also be saved into the storage unit of the mobile terminal, so as to facilitate subsequent generation processing of the three-dimensional image.
S12，从图像数据中提取用于生成待标识联系人的三维形象的特征数据。S12. Extract feature data for generating a three-dimensional image of the contact to be identified from the image data.
在获得待标识联系人的图像数据后，从该图像数据中提取生成该待标识 联系人的三维形象所需的特征数据。本发明实施例中的特征数据可以包括：待标识联系人的面部纹理、发型的360度外观特征、肤色、发色、瞳孔颜色、身高、穿着、四肢比例等特征数据。After obtaining the image data of the contact to be identified, extracting and generating the to-be-identified image from the image data The feature data required for the 3D image of the contact. The feature data in the embodiment of the present invention may include: facial texture of the contact to be identified, 360 degree appearance features of the hairstyle, skin color, hair color, pupil color, height, wearing, limb ratio, and the like.
S13，根据所提取的特征数据，启动三维形象引擎以生成该待标识联系人的三维形象，并保存该三维形象。S13. Start, according to the extracted feature data, a three-dimensional image engine to generate a three-dimensional image of the contact to be identified, and save the three-dimensional image.
在提取到用于生成待标识联系人的三维形象的特征数据后，根据所提取的特征数据，启动三维形象引擎以生成该待标识联系人的三维形象。在实际应用中，将提取的特征数据进行升维计算，计算出在三维空间中整体人物的长宽高以及四肢比例数据，以生成一个初步的人物模型，然后使用从全景图片中相对应的图像数据中所提取的特征数据，比如脸部特征数据、发型特征数据、穿着特征数据、颜色特征数据等对生成的人物模型一一进行渲染。可选地，还可采用全景拼接融合技术将不同朝向的图像信息进行拼接处理，从而最终生成与所拍摄的待标识联系人相对应的三维人物形象。可选地，在生成该待标识联系人的三维形象后，还可以将该三维形象保存到存储单元中，以便在移动终端接收到联系人标识指令时，将该三维形象与该待标识联系人关联起来。After extracting the feature data for generating the three-dimensional image of the contact to be identified, according to the extracted feature data, the three-dimensional image engine is activated to generate a three-dimensional image of the contact to be identified. In practical applications, the extracted feature data is subjected to the ascending dimension calculation, and the length, width, height and limb ratio data of the overall character in the three-dimensional space are calculated to generate a preliminary character model, and then the corresponding image from the panoramic image is used. The feature data extracted in the data, such as facial feature data, hair styling feature data, wearing feature data, color feature data, etc., are used to render the generated character models one by one. Optionally, the panoramic splicing fusion technology may also be used to splicing the image information of different orientations, thereby finally generating a three-dimensional character image corresponding to the captured contact to be identified. Optionally, after the three-dimensional image of the to-be-identified contact is generated, the three-dimensional image may also be saved to the storage unit, so that when the mobile terminal receives the contact identification instruction, the three-dimensional image and the to-be-identified contact are Linked up.
可选地，在本发明实施例中，上述三维形象引擎是为便于生成本发明实施例中的三维形象而开发的一种三维图形引擎。相关技术中一般都是在微机上使用OpenGL或DirectX等大型开发工具编写三维图形应用，但由于三维图形涉及到许多算法和专业知识，要快速的开发三维应用程序仍然具有一定的困难。因此，3D应用程序的开发需要一个封装了硬件操作和图形算法，同时也简单易用且功能丰富的三维图形开发环境，而这个三维图形开发环境可以称作三维图形引擎。该三维图形引擎例如包括：OGRE引擎、OSG引擎等。Optionally, in the embodiment of the present invention, the three-dimensional image engine is a three-dimensional graphics engine developed for facilitating generation of a three-dimensional image in the embodiment of the present invention. In the related art, a three-dimensional graphics application is generally written on a microcomputer using a large development tool such as OpenGL or DirectX. However, since the three-dimensional graphics involve many algorithms and professional knowledge, it is still difficult to rapidly develop a three-dimensional application. Therefore, the development of 3D applications requires a three-dimensional graphics development environment that encapsulates hardware operations and graphics algorithms, as well as an easy-to-use and feature-rich environment. This three-dimensional graphics development environment can be called a three-dimensional graphics engine. The three-dimensional graphics engine includes, for example, an OGRE engine, an OSG engine, and the like.
本发明实施例中，移动终端基于待标识联系人的全景照片进行生成三维形象所需的特征数据的提取，进而根据所提取的特征数据，启动三维形象引擎以生成相应的三维形象。本发明实施例能够方便快捷的生成所拍摄对象的三维形象，并用于标识相应的联系人，从而满足用户的个性化使用需求，提升用户使用体验。 In the embodiment of the present invention, the mobile terminal performs extraction of feature data required for generating a three-dimensional image based on the panoramic photo of the contact to be identified, and then starts the three-dimensional image engine to generate a corresponding three-dimensional image according to the extracted feature data. The embodiment of the invention can conveniently and quickly generate a three-dimensional image of the captured object, and is used to identify the corresponding contact, thereby satisfying the personalized use requirement of the user and improving the user experience.
可选地，基于上述图11所示实施例的联系人标识方法，本发明实施例还提供一种联系人标识方法，如图12所示，为本发明实施例提供的联系人标识方法中一种提取用于生成待标识联系人的三维形象的特征数据的流程示意图。本实施例提供的方法中，S12包括可以包括如下步骤，即S121～S122：Optionally, based on the method for the contact identification of the embodiment shown in FIG. 11 , the embodiment of the present invention further provides a method for identifying a contact, as shown in FIG. 12 , which is a method for identifying a contact provided by an embodiment of the present invention. A flow diagram for extracting feature data for generating a three-dimensional image of a contact to be identified. In the method provided in this embodiment, the step S12 includes the following steps, that is, S121 to S122:
S121，提取图像数据中的人像数据，并对该图像数据中不同朝向的人像数据进行标定。S121, extracting portrait data in the image data, and calibrating the portrait data of different orientations in the image data.
由于拍摄所得到的图像数据中包含有待标识联系人的人像数据以及该人物对象所在环境的图像数据，因此，首先可以将待标识联系人的人像数据从图像数据中提取出来单独进行处理。在实际引用中，由于图像数据中待标识联系人的人像数据为一整体的闭合区域，因此，可采用图像边缘检测算法区分人物与背景环境，进而将所检测确定的像素边缘闭合后所对应的图像数据提取出来即可得到待标识联系人的人像数据。此外，又由于图像数据包括不同朝向的图像数据，因此，还可以对不同朝向的图像数据中的人像数据进行一一标定以用于进行区分。例如，可采用人体朝向检测算法对所述人像数据进行标定，比如以人物对象的正面为参照，每隔45度标定一个人体朝向，则对于360度方位的人物对象的朝向可标定为八个朝向。不同朝向下对应的人物对象图像中的特征数据绝大部分都不相同，因此可进行不同朝向方向下的人物对象特征数据提取。Since the image data obtained by the shooting includes the portrait data of the contact to be identified and the image data of the environment in which the character object is located, the portrait data of the contact to be identified may first be extracted from the image data and processed separately. In the actual reference, since the portrait data of the contact to be identified in the image data is a closed area of the whole, an image edge detection algorithm may be used to distinguish the person from the background environment, and then the corresponding determined pixel edge is closed. The image data is extracted to obtain the portrait data of the contact to be identified. In addition, since the image data includes image data of different orientations, it is also possible to perform one-to-one calibration of the portrait data in the image data of different orientations for distinguishing. For example, the human body image data may be calibrated by using a human body orientation detection algorithm, such as calibrating a human body orientation every 45 degrees with respect to the front side of the human object, and the orientation of the human object for the 360 degree orientation may be calibrated to eight orientations. . The feature data in the corresponding person object image in different orientations is mostly different, so the character object feature data extraction in different orientation directions can be performed.
S122，从待标识联系人在不同朝向下的人像数据中获取特征数据，该特征数据包括人脸纹理特征数据、身体比例的特征数据、发型特征数据、穿着特征数据、颜色特征数据。S122. Acquire feature data from the portrait data of the contact to be identified in different orientations, where the feature data includes face texture feature data, body scale feature data, hair style feature data, wearing feature data, and color feature data.
在实际应用中，脸部图像数据特征是一个非常重要的区别特征，因此，本发明实施例中，可以对不同朝向下所有图像数据进行人脸检测，并确定存在人脸的图像数据中人脸图像所在位置区域，然后再在此基础上定位人脸关键点的位置，例如包括眼睛中心、嘴角、鼻梁等，由于拍摄过程中所选定的拍摄距离、角度的不同，因而对应的图像中人物的头部大小、角度朝向也不相同，因此，可以通过将人脸通过缩放和/或旋转和/或拉伸等处理以得到一个预设的标准大小的正常脸部头像后，再进行脸部区域特征数据的提取。本发明实施例中，对于脸部区域特征数据的提取方式不限，例如可采用LBP算 法，或者HOG算法、Gabor滤波器算法等进行图像的特征提取。In practical applications, the facial image data feature is a very important distinguishing feature. Therefore, in the embodiment of the present invention, face detection can be performed on all image data in different orientations, and the face in the image data of the face is determined. The position of the image is located, and then the position of the key point of the face is located on the basis of the image, for example, including the center of the eye, the corner of the mouth, the bridge of the nose, etc., due to the different shooting distances and angles selected during the shooting, the corresponding characters in the image The head size and the angle orientation are also different. Therefore, the face can be obtained by scaling and/or rotating and/or stretching to obtain a normal standard face avatar of a preset standard size. Extraction of regional feature data. In the embodiment of the present invention, the method for extracting the feature data of the face region is not limited, for example, the LBP calculation may be adopted. The method, or the HOG algorithm, the Gabor filter algorithm, etc., performs feature extraction of the image.
可选地，为使得生成的三维人物形象更为逼真，除需要提取三维人物的脸部特征数据外，还可以确定不同朝向下人像数据中的头部、上半身、下半身及四肢所在区域，以提取该待标识联系人的头部、上半身、下半身及四肢的长度比例的特征数据(即身体比例的特征数据)。在实际应用中，可以从人像数据中根据人体不同部位的相对位置以及相关度函数来区分出人物的头部、上半身、下半身和四肢的区域，然后根据所占像素的比例提取出头部、上半身、下半身以及四肢的长度比例的特征数据。Optionally, in order to make the generated three-dimensional character image more realistic, in addition to extracting the facial feature data of the three-dimensional character, the head, the upper body, the lower body and the limbs in the different orientation image data may be determined to extract Characteristic data of the length ratio of the head, upper body, lower body, and limbs of the contact to be identified (ie, feature data of the body ratio). In practical applications, the head, upper body, lower body and limbs of the person can be distinguished from the portrait data according to the relative positions of different parts of the human body and the correlation function, and then the head and upper body are extracted according to the proportion of the occupied pixels. Characteristic data of the length ratio of the lower body and the limbs.
可选地，为使得生成的三维人物形象更为逼真，还可以提取不同朝向下人像数据中的发型特征数据、穿着特征数据、颜色特征数据等。在实际应用中，可以采用边缘检测及特征提取相结合的方式，获取待标识联系人发型的360度外观特征数据；根据上半身与下半身区域，对待标识联系人的穿着进行特征检测，从而抽取衣着的外形款式以及主要印花等特征数据；可选地，还可以对待标识联系人的头发颜色、皮肤颜色、瞳孔颜色、穿着颜色等颜色特征数据进行抽取。Optionally, in order to make the generated three-dimensional character image more realistic, hair style feature data, wearing feature data, color feature data, and the like in different orientation-oriented portrait data may also be extracted. In practical applications, edge detection and feature extraction may be combined to obtain 360 degree appearance feature data of the contact hairstyle to be identified; according to the upper body and the lower body region, feature detection of the contact of the identified contact is performed, thereby extracting the clothing Characteristic data such as shape and main printing; optionally, color feature data such as hair color, skin color, pupil color, and wearing color of the contact person can be extracted.
本发明实施例中，为使得最终生成的三维人物形象更为逼真，可以从待标识联系人的整体图像数据中提取更多的特征数据，例如包括最具区别性的脸部特征数据以及身体四肢比例数据、发型特征数据、穿着特征数据、颜色特征数据等，从而为用户提供更具可玩性的三维人物形象。In the embodiment of the present invention, in order to make the finally generated three-dimensional character image more realistic, more feature data may be extracted from the overall image data of the contact to be identified, for example, including the most distinctive facial feature data and body limbs. Proportional data, hair styling data, wearing feature data, color characterization data, etc., to provide users with a more playable three-dimensional character image.
可选地，基于上述任一实施例中的联系人标识方法，本发明实施例还提供一种联系人标识方法，如图13所示，为本发明实施例提供的再一种联系人标识方法的流程示意图。本实施例以在图11所示实施例的基础上为例予以示出，本实施例中的S10还可以包括如下步骤，即S14～S15：Optionally, based on the contact identification method in any of the foregoing embodiments, the embodiment of the present invention further provides a contact identification method, as shown in FIG. 13 , which is another contact identification method provided by an embodiment of the present invention. Schematic diagram of the process. This embodiment is shown as an example on the basis of the embodiment shown in FIG. 11. S10 in this embodiment may further include the following steps, namely, S14 to S15:
S14，在侦测到与该待标识联系人的通话时，复制所接收到的该待标识联系人的声音数据，并将复制得到的声音数据作为该待标识联系人的声音样本。S14. When detecting a call with the to-be-identified contact, copy the received voice data of the to-be-identified contact, and use the copied voice data as a sound sample of the to-be-identified contact.
在本发明实施例中，可以在侦测到与待标识联系人语音通话时，通过音频输入接口复制对端传输过来的声音数据，将所复制得到的声音数据作为该 待标识联系人的声音样本；或者，也可以在侦测到录音功能开启时，通过音频输入接口复制麦克风传输的声音数据，将所复制得到的声音数据作为该待标识联系人的声音样本。在实际应用中，可以通过在PCM音频接口处，通过复制的方法进行声音样本的采集，复制一份声音数据作为该待标识联系人的声音样本进行声纹特征信息的提取。In the embodiment of the present invention, when the voice call with the to-be-identified contact is detected, the voice data transmitted by the opposite end is copied through the audio input interface, and the copied sound data is used as the voice data. A sound sample of the contact to be identified; or, when the recording function is detected to be turned on, copy the sound data transmitted by the microphone through the audio input interface, and use the copied sound data as the sound sample of the contact to be identified. In practical applications, the sound sample can be collected by copying at the PCM audio interface, and a voice data is copied as the sound sample of the contact to be identified to extract the voiceprint feature information.
S15，从声音样本中提取声纹特征信息。S15. Extract voiceprint feature information from the sound sample.
在获取到待标识联系人的声音样本后，从该声音样本中提取声纹特征信息。提取该声纹特征信息的技术可以为：基音频谱及其轮廓、基音帧的能量、基音共振峰的出现频率及其轨迹；线性预测倒谱、线谱对、自相关和对数面积比、MFCC、感知线性预测；小波变换技术等。After acquiring the sound sample of the contact to be identified, the voiceprint feature information is extracted from the sound sample. The technique for extracting the voiceprint feature information may be: a base audio spectrum and its contour, the energy of the pitch frame, the appearance frequency of the pitch formant and its trajectory; linear prediction cepstrum, line spectrum pair, autocorrelation and log area ratio, MFCC , perceptual linear prediction; wavelet transform technology.
可选地，在获取到待标识联系人的声纹特征信息后，还可以将该信息保存至移动终端的存储单元中，以供移动终端在接收到联系人标识指令时，将该声纹特征信息与对应的联系人关联起来。Optionally, after acquiring the voiceprint feature information of the to-be-identified contact, the information may be saved to the storage unit of the mobile terminal, where the mobile terminal obtains the voiceprint feature when receiving the contact identifier instruction. The information is associated with the corresponding contact.
本发明实施例通过采集待标识联系人的声纹特征信息，从而为实现根据语音角色对应的声纹特征信息进行语音播报提供了基础。In the embodiment of the present invention, the voiceprint feature information of the contact to be identified is collected, thereby providing a basis for realizing voice broadcast according to the voiceprint feature information corresponding to the voice character.
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明实施例中的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本发明各个实施例和可选实施方式所提供的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better. Implementation. Based on such understanding, the technical solution in the embodiments of the present invention may be embodied in the form of a software product in essence or in a contribution to the related art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic The disc, the optical disc, includes a number of instructions for causing a terminal device (which may be a cell phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods provided by various embodiments and alternative embodiments of the present invention.
以上仅为本发明的实施例和可选实施例方式，并非因此限制本发明实施例的保护范围，凡是利用本文说明书及说明书附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。 The above are only the embodiments and the alternative embodiments of the present invention, and are not intended to limit the scope of the embodiments of the present invention, and the equivalent structure or equivalent process transformations made by the contents of the specification and the drawings herein, or directly or indirectly The use of other related technical fields is equally included in the scope of patent protection of the present invention.
本领域普通技术人员可以理解上述实施例的全部或部分步骤可以使用计算机程序流程来实现，所述计算机程序可以存储于一计算机可读存储介质中，所述计算机程序在相应的硬件平台上(根据***、设备、装置、器件等)执行，在执行时，包括方法实施例的步骤之一或其组合。One of ordinary skill in the art will appreciate that all or a portion of the steps of the above-described embodiments can be implemented using a computer program flow, which can be stored in a computer readable storage medium on a corresponding hardware platform (according to The system, device, device, device, etc. are executed, and when executed, include one or a combination of the steps of the method embodiments.
可选地，上述实施例的全部或部分步骤也可以使用集成电路来实现，这些步骤可以被分别制作成一个个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。Alternatively, all or part of the steps of the above embodiments may also be implemented by using an integrated circuit. These steps may be separately fabricated into individual integrated circuit modules, or multiple modules or steps may be fabricated into a single integrated circuit module. achieve.
上述实施例中的装置/功能模块/功能单元可以采用通用的计算装置来实现，它们可以集中在单个的计算装置上，也可以分布在多个计算装置所组成的网络上。The devices/function modules/functional units in the above embodiments may be implemented by a general-purpose computing device, which may be centralized on a single computing device or distributed over a network of multiple computing devices.
上述实施例中的装置/功能模块/功能单元以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。上述提到的计算机可读取存储介质可以是只读存储器，磁盘或光盘等。When the device/function module/functional unit in the above embodiment is implemented in the form of a software function module and sold or used as a stand-alone product, it can be stored in a computer readable storage medium. The above mentioned computer readable storage medium may be a read only memory, a magnetic disk or an optical disk or the like.
工业实用性Industrial applicability
本发明实施例通过当接收到联系人标识指令时，获取待标识联系人的三维形象及声纹特征信息，并采用获取的三维形象及声纹特征信息对该待标识联系人进行标识，从而有效解决了相关技术中联系人的标识方式十分简单，且不够生动形象的问题。另外，还通过在查看上述待标识联系人信息时，对该待标识联系人信息进行具有该待标识联系人的三维形象及声纹特征信息的语音播报，从而满足了用户日益增长的个性化需求，提高用户的体验及可玩性。 When receiving the contact identifier instruction, the embodiment obtains the three-dimensional image and the voiceprint feature information of the contact to be identified, and uses the acquired three-dimensional image and voiceprint feature information to identify the contact to be identified, thereby effectively The problem that the identification of the contacts in the related art is very simple and not vivid enough is solved. In addition, when viewing the contact information to be identified, the contact information of the to-be-identified contact information is broadcasted with the three-dimensional image and the voiceprint feature information of the contact to be identified, thereby satisfying the increasing personalized demand of the user. Improve user experience and playability.

Claims (20)

  1. 一种移动终端，包括：A mobile terminal includes:
    获取模块，设置为：当接收到联系人标识指令时，获取待标识联系人对应的三维形象及声纹特征信息；Obtaining a module, configured to: obtain a three-dimensional image and voiceprint feature information corresponding to the contact to be identified when receiving the contact identifier instruction;
    标识模块，设置为：采用所述获取模块获取的所述三维形象及声纹特征信息对所述待标识联系人进行标识；The identifier module is configured to: identify, by using the three-dimensional image and voiceprint feature information acquired by the acquiring module, the contact to be identified;
    播报模块，设置为：在查看所述标识模块所标识的待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报。The broadcast module is configured to: when viewing the contact information to be identified that is identified by the identifier module, perform a voice broadcast with the three-dimensional image and the voiceprint feature information of the contact to be identified.
  2. 根据权利要求1所述的移动终端，其中，所述待标识联系人信息包括：联系人应用中待标识联系人信息，或者，短信应用中待标识联系人发的短信信息。The mobile terminal according to claim 1, wherein the to-be-identified contact information comprises: contact information to be identified in the contact application, or short message information sent by the contact to be identified in the short message application.
  3. 根据权利要求2所述的移动终端，其中，所述待标识联系人信息为所述联系人应用中待标识联系人信息时，所述播报模块设置为：在查看所述标识模块所标识的待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报，包括：The mobile terminal according to claim 2, wherein, when the to-be-identified contact information is the contact information to be identified in the contact application, the broadcast module is configured to: view the identifier to be identified by the identification module When the contact information is identified, the voice broadcast of the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
    在查看所述待标识联系人信息时，在所述移动终端的屏幕上显示所述待标识联系人的三维形象，并使用所述待标识联系人的声纹特征信息播报所述待标识联系人信息。Displaying the three-dimensional image of the to-be-identified contact on the screen of the mobile terminal, and using the voiceprint feature information of the to-be-identified contact to broadcast the to-be-identified contact information.
  4. 根据权利要求2所述的移动终端，其中，所述待标识联系人信息为所述短信应用中待标识联系人发的短信信息时，所述播报模块设置为：在查看所述标识模块所标识的待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报，包括：The mobile terminal according to claim 2, wherein, when the to-be-identified contact information is the short message information sent by the contact to be identified in the short message application, the broadcast module is configured to: view the identifier of the identification module When the contact information is to be identified, the voice broadcast of the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
    在查看所述待标识联系人信息时，在所述移动终端的屏幕上显示发送短信的联系人的三维形象，并使用所述联系人的声纹特征信息播报所述短信的内容。When viewing the to-be-identified contact information, displaying a three-dimensional image of the contact that sends the short message on the screen of the mobile terminal, and using the voiceprint feature information of the contact to broadcast the content of the short message.
  5. 根据权利要求1所述的移动终端，其中，所述播报模块包括：The mobile terminal of claim 1, wherein the broadcast module comprises:
    合成单元，设置为：在查看所述待标识联系人信息时，确定待播报文本，并合成所述待播报文本的标准发音； a synthesizing unit, configured to: when viewing the contact information to be identified, determine a text to be broadcasted, and synthesize a standard pronunciation of the text to be broadcasted;
    修饰单元，设置为：根据所述待标识联系人的声纹特征信息对所述合成单元合成的所述标准发音进行修饰，得到所述待标识联系人的发音特征的声音波形；The modifying unit is configured to: modify the standard pronunciation synthesized by the synthesizing unit according to the voiceprint feature information of the to-be-identified contact, to obtain a sound waveform of the pronunciation feature of the contact to be identified;
    输出单元，设置为：输出所述修饰单元得到的所述声音波形进行语音播报，并在所述移动终端的屏幕上显示所述待标识联系人的三维形象。And an output unit, configured to: output the sound waveform obtained by the modifying unit to perform voice broadcast, and display a three-dimensional image of the contact to be identified on a screen of the mobile terminal.
  6. 根据权利要求1所述的移动终端，其中，所述获取模块包括：The mobile terminal of claim 1, wherein the obtaining module comprises:
    拍摄单元，设置为：启动所述移动终端的摄像头应用中的全景拍摄模式，以拍摄并存储所述待标识联系人的图像数据；a shooting unit, configured to: activate a panoramic shooting mode in a camera application of the mobile terminal to capture and store image data of the contact to be identified;
    第一提取单元，设置为：从所述拍摄单元存储的所述图像数据中提取用于生成所述待标识联系人的三维形象的特征数据；a first extracting unit, configured to: extract, from the image data stored by the shooting unit, feature data for generating a three-dimensional image of the contact to be identified;
    生成单元，设置为：根据所述第一提取单元所提取的所述特征数据，启动三维形象引擎以生成所述待标识联系人的三维形象。And a generating unit, configured to: according to the feature data extracted by the first extracting unit, activate a three-dimensional image engine to generate a three-dimensional image of the contact to be identified.
  7. 根据权利要求6所述的移动终端，其中，所述拍摄单元启动所述移动终端的摄像头应用中的全景拍摄模式，以拍摄并存储所述待标识联系人的图像数据，包括：The mobile terminal of claim 6, wherein the capturing unit activates a panoramic shooting mode in a camera application of the mobile terminal to capture and store image data of the contact to be identified, including:
    启动所述移动终端的摄像头应用中的全景拍摄模式，对所述待标识联系人进行对焦后沿同一距离围绕所述待标识联系人进行圆周运动，直到所述摄像头获取到所述待标识联系人的360度方位的图像数据。Activating a panoramic shooting mode in the camera application of the mobile terminal, and performing focus motion on the to-be-identified contact along the same distance after the contact to be identified is focused until the camera acquires the to-be-identified contact 360-degree image data.
  8. 根据权利要求6所述的移动终端，其中，第一提取单元包括：The mobile terminal of claim 6, wherein the first extracting unit comprises:
    标定子单元，设置为：提取所述拍摄单元存储的所述图像数据中的人像数据，并对所述图像数据中不同朝向的人像数据进行标定；The target stator unit is configured to: extract the portrait data in the image data stored by the photographing unit, and calibrate the portrait data of different orientations in the image data;
    获取子单元，设置为：从所述标定子单元标定的所述待标识联系人在不同朝向下的人像数据中获取所述特征数据，所述特征数据包括人脸纹理特征数据、身体比例的特征数据、发型特征数据、穿着特征数据、颜色特征数据。Obtaining the sub-unit, the method is configured to: acquire the feature data from the image data of the to-be-identified contact that is calibrated by the target stator unit in different orientations, where the feature data includes facial texture feature data and body proportion characteristics Data, hair styling data, wearing feature data, color characterization data.
  9. 根据权利要求1～8中任一项所述的移动终端，其中，所述获取模块还包括：The mobile terminal according to any one of claims 1 to 8, wherein the obtaining module further comprises:
    复制单元，设置为：在侦测到与所述待标识联系人的通话时，复制所接收到的所述待标识联系人的声音数据，并将复制得到的所述声音数据作为所 述待标识联系人的声音样本；a copying unit, configured to: when detecting a call with the to-be-identified contact, copying the received voice data of the to-be-identified contact, and copying the obtained sound data as a Describe the sound sample of the identified contact;
    第二提取单元，设置为：从所述复制单元得到的所述声音样本中提取所述声纹特征信息。The second extracting unit is configured to extract the voiceprint feature information from the sound samples obtained by the copying unit.
  10. 根据权利要求9所述的移动终端，其中，所述复制单元，还设置为：在侦测到所述移动终端的录音功能开启时，复制麦克风传输的所述待标识联系人的声音数据，并将复制得到的所述声音数据作为所述待标识联系人的声音样本。The mobile terminal according to claim 9, wherein the copying unit is further configured to: when detecting that the recording function of the mobile terminal is enabled, copying the sound data of the to-be-identified contact transmitted by the microphone, and The copied sound data is used as a sound sample of the contact to be identified.
  11. 一种联系人标识方法，包括：A method for identifying a contact, comprising:
    当接收到联系人标识指令时，获取待标识联系人对应的三维形象及声纹特征信息；Obtaining a three-dimensional image and voiceprint feature information corresponding to the contact to be identified when receiving the contact identification instruction;
    采用获取的所述三维形象及声纹特征信息对所述待标识联系人进行标识，以在查看所述待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报。And identifying the to-be-identified contact by using the acquired three-dimensional image and voiceprint feature information, to perform three-dimensional image and voiceprint feature information of the contact to be identified when viewing the to-be-identified contact information Voice broadcast.
  12. 根据权利要求11所述的联系人标识方法，其中，所述待标识联系人信息包括：联系人应用中待标识联系人信息，或者，短信应用中待标识联系人发的短信信息。The contact identification method according to claim 11, wherein the to-be-identified contact information comprises: contact information to be identified in the contact application, or short message information sent by the contact to be identified in the short message application.
  13. 根据权利要求12所述的联系人标识方法，其中，所述待标识联系人信息为所述联系人应用中待标识联系人信息时，所述在查看所述标识模块所标识的待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报，包括：The contact identification method according to claim 12, wherein when the contact information to be identified is the contact information to be identified in the contact application, the viewing the contact to be identified identified by the identification module The information is broadcasted by the voice broadcast having the three-dimensional image and the voiceprint feature information of the contact to be identified, including:
    在查看所述待标识联系人信息时，在移动终端的屏幕上显示所述待标识联系人的三维形象，并使用所述待标识联系人的声纹特征信息播报所述待标识联系人信息。When the contact information to be identified is viewed, the three-dimensional image of the contact to be identified is displayed on the screen of the mobile terminal, and the contact information to be identified is broadcasted by using the voiceprint feature information of the contact to be identified.
  14. 根据权利要求12所述的联系人标识方法，其中，所述待标识联系人信息为所述短信应用中待标识联系人发的短信信息时，所述在查看所述标识模块所标识的待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报，包括：The contact identification method according to claim 12, wherein, when the to-be-identified contact information is the short message information sent by the to-be-identified contact in the short-message application, the viewing the identifier to be identified by the identification module When the contact information is obtained, a voice broadcast with the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
    在查看所述待标识联系人信息时，在移动终端的屏幕上显示发送短信的 联系人的三维形象，并使用所述联系人的声纹特征信息播报所述短信的内容。When viewing the contact information to be identified, displaying the short message on the screen of the mobile terminal The three-dimensional image of the contact, and the content of the short message is broadcasted using the voiceprint feature information of the contact.
  15. 根据权利要求11所述的联系人标识方法，其中，所述在查看所述待标识联系人信息时，进行具有所述待标识联系人的三维形象及声纹特征信息的语音播报，包括：The contact identification method according to claim 11, wherein, when the contact information to be identified is viewed, the voice broadcast having the three-dimensional image and the voiceprint feature information of the contact to be identified is performed, including:
    在查看所述待标识联系人信息时，确定待播报文本，并合成所述待播报文本的标准发音；When viewing the contact information to be identified, determining a text to be broadcasted, and synthesizing a standard pronunciation of the text to be broadcasted;
    根据所述待标识联系人的声纹特征信息对所述标准发音进行修饰，得到所述待标识联系人的发音特征的声音波形；And modifying the standard pronunciation according to the voiceprint feature information of the to-be-identified contact, to obtain a sound waveform of the pronunciation feature of the contact to be identified;
    输出所述声音波形进行语音播报，并在移动终端的屏幕上显示所述待标识联系人的三维形象。The sound waveform is output for voice broadcast, and the three-dimensional image of the contact to be identified is displayed on the screen of the mobile terminal.
  16. 根据权利要求11所述的联系人标识方法，其中，所述获取待标识联系人对应的三维形象，包括：The contact identification method according to claim 11, wherein the obtaining the three-dimensional image corresponding to the contact to be identified comprises:
    启动移动终端的摄像头应用中的全景拍摄模式，以拍摄并存储所述待标识联系人的图像数据；Activating a panoramic shooting mode in a camera application of the mobile terminal to capture and store image data of the contact to be identified;
    从所述图像数据中提取用于生成所述待标识联系人的三维形象的特征数据；Extracting, from the image data, feature data for generating a three-dimensional image of the contact to be identified;
    根据所提取的所述特征数据，启动三维形象引擎以生成所述待标识联系人的三维形象。Based on the extracted feature data, a three-dimensional image engine is launched to generate a three-dimensional image of the contact to be identified.
  17. 根据权利要求16所述的联系人标识方法，其中，所述启动移动终端的摄像头应用中的全景拍摄模式，以拍摄并存储所述待标识联系人的图像数据，包括：The contact identification method according to claim 16, wherein the activating the panoramic shooting mode in the camera application of the mobile terminal to capture and store the image data of the contact to be identified includes:
    启动所述移动终端的摄像头应用中的全景拍摄模式，对所述待标识联系人进行对焦后沿同一距离围绕所述待标识联系人进行圆周运动，直到所述摄像头获取到所述待标识联系人的360度方位的图像数据。Activating a panoramic shooting mode in the camera application of the mobile terminal, and performing focus motion on the to-be-identified contact along the same distance after the contact to be identified is focused until the camera acquires the to-be-identified contact 360-degree image data.
  18. 根据权利要求16所述的联系人标识方法，其中，所述从所述图像数据中提取用于生成所述待标识联系人的三维形象的特征数据，包括：The contact identification method according to claim 16, wherein the extracting the feature data for generating the three-dimensional image of the contact to be identified from the image data comprises:
    提取所述图像数据中的人像数据，并对所述图像数据中不同朝向的人像 数据进行标定；Extracting portrait data in the image data and differently facing portraits in the image data Data calibration;
    从所述待标识联系人在不同朝向下的人像数据中获取所述特征数据，所述特征数据包括人脸纹理特征数据、身体比例的特征数据、发型特征数据、穿着特征数据、颜色特征数据。The feature data is acquired from the portrait data of the contact to be identified in different orientations, and the feature data includes face texture feature data, body scale feature data, hair style feature data, wearing feature data, and color feature data.
  19. 根据权利要求11～18中任一项所述的联系人标识方法，其中，所述获取待标识联系人对应的声纹特征，包括：The contact identification method according to any one of claims 11 to 18, wherein the acquiring the voiceprint feature corresponding to the contact to be identified includes:
    在侦测到与所述待标识联系人的通话时，复制所接收到的所述待标识联系人的声音数据，并将复制得到的所述声音数据作为所述待标识联系人的声音样本；And detecting the received voice data of the to-be-identified contact, and using the copied voice data as a sound sample of the to-be-identified contact;
    从所述声音样本中提取所述声纹特征信息。Extracting the voiceprint feature information from the sound samples.
  20. 根据权利要求19所述的联系人标识方法，其中，所述获取待标识联系人对应的声纹特征，还包括：The contact identification method according to claim 19, wherein the obtaining the voiceprint feature corresponding to the contact to be identified further comprises:
    在侦测到所述移动终端的录音功能开启时，复制麦克风传输的所述待标识联系人的声音数据，并将复制得到的所述声音数据作为所述待标识联系人的声音样本。 When the recording function of the mobile terminal is detected, the sound data of the to-be-identified contact transmitted by the microphone is copied, and the copied sound data is used as the sound sample of the contact to be identified.
PCT/CN2016/107329 2015-12-28 2016-11-25 Mobile terminal and method for identifying contact WO2017114048A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201511004834.0 2015-12-28
CN201511004834.0A CN105635452B (en) 2015-12-28 2015-12-28 Mobile terminal and its identification of contacts method

Publications (1)

Publication Number Publication Date
WO2017114048A1 true WO2017114048A1 (en) 2017-07-06

Family

ID=56049914

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/107329 WO2017114048A1 (en) 2015-12-28 2016-11-25 Mobile terminal and method for identifying contact

Country Status (2)

Country Link
CN (1) CN105635452B (en)
WO (1) WO2017114048A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112820270A (en) * 2020-12-17 2021-05-18 北京捷通华声科技股份有限公司 Voice broadcasting method and device and intelligent equipment
US11274102B2 (en) 2017-10-30 2022-03-15 Glaxosmithkline Intellectual Property Development Limited Compounds useful in HIV therapy

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105635452B (en) * 2015-12-28 2019-05-10 努比亚技术有限公司 Mobile terminal and its identification of contacts method
CN106686184A (en) * 2017-01-13 2017-05-17 北京搜狗科技发展有限公司 Contact person adding method, electronic devices and server
CN107680598B (en) * 2017-09-04 2020-12-11 百度在线网络技术（北京）有限公司 Information interaction method, device and equipment based on friend voiceprint address list
CN111261139B (en) * 2018-11-30 2023-12-26 上海擎感智能科技有限公司 Literal personification broadcasting method and system
CN110308792B (en) * 2019-07-01 2023-12-12 北京百度网讯科技有限公司 Virtual character control method, device, equipment and readable storage medium
CN111798829A (en) * 2020-06-30 2020-10-20 中国联合网络通信集团有限公司 Method, system, computer equipment and storage medium for reading text information by voice
CN113012679A (en) * 2021-02-20 2021-06-22 北京百度网讯科技有限公司 Method, apparatus and medium for broadcasting message by voice
CN114549706A (en) * 2022-02-21 2022-05-27 成都工业学院 Animation generation method and animation generation device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1379392A (en) * 2001-04-11 2002-11-13 国际商业机器公司 Feeling speech sound and speech sound translation system and method
CN103067558A (en) * 2013-01-17 2013-04-24 深圳市中兴移动通信有限公司 Method and device associating pictures of contact person in address book
US20150046164A1 (en) * 2013-08-07 2015-02-12 Samsung Electronics Co., Ltd. Method, apparatus, and recording medium for text-to-speech conversion
CN104464716A (en) * 2014-11-20 2015-03-25 北京云知声信息技术有限公司 Voice broadcasting system and method
CN105635452A (en) * 2015-12-28 2016-06-01 努比亚技术有限公司 Mobile terminal and contact person identification method thereof

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101645954A (en) * 2009-08-31 2010-02-10 深圳华为通信技术有限公司 Terminal address book management method and terminal device
CN102170490B (en) * 2011-03-18 2014-11-05 惠州Tcl移动通信有限公司 Mobile phone for old people and dialing method thereof
CN102915350A (en) * 2012-09-29 2013-02-06 北京小米科技有限责任公司 Method, device and equipment for searching contact information
CN103035247B (en) * 2012-12-05 2017-07-07 北京三星通信技术研究有限公司 Based on the method and device that voiceprint is operated to audio/video file

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1379392A (en) * 2001-04-11 2002-11-13 国际商业机器公司 Feeling speech sound and speech sound translation system and method
CN103067558A (en) * 2013-01-17 2013-04-24 深圳市中兴移动通信有限公司 Method and device associating pictures of contact person in address book
US20150046164A1 (en) * 2013-08-07 2015-02-12 Samsung Electronics Co., Ltd. Method, apparatus, and recording medium for text-to-speech conversion
CN104464716A (en) * 2014-11-20 2015-03-25 北京云知声信息技术有限公司 Voice broadcasting system and method
CN105635452A (en) * 2015-12-28 2016-06-01 努比亚技术有限公司 Mobile terminal and contact person identification method thereof

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11274102B2 (en) 2017-10-30 2022-03-15 Glaxosmithkline Intellectual Property Development Limited Compounds useful in HIV therapy
CN112820270A (en) * 2020-12-17 2021-05-18 北京捷通华声科技股份有限公司 Voice broadcasting method and device and intelligent equipment

Also Published As

Publication number Publication date
CN105635452B (en) 2019-05-10
CN105635452A (en) 2016-06-01

Similar Documents

Publication Publication Date Title
WO2017114048A1 (en) Mobile terminal and method for identifying contact
TWI788630B (en) Method, device, computer equipment, and storage medium for generating 3d face model
CN108769562B (en) Method and device for generating special effect video
CN110502954B (en) Video analysis method and device
WO2019109801A1 (en) Method and device for adjusting photographing parameter, storage medium, and mobile terminal
JP7408678B2 (en) Image processing method and head mounted display device
KR102018887B1 (en) Image preview using detection of body parts
WO2017088714A1 (en) Mobile terminal and three-dimensional image generation method therefor
KR102114377B1 (en) Method for previewing images captured by electronic device and the electronic device therefor
WO2020238380A1 (en) Panoramic photography method and terminal device
CN110572711B (en) Video cover generation method and device, computer equipment and storage medium
WO2017124899A1 (en) Information processing method, apparatus and electronic device
US8400532B2 (en) Digital image capturing device providing photographing composition and method thereof
WO2021043121A1 (en) Image face changing method, apparatus, system, and device, and storage medium
CN109360222B (en) Image segmentation method, device and storage medium
KR102004884B1 (en) Method and apparatus for controlling animated image in an electronic device
WO2022033272A1 (en) Image processing method and electronic device
WO2020093798A1 (en) Method and apparatus for displaying target image, terminal, and storage medium
CN110827195A (en) Virtual article adding method and device, electronic equipment and storage medium
JP2014146989A (en) Image pickup device, image pickup method, and image pickup program
CN113850709A (en) Image transformation method and device
CN115150542B (en) Video anti-shake method and related equipment
CN110675473A (en) Method, device, electronic equipment and medium for generating GIF dynamic graph
JP2012257112A (en) Imaging apparatus and program
CN112235650A (en) Video processing method, device, terminal and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16880838

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16880838

Country of ref document: EP

Kind code of ref document: A1