CN109559576A - A kind of children companion robot and its early teaching system self-learning method - Google Patents
A kind of children companion robot and its early teaching system self-learning method Download PDFInfo
- Publication number
- CN109559576A CN109559576A CN201811367002.9A CN201811367002A CN109559576A CN 109559576 A CN109559576 A CN 109559576A CN 201811367002 A CN201811367002 A CN 201811367002A CN 109559576 A CN109559576 A CN 109559576A
- Authority
- CN
- China
- Prior art keywords
- things
- new
- image
- identified
- layer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B5/00—Electrically-operated educational appliances
- G09B5/06—Electrically-operated educational appliances with both visual and audible presentation of the material to be studied
- G09B5/065—Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of children with robot and its early teaching system self-learning method is learned, and self-learning method includes: step A10, training convolutional neural networks；Step A20, using convolutional neural networks to the image zooming-out feature vector of input；Step A30, using product quantification technique to feature vector group quantization；Step A40, according to Imagenet data set generation benchmark alphabet；Step A50 obtains the image and classification of unknown new things, extracts the feature vector and group quantization of new things image, and matched new things character string is searched in benchmark alphabet；New things character string is connect with categorical match in associative memory model, is realized new things study into early teaching system；Step A60 obtains the image of things to be identified, and it is other that early education system identification obtains class of things to be identified.The present invention may be implemented the studying new knowledge together with children and know, and common contest improves the enjoyment of children for learning.
Description
Technical field
The present invention relates to smart machines, particularly relate to a kind of children companion robot and its early teaching system self-learning method.
Background technique
Preschool children not completely, need adult to accompany and keep an eye on for a long time due to intelligence and physical development.And
And child stage is the ability developments such as movement, language, mathematics most fast sensitive periods, importance is self-evident, this just needs house
It is long to pay a large amount of energy and accompany and educate children at original.Early teaching system existing at present only has audio, video mostly
Playing function interacted although the early teaching system in part can simply be interacted by way of voice or touch-control with it
Content must be existing material in system database.That is, it is " program request " system that teaching system, which mainly has, existing morning
Existing material function in system causes system without calligraphy learning in practical applications since early teaching system does not have learning ability
The system database encountered fails the new knowledge covered, and system is unable to reach in turn to be learnt to know together with children with teaching through lively activities
Object, the basic functions such as become literate, count, are unable to satisfy the requirement cooperatively grown up with children.
Summary of the invention
Do not have caused by self-learning function for current children's morning teaching system be not covered by without calligraphy learning to system it is new
The technical issues of knowledge, the present invention provide a kind of early teaching system self-learning method of children companion robot, pass through with machine
The self-learning function of the early teaching system of people improves children's study enjoyment, to realize study, common growth together with children
Purpose, and the fear for eliminating children is weary of studying mood.
To realize the above-mentioned technical purpose, the present invention adopts the following technical scheme:
A kind of children companion learns the early teaching system self-learning method of robot, comprising the following steps:
Step A10, training convolutional neural networks；
Convolutional neural networks model is constructed, using all sample images of Imagenet data set as input, sample image
Classification as label, training convolutional neural networks；
Step A20 extracts image feature information；
Feature extraction is carried out using image of the step A10 obtained convolutional neural networks of training to input, output feature to
Amount；
Step A30, to feature vector group quantization；
Feature vector is grouped by quantization using product quantification technique, forms m sub- feature vectors；
Step A40 generates benchmark alphabet；
By all sample images of Imagenet data set, is handled by step A20 and A30, obtain each sample image
M sub- feature vectors；
To all sample images, the identical subcharacter vector of sequence is taken to constitute 1 packet data collection, total total m grouping
Data set；
K is calculated using K-means algorithm to each packet data collectionsA class center records each packet data collection
KsA class center is 1 class set, and m class set constitutes benchmark alphabet in total；
Benchmark alphabet is preset in the input layer of associative memory model；
Step A50 learns new things；
The image and classification of unknown new things are obtained from early education exterior；
The new things image that will acquire is handled by step A20 and step A30, obtains m sub- feature vectors of new things；Time
M sub- feature vectors for going through new things, in the class set identical with the current subcharacter sequence vector of new things of benchmark alphabet
Matched letter is searched in conjunction, obtains the new things character string that length is m；
The node of each letter of new things character string is stored in activation associative memory model input layer, associative memory model will
The node that input layer is activated is connect with the output layer node matching for indicating new things classification；Wherein, the associative memory model
It is the binary neural network for including input layer and output layer, the node of the output layer presets things classification；
Step A60 identifies things to be identified；
The image of things to be identified is obtained from early education exterior；
The image for the things to be identified that will acquire is handled by step A20 and step A30, obtains m son of things to be identified
Feature vector；M sub- feature vectors for traversing things to be identified, in the current subcharacter with things to be identified of benchmark alphabet
Matched letter is searched in the identical class set of sequence vector, obtains the things character string to be identified that length is m；
The node of each letter of things character string to be identified, associative memory mould are stored in activation associative memory model input layer
Type searches the output node layer connecting with the node matching that input layer is activated, and exports the output corresponding things class of node layer
Not, it is other to obtain class of things to be identified for identification.
For the new things not having in early teaching system, this programme can get new things by self-learning method
In the case that image and other children/introduction personnel inform new things classification, the image of new things is connect with categorical match, is joined
Think that the matching relationship between the things image stored in memory models and classification is updated, children realize with robot is learned to things
Incremental learning.To under the introduction of children or other staff, study identification new object, new text, realize with
Children together know by studying new knowledge, mutual contest, to improve the enjoyment of children for learning.
Further, the output layer of the associative memory model is that each things classification distributes at least two node；Work as study
When new things, matched with the input layer of activation connection output node layer be under current things classification not yet with input layer section
First node of point connection.
The output layer class of things of associative memory model not Fen Bie multiple nodes, things of the like description can repeatedly be learned
It practises, records the connection relationship of a plurality of such things and classification, to improve the identification of things identification.
Further, the neural network that the associative memory model is two layers.
Further, it is searched respectively in m class set of benchmark alphabet matched with the m of things sub- feature vector
The method of letter are as follows: calculate separately current subcharacter vector and the k in corresponding class setsThe distance between a class center, takes
1 nearest class center of distance obtains m letter as 1 letter corresponding with things, by m sub- feature vectors of things,
To obtain the character string that length corresponding with things is m.
Further, class of things is not stored in the key-value pair of .txt format between the real name of things with key-value pair
In file；When learning new things, first from the extraneous real name for obtaining new things, then found in key-value pair file with very
Real name claims corresponding classification；Category information if it does not exist is then inserted into a new key-value pair, to indicate the classification newly learnt,
Class of things associative memory model is not inputed into again；When identifying things to be identified, associative memory model output is to be identified
Class of things is other, and by finding real name corresponding with classification in key-value pair file, early teaching system exports thing to be identified
The real name of object.
This programme can save memory space by storing key-value pair.
Further, cross entropy is used to carry out training convolutional neural networks model as loss function, and according to loss function
Calculated value L update each layer of convolutional neural networks weight matrix；
Wherein, the loss function is as shown in Equation 1:
Wherein, N indicates the quantity of the sample image of input convolutional neural networks every time, yiFor the class of i-th of sample image
Other true tag, y'iIt is convolutional neural networks to the predicted value of i-th of sample image.
Further, the convolutional neural networks include input layer, convolutional layer, skip floor connection, pond layer, full articulamentum and
Classification layer；When training convolutional neural networks, with the predictor calculation loss function value for layer output of classifying；Use convolutional neural networks
When extracting image feature information, the feature vector of image is constituted with the feature of full articulamentum output.
Corresponding with early teaching system self-learning method, the present invention also provides a kind of children with robot, comprising:
Function selecting module, for starting new things learning functionality or things identification function in early education system function；
Memory module, for store class of things not with the key-value pair file of real name and Memory Reference alphabet；
MIM message input module obtains the image and real name of new things when being activated for new things learning functionality, or
Things identification function obtains the image of things to be identified when being activated；
Message processing module, when new things learning functionality is activated, for according to the real name of new things got,
Classification corresponding with real name is searched in key-value pair file, then according to the image of new things and classification, according to above-mentioned side
Calligraphy learning new things；When things identification function is activated, for the image according to the things to be identified got, according to above-mentioned side
Method identifies things to be identified, and the corresponding real name of classification is then found in key-value pair file according to the classification recognized；
Message output module, for exporting the real name of things to be identified when things identification function is activated.
Further, the MIM message input module includes camera and voice-input unit.
Further, the message output module includes display screen and voice-output unit.
Beneficial effect
The present invention provides a kind of children companion robot and its early teaching system self-learning method, for not having in early teaching system
New things, this programme can informs in image and other children/the introduction personnel for getting new things by self-learning method
In the case where the real name or classification of new things, according to character of the new things image after convolutional neural networks and product quantization
The input layer of series excitation associative memory model living, and the output layer node matching of input layer and respective classes is connected,
Matching relationship between the things image stored in associative memory model and classification is updated, and children realize with robot is learned to thing
The incremental learning of object.To which under the introduction of children or other staff, study identifies new object, text, formula etc., real
Now studying new knowledge is known together with children, mutual contest, to improve the enjoyment of children for learning.
Detailed description of the invention
A specific embodiment of the invention is described in detail by way of example and not limitation with reference to the accompanying drawings hereinafter.
Attached drawing is as follows:
Fig. 1 is the children according to the embodiment of the present invention with the schematic configuration figure for learning robot；
Fig. 2 is the children according to the embodiment of the present invention with the module composition block diagram for learning robot；
Fig. 3 is the embodiment schematic diagram that children of the present invention identify object, text, formula with robot；
Fig. 4 is the learning framework schematic diagram of study module；
Fig. 5 is that the alphabet in study module generates exemplary diagram；
Fig. 6 is the result exemplary diagram of feature product quantization in study module；
Fig. 7 is with the study or identification process schematic diagram for learning robot；
Fig. 8 is convolutional neural networks model schematic, wherein (a) is the structural schematic diagram of residual block, (b) is convolutional layer
Schematic diagram, be (c) maximum pond layer schematic diagram, (d) be the schematic diagram of full articulamentum；
Fig. 9 is convolutional neural networks training flow chart；
Figure 10 is the simple neural network model that the present invention enumerates.
Drawing reference numeral: 1- camera, 2- Interactive function select area, 3- phonetic incepting and output area, 4- display screen.
Specific embodiment
Below in conjunction with attached drawing and example, the present invention is described further.
Children of the invention are with robot is learned, and schematic diagram is as shown in Figure 1, include camera 1, interactive function choosing
Select area 2, phonetic incepting and output area 3 and display screen 4.Wherein, camera 1 is located at the overhead for learning robot, main to use
The various image informations for things identification are obtained from ambient enviroment in robot.Interactive function selects area 2 to be located at machine
The front of device human body, user can be by selecting the robot in operating interactives formulas such as region click, dragging, slidings
Various functions.Phonetic incepting and output area 3 are located at the back of head that the companion learns robot, are mainly used for knowing from ambient enviroment
The feedback sound not operated respectively with the reply voice or response system of the voice and output response voice messaging that receive user
Sound.Display screen 4 is located at the response message and robot for being mainly used for showing robot with the head front for learning robot
Related expression.The present embodiment uses touch screen, has both the function in interactive function selection area and display screen.For example, if user returns
Answer that question answering is correct, then can be with one smiling face of display screen display for learning robot, other emotional expressions are similar therewith.
According to above-mentioned children with the structure for learning robot, early education system principle diagram is as shown in Figure 2.Children companion of the present invention
Learning robot morning teaching system includes: function selecting module, memory module, MIM message input module, message processing module, information output
Module.
Children of the invention have multiple functions with the early teaching system for learning robot: knowing object, become literate, count, singing, learning
Etc. functions, user select corresponding function in interactive function selection area, the function selecting module of early teaching system just starts this
Function.The interactive interface of the function selecting module both can use touch screen form, can also by way of voice command with
Convenient for users to selection.
Early teaching system has memory as memory module, for storing the parameter of convolutional neural networks, in Imagenet
Data set is gone to school benchmark alphabet that acquistion arrives, the connection type of two-value associative memory model, the output of two-value associative memory model
Corresponding classification of node (i.e. the other information of class of things namely key-value pair relationship) etc..
MIM message input module, for inputting the voice messaging of user and being extracted from robot
Object, text, formula etc. be used for the image information of system identification.
Message processing module is mainly handled the collected voice of MIM message input module and image information, completion pair
The study and identification work of relevant target object in input picture, study module therein, which can allow, to be learned with robot by related
Habit mechanism knows the Subject information of exterior.The processing of voice messaging includes the identification and understanding of voice；Image information
Processing include image preprocessing, image characteristics extraction and image recognition.The treatment process includes following two categories situation:
First is that simple identification, calls directly the model trained in existing database and identifies to image, and handle
The result of identification returns to system, can recognize common object and text, and can do simple arithmetic.
Second is that study new things, for object, the text etc. not having in system database, which, which learns robot, to pass through
The learning algorithm of study module is in the case where other children or introduction personnel inform the classification or text information of new things, benefit
With the new and old model of new data, the study of increment type is realized.After study several times, so that it may know to new things
Not, learn together with the children as user to realize, progressive purpose, is mentioned in a manner of accompanying children for learning together
Its high learning interest.
Message output module, including voice output module and display output module.Voice output module, for exporting machine
The response voice of people responds the feedback sound of this system and plays music in singing function.It shows output module, uses
The pattern that matches with voice messaging and response voice in output or with the related expression of robot, and to monitoring
The warning message etc. that human hair is sent.
Wherein, message processing module includes study module and alert module.
Alert module, by children with learn robot interaction time judge, when children's regular hour not
When with being interacted with robot, then it is assumed that children desert for a long time, the side shown by voice prompting or display screen
Formula sends warning message to guardian in time.
Study module, for knowing the new things not having in early education system database by way of self study, the new thing
Object refers to object or text information.Early teaching system of the invention, can be other staff's using function possessed by study module
Under introduction, study identifies new object, new text etc., can know studying new knowledge together with children, mutual contest, jointly into
Step, to improve the enjoyment of children for learning.
Present invention morning teaching system realizes learning functionality by study module, can also may be used while serving as children playfellow
To teach through lively activities learn the basic skills such as to know object, become literate, count together with children, and can cooperatively grow up with children.For
This, does not have the new object of data or new literacy when the present invention is for original training identification model.In system information processing module
A learning framework is constructed in study module, as shown in Figure 4.Study module is joined using convolutional neural networks (CNNS) and two-value
Think the method that memory models combine, feature extraction carried out to new samples first with the convolutional neural networks model of pre-training,
Then two-value associative memory is finally used by the Feature Mapping of extraction to a limited alphabet using product quantification technique
Model stores new samples, to realize the study to new samples.It is carried out most in identification and prediction with two-value associative memory model
Neighbor search realizes the identification to new category.The former new object not having can be known by way of self study in robot as a result,
Body, text information, thus learn together with children, it is common to grow up.As shown in Figure 4, learning framework shown in mainly includes convolution
Neural network (CNNS) and two-value associative memory model.Wherein convolutional neural networks have powerful learning ability and expression energy
Power can be very good to express using the feature that the convolutional network of data set pre-training up to a million in Imagenet data set extracts
Image.
Children of the invention are with the early teaching system self-learning method for learning robot, including step as described below:
Step A10, training convolutional neural networks:
Convolutional neural networks model is constructed first, and model used in the present embodiment is carried out on the basis of ResNet
It is correspondingly improved, has used 50 residual blocks altogether, shown in the structure of residual block such as Fig. 8 (a), each residual block includes three convolution
Layer and a jump connection, for the structural schematic diagram of convolutional layer as shown in figure (b), jump connection can be with effective solution training process
In gradient disappearance problem；Every 10 residual blocks connect a maximum pond layer, and shown in maximum pond layer such as Fig. 8 (c), pond layer can
To reduce characteristic size, and extract most notable one feature.Last then full articulamentum, shown in full articulamentum such as Fig. 8 (d),
Network used herein has used three full articulamentums altogether, and first full articulamentum has 1024 nodes, and second connects entirely
Connecing layer has 2048 nodes, the last one full articulamentum has 1000 (Imagenet data set shares 1000 classes) a nodes, most
The result of the full articulamentum of the latter just obtains the probability of 1000 classifications in Imagenet data set after softmax.?
When the subsequent extraction feature using the network, use the output of second full articulamentum as feature.
Then cross entropy is used to carry out training convolutional neural networks model as loss function, loss function is as shown in Equation 1,
Middle yiFor the classification true tag of i-th of sample image, y'iIt is convolutional neural networks to the predicted value of i-th of sample image.Mould
The process of type training is exactly that convolutional Neural net is updated further according to the difference condition of the two by comparing predicted value and true tag
The weight matrix that each layer of network, the present embodiment measure predicted value and true as loss function using cross entropy shown in formula 1
Then difference between label updates each layer in convolutional neural networks model of weight matrix according to loss function value.
Wherein N is batchsize, i.e., the quantity of the primary sample image for being sent into training.
Specifically, include propagated forward to the process of convolutional neural networks training, calculate loss function and backpropagation three
Part.Propagated forward is to obtain predicted value by convolution, pond and full articulamentum to the image for being input to convolutional neural networks
y'；Due to the true tag y of known sample, i.e. class of things shown in known sample is other, thus can according to sample true tag y,
Penalty values L is calculated by formula 1 in predicted value y'；Then backpropagation is carried out using gradient descent method and update the power of convolutional neural networks
Weight matrix.It constantly repeats the above process, until loss function value L is less than default desired value or reaches preset exercise wheel number.It is whole
A trained process is as shown in figure 9, be wherein with chain type Rule for derivation, from loss function toward forward pass during derivation.Below with complete
Back-propagation process is described in detail for articulamentum.
Figure 10 is the neural network being made of two full articulamentums, and wherein i is input, and h is middle layer, and o is output
Layer, w is weight, and the calculating of propagated forward is as follows:
h1=w1*i1+w3*i2+w5*i3 (2)
h2=w2*i1+w4*i2+w6*i3 (3)
o1=w7*h1+w9*h2 (4)
o2=w8*h1+w10*h2 (5)
Know o=[o1,o2], it is y=[y' that predicted value is obtained after the activation of softmax function1,y'2], it is specific to calculate
Such as following formula:
Assuming that the true tag of the sample is y=[y1,y2] then have penalty values as follows:
Above it is exactly propagated forward and the process for calculating penalty values, is updated the following detailed description of backpropagation and weight parameter
Process.
By penalty values L known to chain type Rule for derivation to parameter w1Gradient calculation formula it is as follows:
The gradient value of other weight parameters can be similarly calculated, the formula that parameter updates is as follows:
Wherein α is learning rate, the wheel study to neural network weight parameter is just completed by the above process, by not
Disconnected study, penalty values L can constantly decline, until converging to a certain value or certain wheel number being trained to complete convolutional neural networks
Training process, the weight matrix optimized, the convolutional neural networks of the weight matrix are the convolutional Neural net after training
Network.In the subsequent output feature vector using convolutional neural networks, feature vector is exported by the previous full articulamentum of classification layer.
Step A20 extracts image feature information: carrying out feature extraction, shape using image of the convolutional neural networks to input
The feature vector for being d at dimension.Wherein, the feature vector extracted is by the previous complete of the classification layer of relationship neural network
Articulamentum exports to obtain.
Step A30, to feature vector group quantization: using product quantification technique, the feature vector amount of being grouped that d is tieed up
Change, form m sub- feature vectors, the dimension of each subcharacter vector is d/m.
Step A40 generates benchmark alphabet:
All sample images of Imagenet data set are obtained, quantity amounts to n, and each sample image presses step A20
With A30 processing, then n sample image obtains m sub- feature vectors；
To all sample images, take the identical subcharacter vector of sequence, the identical subcharacter of the sequence of n sample image to
Amount constitutes 1 packet data collection, and each sample standard deviation has m sub- feature vectors, therefore always amounts to m packet data collection.Sequence herein
Identical to refer to, each sample image has m sub- feature vectors, numbers in order to m sub- feature vectors, takes each sample graph
The identical subcharacter vector of sequence number as in, to obtain the packet data collection of the sequence number.Due to each sample data
There are m sub- feature vectors, therefore m packet data collection can be obtained in all sample images.For example, each sample image, take its 1st
1st sub- feature vector of a sub- feature vector, all sample images constitutes the 1st packet data collection；Then, each sample graph
Picture, takes its 2nd sub- feature vector, and the 2nd sub- feature vector of all sample images constitutes the 2nd packet data
Collection；……；Finally obtain m packet data collection.
Each packet data collection is calculated using K-means algorithm the k of each packet data collectionsA class center, note
The ksA class center is 1 class set, then m class set constitutes benchmark alphabet in total.
Example as shown in Figure 5 assumes a total of n sample in figure, the characteristic dimension d=12 extracted, packet count m=4,
Class center is k in each groupsA, the result quantified is as shown in Figure 6, it is assumed here that ks=4 include 4 to get the alphabet arrived
A letter group (i.e. m class set, respectively J=1,2,3,4), each letter group includes 4 letter (i.e. k againsA class center point
It Wei not Cij, indicate j-th of class center in i-th group of subcharacter vector), we utilize the letter obtained in Imagenet data set
For table as benchmark alphabet, Imagenet data include thousands of classes, picture up to a million, are gone to school the letter that acquistion arrives in these data
Table can very effective summary characteristics of image, thus improve things identification when identification.
Then benchmark alphabet is preset in the input layer of associative memory model again.
Step A50 learns new things:
When the learning functionality of early teaching system starting new things, as shown in figure 4, passing through camera from system with robot is learned
The image of the unknown new things of external acquisition system, other children or introduction personnel are unknown new by voice informing robot system
The real name of things.
The new things image that will acquire is handled to obtain the feature vector x of new things by step A20d, then by step A30 processing
Obtain m sub- feature vectors of new thingsThe serial number of i expression subcharacter vector.
In the k of i-th of class set of benchmark alphabetsSearched inside a class center with i-th of subcharacter of new things to
Most matched letter is measured, until m sub- feature vectors of new thingsMost matched letter is all found, so that obtaining length is m
New things character string.Since each class set is obtained by the identical subcharacter vector of sequence of sample, new things
When study, and in the identical class set of sequence search matched letter.
Wherein, the method for most matched letter is searched are as follows: calculate separately i-th of subcharacter vector and i-th class set
ksEuclidean distance between a class center takes class center corresponding to distance minimum as corresponding 1 letter of thing new things, newly
Total m sub- feature vectors of things then obtain m letter, so that obtaining length corresponding with new things is the character string of m, and export
To in associative memory model.Such as in Fig. 7, the 1st sub- feature vectorIt is found at 4 class centers and is apart from nearest letter
C11, the 2nd sub- feature vectorFinding at 4 class centers apart from nearest letter is C23, the 3rd sub- feature vectorAt 4
It is C that class center, which is found apart from nearest letter,34, the 4th sub- feature vectorIt is found at 4 class centers and is apart from nearest letter
C42, the character string which obtains after product quantifies is C11C23C34C42,
It is right with the real name of new things institute to search in the key-value pair file for the .txt format being stored in memory module
The classification answered, and node corresponding with the new things classification is found in the output layer of associative memory model.If in key-value pair text
Category information is not found in part, is indicated that the things did not learnt before being, is then inserted into a new key in key-value pair file
Value pair.
Respective nodes in the input layer of each letter activation associative memory model of new things character string, associative memory model
The input layer that new things character string activates is connect with the output layer node matching for indicating new things classification.
In the present embodiment, associative memory model be include the binary neural network of input layer and output layer, and output layer
Node preset things classification.
The output layer of associative memory model is that each things classification distributes at least two node, and sharp when learn new things
The output node layer of input layer matching connection living is first not yet connect with input layer under current things classification
A node illustrates the things sufficiently study of the category into system until the node of all categories has been assigned.By right
Things of the like description is repeatedly learnt, and the connection relationship between such a plurality of things and classification is recorded, thus when improving things identification
Identification.
Associative memory model in the present embodiment is realized using two layers neural network: input layer is by m group node
Composition, every group includes ksA node, input layer are used to correspond to each k of m class set of Memory Reference alphabetsClass center；
Output layer includes RC node, and wherein C indicates that the new things classification sum that can learn, R indicate that associative memory model is every class thing
The number of nodes of object distribution, C=9, R=2 are assumed in Fig. 7, R and C can be taken to a very big value in practice, therefore only need
Seldom storage demand can be achieved with the incremental learning of very multiclass and data volume.
Such as in Fig. 7, letter C that the new things of study obtain after product quantifies11、C23、C34、C42It is input to association's note
Recall model, activates the respective nodes in the input layer of associative memory model (since benchmark alphabet is preset in associative memory respectively
The input layer of model, therefore respective nodes here refer to and are stored with letter C in input layer respectively11、C23、C34、C42
Each node)；Associative memory model is that the new things classification (assuming that classification is ' 6 ') is exporting the 6th category node of Layer assignment,
And activate the 6th classification first node that connection is not present, i.e. ' η61'；Again and by each node C of input layer11、C23、C34、C42
Respectively with new things category node η61Carry out matching connection.Since the associative memory model in the present invention is two-value memory mould
Type, therefore there is no connection weight, only there is connection and there is no connections.When R node of the category has been assigned, indicate
Sufficiently study thereby realizes the study to new samples into system to the things of the category.
Companion robot is similar to children's study process to the learning process of new things, with repetition learning, to thing
The accuracy of object identification can step up, and form a kind of situation for learning to improve jointly jointly with children, this with children classmate
Role accompanies the mode of children for learning, rather than in a manner of instructing children for learning, can be disappeared by the role of the teacher of children
Except the fear of children is weary of studying mood, children's study interest is improved.
The contents such as object, text are stored in internal system in various forms by current existing system, and when use recalls,
Lack entity object.And the present invention is external Subject with the things that robot is identified is learned, and passes through computer vision skill
The image that art, i.e. borrow camera obtain things, by self study come object, text, the formula outside identifying system, and pronounces
Object names are said, the calculated result of text and formula is read out, therefore the interest and the sense of reality of existed system can be improved.
Step A60 identifies things to be identified:
When the things identification function of early teaching system is activated, with learn robot by camera from exterior obtain to
Identify the image of things；
The image for the things to be identified that will acquire is handled to obtain the feature vector x of things to be identified by step A20d, then press
Step A30 handles to obtain m sub- feature vectors of things to be identifiedThe serial number of i expression subcharacter vector.
In the k of i-th of class set of benchmark alphabetsIt is searched inside a class center special with i-th of son of things to be identified
The most matched letter of vector is levied, until m sub- feature vectors of things to be identified all find most matched letter, to be grown
Degree is the character string of the things to be identified of m.Wherein, search the method for most matched letter are as follows: calculate separately i-th of subcharacter to
Amount and ksEuclidean distance between a class center takes class center corresponding to distance minimum corresponding 1 as things to be identified
Letter, the total m of things to be identified sub- feature vectors then obtain m letter, so that obtaining length corresponding with things to be identified is m
Character string, and export in associative memory model.
The respective nodes of the input layer of each letter activation associative memory model of things character string to be identified, associative memory mould
Type searches the output node layer that connection is matched with the input layer of activation, and exports the output corresponding things classification of node layer.
In the key-value pair file for the .txt format being stored in memory module search with class of things to be identified it is other pair
Then the real name of things to be identified is learned robot by display screen or voice output children companion by the real name answered.
In the present invention, things can be any one in the various forms such as object, text, formula, and class of things is not
Refer to the coded sequence for each different things.When things refers specifically to object, the real name of things refers to the Chinese name of object
Title, phonetic and/or English name；When things specifically refers to text, the real name of things refers to the phonetic of text, paraphrase
Deng when things specifically refers to formula, the real name of things refers to operation result.
Fig. 3 is the embodiment schematic diagram that children of the present invention identify object, text, number with robot.Such as this
Shown in Fig. 3, when with learning robot and seeing object by the camera of overhead, it will this object of automatic identification, and by the object
The title of body informs user.For example, being seen when being in things identification function with robot morning teaching system when with robot
When to the apple for being placed in front, will automatically identify the object is apple, and display screen actively exports the object names, or
When user inquires that this is to give what when to answer with robot is learned.Inform that the mode of the real name of user's things both may be used
To be output by voice, " apple " two word and its corresponding phonetic and English word can also be shown on a display screen, or
The two haves both at the same time.It is similar therewith, when being in character identification function, see the Chinese character in front of being placed in (such as when companion learns robot
" playing chess " word) when, the Chinese character can be also automatically identified, and can actively export phonetic, the paraphrase of the Chinese character, or inquire in user
When give and answer.In addition, children of the present invention can also carry out addition subtraction multiplication and division four with robot is learned other than it can know object, character learning
Then operation.For example, being placed in front with the card or machine of " 6+4 " when seeing with robot when being in formula function
People hear user inquire its " 6+4=? " when, this robot can voice informing user's operation result be " 6+4=10 ", simultaneously
Digital " 6+4=10 " is shown on a display screen.
If things identification target object system according to things classification existing in Imagenet data set picture, or
The things classification learnt by new things, which, which learns robot, can accomplish to identify at any time, that is, informs and use at once after identifying
The real name and relevant information of person's things.Wherein existing things classification in Imagenet data set picture is with machine
Device people's system learns into associative memory model in advance, increasing when learning method and above-mentioned steps A50 are used with robot
The method of amount formula study is identical.If target object is system not and did not learn, which, which learns robot, can star morning
The new things learning functionality of teaching system learns the new things that was met in the case where user informs new things classification
Classification enables correctly to identify with robot when seeing this object next time.
For example, children of the present invention are with robot if there is the data of pear in initial Imagenet data set picture
After seeing pear, it can identify immediately and inform user this is pear.And for not having in initial tranining database
Apple, parent or other introductions personnel can inform companion robot this are with learning when robot camera sees apple
Apple, which can realize the study of increment type by phase new things learning functionality, so as to next time and as user
Children recognize the object together.
The present invention has supervisory role with robot is learned, by sentencing to children with the interaction time for learning robot
It is disconnected, when children's regular hour with robot with not interacting, then it is assumed that children desert for a long time, then pass through
The mode that voice prompting or display screen are shown sends warning message to guardian in time, avoid children do it is some unsafe
Thing；Voice prompting and display screen show the interest that can trigger children simultaneously, and children is allowed more to carry out with playfellow robot
Interaction.
The children of the present embodiment know robot technology, image procossing, mode with robot and children's morning teaching system
Other Technology application intelligently responds the various movements and state of user, it is good to give user in the company of preschool child
Good usage experience.Compared to existing early education robot system, the invention patent system has the advantage that 1) this system has
Learning functionality can learn identification new object, new text, learn together with children to realize under the guidance of related personnel
The target practise, grown up jointly, saves parent the time it takes and energy in terms of children's getting up early education；2) this system is more
The classmate for being positioned at children, rather than the teacher of children, the mood so that fear for being conducive to eliminate children is weary of studying；3) this system
It can use the Subject in the image recognition external world of camera acquisition, therefore this system is with relatively good interest and really
Sense；4) this system has supervisory role, can send warning letter to guardian in child behavior exception, as deserted for a long time
Breath.
So far, although those skilled in the art will appreciate that present invention has been shown and described in detail herein is exemplary
Embodiment still without departing from the spirit and scope of the present invention, can still directly determine according to the present disclosure
Or derive many other deformations or modification for meeting the principle of the invention.Therefore, the scope of the present invention is it should be understood that and assert
To cover other all these deformations or modification.
Claims (10)
1. a kind of children are with the early teaching system self-learning method for learning robot, which comprises the following steps:
Step A10, training convolutional neural networks；
Convolutional neural networks model is constructed, using all sample images of Imagenet data set as input, the class of sample image
Not Zuo Wei label, training convolutional neural networks；
Step A20 extracts image feature information；
The convolutional neural networks obtained using step A10 training carry out feature extraction to the image of input, export feature vector；
Step A30, to feature vector group quantization；
Feature vector is grouped by quantization using product quantification technique, forms m sub- feature vectors；
Step A40 generates benchmark alphabet；
It by all sample images of Imagenet data set, is handled by step A20 and A30, obtains m of each sample image
Subcharacter vector；
To all sample images, takes the identical subcharacter vector of sequence to constitute 1 packet data collection, always amount to m packet data
Collection；
K is calculated using K-means algorithm to each packet data collectionsA class center records the k of each packet data collectionsIt is a
Class center is 1 class set, and m class set constitutes benchmark alphabet in total；
Benchmark alphabet is preset in the input layer of associative memory model；
Step A50 learns new things；
The image and classification of unknown new things are obtained from early education exterior；
The new things image that will acquire is handled by step A20 and step A30, obtains m sub- feature vectors of new things；Traversal is new
M sub- feature vectors of things, in the class set identical with the current subcharacter sequence vector of new things of benchmark alphabet
Matched letter is searched, the new things character string that length is m is obtained；
The node of each letter of new things character string is stored in activation associative memory model input layer, associative memory model will input
The node that layer is activated is connect with the output layer node matching for indicating new things classification；Wherein, the associative memory model is packet
The binary neural network of input layer and output layer is included, the node of the output layer presets things classification；
Step A60 identifies things to be identified；
The image of things to be identified is obtained from early education exterior；
The image for the things to be identified that will acquire is handled by step A20 and step A30, obtains m subcharacter of things to be identified
Vector；M sub- feature vectors for traversing things to be identified, in the current subcharacter vector with things to be identified of benchmark alphabet
Matched letter is searched in the identical class set of sequence, obtains the things character string to be identified that length is m；
The node of each letter of things character string to be identified is stored in activation associative memory model input layer, associative memory model is looked into
The output node layer connecting with the node matching that input layer is activated is looked for, and exports the output corresponding things classification of node layer, is known
It is other that class of things to be identified is not obtained.
2. the method according to claim 1, wherein the output layer of the associative memory model is each things class
It Fen Pei not at least two node；When learning new things, it is current that the output node layer of connection is matched with the input layer of activation
First node not yet being connect with input layer under things classification.
3. the method according to claim 1, wherein the neural network that the associative memory model is two layers.
4. the method according to claim 1, wherein in m class set of benchmark alphabet respectively search with
The method of the m of the things sub- matched letters of feature vector are as follows: calculate separately current subcharacter vector in corresponding class set
KsThe distance between a class center, 1 class center for taking distance nearest is as 1 letter corresponding with things, by the m of things
A sub- feature vector obtains m letter, to obtain the character string that length corresponding with things is m.
5. the method according to claim 1, wherein class of things is not between the real name of things with key assignments
To in the key-value pair file for being stored in .txt format；When learning new things, first from the extraneous real name for obtaining new things, then
Classification corresponding with real name is found in key-value pair file；Category information if it does not exist is then inserted into a new key
Class of things to indicate the classification newly learnt, then is not inputed to associative memory model by value pair；When identifying things to be identified, connection
Think memory models output is that class of things to be identified is other, by finding true name corresponding with classification in key-value pair file
Claim, early teaching system exports the real name of things to be identified.
6. the method according to claim 1, wherein cross entropy is used to carry out training convolutional nerve as loss function
Network model, and according to the weight matrix of the calculated value L of loss function update each layer of convolutional neural networks；
Wherein, the loss function is as shown in Equation 1:
Wherein, N indicates the quantity of the sample image of input convolutional neural networks every time, yiClassification for i-th of sample image is true
Label, y'iIt is convolutional neural networks to the predicted value of i-th of sample image.
7. according to the method described in claim 6, it is characterized in that, the convolutional neural networks include input layer, convolutional layer, jump
Layer connection, pond layer, full articulamentum and classification layer；When training convolutional neural networks, with the predictor calculation damage for layer output of classifying
Lose functional value；When extracting image feature information using convolutional neural networks, image is constituted with the feature of full articulamentum output
Feature vector.
8. a kind of children are with robot characterized by comprising
Function selecting module, for starting new things learning functionality or things identification function in early education system function；
Memory module, for store class of things not with the key-value pair file of real name and Memory Reference alphabet；
MIM message input module, the image of acquisition new things and real name or things when being activated for new things learning functionality
The image of things to be identified is obtained when identification function is activated；
Message processing module, when new things learning functionality is activated, for the real name according to the new things got, in key
Value is to classification corresponding with real name is searched in file, then according to the image of new things and classification, according to claim 1 institute
The method stated learns new things；When things identification function is activated, for according to the image of things to be identified got, according to
According to method described in claim 1, things to be identified is identified, then found in key-value pair file according to the classification recognized
The corresponding real name of classification；
Message output module, for exporting the real name of things to be identified when things identification function is activated.
9. a kind of children according to claim 8 are with robot, which is characterized in that the MIM message input module includes taking the photograph
As head and voice-input unit.
10. a kind of children according to claim 8 are with learning robot, which is characterized in that the message output module includes
Display screen and voice-output unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811367002.9A CN109559576B (en) | 2018-11-16 | 2018-11-16 | Child accompanying learning robot and early education system self-learning method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811367002.9A CN109559576B (en) | 2018-11-16 | 2018-11-16 | Child accompanying learning robot and early education system self-learning method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109559576A true CN109559576A (en) | 2019-04-02 |
| CN109559576B CN109559576B (en) | 2020-07-28 |
Family
ID=65866326
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811367002.9A Active CN109559576B (en) | 2018-11-16 | 2018-11-16 | Child accompanying learning robot and early education system self-learning method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109559576B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111078008A (en) * | 2019-12-04 | 2020-04-28 | 东北大学 | Control method of early education robot |
| CN112036208A (en) * | 2019-05-17 | 2020-12-04 | 深圳市希科普股份有限公司 | Artificial intelligence sprite based on interactive learning system |
| WO2021089059A1 (en) * | 2019-11-06 | 2021-05-14 | 昆山提莫智能科技有限公司 | Method and apparatus for smart object recognition, object recognition device, terminal device, and storage medium |
| CN113673795A (en) * | 2020-05-13 | 2021-11-19 | 百度在线网络技术（北京）有限公司 | Method and device for acquiring online teaching material content and intelligent screen equipment |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103729459A (en) * | 2014-01-10 | 2014-04-16 | 北京邮电大学 | Method for establishing sentiment classification model |
| CN104866855A (en) * | 2015-05-07 | 2015-08-26 | 华为技术有限公司 | Image feature extraction method and apparatus |
| CN105512685A (en) * | 2015-12-10 | 2016-04-20 | 小米科技有限责任公司 | Object identification method and apparatus |
| US20160140438A1 (en) * | 2014-11-13 | 2016-05-19 | Nec Laboratories America, Inc. | Hyper-class Augmented and Regularized Deep Learning for Fine-grained Image Classification |
| CN105913087A (en) * | 2016-04-11 | 2016-08-31 | 天津大学 | Object identification method based on optimal pooled convolutional neural network |
| CN106409290A (en) * | 2016-09-29 | 2017-02-15 | 深圳市唯特视科技有限公司 | Infant intelligent voice education method based on image analysis |
| US20170098153A1 (en) * | 2015-10-02 | 2017-04-06 | Baidu Usa Llc | Intelligent image captioning |
| CN107918782A (en) * | 2016-12-29 | 2018-04-17 | 中国科学院计算技术研究所 | A kind of method and system for the natural language for generating description picture material |
| CN108304846A (en) * | 2017-09-11 | 2018-07-20 | 腾讯科技（深圳）有限公司 | Image-recognizing method, device and storage medium |
| CN108460399A (en) * | 2017-12-29 | 2018-08-28 | 华南师范大学 | A kind of child building block builds householder method and system |
-
2018
- 2018-11-16 CN CN201811367002.9A patent/CN109559576B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103729459A (en) * | 2014-01-10 | 2014-04-16 | 北京邮电大学 | Method for establishing sentiment classification model |
| US20160140438A1 (en) * | 2014-11-13 | 2016-05-19 | Nec Laboratories America, Inc. | Hyper-class Augmented and Regularized Deep Learning for Fine-grained Image Classification |
| CN104866855A (en) * | 2015-05-07 | 2015-08-26 | 华为技术有限公司 | Image feature extraction method and apparatus |
| US20170098153A1 (en) * | 2015-10-02 | 2017-04-06 | Baidu Usa Llc | Intelligent image captioning |
| CN105512685A (en) * | 2015-12-10 | 2016-04-20 | 小米科技有限责任公司 | Object identification method and apparatus |
| CN105913087A (en) * | 2016-04-11 | 2016-08-31 | 天津大学 | Object identification method based on optimal pooled convolutional neural network |
| CN106409290A (en) * | 2016-09-29 | 2017-02-15 | 深圳市唯特视科技有限公司 | Infant intelligent voice education method based on image analysis |
| CN107918782A (en) * | 2016-12-29 | 2018-04-17 | 中国科学院计算技术研究所 | A kind of method and system for the natural language for generating description picture material |
| CN108304846A (en) * | 2017-09-11 | 2018-07-20 | 腾讯科技（深圳）有限公司 | Image-recognizing method, device and storage medium |
| CN108460399A (en) * | 2017-12-29 | 2018-08-28 | 华南师范大学 | A kind of child building block builds householder method and system |
Non-Patent Citations (1)
| Title |
|---|
| 谢振平等: "自组织决策树的联想记忆在线学习模型", 《模式识别与人工智能》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112036208A (en) * | 2019-05-17 | 2020-12-04 | 深圳市希科普股份有限公司 | Artificial intelligence sprite based on interactive learning system |
| CN112036208B (en) * | 2019-05-17 | 2021-10-08 | 深圳市希科普股份有限公司 | Artificial intelligence sprite based on interactive learning system |
| WO2021089059A1 (en) * | 2019-11-06 | 2021-05-14 | 昆山提莫智能科技有限公司 | Method and apparatus for smart object recognition, object recognition device, terminal device, and storage medium |
| CN111078008A (en) * | 2019-12-04 | 2020-04-28 | 东北大学 | Control method of early education robot |
| CN113673795A (en) * | 2020-05-13 | 2021-11-19 | 百度在线网络技术（北京）有限公司 | Method and device for acquiring online teaching material content and intelligent screen equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109559576B (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109447140B (en) | Image identification and cognition recommendation method based on neural network deep learning | |
| US20220239988A1 (en) | Display method and apparatus for item information, device, and computer-readable storage medium | |
| CN108021616B (en) | Community question-answer expert recommendation method based on recurrent neural network | |
| US11113598B2 (en) | Dynamic memory network | |
| CN110148318B (en) | Digital teaching assistant system, information interaction method and information processing method | |
| US20160350653A1 (en) | Dynamic Memory Network | |
| CN109559576A (en) | A kind of children companion robot and its early teaching system self-learning method | |
| CN110379225A (en) | The system and method for interactive language acquisition | |
| CN110060097A (en) | User behavior sequence of recommendation method based on attention mechanism and convolutional neural networks | |
| CN108536679A (en) | Name entity recognition method, device, equipment and computer readable storage medium | |
| CN109460479A (en) | A kind of prediction technique based on reason map, device and system | |
| CN109213864A (en) | Criminal case anticipation system and its building and pre-judging method based on deep learning | |
| CN111915460B (en) | AI vision-based intelligent scoring system for experimental examination | |
| CN109829049A (en) | The method for solving video question-answering task using the progressive space-time attention network of knowledge base | |
| CN112527993B (en) | Cross-media hierarchical deep video question-answer reasoning framework | |
| US20210366022A1 (en) | Matching users with visual items | |
| Sosnovshchenko et al. | Machine learning with Swift: artificial intelligence for iOS | |
| CN115545160B (en) | Knowledge tracking method and system for multi-learning behavior collaboration | |
| CN113360635A (en) | Intelligent teaching method and system based on self-attention and pre-training mechanism | |
| CN112948710A (en) | Graph neural network-based punishment education recommendation method, system and storage medium | |
| Sha et al. | Neural knowledge tracing | |
| CN116187349A (en) | Visual question-answering method based on scene graph relation information enhancement | |
| Doering et al. | Neural-network-based memory for a social robot: Learning a memory model of human behavior from data | |
| CN113705159A (en) | Merchant name labeling method, device, equipment and storage medium | |
| CN108154165A (en) | Love and marriage object matching data processing method, device, computer equipment and storage medium based on big data and deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |